A "@type copy" statement is already present at the beginning of each
match element, so extra "type copy" are not needed. They are causing the
following warnings in fluentd logs:
[warn]: parameter 'type' in <match syslog.local0.**>
[warn]: parameter 'type' in <match syslog.local1.**>
This commit also harmonizes indentation of the Monasca config block.
Change-Id: I779c2b942d007acbdd43d999f2fc0cdc131d431f
Related-Bug: #1885873
The common role was previously added as a dependency to all other roles.
It would set a fact after running on a host to avoid running twice. This
had the nice effect that deploying any service would automatically pull
in the common services for that host. When using tags, any services with
matching tags would also run the common role. This could be both
surprising and sometimes useful.
When using Ansible at large scale, there is a penalty associated with
executing a task against a large number of hosts, even if it is skipped.
The common role introduces some overhead, just in determining that it
has already run.
This change extracts the common role into a separate play, and removes
the dependency on it from all other roles. New groups have been added
for cron, fluentd, and kolla-toolbox, similar to other services. This
changes the behaviour in the following ways:
* The common role is now run for all hosts at the beginning, rather than
prior to their first enabled service
* Hosts must be in the necessary group for each of the common services
in order to have that service deployed. This is mostly to avoid
deploying on localhost or the deployment host
* If tags are specified for another service e.g. nova, the common role
will *not* automatically run for matching hosts. The common tag must
be specified explicitly
The last of these is probably the largest behaviour change. While it
would be possible to determine which hosts should automatically run the
common role, it would be quite complex, and would introduce some
overhead that would probably negate the benefit of splitting out the
common role.
Partially-Implements: blueprint performance-improvements
Change-Id: I6a4676bf6efeebc61383ec7a406db07c7a868b2a
Time format in Ruby Time.strptime is not accepting padding flags,
therefore we need to remove them for the Fluentd to be able
to parse MariaDB xinetd logs properly.
Change-Id: Iabfa9afdcad505106a5580eb2d058273ee5f7c1f
Closes-Bug: #1886002
In Fluentd v0.12, both the in memory and file buffer chunk size default
to 8MB. In v1.0 the file buffer defaults to 256MB. This can exceed the
Monasca Log or Unified API maximum chunk size which is set to 10MB.
This can result in logs being rejected and filling the local buffer
on disk.
Change-Id: I9c495773db726a3c5cd94b819dff4141737a1d6e
Closes-Bug: #1885885
Co-Authored-By: Sebastian Luna Valero <sebastian.luna.valero@gmail.com>
Resolve trivial syntax error in Fluentd output config for Monasca.
Change-Id: I20b37bb83a76bfabb1126925a1b4f1f59767b7a3
Co-Authored-By: Sebastian Luna Valero <sebastian.luna.valero@gmail.com>
Closes-Bug: #1885873
Currently there is no way to configure a CA certificate bundle file for
fluentd to Elasticsearch communication. This change adds a new variable,
'fluentd_elasticsearch_cacert' with a default value set to the value of
'openstack_cacert.
Closes-Bug: #1885109
Change-Id: I5bbf55a4dd4ccce9fa2635cee720139c088268e3
Replaced "kolla_external_fqdn_cacert" and "kolla_internal_fqdn_cacert" with
"kolla_admin_openrc_cacert". OS_CACERT is now set to the value of
"kolla_admin_openrc_cacert" in the generated admin-openrc.sh file.
Change-Id: If195d5402579cee9a14b91f63f5fde84eb84cccf
Partially-Implements: blueprint add-ssl-internal-network
Depends-On: https://review.opendev.org/#/c/731344/
The Monasca Log API has been removed and in this change we switch
to using the unified API. If dedicated log APIs are required then
this can be supported through configuration. Out of the box the
Monasca API is used for both logs and metrics which is envisaged to
work for most use cases.
In order to use the unified API for logs, we need to disable the
legacy Kafka client. We also rename the Monasca API config file
to remove a warning about using the old style name.
Depends-On: https://review.opendev.org/#/c/728638
Change-Id: I9b6bf5b6690f4b4b3445e7d15a40e45dd42d2e84
Add TLS support for Glance api using HAProxy to perform TLS termination.
Change-Id: I77051baaeb5d3f7dd9002262534e7d35f3926809
Partially-Implements: blueprint add-ssl-internal-network
Zun has a new component "zun-cni-daemon" which should be
deployed in every compute nodes. It is basically an implementation
of CNI (Container Network Interface) that performs the neutron
port binding.
If users is using the capsule (pod) API, the recommended deployment
option is using "cri" as capsule driver. This is basically to use
a CRI runtime (i.e. CRI plugin for containerd) for supporting
capsules (pods). A CRI runtime needs a CNI plugin which is what
the "zun-cni-daemon" provides.
The configuration is based on the Zun installation guide [1].
It consits of the following steps:
* Configure the containerd daemon in the host. The "zun-compute"
container will use grpc to communicate with this service.
* Install the "zun-cni" binary at host. The containerd process
will invoke this binary to call the CNI plugin.
* Run a "zun-cni-daemon" container. The "zun-cni" binary will
communicate with this container via HTTP.
Relevant patches:
Blueprint: https://blueprints.launchpad.net/zun/+spec/add-support-cri-runtime
Install guide: https://review.opendev.org/#/c/707948/
Devstack plugin: https://review.opendev.org/#/c/705338/
Kolla image: https://review.opendev.org/#/c/708273/
[1] https://docs.openstack.org/zun/latest/install/index.html
Depends-On: https://review.opendev.org/#/c/721044/
Change-Id: I9c361a99b355af27907cf80f5c88d97191193495
Kolla Ansible was missing vitrage-persistor service
required by Vitrage for data storage.
Depends on fixing availability of Kolla image.
Change-Id: I8158ba66b8b624f6bcb89da9c990a30a68b7187b
Depends-On: Id5e143636f9a81e7294b775f3d8b9134bee58054
Closes-Bug: #1869319
Refactor service configuration to use the copy certificates task. This
reduces code duplication and simplifies implementing encrypting backend
HAProxy traffic for individual services.
Change-Id: I0474324b60a5f792ef5210ab336639edf7a8cd9e
Some services look for /etc/timezone on Debian/Ubuntu, so we should
introduce it to the containers.
In addition, added prechecks for /etc/localtime and /etc/timezone.
Closes-Bug: #1821592
Change-Id: I9fef14643d1bcc7eee9547eb87fa1fb436d8a6b3
kolla-toolbox container name variable is fixed in some places,
but in the defaults directory, kolla-toolbox container_name variable
is variable. If the kolla-toolbox container_name variable is changed
during deployment, it will not be assigned to kolla-toolbox,
but a fixed 'kolla-toolbox' name.
Change-Id: I9579017761ff47477dba597282be9ae6fab4242a
The 'kolla-ansible stop' command can be used to stop the services
running on hosts. However, if you run this command in an environment
with heterogeneous nodes (most real world scenarios have at least
control/compute), then it fails. This is because it only checks
whether a container is enabled, and not whether the host is in the
correct group. For example, it fails with nova-libvirt:
No such container: nova_libvirt to stop.
This change fixes the issue by only attempting to stop containers on
hosts to which they are mapped.
Change-Id: Ibecac60d1417269bbe25a280996ca9de6e6d018f
Closes-Bug: #1868596
Fluentd cannot accept empty 'path' parameter.
I refactored the service list following the general pattern
we have.
Change-Id: I83d820efcc7e86bac9f8bda26a8f8bece72159e6
Closes-bug: #1867953
Currently, config folders lack the execute bit so Fluentd
cannot read the config and just does nothing when it starts up. This
change explicitly sets the execute bit on folders which need it,
rather than doing it in a more generic way which is more risky from
a security perspective.
Change-Id: Ia840f4b67043df4eaa654f47673dcdc973f13d9c
Closes-Bug: #1867754
I didn't use a for loop as the logic for omitting the
comma for the final element dirties the logic.
Change-Id: Id29d5deebcc5126d69a1bd8395e0df989f2081f0
When change the cert file in /etc/kolla/certificate/.
The certificate in the container has not changed.
So I think can use kolla-ansible deploy when certificate is
changed. restart <container>
Partially-Implements: blueprint custom-cacerts
Change-Id: Iaac6f37e85ffdc0352e8062ae5049cc9a6b3db26
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
We already only include .conf files in fluent.conf:
(fluentd)[fluentd@cpu-e-1041 /etc/fluentd]$ cat fluent.conf
@include input/*.conf
@include filter/*.conf
@include format/*.conf
@include output/*.conf
so this change should not cause ill effect. This works because of the
merge option in config files:
merge: merges the source directory into the target directory instead of
replacing it. Boolean, defaults to false.
see https://docs.openstack.org/kolla/latest/admin/kolla_api.html#kolla-api-external-config
Change-Id: I28f63ec81f1ea5bc4a213d053bfb2c04388d5925
Closes-Bug: #1862211
The logrotate rotation interval and count are not configurable.
Currently, the configuration is a "default" that keeps 6 weeks of logs.
Change-Id: I4f55ee2a98f7861cb8de2724f5edc32da6d2f9ee
Both include_role and import_role expect role's name to be given
via "name" param instead of "role".
This worked but caused errors with ansible-lint.
See: https://review.opendev.org/694779
Change-Id: I388d4ae27111e430d38df1abcb6c6127d90a06e0
We assume that all groups are present in the inventory, and quite obtuse
errors can result if any are not.
This change adds a precheck that checks for the presence of all expected
groups in the inventory for each service. It also introduces a common
service-precheck role that we can use for other common prechecks.
Change-Id: Ia0af1e7df4fff7f07cd6530e5b017db8fba530b3
Partially-Implements: blueprint improve-prechecks
Kolla-Ansible Ceph deployment mechanism has been deprecated in Train [1].
This change removes the Ansible code and associated CI jobs.
[1]: https://review.opendev.org/669214
Change-Id: Ie2167f02ad2f525d3b0f553e2c047516acf55bc2
By default a retry limit of 17 exists. When the limit is reached buffered
logs are discarded. To avoid this, we disable the retry limit. The risk of
bringing down the host by filling the Fluent data docker volume is managed
by the maximum buffer size which is 2GB by default.
In summary, after this change, the net behaviour is that Fluentd should
buffer up to a maximum of 2GB of logs locally, and attept to post them to
the Monasca Log API at intervals not exceeding 30 minutes.
Closes-Bug: #1855702
Change-Id: I0d5a3dab29635c00411f4f51e5a0721726df2abd
This enables buffering to file, rather than memory for Monasca logs.
A dedicated docker volume is used for the file buffer. If a post
to the Monasca Log API fails, retries will be made using an exponential
backoff algorithm with a maximum retry interval of 30mins. The maximum
interval is set relatively low to try and reduce the risk of large
buffers accumulating, and therefore the risk of overloading the Monasca
Log API.
Closes-Bug: #1855700
Change-Id: Ib5286e9dbaf2bc92d2f4960b2131223ab5dbdbec
When kolla_copy_ca_into_containers is set to "yes", the Certificate
Authority in /etc/kolla/certificates will be copied into service
containers to enable trust for that CA. This is especially useful when
the CA is self signed, and would not be trusted by default.
Partially-Implements: blueprint custom-cacerts
Change-Id: I4368f8994147580460ebe7533850cf63a419d0b4
