128 Commits

Author SHA1 Message Date
LinPeiWen
5c2d1c90be Use Docker healthchecks for octavia services
This change enables the use of Docker healthchecks for octavia services.
Implements: blueprint container-health-check

Change-Id: I2ec11544b0e9c66a1821bfe0f9a40a9e9d88033b
2021-03-03 11:08:16 +00:00
Carsten Koester
bf6d9308aa Add IPv6 configuration options to Octavia management network
If the Octavia/Amphora management network is created by Kolla, support
setting the IP address family and IPv6 address/RA mode.

Closes-Bug: 1913409

Change-Id: I9f2ef2196654c91596cb5c4b3c157bcee267226a
2021-02-03 08:24:04 -08:00
Piotr Parczewski
5db72659a0 [docs] Unify project's naming convention
There are inconsitencies across the documentation and the source code files
when it comes to project's name (Kolla Ansible vs. Kolla-Ansible). This
commit aims at unifying it so that the naming becomes consistent everywhere.

Change-Id: I903b2e08f5458b1a1abc4af3abefe20b66c23a54
2021-01-27 20:08:41 +01:00
Zuul
07659dd96f Merge "Register resources info for octavia upgrading" 2021-01-12 13:45:57 +00:00
wu.chunyang
7f7bb1b448 Register resources info for octavia upgrading
kolla-ansible upgrade failed when octavia_auto_configure set
to true. because upgrade action don't register the resources
info.
this change adds some tasks to query the resources info
for upgrade action in octavia role.

Change-Id: I4b0ac001b38bee81d983dd68534b9d0e78b4f6d7
2021-01-09 18:34:49 +08:00
Christian Berendt
b42688f87b octavia: fix typo in defaults
Signed-off-by: Christian Berendt <berendt@betacloud-solutions.de>
Change-Id: Ib0c96479c79ce3dfa7469a58b1ea8ca038defe1e
2021-01-04 15:05:13 +01:00
Zuul
860c32de76 Merge "Revert "Performance: Use import_tasks in the main plays"" 2020-12-15 19:52:24 +00:00
Mark Goddard
db4fc85c33 Revert "Performance: Use import_tasks in the main plays"
This reverts commit 9cae59be51e8d2d798830042a5fd448a4aa5e7dc.

Reason for revert: This patch was found to introduce issues with fluentd customisation. The underlying issue is not currently fully understood, but could be a sign of other obscure issues.

Change-Id: Ia4859c23d85699621a3b734d6cedb70225576dfc
Closes-Bug: #1906288
2020-12-14 10:36:55 +00:00
Radosław Piliszek
71e9c603b8 Do not set 'always' tag where unnecessary
Makes 'import_tasks' not change behaviour compared to
'include_tasks'.

Change-Id: I600be7c3bd763b3b924bd4a45b4e7b4dca7a33e3
2020-10-27 19:51:46 +01:00
Radosław Piliszek
9cae59be51 Performance: Use import_tasks in the main plays
Main plays are action-redirect-stubs, ideal for import_tasks.

This avoids 'include' penalty and makes logs/ara look nicer.

Fixes haproxy and rabbitmq not to check the host group as well.

Change-Id: I46136fc40b815e341befff80b54a91ef431eabc0
Partially-Implements: blueprint performance-improvements
2020-10-27 19:09:32 +01:00
Radosław Piliszek
3411b9e420 Performance: optimize genconfig
Config plays do not need to check containers. This avoids skipping
tasks during the genconfig action.

Ironic and Glance rolling upgrades are handled specially.

Swift and Bifrost do not use the handlers at all.

Partially-Implements: blueprint performance-improvements
Change-Id: I140bf71d62e8f0932c96270d1f08940a5ba4542a
2020-10-12 19:30:06 +02:00
wu.chunyang
4fe4820b1f remove redundant line and replace octavia user
remove redundant space line
replace octavia user with {{ octavia_keystone_user }}

Change-Id: I284acc580a1a530eede3e0227febe8667dea5d47
2020-10-09 17:26:37 +08:00
wu.chunyang
195269d758 add octavia openrc file
we use octavia user to upload image currently, so it is better to
create a octavia openrc file for user

Implements: blueprint implement-automatic-deploy-of-octavia

Change-Id: Ib53d00fa4a6ee59b8a0b2245f83786a6af0cbf53
2020-10-08 14:50:52 +00:00
wu.chunyang
4a58f4238c Implement automatic deploy of octavia
this patchset has implemented:
  - network (lb-mgmt-net)
  - security groups and rules (used by amphora and health manager)
  - amphora flavor (used by amphora)
  - nova keypair (used by amphora at the time of debugging)

Add a octavia_amp_listen_port variable which used by amphora
Add amp_image_owner_id in octavia.conf

Implements: blueprint implement-automatic-deploy-of-octavia
Co-Authored-By: zhangchun <zhangchun@yovole.com>

Depends-On: https://review.opendev.org/652030

Change-Id: I67009d046925cfc02c1e0073c80085c1471975f6
2020-10-02 14:05:00 +02:00
Zuul
ba933f16e9 Merge "Support TLS encryption of RabbitMQ client-server traffic" 2020-09-29 11:31:03 +00:00
Pierre Riteau
c81772024c Reduce the use of SQLAlchemy connection pooling
When the internal VIP is moved in the event of a failure of the active
controller, OpenStack services can become unresponsive as they try to
talk with MariaDB using connections from the SQLAlchemy pool.

It has been argued that OpenStack doesn't really need to use connection
pooling with MariaDB [1]. This commit reduces the use of connection
pooling via two configuration options:

- max_pool_size is set to 1 to allow only a single connection in the
  pool (it is not possible to disable connection pooling entirely via
  oslo.db, and max_pool_size = 0 means unlimited pool size)
- lower connection_recycle_time from the default of one hour to 10
  seconds, which means the single connection in the pool will be
  recreated regularly

These settings have shown better reactivity of the system in the event
of a failover.

[1] http://lists.openstack.org/pipermail/openstack-dev/2015-April/061808.html

Change-Id: Ib6a62d4428db9b95569314084090472870417f3d
Closes-Bug: #1896635
2020-09-22 17:54:45 +02:00
Mark Goddard
761ea9a333 Support TLS encryption of RabbitMQ client-server traffic
This change adds support for encryption of communication between
OpenStack services and RabbitMQ. Server certificates are supported, but
currently client certificates are not.

The kolla-ansible certificates command has been updated to support
generating certificates for RabbitMQ for development and testing.

RabbitMQ TLS is enabled in the all-in-one source CI jobs, or when
The Zuul 'tls_enabled' variable is true.

Change-Id: I4f1d04150fb2b5af085b762890092f87ae6076b5
Implements: blueprint message-queue-ssl-support
2020-09-17 12:05:44 +01:00
Mark Goddard
496904d650 Performance: use import_tasks for register and bootstrap
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. In the case of the register.yml and bootstrap.yml
includes, all of the tasks in the included file use run_once: True.
The run_once flag improves performance at scale drastically, so
importing these tasks unconditionally will have a lower overhead than a
conditional include task.  It therefore makes sense to switch to use
import_tasks there.

See [1] for benchmarks of run_once.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/run-once.md

Change-Id: Ic67631ca3ea3fb2081a6f8978e85b1522522d40d
Partially-Implements: blueprint performance-improvements
2020-08-28 16:31:04 +00:00
Mark Goddard
b685ac44e0 Performance: replace unconditional include_tasks with import_tasks
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. For unconditionally included tasks, switching to
import_tasks provides a clear benefit.

Benchmarking of include vs. import is available at [1].

This change switches from include_tasks to import_tasks where there is
no condition applied to the include.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/include-and-import.md#task-include-and-import

Partially-Implements: blueprint performance-improvements

Change-Id: Ia45af4a198e422773d9f009c7f7b2e32ce9e3b97
2020-08-28 16:12:03 +00:00
wu.chunyang
60dc012817 add region name for tasks
when use multiple regions, those tasks always get a wrong endpoint.

Change-Id: Ie164687b2ffd80cedf8a00e7f705b73fccd416e2
2020-08-25 11:25:41 +08:00
Rafael Weingärtner
f425c0678f Standardize use and construction of endpoint URLs
The goal for this push request is to normalize the construction and use
 of internal, external, and admin URLs. While extending Kolla-ansible
 to enable a more flexible method to manage external URLs, we noticed
 that the same URL was constructed multiple times in different parts
 of the code. This can make it difficult for people that want to work
 with these URLs and create inconsistencies in a large code base with
 time. Therefore, we are proposing here the use of
 "single Kolla-ansible variable" per endpoint URL, which facilitates
 for people that are interested in overriding/extending these URLs.

As an example, we extended Kolla-ansible to facilitate the "override"
of public (external) URLs with the following standard
"<component/serviceName>.<companyBaseUrl>".
Therefore, the "NAT/redirect" in the SSL termination system (HAproxy,
HTTPD or some other) is done via the service name, and not by the port.
This allows operators to easily and automatically create more friendly
 URL names. To develop this feature, we first applied this patch that
 we are sending now to the community. We did that to reduce the surface
  of changes in Kolla-ansible.

Another example is the integration of Kolla-ansible and Consul, which
we also implemented internally, and also requires URLs changes.
Therefore, this PR is essential to reduce code duplicity, and to
facility users/developers to work/customize the services URLs.

Change-Id: I73d483e01476e779a5155b2e18dd5ea25f514e93
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2020-08-19 07:22:17 +00:00
Mark Goddard
146b00efa7 Mount /etc/timezone based on host OS
Previously we mounted /etc/timezone if the kolla_base_distro is debian
or ubuntu. This would fail prechecks if debian or ubuntu images were
deployed on CentOS. While this is not a supported combination, for
correctness we should fix the condition to reference the host OS rather
than the container OS, since that is where the /etc/timezone file is
located.

Change-Id: Ifc252ae793e6974356fcdca810b373f362d24ba5
Closes-Bug: #1882553
2020-08-10 10:14:18 +01:00
Mark Goddard
9702d4c3c3 Performance: use import_tasks for check-containers.yml
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. In the case of the check-containers.yml include, the
included file only has a single task, so the overhead of skipping this
task will not be greater than the overhead of the task import. It
therefore makes sense to switch to use import_tasks there.

Partially-Implements: blueprint performance-improvements

Change-Id: I65d911670649960708b9f6a4c110d1a7df1ad8f7
2020-07-28 12:10:59 +01:00
Zuul
39909a600c Merge "Performance: remove unnecessary conditions from includes" 2020-07-24 07:52:37 +00:00
wu.chunyang
5c0f18793c add missing conditional for octavia
octavia deploy failed due to mount a empyt directroy into container

Change-Id: Ifd95126da59f649b02ab39c0b209df4750bdcfce
2020-07-23 20:05:16 +08:00
Zuul
9a8341c2a7 Merge "Performance: Run common role in a separate play" 2020-07-17 15:43:22 +00:00
Jeffrey Zhang
1cba949f84 Manage octavia health manager worker through openstack_service worker
Change-Id: Idfa7feb9157cdf12af9641be10c8eb8478930f15
Closes-Bug: #1886747
2020-07-08 11:36:04 +08:00
Mark Goddard
56ae2db7ac Performance: Run common role in a separate play
The common role was previously added as a dependency to all other roles.
It would set a fact after running on a host to avoid running twice. This
had the nice effect that deploying any service would automatically pull
in the common services for that host. When using tags, any services with
matching tags would also run the common role. This could be both
surprising and sometimes useful.

When using Ansible at large scale, there is a penalty associated with
executing a task against a large number of hosts, even if it is skipped.
The common role introduces some overhead, just in determining that it
has already run.

This change extracts the common role into a separate play, and removes
the dependency on it from all other roles. New groups have been added
for cron, fluentd, and kolla-toolbox, similar to other services. This
changes the behaviour in the following ways:

* The common role is now run for all hosts at the beginning, rather than
  prior to their first enabled service
* Hosts must be in the necessary group for each of the common services
  in order to have that service deployed. This is mostly to avoid
  deploying on localhost or the deployment host
* If tags are specified for another service e.g. nova, the common role
  will *not* automatically run for matching hosts. The common tag must
  be specified explicitly

The last of these is probably the largest behaviour change. While it
would be possible to determine which hosts should automatically run the
common role, it would be quite complex, and would introduce some
overhead that would probably negate the benefit of splitting out the
common role.

Partially-Implements: blueprint performance-improvements

Change-Id: I6a4676bf6efeebc61383ec7a406db07c7a868b2a
2020-07-07 15:00:47 +00:00
Mark Goddard
7ff27de7ac Performance: remove unnecessary conditions from includes
There are a number of tasks where we conditionally use include_tasks
with a condition, and the condition is always true. This change removes
these conditions, in preparation for switching unconditional task
includes to task imports.

Partially-Implements: blueprint performance-improvements

Change-Id: I3804c440fe3552950d9d434ef5409f685c39bbcf
2020-07-07 15:50:58 +01:00
wu.chunyang
36b93dd6e2 Add support of octavia dev mod
Similarly to other OpenStack services octavia should support
kolla dev mod for debugging.

Change-Id: I81b79dc0a4c5e40a67af7120a4109dfe11098a97
2020-06-18 22:27:23 +08:00
Zuul
e2acee87ee Merge "Clean up the unnecessary "" for include_tasks" 2020-06-17 20:27:49 +00:00
Zuul
e2e77ccf01 Merge "Switch octavia to use service project in service_auth" 2020-06-17 17:46:27 +00:00
gugug
f220970d46 Clean up the unnecessary "" for include_tasks
The double quotation is not necessary for include_tasks, this
ps to cleanup it.

Change-Id: I0701035d185fdf19286cced7fe51fc277511e4c1
2020-06-16 23:36:42 +08:00
Xing Zhang
c2037885e7 Switch octavia to use service project in service_auth
Recently a patch [1] was merged to stop adding the octavia user to the
admin project, and remove it on upgrade. However, the octavia
configuration was not updated to use the service project, causing load
balancer creation to fail.

There is also an issue for existing deployments in simply switching to
the service project. While existing load balancers appear to continue to
work, creating new load balancers fails due to the security group
belonging to the admin project. At a minimum, the deployer needs to
create a security group in the service project, and update
'octavia_amp_secgroup_list' to match its ID. Ideally the flavor and
network would also be recreated in the service project, although this
does not seem to impact operation and will result in downtime for
existing Amphorae.

This change adds a new variable, 'octavia_service_auth_project', that
can be used to set the project. The default in Ussuri is 'service',
switching to the new behaviour. For backports of this patch it should be
switched to 'admin' to maintain compatibility.

If a deployer sets 'octavia_service_auth_project' to 'admin', the
octavia user will be assigned the admin role in the admin project, as
was done previously.

Closes-Bug: #1882643
Related-Bug: #1873176

[1] https://review.opendev.org/720243/

Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Change-Id: I1efd0154ebaee69373ae5bccd391ee9c68d09b30
2020-06-16 12:57:56 +01:00
gugug
a7c290765b Remove the unnecessary task for octavia
Since the Victoria released, the unnecessary task should be clean up.

Change-Id: Idd2a05ed0594dcca6fa9881dee63f5550cb6dc0e
2020-06-10 00:11:15 +08:00
Zuul
8d4157a510 Merge "Adapt to Octavia Certificate Configuration Guide." 2020-04-29 11:46:54 +00:00
Noboru Iwamatsu
e84c968ed2 Adapt to Octavia Certificate Configuration Guide.
This patch updates the octavia controller deployment to use the
latest octavia certificate configuration guide [1]. The dual CA changes
were introduced in Train.

[1] https://docs.openstack.org/octavia/latest/admin/guides/certificates.html

Change-Id: If89ec0d631568db70690f1a69d00115c59abe678
Closes-Bug: #1862133
2020-04-29 08:30:12 +03:00
Xing Zhang
01ae01ec26
Make sure octavia uses internal endpoint to barbican
The octavia service communicates to the barbican service with
public endpoint_type by default[1], it should use internal
like other services.

[1] 0056b5175f/octavia/common/config.py (L533-L537)

Closes-Bug: #1875618
Change-Id: I90d2b0aeac090a3e2366341e260232fc1f0d6492
2020-04-28 18:55:32 +08:00
Zuul
1939c3ce68 Merge "[octavia] Adds region_name if enable_barbican" 2020-04-27 13:51:12 +00:00
ramboman
6c372312ad [octavia] Adds region_name if enable_barbican
Adds necessary "region_name" to octavia.conf when
"enable_barbican" is set to "true".

Closes-Bug: #1867926

Change-Id: Ida61cef4b9c9622a5e925bac4583fba281469a39
2020-04-27 09:53:57 +03:00
Xing Zhang
bb7e1e8660 Remove octavia user from admin project
It is unnecessary to add octavia user into admin project.
Octavia project does not require this action. Like other projects,
octavia user in service project with admin role is enough.

[1] https://docs.openstack.org/octavia/latest/install/install-ubuntu.html
[2] https://docs.openstack.org/octavia/latest/contributor/guides/dev-quick-start.html#production-deployment-walkthrough
[3] https://github.com/openstack/octavia/blob/master/devstack/plugin.sh

Closes-Bug: #1873176
Change-Id: I35d35177aaabfc6f0abc533a1f756b363bd02308
2020-04-17 09:50:33 +00:00
Zuul
87984f5425 Merge "Add Ansible group check to prechecks" 2020-04-16 15:33:46 +00:00
Zuul
7f42813159 Merge "Refactor copy certificates task" 2020-04-16 14:03:37 +00:00
lixuehai
ea4505f17a Fix Octavia CA cert paths
This fixes Octavia in scenarios requiring providing
CA cert (self-signed, internally-signed).

Change-Id: I60b7ec85f4fd8bbacf5df0ab7ed9a00658c91871
Closes-Bug: #1872404
2020-04-16 11:11:46 +02:00
James Kirsch
4d155d69cd Refactor copy certificates task
Refactor service configuration to use the copy certificates task. This
reduces code duplication and simplifies implementing encrypting backend
HAProxy traffic for individual services.

Change-Id: I0474324b60a5f792ef5210ab336639edf7a8cd9e
2020-04-14 17:26:19 +00:00
Dincer Celik
4b5df0d866 Introduce /etc/timezone to Debian/Ubuntu containers
Some services look for /etc/timezone on Debian/Ubuntu, so we should
introduce it to the containers.

In addition, added prechecks for /etc/localtime and /etc/timezone.

Closes-Bug: #1821592
Change-Id: I9fef14643d1bcc7eee9547eb87fa1fb436d8a6b3
2020-04-09 18:53:36 +00:00
Mark Goddard
0edad7138c Remove default(omit) from openstack_cacert in templates
The use of default(omit) is for module parameters, not templates. We
define a default value for openstack_cacert, so it should never be
undefined anyway.

Change-Id: Idfa73097ca168c76559dc4f3aa8bb30b7113ab28
2020-04-03 14:49:11 +01:00
Zuul
2a2ce059dc Merge "Add notify restart container when cert changed" 2020-03-10 12:12:55 +00:00
yj.bai
d3cc2f670e Add notify restart container when cert changed
When change the cert file in /etc/kolla/certificate/.
The certificate in the container has not changed.
So I think can use kolla-ansible deploy when certificate is
changed. restart <container>

Partially-Implements: blueprint custom-cacerts

Change-Id: Iaac6f37e85ffdc0352e8062ae5049cc9a6b3db26
Signed-off-by: yj.bai <bai.yongjun@99cloud.net>
2020-03-10 16:23:09 +08:00
Radosław Piliszek
266fd61ad7 Use "name:" instead of "role:" for *_role modules
Both include_role and import_role expect role's name to be given
via "name" param instead of "role".
This worked but caused errors with ansible-lint.
See: https://review.opendev.org/694779

Change-Id: I388d4ae27111e430d38df1abcb6c6127d90a06e0
2020-03-02 10:01:17 +01:00