5589 Commits

Author SHA1 Message Date
Pierre Riteau
c81772024c Reduce the use of SQLAlchemy connection pooling
When the internal VIP is moved in the event of a failure of the active
controller, OpenStack services can become unresponsive as they try to
talk with MariaDB using connections from the SQLAlchemy pool.

It has been argued that OpenStack doesn't really need to use connection
pooling with MariaDB [1]. This commit reduces the use of connection
pooling via two configuration options:

- max_pool_size is set to 1 to allow only a single connection in the
  pool (it is not possible to disable connection pooling entirely via
  oslo.db, and max_pool_size = 0 means unlimited pool size)
- lower connection_recycle_time from the default of one hour to 10
  seconds, which means the single connection in the pool will be
  recreated regularly

These settings have shown better reactivity of the system in the event
of a failover.

[1] http://lists.openstack.org/pipermail/openstack-dev/2015-April/061808.html

Change-Id: Ib6a62d4428db9b95569314084090472870417f3d
Closes-Bug: #1896635
2020-09-22 17:54:45 +02:00
Radosław Piliszek
3916c156be Add support for with_frontend and with_backend
This allows for more config flexibility - e.g. running multiple
backends with a common frontend.

Note this is a building block for future work on letsencrypt
validator (which should offer backend and share frontend with
any service running off 80/443 - which would be only horizon
in the current default config), as well as any work towards
single port (that is single frontend) and multiple services
anchored at paths of it (which is the new recommended default).

Change-Id: Ie088fcf575e4b5e8775f1f89dd705a275725e26d
Partially-Implements: blueprint letsencrypt-https
2020-09-22 17:26:42 +02:00
Radosław Piliszek
9451ac61a0 Change the default haproxy template to split variant
This allows for more config flexibility - e.g. running multiple
backends with a common frontend.
It is not possible with the 'listen' approach (which enforces
frontend).
Additionally, it does not really make sense to support two ways
to do the exact same thing as the process is automated and
'listen' is really meant for humans not willing to write separate
sections.
Hence this deprecates 'listen' variant.

At the moment both templates work exactly the same.
The real flexibility comes in following patches.

Note this is a building block for future work on letsencrypt
validator (which should offer backend and share frontend with
any service running off 80/443 - which would be only horizon
in the current default config), as well as any work towards
single port (that is single frontend) and multiple services
anchored at paths of it (which is the new recommended default).

Change-Id: I2362aaa3e8069fe146d42947b8dddf49376174b5
Partially-Implements: blueprint letsencrypt-https
2020-09-22 16:49:36 +02:00
Radosław Piliszek
a45ef7ccaa Fix default mode in haproxy_single_service_split
haproxy_single_service_listen (the default template) was already fine.

Closes-Bug: #1896591
TrivialFix

Change-Id: Id68fe19ea87565aa36fb74f2a2ca66cb951169f6
2020-09-22 11:58:38 +02:00
Michal Nasiadka
f257e79aff Allow setting container_proxy per service
Currently there is no option to set container_proxy only for one service
(e.g. magnum). This change adds this option.

Change-Id: Ia938ee660ebe8ce84321f721b6292b0b58a06e20
2020-09-22 10:54:40 +02:00
Radosław Piliszek
bce266201b Allow to skip and unset sysctl vars
via KOLLA_SKIP and KOLLA_UNSET

Change-Id: I7d9af21c2dd8c303066eb1ee4dff7a72bca24283
Related-Bug: #1837551
2020-09-21 13:13:58 +02:00
Radosław Piliszek
6be51fa67a Add support for changing sysctl.conf path
via kolla_sysctl_conf_path

Change-Id: I09b20fa008a7fecedcb599b4792f24215179b853
2020-09-21 11:47:05 +02:00
Zuul
cccfa8f378 Merge "Fix glance-tls-proxy logrotate and fluentd log permissions" 2020-09-21 09:04:53 +00:00
wu.chunyang
88de8feb7b replace internal with openstack_interface
replace harcode 'internal' with {{ openstack_interface }}

Change-Id: I885622967ffde2a7a1a08fedbde2eb0e4e330e22
2020-09-18 21:42:52 +08:00
Michal Nasiadka
aed9f84fe9 Fix glance-tls-proxy logrotate and fluentd log permissions
Change-Id: Iabc0115d3476a626df134cc70cb473bf6e72487e
Closes-Bug: #1890439
2020-09-18 08:51:36 +00:00
Zuul
91f5861769 Merge "Support neutron_sriov_physnet_mappings to support multiple devices" 2020-09-17 16:53:26 +00:00
Zuul
90e4795f50 Merge "Change mariadb image to mariadb-server" 2020-09-17 16:53:21 +00:00
Bharat Kunwar
c24a280bee Support neutron_sriov_physnet_mappings to support multiple devices
Change-Id: Ifcedcc72307732393a92a702a7567addc043b5b2
2020-09-17 13:26:30 +00:00
Mark Goddard
761ea9a333 Support TLS encryption of RabbitMQ client-server traffic
This change adds support for encryption of communication between
OpenStack services and RabbitMQ. Server certificates are supported, but
currently client certificates are not.

The kolla-ansible certificates command has been updated to support
generating certificates for RabbitMQ for development and testing.

RabbitMQ TLS is enabled in the all-in-one source CI jobs, or when
The Zuul 'tls_enabled' variable is true.

Change-Id: I4f1d04150fb2b5af085b762890092f87ae6076b5
Implements: blueprint message-queue-ssl-support
2020-09-17 12:05:44 +01:00
Zuul
127c3072b4 Merge "replace openstackclient with ansible module" 2020-09-17 11:03:28 +00:00
Zuul
fbef9b36d6 Merge "Performance: use a single config file for fluentd" 2020-09-17 11:03:26 +00:00
Zuul
bc388d5657 Merge "Performance: use a single config file for logrotate" 2020-09-17 10:55:01 +00:00
Michal Nasiadka
a7941e2498 Change mariadb image to mariadb-server
Since change [1] merged we have two mariadb images (mariadb and mariadb-server)
Let's use mariadb-server in kolla-ansible, so we can deprecate mariadb image.

[1]: https://review.opendev.org/#/c/710217/

Change-Id: I4ae2ccaaba8fb516f469f4ce8628e8c61de03f0d
2020-09-17 10:42:21 +00:00
wu.chunyang
0bb16b52f6 replace openstackclient with ansible module
replace 'openstack aggregate create' command with ansible
os_nova_host_aggregate module and remove TODO

Change-Id: I727f9e4acc9e22f59735c65190ac38cc75e5f781
2020-09-17 11:41:27 +08:00
Pierre Riteau
3d30624cc1 Revert "Add support for encrypting Ironic API"
This reverts commit 316b0496b3dd7a9b33692b171391d9d17d535116, because
ironic-inspector is not ready to use WSGI. It would need to be split
into two separate containers, one running ironic-inspector-api-wsgi and
another running ironic-inspector-conductor.

Change-Id: I7e6c59dc8ad4fdee0cc6d96313fe66bc1d001bf7
2020-09-10 15:26:06 +00:00
Zuul
ec34132b25 Merge "Synchronize REST_API_REQUIRED_SETTINGS with Horizon" 2020-09-09 09:17:35 +00:00
Zuul
f10b5336cc Merge "Set neutron-ovn-metadata-agent metadata_workers to 2" 2020-09-09 09:03:59 +00:00
Zuul
b9fd7d8e92 Merge "Add support for encrypting Ironic API" 2020-09-08 08:50:50 +00:00
Michal Nasiadka
dcc417dbec Set neutron-ovn-metadata-agent metadata_workers to 2
As per [1] and [2] - it solves a problem, where neutron-ovn-metadata-agent will
spawn high number of workers (defaults to half number of CPUs).

[1]: http://lists.openstack.org/pipermail/openstack-discuss/2020-September/016960.html
[2]: https://bugs.launchpad.net/neutron/+bug/1893656

Change-Id: Id69f9399fe76ff7c4e2e17b5ab5ec7df1a01c5c9
2020-09-07 10:22:57 +00:00
Pierre Riteau
295f8d1b43 Remove unused configuration for prometheus-openstack-exporter
The Prometheus OpenStack exporter was needlessly configured to use the
prometheus Docker volume and change permissions of /data, which does
not exist in the container image.

This must have been copy-pasted from existing Prometheus code.

Change-Id: I96017c17e68ca7a00a2d5ac41f2f43ef87694514
2020-09-01 14:15:52 +02:00
Zuul
3316daad83 Merge "Performance: use import_tasks for register and bootstrap" 2020-08-31 11:30:59 +00:00
James Kirsch
316b0496b3 Add support for encrypting Ironic API
This patch introduces an optional backend encryption for the Ironic API
and Ironic Inspector service. When used in conjunction with enabling
TLS for service API endpoints, network communcation will be encrypted
end to end, from client through HAProxy to the Ironic service.

Change-Id: I3e82c8ec112e53f907e89fea0c8c849072dcf957
Partially-Implements: blueprint add-ssl-internal-network
Depends-On: https://review.opendev.org/#/c/742776/
2020-08-29 15:25:49 +00:00
Mark Goddard
496904d650 Performance: use import_tasks for register and bootstrap
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. In the case of the register.yml and bootstrap.yml
includes, all of the tasks in the included file use run_once: True.
The run_once flag improves performance at scale drastically, so
importing these tasks unconditionally will have a lower overhead than a
conditional include task.  It therefore makes sense to switch to use
import_tasks there.

See [1] for benchmarks of run_once.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/run-once.md

Change-Id: Ic67631ca3ea3fb2081a6f8978e85b1522522d40d
Partially-Implements: blueprint performance-improvements
2020-08-28 16:31:04 +00:00
Mark Goddard
3c02c966cb Performance: remove one include_tasks in nova-cell
Including tasks has a performance penalty when compared with importing
tasks. The nova-cell role uses include_tasks twice when generating
certificates and keys for libvirt TLS. While a dynamic include makes
sense here for a non-default feature, we can use one include rather than
two with the same effect. Since this task runs against compute nodes the
overhead is significant.

See [1] for benchmarks of include_tasks and import_tasks.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/include-and-import.md

Partially-Implements: blueprint performance-improvements

Change-Id: Ic687d2f7d4625aede386e576ebb174da72142756
2020-08-28 16:16:56 +00:00
Mark Goddard
b685ac44e0 Performance: replace unconditional include_tasks with import_tasks
Including tasks has a performance penalty when compared with importing
tasks. If the include has a condition associated with it, then the
overhead of the include may be lower than the overhead of skipping all
imported tasks. For unconditionally included tasks, switching to
import_tasks provides a clear benefit.

Benchmarking of include vs. import is available at [1].

This change switches from include_tasks to import_tasks where there is
no condition applied to the include.

[1] https://github.com/stackhpc/ansible-scaling/blob/master/doc/include-and-import.md#task-include-and-import

Partially-Implements: blueprint performance-improvements

Change-Id: Ia45af4a198e422773d9f009c7f7b2e32ce9e3b97
2020-08-28 16:12:03 +00:00
Zuul
48da350000 Merge "Drop support for /etc/modules" 2020-08-28 15:39:15 +00:00
Michal Nasiadka
8464046eac Fix bifrost deployment after removal of env-vars
Change-Id: I1318a5ea7f746012cc3055d4c03fd0508df921ff
2020-08-28 10:19:13 +02:00
Zuul
fa48cc7eaf Merge "Use iSCSI multipath for libvirt" 2020-08-26 13:57:47 +00:00
Zuul
68471ada13 Merge "Update conf for magnum" 2020-08-26 13:37:02 +00:00
Zuul
a8eb112838 Merge "Fix tz handling for newly introduced services" 2020-08-26 13:35:37 +00:00
Zuul
50999f4af3 Merge "Drop option http-tunnel for neutron-server" 2020-08-26 13:22:11 +00:00
Mark Goddard
98e8a43668 Drop support for /etc/modules
We no longer support any OS distros that use /etc/modules.

Change-Id: Ic41cad64ed0479a0cef7321aeed191fe06a30e8f
2020-08-25 20:20:57 +01:00
Zuul
36c388bb5e Merge "add region name for tasks" 2020-08-25 09:59:00 +00:00
Zuul
d6b7f648de Merge "Disable arp_responder option for LinuxBridge agent" 2020-08-25 09:58:56 +00:00
wu.chunyang
60dc012817 add region name for tasks
when use multiple regions, those tasks always get a wrong endpoint.

Change-Id: Ie164687b2ffd80cedf8a00e7f705b73fccd416e2
2020-08-25 11:25:41 +08:00
Zuul
3f54490781 Merge "Fix fernet bootstrap and key distribution - follow up" 2020-08-24 19:55:22 +00:00
Nick Jones
e721c6ae0b
Disable arp_responder option for LinuxBridge agent
Enabling both l2_population and arp_responder for LinuxBridge can cause
problems in some configurations [0].  This commit removes the explicit
'true', reverting it to the default which is 'False'.

Closes-Bug: #1892776

[0] https://bugs.launchpad.net/neutron/+bug/1661717

Change-Id: Ia9445a651fd7a082835a858964bcb9e8e325338d
Signed-off-by: Nick Jones <nick@dischord.org>
2020-08-24 20:34:53 +01:00
Zuul
0d63e380b4 Merge "Drop python-path" 2020-08-24 11:03:08 +00:00
Zuul
02ba8bb847 Merge "Prevent overwriting existing Keystone Fernet keys" 2020-08-24 10:59:13 +00:00
Zuul
42f57166d4 Merge "replace os-tenant-name with os-project-name in openstackclient" 2020-08-24 10:27:40 +00:00
Zuul
4bde4f4349 Merge "remove useless files from destroy role" 2020-08-24 10:27:38 +00:00
Zuul
d4ef57130c Merge "change the timezone precheck task's condition" 2020-08-24 10:27:36 +00:00
Michal Nasiadka
793a37e2ff Fix fernet bootstrap and key distribution - follow up
As per post-merge comments in [0] - following up.

[0]: https://review.opendev.org/#/c/698710

Change-Id: I92b3de7fb792f1fffe298ffaf6bbafab8e640742
2020-08-24 10:43:47 +01:00
Radosław Piliszek
7c42204a65 Drop option http-tunnel for neutron-server
Change-Id: I6910bcdb122e42337d7938405f792fa0fe32e49b
Closes-Bug: #1892686
2020-08-24 10:01:27 +02:00
Radosław Piliszek
9c38a0c77b Drop python-path
It was found to be useless in [1].

It is one of distro_python_version usages.

Note Freezer and Horizon still use python_path (and hence
distro_python_version) for different purposes.

[1] https://review.opendev.org/675822

Change-Id: I6d6d9fdf4c28cb2b686d548955108c994b685bb1
Partially-Implements: blueprint drop-distro-python-version
2020-08-24 07:38:21 +00:00