Build upon changes in kolla which change strategy of installing projects
in containers when in dev mode. This fixes problems where when package
file manifest changes, the changes were not reflected in to
devmode-enabled container.
It changes the strategy of installing projects in dev mode in containers.
Instead of bind mounting the project's git repository to the venv
of the container, the repository is bind mounted to
/dev-mode/<project_name> from which the it is installed using pip
on every startup of the container using kolla_install_projects script.
Also updates docs to reflect the changes.
Depends-On: https://review.opendev.org/c/openstack/kolla/+/925712
Closes-Bug: #1814515
Singed-off-by: Roman Krček <roman.krcek@tietoevry.com>
Change-Id: If191cd0e3fcf362ee058549a1b6c244d109b6d9a
Refactor that prepares kolla_container_facts
module for introducing more actions that will be moved
from kolla_container module and kolla_container_volume_facts.
This change is based on a discussion about adding a new action
to kolla_container module that retrieves all names of the running
containers. It was agreed that kolla-ansible should follow Ansible's
direction of splitting modules between action modules and facts
modules. Because of this, kolla_container_facts needs to be able
to handle different requests for data about containers or volumes.
Change-Id: Ieaec8f64922e4e5a2199db2d6983518b124cb4aa
Signed-off-by: Ivan Halomi <ivan.halomi@tietoevry.com>
Change I60162b54bc06e158534d29311d4474b34750c64d
removed the '/v3' suffix from horizon_keystone_url variable,
but the version is needed for some operations.
This patch fixes the "Change password" Horizon function
until Horizon bug #2073639 is resolved.
Closes-Bug: #2073159
Change-Id: I6ff46b47e9109d0757f2e5ce8019ba591b9892e1
The Kolla project supports building images with
user-defined prefixes. However, Kolla-ansible is unable
to use those images for installation.
This patch fixes that issue.
Closes-Bug: #2073541
Change-Id: Ia8140b289aa76fcd584e0e72686e3786215c5a99
Most roles are not leveraging the jinja filters available.
According to [1] filtering the list of services makes the execution
faster than skipping the tasks.
This patchset also includes some cosmetic changes to genconfig.
Individual services are now also using a jinja filter. This has
no impact on performance, just makes the tasks look cleaner.
Naming of some vars in genconfig was changed to "service" to make
the tasks more uniform as some were previously using
the service name and some were using "service".
Three metrics from the deployment were taken and those were
- overall deployment time [s]
- time spent on the specific role [s]
- CPU usage (measured with perf) [-]
Overall genconfig time went down on avg. from 209s to 195s
Time spent on the loadbalancer role went down on avg. from 27s to 23s
Time spent on the neutron role went down on avg from 102s to 95s
Time spent on the nova-cell role went down on avg. from 54s to 52s
Also the average CPUs utilized reported by perf went down
from 3.31 to 3.15.
For details of how this was measured see the comments in gerrit.
[1] - https://github.com/stackhpc/ansible-scaling/blob/master/doc/skip.md
Change-Id: Ib0f00aadb6c7022de6e8b455ac4b9b8cd6be5b1b
Signed-off-by: Roman Krček <roman.krcek@tietoevry.com>
consider this a security hardening
as it would be possible to write to host
owned private tmp files e.g. of systemd-logind
when you are able to highjack the apache2 process
inside the horizon container, which runs as root.
see the bug report for a demonstration of this.
I checked the horizon code, it only facilitates
python tempfiles module for temp file usage.
I also checked the horizon container we build
via `kolla-build -b ubuntu horizon`, which has
a /tmp/ directory.
So no mountpoint should be needed.
Closes-Bug: #2068126
Signed-off-by: Sven Kieske <kieske@osism.tech>
Change-Id: I7ae1db8d42c83b773047bb01e846d4abee02710a
New horizon release use [1] for cache backend
instead of [2] as it was in previous versions.
This patch:
1. Removes override from config and
configure only memcached endpoints, not backend
specification itself. This will avoid bugs
in future in case BACKEND will be switched again.
2. Remove 'memcached' context from kolla_address filter
and use 'url' as [1] don't support inet6:[{address}]
for ipv6 but supports [{address}] which 'url' provides.
[1] django.core.cache.backends.memcached.PyMemcacheCache
[2] django.core.cache.backends.memcached.MemcachedCache
Change-Id: Ie3a8f47e7b776b6aa2bb9b1522fdd4514ea1484b
The purpose of this patch is to make it easier to
review changes, because renaming and changing the
file in one patch will generate diff when the entire
file will be deleted on the one hand and new file
(actually just renamed) will be new on the other hand,
which is hard to review.
Change-Id: I17a16ce746faa8898a457cadbb6f996f964a5b6f
Starting with ansible-core 2.13, list concatenation format is changed
and does not support concatenation operations outside of the jinja template.
The format change:
"[1] + {{ [2] }}" -> "{{ [1] + [2] }}"
This affects the horizon role that iterates over existing policy files to
override and concatenate them into a single variable.
Co-Authored-By: Dr. Jens Harbott <harbott@osism.tech>
Closes-Bug: #2045660
Change-Id: I91a2101ff26cb8568f4615b4cdca52dcf09e6978
Like other WSGI services in Kolla Ansible, the Horizon WSGI application
handles log output via the `wsgi.errors` object. See [1] for further
information. The problem is that this log output is written to a file called
`horizon.log`, causing it to processed as an 'Oslo log' in the Fluentd
processing pipeline. Since the log format doesn't match the expected format,
this results in parsing errors.
This fix renames the log file and adjusts the format to match other WSGI
applications. The logs are then processed in the same way as other WSGI
application logs, resolving the issue.
[1] https://modwsgi.readthedocs.io/en/master/user-guides/debugging-techniques.html
Change-Id: I93777d1c53920f5470c78356e6b3a4064fbe04b4
Closes-Bug: #1898174
Changes name of ansible module kolla_docker to
kolla_container.
Change-Id: I13c676ed0378aa721a21a1300f6054658ad12bc7
Signed-off-by: Martin Hiner <m.hiner@partner.samsung.com>
docker_restart_policy: no causes systemd units to not get created
and we use it in CI to disable restarts on services.
Introducing oneshot policy to not create systemd unit for oneshot
containers (those that are running bootstrap tasks, like db
bootstrap and don't need a systemd unit), but still create systemd
units for long lived containers but with Restart=No.
Change-Id: I9e0d656f19143ec2fcad7d6d345b2c9387551604
Change I60162b54bc06e158534d29311d4474b34750c64d
removed the `/v3` prefix from the WEBSSO_KEYSTONE_URL
variable. However, keystone endpoints do in fact
have the `/v3` prefix, and Horizon expects the
WEBSSO_KEYSTONE_URL variable to already contain
the prefix.
This patch adds the prefix again so that SSO
works with Horizon again.
Closes-Bug: #2028921
Change-Id: I5799f7a6fa4f52c2904b14fc02ed18443f4194bd
Signed-off-by: Juan Pablo Suazo <jsuazo@whitestack.com>
This change block access to the public /server-status url on all
http services exposed by HAProxy, also fixes an issue with Horizon
where 'Require all granted' open access to the /server-status in
the HAProxy-less configurations. Without this change the issue
affects only Ubuntu/Debian installations where mod_status in Apache2
enabled by default.
Closes-Bug: #1996913
Change-Id: I3ec1af6353c3ecc64589599abe375b0ae9b14d5c
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
Use case: exposing single external https frontend and
load balancing services using FQDNs.
Support different ports for internal and external endpoints.
Introduced kolla_url filter to normalize urls like:
- https://magnum.external:443/v1
- http://magnum.external:80/v1
Change-Id: I9fb03fe1cebce5c7198d523e015280c69f139cd0
Co-Authored-By: Jakub Darmach <jakub@stackhpc.com>
Since CVE-2022-29404 is fixed [1,2] the default value for the
LimitRequestBody directive in the Apache HTTP Server has been changed
from 0 (unlimited) to 1 GiB. This limits the size of images (for
example) uploaded in Horizon. This change add the ability to
configure the limit.
1. https://access.redhat.com/articles/6975397
2. https://ubuntu.com/security/CVE-2022-29404
Closes-Bug: #2012588
Change-Id: I4cd9dd088cbcf38ff6f8d188ebcc56be7d9ea1c9
Signed-off-by: Maksim Malchuk <maksim.malchuk@gmail.com>
Regularly, we experience issues in Kolla Ansible deployments because we
use wrong options in OpenStack configuration files. This is because
OpenStack services ignore unknown options. We also need to keep on top
of deprecated options that may be removed in the future. Integrating
oslo-config-validator into Kolla Ansible will greatly help.
Adds a shared role to run oslo-config-validator on each service. Takes
into account that services have multiple containers, and these may also
use multiple config files. Service roles are extended to use this shared
role. Executed with the new command ``kolla-ansible validate-config``.
Change-Id: Ic10b410fc115646d96d2ce39d9618e7c46cb3fbc
Kolla Ansible is switching to OpenSearch and is dropping support for
deploying ElasticSearch. This is because the final OSS release of
ElasticSearch has exceeded its end of life.
Monasca is affected because it uses both Logstash and ElasticSearch.
Whilst it may continue to work with OpenSearch, Logstash remains an
issue.
In the absence of any renewed interest in the project, we remove
support for deploying it. This helps to reduce the complexity
of log processing configuration in Kolla Ansible, freeing up
development time.
Change-Id: I6fc7842bcda18e417a3fd21c11e28979a470f1cf
Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/
in which was suggested to split patch into smaller ones.
THis change adds container_engine to module parameters
so when we introduce podman, kolla_toolbox can be used
for both engines.
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: Ic2093aa9341a0cb36df8f340cf290d62437504ad
Second part of patchset:
https://review.opendev.org/c/openstack/kolla-ansible/+/799229/
in which was suggested to split patch into smaller ones.
This change adds container_engine variable to kolla_container_facts
module, this prepares module to be used with docker and podman as well
without further changes in roles.
Signed-off-by: Ivan Halomi <i.halomi@partner.samsung.com>
Co-authored-by: Martin Hiner <m.hiner@partner.samsung.com>
Change-Id: I9e8fa30646844ab4a288555f3aafdda345b3a118
Currently kolla-ansible sets haproxy balance algorithm to source for
horizon. We can set it to round-robin if the cache backend is memcached
or using the database as the session storage backend. So we can
distribute http requests evenly to all available horizon instances.
Closes-Bug: #1990523
Change-Id: I0721cadcf53d59947bc0db6a193bfafe49c41ad3
This patch adds loadbalancer-config role
which is "wrapper" around haproxy-config
and proxysql-config role which will be added
in follow-up patches.
Change-Id: I64d41507317081e1860a94b9481a85c8d400797d
Role vars have a higher precedence than role defaults. This allows to
import default vars from another role via vars_files without overriding
project_name (see related bug for details).
Change-Id: I3d919736e53d6f3e1a70d1267cf42c8d2c0ad221
Related-Bug: #1951785
