This patch adds two new jobs:
* kolla-ansible-centos-source-upgrade
* kolla-ansible-ubuntu-source-upgrade
These jobs first deploy a control plane using the previous release of
Kolla Ansible, then upgrade to the current release.
Because we can't change the branch of the git repository on the Zuul
executor, we change the branch of the kolla-ansible repository on the
primary node to the branch of the previous release, in this case
stable/rocky. A new remote-template role has been added that supports
generating templates using a remote template source, to generate config
files using the previous kolla-ansible branch.
If the change being tested depends on a kolla change for the current
branch, then we build images. Rather than using the current
kolla-ansible version to tag the images, we now tag them with
change_<gerrit change ID>. This is because the version of kolla-ansible
will change from the previous release to the current one as we upgrade
the system.
Finally, it should be noted that the 'previous_release' variable in the
Zuul config needs to be updated with each release, since this sets the
release of kolla-ansible that is installed initially.
Depends-On: https://review.openstack.org/645089/
Depends-On: https://review.openstack.org/644250/
Depends-On: https://review.openstack.org/645816/
Depends-On: https://review.openstack.org/645840/
Change-Id: If301e0affcd55360fefe3b105f023ae5c47b0853
incorrect path when generating certificates.
The 'setting permissions on key' task fails because the task looks for
the haproxy.key in an invalid path. The certificates_dir is defined as
'{{ node_config }}/certificates' in the main.yml . The 'Setting
permissions on Key' task has a path of '{{ certificates_dir
}}/certificates/private/haproxy.key which is incorrect. Removing the
'certificates' in the path corrects this problem and allows the user to
successfully create certificates using 'kolla-ansible certificates'.
Change-Id: I37b10b994b05d955b6f67c908df1472231a91160
Closes-Bug: 1821805
The synced flush fails due to concurrent indexing operations.
The HTTP status code in that case will be 409 CONFLICT. We can
retry this task until returns success.
Change-Id: I57f9a009b12715eed8dfcf829a71f418d2ce437b
Upgrading MariaDB from Rocky to Stein currently fails, with the new
container left continually restarting. The problem is that the Rocky
container does not shutdown cleanly, leaving behind state that the new
container cannot recover. The container does not shutdown cleanly
because we run dumb-init with a --single-child argument, causing it to
forward signals to only the process executed by dumb-init. In our case
this is mysqld_safe, which ignores various signals, including SIGTERM.
After a (default 10 second) timeout, Docker then kills the container.
A Kolla change [1] removes the --single-child argument from dumb-init
for the MariaDB container, however we still need to support upgrading
from Rocky images that don't have this change. To do that, we add new
handlers to execute 'mysqladmin shutdown' to cleanly shutdown the
service.
A second issue with the current upgrade approach is that we don't
execute mysql_upgrade after starting the new service. This can leave the
database state using the format of the previous release. This patch also
adds handlers to execute mysql_upgrade.
[1] https://review.openstack.org/644244
Depends-On: https://review.openstack.org/644244
Depends-On: https://review.openstack.org/645990
Change-Id: I08a655a359ff9cfa79043f2166dca59199c7d67f
Closes-Bug: #1820325
After upgrading from Rocky to Stein, nova-compute services fail to start
new instances with the following error message:
Failed to allocate the network(s), not rescheduling.
Looking in the nova-compute logs, we also see this:
Neutron Reported failure on event
network-vif-plugged-60c05a0d-8758-44c9-81e4-754551567be5 for instance
32c493c4-d88c-4f14-98db-c7af64bf3324: NovaException: In shutdown, no new
events can be scheduled
During the upgrade process, we send nova containers a SIGHUP to cause
them to reload their object version state. Speaking to the nova team in
IRC, there is a known issue with this, caused by oslo.service performing
a full shutdown in response to a SIGHUP, which breaks nova-compute.
There is a patch [1] in review to address this.
The workaround employed here is to restart the nova compute service.
[1] https://review.openstack.org/#/c/641907
Change-Id: Ia4fcc558a3f62ced2d629d7a22d0bc1eb6b879f1
Closes-Bug: #1821362
When Nova, Glance, or Cinder are deployed alongside an external Ceph deployment
handlers will fail to trigger if keyring files are updated, which results in the
containers not being restarted.
This change adds the missing 'when' conditions for nova-libvirt, nova-compute,
cinder-volume, cinder-backup, and glance-api containers.
Change-Id: I8e183aac9a72e7a7210f7edc7cdcbaedd4fbcaa9
Fixes a race condition where sometimes a volume would still be in the
'creating' state when trying to attach it to a server.
Invalid volume: Volume <id> status must be available or downloading to
reserve, but the current status is creating.
Change-Id: I0687ddfd78c384650cb361ff07aa64c5c3806a93
Services were being passed as a JSON list, then iterated over in the
neutron-server container's extend_start.sh script like this:
['neutron-server'
'neutron-fwaas'
'neutron-vpnaas']
I'm not actually sure why we have to specify services explicitly, it
seems liable to break if we have other plugins that need migrating.
Change-Id: Ic8ce595793cbe0772e44c041246d5af3a9471d44
RDO is packaging placement-api with bundled httpd config
and it conflicts with kolla-ansible generated one.
Change-Id: I018a4ed1b2282e8a789b63e3893e61db2fde8cf2
Reconfiguring Swift currently fails to restart containers if
configuration changes. This is because kolla_set_configs is executed in
the containers as the default swift user, which does not have permission
to access all necessary files.
This change uses the root user to execute the command instead, which
allows it to exit with the correct status of 1 if the config files
differ.
Change-Id: I2a2363c71430a7173bb5253662412ae5dba09654
When adding the rolling upgrade support, some upgrade procedures were
modified to pull images explicitly. This is done inconsistently between
services, and is a change in behaviour from Rocky and earlier releases.
This change removes all image pulling from upgrade tasks.
Change-Id: Id0fed17714235e1daed60b83b1f30620f097eb97
Migrate to the latest Ubuntu LTS release 18.04 aka Bionic. See [0] for
the big picture.
Also test running tox jobs on Bionic.
[0] https://etherpad.openstack.org/p/devstack-bionic
Change-Id: I96e7b8d17bc1e92716c04fdcf362c2adb08a2212
All Prometheus services should use the Prometheus install type which
defaults to the Kolla install type, rather than directly using the
Kolla install type.
Change-Id: Ieaa924986dff33d4cf4a90991a8f34534cfc3468
The api_endpoint option was deprecated, and will be removed by
https://review.openstack.org/643483.
Change-Id: Ie56a8ab07ab21d2e7d678e636c1408099d8ab3aa
Fix filemode in the merge_configs and merge_yaml action plugin to
be compatible with python3
Change-Id: Ief64c5bdcd717141281e23c255a49ec02a96aef2
Closes-Bug: #1820134
Adds support to seperate Swift access and replication traffic from other storage traffic.
In a deployment where both Ceph and Swift have been deployed,
this changes adds functionalality to support optional seperation
of storage network traffic. This adds two new network interfaces
'swift_storage_interface' and 'swift_replication_interface' which maintain
backwards compatibility.
The Swift access network interface is configured via 'swift_storage_interface',
which defaults to 'storage_interface'. The Swift replication network
interface is configured via 'swift_replication_interface', which
defaults to 'swift_storage_interface'.
If a separate replication network is used, Kolla Ansible now deploys separate
replication servers for the accounts, containers and objects, that listen on
this network. In this case, these services handle only replication traffic, and
the original account-, container- and object- servers only handle storage
user requests.
Change-Id: Ib39e081574e030126f2d08f51de89641ddb0d42e
