Currently the curl commands associated with this command run on all
controller nodes of the overcloud. Becuase the ODL URI is always the
same it's actually doing it against the same REST API every time,
causing problems with the optimistic locking ODL uses.
This patch adds extra error information and limits the execution of this
task to only once per playbook (i.e. just one controller).
Change-Id: I75aed2c0f412961c1eed2ff14e039a0baca09e8a
ODL upgrade/update logic has docker CLIs.
Make that logic to use CLI based on aforementioned
podman or docker.
Change-Id: If7de9c496214f6efe87820649d66489f20d0b489
Closes-Bug: #1792469
We don't need upgrade_tasks that stop systemd services since all
services are now containerized.
However, we decided to keep the tasks that remove the rpms in case some
of deployments didn't cleanup them in previous releases, they can still
do it now.
Change-Id: I6abdc9e37966cd818306f7af473958fd4662ccb5
Related-Bug: #1806733
"data" folder mounted on host sometimes makes ODL
believe that it has boot features installed. So ODL
doesnot generate required files.
So don't mount that folder so that ODL can boot
properly on restart.
Closes-Bug: #1805859
Change-Id: Icb415f6ac379e757edfe6dd73f80484a6697d152
For all containers where restart=always is configured and that are not
managed by Pacemaker (this part will be handled later), we remove these
containers at step 1 of post_upgrade_tasks.
Change-Id: Id446dbf7b0a18bd1d4539856e6709d35c7cfa0f0
Mounting empty /var/lib/config-data/puppet-generated/
opendaylight/opt/opendaylight/etc/opendaylight/karaf
from host to /opt/opendaylight/etc/opendaylight/karaf
on container empties the folder on container itself.
Hence on restart/redeploy, ODL can't find files inside
that folder and can't start leading to deployment failure.
So delete the empty karaf folder on host so that the contents
of karaf folder inside the container is intact durin redeploy.
Change-Id: I75aabaa2abc0fc9ba789c53f27fc37cfb7769a8b
Closes-Bug: #1799395
This has been unused for a while, and even deprecation was scheduled
(although the patch never merged [1]). So, in order to stop folks
getting confused with this, it's being removed.
[1] https://review.openstack.org/#/c/543871/
Change-Id: Iada64874432146ef311682f26af5990469790ed2
This has been unused for a while, and even deprecation was scheduled
(although the patch never merged [1]). So, in order to stop folks
getting confused with this, it's being removed.
[1] https://review.openstack.org/#/c/543871/
Change-Id: Icc6b51044ccc826f5b629eb1abd3342813ed84c0
1. Spelling mismatch in ODL net mapping doesnot generate
its endpoitn clearly.
2. Rectified another small syntax correction for string
replace.
3. Delete upgrade flag and then set it to false. ODL
doesnot allow it to be set without deleting.
4. Other changes in accordance with ODL update process.
Change-Id: Ib549e29f50bbbb434183ac9423bcf776807ff1a4
Closes-Bug: 1779410
ODL is configured to log to console and then access the logs
via docker logs. Older logs are being deleted too fast. So
moving back to logging to file which is default setting in ODL
THT.
https://github.com/openstack/tripleo-heat-templates/blob/
master/puppet/services/opendaylight-api.yaml#L60
Change-Id: I29724cfb23e13c8293d1b7b4341cbd409ae1b7a6
Closes-Bug: 1779629
On minor update, when docker RPM or configuration gets updated, docker
daemon restarts, and if container has restart policy `always`, it is
started automatically. This meant that our previous aim of stopping
the container and cleaning its cache was effectively undone by the
docker daemon restart later during execution of update_tasks.
Setting the restart policy to `unless-stopped` keeps the container
stopped on docker daemon restarts if the container was stopped
intentionally.
Co-Authored-By: Janki Chhatbar <jchhatba@redhat.com>
Co-Authored-By: Tim Rozet <trozet@redhat.com>
Change-Id: Id033cbeef46bded13f4fd6c06eb3a244634f0e7b
Partial-Bug: #1775919
OpenDaylight creates multiple files the first time it boots, which we do
not mount to the host. After the first boot, it creates a cache which we
do mount to the host. This means that on a config change or
update/upgrade of ODL the cache will not be removed, but the files will
be. This causes ODL to fail to start.
The solution is to stop the container in update/upgrade and then remove
the cache before the update happens. This will trigger the new ODL to
rebuild the cache with the new ODL version. For config change, we also
need to remove the cache in the host_prep_tasks so that we do not end up
in a similar state.
Closes-Bug: 1775919
Change-Id: Ia457b90b765617822e9adbf07485c9ea1fe179e5
Signed-off-by: Tim Rozet <trozet@redhat.com>
To not to redefine variable multiple times in each service we
run check only once and we set fact. To increase readability of
generated playbook we add block per strep in services.
Change-Id: I2399a72709d240f84e3463c5c3b56942462d1e5c
The new master branch should point now to rocky.
So, HOT templates should specify that they might contain features
for rocky release [1]
Also, this submission updates the yaml validation to use only latest
heat_version alias. There are cases in which we will need to set
the version for specific templates i.e. mixed versions, so there
is added a variable to assign specific templates to specific heat_version
aliases, avoiding the introductions of error by bulk replacing the
the old version in new releases.
[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#rocky
Change-Id: Ib17526d9cc453516d99d4659ee5fa51a5aa7fb4b
This patch mounts karaf.log file at /var/log/
container/opendaylight/. So the logs are now available
via docker logs and in file both.
Change-Id: Ib1275d42daf2896db9e6de6513070a0fba47ed40
Indentation for few lines is corrected and
correct path for config file is updated. data
folder is deleted during update/upgrade. Set
correct permissions (42462:42462) for
genius-mdsalutil-config.xml. 42462 comes from
kolla and is id for odl user inside the container.
Closes-Bug: 1764603
Change-Id: Ie343cd4cab7cc009b1940a98fa73b1ac15b3b56d
The ODL etc directory was being overridden with an empty mount directory
before kolla_start copied the puppet-generated config files. The
puppet-generated config files only include modified configuration files
and not all of the default config files. Therefore ODL was missing
config files when it started so the container was constantly crashing.
This patch removes the unwanted mount erasing the /opt/opendaylight/etc
directory and moves the upgrade file to be created in puppet-generated,
which will be copied at kolla start time for upgrade. The
puppet-generated dir is read-only, so the REST call to disable upgrade
flag in ODL will only disable it for the running instance. Therefore we
have to use ansible to write the file again to disable it incase ODL is
rebooted.
Closes-Bug: 1755916
Change-Id: Ie380cc41ca50a294a2647d673f339d02111bf6b3
Signed-off-by: Tim Rozet <trozet@redhat.com>
Updating OpenStack (within release) means updating ODL from v1 to v1.1.
This is done by "openstack overcloud update" which collects
update_tasks. ODL needs 2 different steps to achieve this
minor update. These are called Level1 and Level2. L1 is
simple - stop ODL, update, start. This is taken care by paunch
and no separate implementation is needed. L2 has extra steps
which are implemented in update_tasks and post_update_tasks.
Updating ODL within the same major release (1->1.1) consists of either
L1 or L2 steps. These steps are decided from ODLUpdateLevel parameter
specified in environments/services-docker/update-odl.yaml.
Upgrading ODL to the next major release (1.1->2) requires
only the L2 steps. These are implemented as upgrade_tasks and
post_upgrade_tasks in https://review.openstack.org/489201.
Steps involved in level 2 update are
1. Block OVS instances to connect to ODL
2. Set ODL upgrade flag to True
3. Start ODL
4. Start Neutron re-sync and wait for it to finish
5. Delete OVS groups and ports
6. Stop OVS
7. Unblock OVS ports
8. Start OVS
9. Unset ODL upgrade flag
These steps are exactly same as upgrade_tasks.
The logic implemented is:
follow upgrade_tasks; when update_level == 2
Change-Id: Ie532800663dd24313a7350b5583a5080ddb796e7
If we use variables defined in later step in conditional before
checking which step are we on we will fail.
Resolves: rhbz#1535457
Closes-Bug: #1743764
Change-Id: Ic21f6eb5c4101f230fa894cd0829a11e2f0ef39b
The ODL private key, cert, and CA cert were missing in the puppet
container config. These are required during puppet stage in order to
create the proper keystores in ODL. The files are not needed during the
service bringup time, because the information is built into the
keystore created by puppet.
Closes-Bug: 1747700
Change-Id: If548fdba836104412bf20e8e05ecf6a5058aa318
Signed-off-by: Tim Rozet <trozet@redhat.com>
Major upgrade (Q -> R) is complex in ODL. There are multiple components
involved.
This patch enables major upgrade of ODL. Steps involved are:
1. Block OVS instances to connect to ODL
2. Set ODL upgrade flag to True
3. Start ODL
4. Start Neutron re-sync and wait for it to finish
5. Delete OVS groups and ports
6. Stop OVS
7. Unblock OVS ports
8. Start OVS
9. Unset ODL upgrade flag
Change-Id: Icf98a1215900762a0677aabee1cccbf1d130e5bd
Enables TLS encryption between ODL and OVS, as well as Northbound ODL
communication with Neutron.
Implements: blueprint opendaylight-ssl-support
Depends-On: Id579aea77bf8d679b514ef9851af36d9170e93a1
Change-Id: I7c43f1358807f3ffeef2ddf29d0085ad55151dfe
Signed-off-by: Tim Rozet <trozet@redhat.com>
ODL logs to either console for containarised deployment and to file
for non-containarised deployment. For containarised deployments, logs
can then be read via "docker logs".
We need both JVM and karaf (ODL application) logs together to debug
any failure scenario and for them to correlate. JVM logs to console
and not to a file. Karaf can log to file and console. So we make
karaf and JVM both log to console and read these logs via docker logs.
In cases when ODL container restarts, "docker logs" retain logs from
previous broken container so there is no loss of logs in these cases.
Change-Id: I2fe56df082c5d9206015f156f5f1b3cfca63c982
This converts "tags: stepN" to "when: step|int == N" for the direct
execution as an ansible playbook, with a loop variable 'step'.
The tasks all include the explicit cast |int.
This also adds a set_fact task for handling of the package removal
with the UpgradeRemovePackages parameter (no change to the interface)
The yaml-validate also now checks for duplicate 'when:' statements
Q upgrade spec @ Ibde21e6efae3a7d311bee526d63c5692c4e27b28
Related Blueprint: major-upgrade-workflow
[0]: 394a92f761/tripleo_common/utils/config.py (L141)
Change-Id: I6adc5619a28099f4e241351b63377f1e96933810
Step config is only required within the puppet_configs section
of docker/services/*. This patch drops the top level 'step_config'
and updates the unit tests accordingly.
Change-Id: I7dc7cfae3ef1965ec95b1d9ef23e7f162418c034
This should help operators find the new log files. We do have them
documented, but not everybody reads every word in the docs :)
The readme creation has ignore_errors: true so that if the directory
isn't present at all (e.g. on deployed server environments, which
don't have openstack packages installed), we don't fail the deployment
when we're not able to create the readme.
Change-Id: I6b36db7b7ce8b3e4da566eb7828d0c3b8646a14f
Partial-Bug: #1730957
Docker services are missing the pre-upgrade validation task
in the upgrade_tasks section which verifies if the service
is running before going on with the upgrade.
Change-Id: Iee10761330311d40825f0ad11b676d8f1f20355f
Partial-Bug: #1704389
We were setting them in the Dockerfile's previously. However this
caused the healtcheck commands to always run regardless of which
process we were running in the container. This caused 'unhealthy'
containers at times they were never intended to be checked. This
change makes it so they are explicitly set.
Change-Id: I7bc12d236b3cc7a52d3e6aa706fd04675dad3a9a
The services that docker depends on, have logging_sources and logging_groups;
but those are not set on the docker outputs so they are not used when dockers
are deployed.
Added logging_source & logging_groups as docker optional parameters in
tools/yaml-validate.py
Closes-Bug: #1718110
Change-Id: I8795eaf4bd06051e9b94aa50450dee0d8761e526
journal and snapshots folders hold data needed for update. This
patch mounts these folders and adds ODL log file in
/var/log/containers/opendaylight
Change-Id: I65c6183c2867b2ced6e6ef25896d80154857b7dc
Closes:Bug: #1714231
Now that ODL clustering is fixed to not use an exec by:
https://git.opendaylight.org/gerrit/#/c/60491
We no longer need to use the workaround puppet-tripleo
tag to configure clustering.
Change-Id: I21c1eb2eff6d4cb855eff4a1122f55ad625d84cc
Signed-off-by: Tim Rozet <trozet@redhat.com>
This patch removes more of the DockerNamespace references as part
of the cleanup/reorg of the container configuration patches.
This also adds a centos-rdo environment file for use with
the new interface. This file was generated with the command
"openstack overcloud container image prepare"
Depends-On: I729fa00175cb36b02b882d729aae5ff06d0e3fbc
Depends-On: I292162d66880278de09f7acbdbf02e2312c5bb2b
Co-Authored-By: Dan Prince <dprince@redhat.com>
Change-Id: Ice7b57c25248634240a6dd6e14e6d411e7806326
Makes it possible to resolve network subnets within a service
template; the data is transported into a new property ServiceData
wired into every service which hopefully is generic enough to
be extended in the future and transport more data.
Data can be consumed in service templates to set config values
which need to know what is the subnet where a deamon operates (for
example the Ceph Public vs Cluster network).
Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
This solves a problem with bind-mounts when the containers are holding
files descriptors open.
At the same time this makes the template more robust to puppet changes
since new config files will be available in the containers without
needing to update the templates.
Partial-Bug: #1698323
Change-Id: Ia4ad6d77387e3dc354cd131c2f9756939fb8f736