grenade by default enable GLOBAL_VENV which means it
install and run everything from virtual env
- https://review.opendev.org/c/openstack/grenade/+/930507
We faced the error in ironic grenade scripts in virtual env
so GLOBAL_VENV was disabled explicitly. This fixing the scripts
and enable GLOBAL_VENV in ironic jobs also.
Change-Id: I48ee1dd4adc2e5bcc18c5f116d979e7524248495
Change the default RBAC policy in ironic such that the new RBAC
policy is enforced by default and the legacy policy is not usable
unless explicitly re-enabled.
Depends-On: https://review.opendev.org/c/openstack/metalsmith/+/905012
Change-Id: Id559f1d8b9a76c8a570b598585c2d58c56d08837
Grenade is currently failing not finding the neutron command, we should
likely not be using it anyway since the deprecation message says it may
disappear after Z.
Change-Id: Ic24d59379bafcc5a630fe5c074fcc13303902965
In [1] we finally got rid of the unfinished lib/neutron module and kept
only lib/neutron-legacy. It's renamed to lib/neutron now and it's the
only neutron related module in Devstack.
So this patch removes leftovers related to the old lib/neutron-legacy.
[1] https://review.opendev.org/c/openstack/devstack/+/865014
Change-Id: Id938deab7188743e754d028dee8e0b2591ab6f7b
The upgrade path logic was built to force a developer pattern to break
things such as new tables and features across multiple patches, and
the status check *can* explicitly fail if we don't explicitly go
hint to it that we've added table. Yes, we have a hard coded list...
Anyway, a better pattern is allow the db sync process to do the
appropriate needful. Run the status check, if it fails, fallback
and update the schema.
Also handles the explicit failure error and tries to return a human
friendly error message for when the table is not present.
In the end this allows us to merge schema changes such as additional
tables with their underlying objects and properly handle things as
long as the schema update works as expected. This allows us to
leverage an operational model of performing upgrades.
Change-Id: Id5f2a8068bc064e1ed1e376524850e4739f79ef2
When I merged the db status check in for database indexes, I missed
the most improtant line, which is where the object is populated with
the method name to be executed by the upgrade checks framework.
In the rush to try and clean-up after the impact of the Secure RBAC
work, I just didn't manually test the final file I uploaded into
review. I assumed it just worked because the job passed, but didn't
think about the resulting return codes which we *should* experience
on an upgrade from a prior version. Later on, I noticed that because
of the way the status checks are invoked, I also added the code to
do the index check in the wrong order, so I had to restructure things
so the method definition was known by the object on the class which
holds the method names list.
I guess I copied/pasted this over from another file I was testing
in just didn't run the final file. :( Funny enough, the index check
works like a charm now.
Also updates the status check invocation check in the upgrade script
for grenade, *as* warnings *are* permissible and not fatal.
Change-Id: Ifa9da65dc8df5bf9a369d6faeab310386dfd944a
It seems, however variables are getting set with grenade
in native zuulv3 jobs, the environment variables are no
longer comparing as they once did in standard bash jobs.
What worked previously, was [ "$VAR" == 'value' ], however
when comparing the newer grenade node jobs we see things like
[[ subnode != \s\u\b\n\o\d\e ]] returning true when single
quotes are used.
Comparison in the logs to where double quotes were used for
functionally the same comparision, revealed proper processing
and execution as expected.
As such, changing our single quotes to double quotes where
applicable.
For the record, I also quickly checked codesearch and it looks
like we were the only project to use single quotes for this
comparison.
Change-Id: I2db2e870e2c5f32aa061af025ee7ce12c4f7c049
Adds a check for compatibility of the object versions (in the DB)
with the release of ironic. This check is used by the
'ironic-status upgrade check' command.
The ironic-status command is added to devstack and grenade (when
upgrading).
Change-Id: I2043dc01856106d50356637db327a2817db90366
Story: 2004990
Task: 29459
As of 988243c93fb87190e1867ef913136ecf139e7cb8 in ipxe
"[virtio] Add virtio-net 1.0 support", VMs don't attempt
to boot over the network when using the virtio-net rom.
The ipxe-qemu package used in the grenade jobs contains
this problem. Switch to e1000 to avoid baremetal VMs
working on some hosts and not others.
Change-Id: I4ec297281380896ffe9004c4de7117586ded7149
Story: #2003808
Task: #26603
This was disabled when grenade switched to using rocky
as its base install as it didn't work with wsgi. Restore it
using a version that compatable with WSGI.
Task: #26604
Story: 2003808
Change-Id: I968339b48191df46a44f45c1794478aa0d1039f6
Downgrade ipxe-qemu during grenade job
Temporary workaround while we find the problem with the
newer ipxe-qemu package.
Task: #26553
Switch grenade job to wsgi
Now that grenade is using Rocky as its base we
can switch to WSGI for ir-api. Also now required
for CI as IRONIC_USE_WSGI was getting flipflopped
from False -> True -> False during the grenade job.
Task: #26594
Disable the ironic-api redirect as it doesn't work with wsgi
Task: #26595
Increase the keystone token timeout, for grenade jobs only
Task: #26596
Story: #2003808
Change-Id: Icc380e6e5128f15f90b3a86cb773e7bb8d60d076
Depends-On: Ib7d68d1360e3cdbfdc2ff185a9760461e75ac8c4
This change removes pxe_ipmitool and agent_ipmitool. Some tests
and documentation bits are removed, since we no longer have any
classic drivers to provide examples.
The Inspector.create_if_enabled method is no longer used and was
removed. The [inspector]enabled option will be deprecated in
a follow-up.
IRONIC_ENABLED_DRIVERS is removed from devstack to prevent
devstack-gate from injecting the removed drivers into it.
A temporary migration from queens is updated to disable all
classic drivers. It will be removed when the enabled_drivers
option is completely removed.
Change-Id: Iab247e3b3336f90446ef0687edb340cf8282a925
This patch switches ironic API to run under uwsgi,
and drops support for running ironic API under mod_wsgi
from ironic's devstack plugin.
It also effectively moves all jobs except grenade ones to
run ironic-api under UWSGI as we start to honor the devstack's
ENABLE_HTTPD_MOD_WSGI_SERVICES variable (which is True by default).
The new variable in devstack plugin to toggle deployment with uwsgi
is IRONIC_USE_WSGI, which defaults to (now confusingly named)
IRONIC_USE_MOD_WSGI for backward compatibility.
Related-Bug: #1719260
Co-Authored-By: anascko <ovoshchana@mirantis.com>
Change-Id: I9ef3aa48db6efe8e2216af785cc13fdb7f754a02
This change moves our devstack plugin away from using the old
scheduling properties: memory_mb, local_gb and cpus. Nova will
stop using them for scheduling in Queens.
The cpu_arch property is left intact, as it's still useful.
The Placement API is polled to determine when nodes are fully exposed
to nova instead of polling the hypervisor stats. cURL + jq are used,
as the Placement OSC plugin is not ready yet.
Change-Id: I497a992bd86e8a359f333d6936621d1136d68061
Run ironic-dbsync online_data_migrations on single node grenade
right after usual dbsync.
Do not set pin_relase_version for single node grenade as we
do cold upgrade and it is not required.
Related-Bug: 1526283
Related-Bug: 1502887
Change-Id: I2e331252aeb66376c2d6cc3c16b6de0aefca5434
This reverts part of commit 30bc3c0933a1072e5e4c48b4614f5f387ffd8327.
On master, when upgrading from pike to queens in grenade, we already
create all the nodes with a resource class being set, so no need to
update it. We still pass the parameters to verify and verify_noapi as
that's what grenade does.
Change-Id: Ic7c3218fe95ef9f73d358216831158a1ec025fba
This is done to trigger a bit of code from commit
c3118b91db4b97a689c9a2f25a9554a0ccb29252 to nova for active nodes.
Change-Id: I1ac1b8501cbed901941e7c89fc55c5b9bea0cb4f
As we do not upgrade nova on multinode grenade, there is no need to
restart it. This way we avoid breaking n-cpu service, as new
dependencies are installed and they may not work with old code.
Change-Id: Id0c13704654d43a663458b9c099a1a0477c746a6
Nova do not wait for cleaning is completed when deleting instance
as result after destroy phase ironic may still perform cleaning.
This patch ensures all nodes are available again after base smoke tests
finished and hypervisor stats are updated.
We still need to wait for resources after destroy phase which will be
done in separate patch as changes to grenade workflow are needed.
Change-Id: Ib6c5a0543533893664cb3e895286d0e226e364d2
Instead, use the common_settings file to source all the variables
we need at the early stages of devstack setup and during the upgrade
itself. This also fixes an issue with the networking-generic-switch
upgrade script, which is run from the old release instead of the new
one.
It depends on a patch from an older release, as grenade is setup
using settings from an older release, we need common_setting to be
populated there.
Change-Id: I0d6d3e9e60785bb9b6a10c8c7a07867a5b2a5e6e
Depends-On: I34fb68c150b753824bd2687a1f522ac87771a5e3
We now run services under systemd, and this check is not necessary in
this case. Follow-up to commit ee538d1e7c1a92cb8412713e03cdfbcdacad7fab
to grenade.
Change-Id: I84ca621f0200838c04e267101848dc7ca16a5cfe
The reschedule, that happens in multinode jobs quite often, does not
work with multicell setup. Don't use multicell setup for our Ironic
jobs.
Increase the http timeout for clients in order to work around neutron
issue.
Change-Id: I2bbdac5cd431e72517cb1c72bb925660ec8e868b
Multi-node grenade jobs will fail if we upgrade nova as the minimum
required API for nova to communicate with ironic was upgraded.
Our goal for multi-node grenade is to test rolling upgrades. The
scenario we envision for rolling upgrades to be done is:
1) Upgrade ironic-conductor services one by one. This means old
ironic-api services communicating with new and old
ironic-conductor services. This is what the multi-node grenade job
tests.
2) Upgrade ironic-api services.
3) Upgrade nova
In this upgrade procedure we should not be running updated nova with
old ironic-api services. This patch fixes that.
Closes-Bug: #1706418
Change-Id: I85c607ae7d031e4cabcb505915aca7e88df166aa
This patch set nomulitcell flag when starting nova-compute because
multi-cell rabbit setup is not necessary in Grenade. This nomulticell
flag will be introduced with I08d7da843d18b426dda8a8a231039d950a4c0ce5.
Closes-Bug: 1700548
Change-Id: Ib55f6359751052d41b46e31ec8e15f46e33c1193
Ironic upgrade process has pre-requirement that ironic-conductors
should be upgraded before ironic-api. Grenade is launched on primary
node, where we have ir-api and ir-cond runnning. Do not start ir-api
on primary node after upgrade to avoid situation when we have new ir-api
(primary) and old ir-cond (subnode) as it is not allowed by upgrade
procedure. Instead redirect all requests destinied to ir-api on primary
node to subnode.
Depends-On: I026121121059768aa74389add7eee6e63fdb214d
Change-Id: I9e496353a1d3d6547e4881f96fb187e488cff83f
We enable plugin and services in project-config.
Drop those duplications from devstack/upgrade/settings
Change-Id: I95d83d6b1f2914cbb0373099e128ab462fc3b492
Depends-On: Icdd89453ec3ebecd8375a5f2efd1a0d4bf6b4064
To support multi-node grenade jobs we need to move enabling of the
ironic devstack plugin out of the ironic grenade settings file
devstack/upgrade/settings. But if we add enabling of the ironic devstack
plugin to project-config it will cause the gate to break as the plugin
will be enabled twice.
This checks to see if the ironic devstack plugin has already been
enabled:
* If it has already been enabled it will not enable it again.
* If it has not yet been enabled it will enable it.
This will allow us to update project-config to enable the ironic
devstack plugin and not break the gate.
After all the changes have landed we will propose a follow-up patch to
remove all this additonal logic and the enabling of the ironic devstack
plugin.
Change-Id: I09ed78ef918eb0359045350e89d42c6df9f3e490
Adds handling of running ironic API under Apache as WSGI app to
devstack plugin.
New variable IRONIC_USE_MOD_WSGI (False by default) is added.
Another new variable IRONIC_WSGI_DIR (default is distro specific) is
also added, which specifies location for WSGI scripts.
Change-Id: I9c5ad56e1acd292ff0f9cc9b460125fc420abda5
Closes-Bug: #1513005
We set global variables in Ironic devstack plugin like
DEFAULT_IMAGE_NAME. Ensure that grenade uses them by
sourcing lib/ironic from upgrade/settings.
Set IRONIC_DEPLOY_DRIVER by default to pxe_ipmitool. As we no longer
add IRONIC_DEPLOY_DRIVER to IRONIC_ENABLED_DRIVERS and defaults are
conflicting.
Closes-Bug: #1663371
Change-Id: I462ac792ad7920beff64c15a7c005406fcfa5cce
When running grenade with multitenancy, we are able to boot instances
in the network neutron has created in its upgrade phase, and we don't
have to change that setup.
Change-Id: I1038548ad4f93788fca3e2b7dd8ed93da0ce7702
Starting with Idede322190d57491d4f7e18090aa8c29b5b87353 base and target
releases has the same tempest config file. This patch drops setting
fixed_network_name in target tempest.conf workaround.
Change-Id: I661ace60bf9b4817f73aaccda9c18e24adfc4e56
Grenade is designed to upgrade services. We shouldn't add them during
upgrade. This patch removes case when we trying to add Ironic to cluster
on upgrade.
Change-Id: Id7159aac8c8a343e3d20561cedc4722d87d3e776
By sourcing localrc second time we try to enable ironic plugin
one more time witch is not allowed by [0]
This patch checks if ironic was enabled or not in base environment
without sourcing localrc.
[0] https://review.openstack.org/#/c/396794/
Change-Id: I432c243c66c0d11adcf4c6568968fbeaaf9acdcd
This changes up devstack code to make sure OVS does the right thing
after upgrading. After the neutron upgrade, it uses a new OVS tag for
networking, so this changes things over to the new one.
Make sure that we remove all resources that were created during
resources phase.
Set tempest fixed_network_name explicitly as it is prerequirement for
ironic, see reference.
Reference:
https://github.com/openstack/tempest/blob/039b2fe/tempest/scenario/manager.py#L1161
Change-Id: I35789602fb542bb33533211a742c3e4d265a3b91