11336 Commits

Author SHA1 Message Date
Zuul
c3a80353c8 Merge "Prevent redfish-virtual-media from being used with Dell nodes" 2021-02-08 07:47:40 +00:00
Lance Bragstad
94ba2641e3 Update oslo.policy requirement to version 3.6.2
This version of oslo.policy includes fixes that ensure the Enforcer only
modifies copies of the rules, making it safer in environments that run
tests in parallel where the Enforcer is configured differently.

Change-Id: I8d7a06558cbf073487707fc33219e43beb5fc043
2021-02-05 21:03:39 +00:00
Zuul
f81843aa05 Merge "ilo: do not change deploy_boot_mode in instance_info" 2021-02-05 14:30:27 +00:00
Dmitry Tantsur
cf22604c58 Prevent redfish-virtual-media from being used with Dell nodes
Indicate that idrac-redfish-virtual-media must be used instead,
otherwise a confusing failure will happen.

Change-Id: I3b6ced6dcf03580903f5ea7237fc057f372999f9
2021-02-05 12:09:00 +01:00
Derek Higgins
4287951d71 Don't mark an agent as alive if rebooted
If 'agent_url' has been cleared from internal_info
it indicates that the node has been powered off.

Change-Id: Idba486c98e1e92d35fca2e2d156866566acb9e40
Story: 2008583
Task: 41736
2021-02-04 13:01:50 +00:00
Zuul
058b97b59c Merge "Trivial: log the newly detected vendor" 2021-02-04 08:46:18 +00:00
Zuul
e0ae9cdb8b Merge "Swap Metalsmith job out for centos8-uefi" 2021-02-03 19:15:13 +00:00
Zuul
f682dee6b0 Merge "Update python packages to python3 in quickstart.rst" 2021-02-03 17:41:41 +00:00
Aija Jauntēva
3138acc836 Add 'deploy steps' parameter for provisioning API
Story: 2008043
Task: 40705
Change-Id: I3dc2d42b3edd2a9530595e752895e9d113f76ea8
2021-02-03 11:47:53 -05:00
Dmitry Tantsur
b8a2dcaf86 Trivial: log the newly detected vendor
Change-Id: Ib751316a98d7a1c4469b405117c8e1fd1f296757
2021-02-03 17:31:54 +01:00
Zuul
f079531b98 Merge "Raw image size estimation improved" 2021-02-03 14:49:15 +00:00
Zuul
f4197a12ef Merge "Redfish secure boot management" 2021-02-03 14:43:06 +00:00
Zuul
f4142d4930 Merge "Apply force_persistent_boot_device to all boot interfaces" 2021-02-03 07:51:20 +00:00
Zuul
398872e756 Merge "Fix Mis-Ordering of Bash Variable Definition in DevStack" 2021-02-03 04:26:51 +00:00
Zuul
8d0cef5cb8 Merge "Add release version to release notes" 2021-02-03 03:19:19 +00:00
Zuul
d6bdf1adb0 Merge "Pass context objects directly to policy enforcement" 2021-02-02 16:17:16 +00:00
Julia Kreger
561ed90390 Swap Metalsmith job out for centos8-uefi
Depends-On: https://review.opendev.org/c/openstack/metalsmith/+/773701
Change-Id: Ide1a8988e12958e684670a340bf3c09d516ffa23
2021-02-02 07:01:43 -08:00
Mudit
1aea2040d0 More GPU support in idrac-wsman inspect interface
Enhance idrac wsman inspect interface to report an additional
GPU device, namely GV100GL [Tesla V100 PCIe 16GB].

Story: 2008552
Task: 41682
Change-Id: I368180a15bfa7d166d35cef4cdf9741e787046e8
2021-02-02 08:50:21 -05:00
Dmitry Tantsur
ccc6c551c3 Make boot_mode more consistent with other capabilities
All capabilities, except for boot_mode, are read from instance_info.
This change makes instance_info.capabilities[boot_mode] work as well
and deprecates instance_info.deploy_boot_mode.

Note that the special handling of properties.capabilities[boot_mode]
is kept in this patch.

Change-Id: Ic2e7fd4c71b7a7bc2950d17f7e1bbdad73bbb8a7
2021-02-02 12:06:17 +01:00
Dmitry Tantsur
6c8dad9465 ilo: do not change deploy_boot_mode in instance_info
instance_info is the input from an operator, we should not change that.
Use driver_internal_info instead.

Change-Id: I12b4bc0d4599ccf5ef6fdca91f54f4294b127f9d
2021-02-02 11:40:14 +01:00
Dmitry Tantsur
a5f7d75ba2 Apply force_persistent_boot_device to all boot interfaces
For some (likely historical) reasons we only use it for PXE and iPXE,
but the same logic applies to any boot interface (since it depends
on how the management interface and the BMC work, not on the boot
method). This change moves its handling to conductor utils.

Change-Id: I948beb4053034d3c1b4c5b7c64100e41f6022739
2021-02-01 13:37:20 +01:00
Riccardo Pittau
64ff84bd2b Add release version to release notes
Change-Id: I8ab394ebfde2f5d78255d293367b38d9140d7e34
2021-02-01 10:08:29 +01:00
Vanou Ishii
13e77e2179 Fix Mis-Ordering of Bash Variable Definition in DevStack
In devstack/lib/ironic, IRONIC_DEPLOY_DRIVER is defined at line 341.
However variables which use IRONIC_DEPLOY_DRIVER in default value
(e.g. IRONIC_DEPLOY_RAMDISK, IRONIC_DEPLOY_KERNEL, IRONIC_DEPLOY_ISO
and IRONIC_EFIBOOT) are defined at line 276-282.

This will cause problem at line 295-296:

 if [[ "$IRONIC_BUILD_DEPLOY_RAMDISK" == "False" && \
         ! (-e "$IRONIC_DEPLOY_RAMDISK" && -e "$IRONIC_DEPLOY_KERNEL")

So, this commit moves definition of IRONIC_DEPLOY_DRIVER before
its first use.

Change-Id: I74acb32714ce8830d4697fc796146b894aa7d8c9
2021-02-01 10:17:39 +09:00
ankit
2e6777d757 Fixes issue of redfish firmware update
Currently ilo5 based hardware does not support redfish
based firmware update. This patch fixes the issue by
making the change to to check whether sushy_task.messages
is present. It was also not calling prepare_ramdisk()
before rebooting the system to update the firmware which
has been fixed in this patch.

Change-Id: I9d70fed0de1829973748c06a1342d7a7af0f93d4
Story: #2008403
Task: #41339
2021-01-31 09:58:48 +00:00
likui
1162f2e94f Update python packages to python3 in quickstart.rst
Change-Id: I80c82fdaaf759e7f6758147e05945317aa80ee58
2021-01-30 21:38:25 +08:00
Julia Kreger
5f6a51e178 Set default to prevent out of memory conditions
Change-Id: I00efd4784346adb89b137424ed35bdeafe9b6f24
2021-01-29 22:34:54 +00:00
Julia Kreger
d9913370de Guard conductor from consuming all of the ram
One of the biggest frustrations larger operators have is when they
trigger a massive number of concurrent deployments. As one would
expect, the memory utilization of the conductor goes up. Except,
even with the default number of worker threads, if we're requested
to convert 80 images at the same time, or to perform the write-out
to the remote node at the same time, we will consume a large amount
of system RAM. Or more specifically, qemu-img will consume a large
amount of memory.

If the amount of memory goes too low, the system can trigger
OOMKiller which will slay processes using ram. Ideally, we do not
want this to happen to our conductor process, much less the work
that is being performed, so we need to add some guard rails to help
keep us from entering into situations where we may compromise the
conductor by taking on too much work.

Adds a guard in the conductor to prevent multiple parallel
deployment operations from running the conductor out of memory.

With the defaults, the conductor will attempt to throttle back
automatically and hold worker threads which will slow down the
amount of work also proceeding through the conductor, as we are
in a memory condition where we should be careful about the work.

The defaults allow this to occur for a total of 15 seconds between
re-check of available RAM, for a total number of six retries.
The minimum default is 1024 (MB), as this is the amount of memory
qemu-img allocates when trying to write images. This quite literally
means no additional qemu-img process can spawn until the default
memory situation has resolved itself.

Change-Id: I69db0169c564c5b22abd0cb1b890f409c13b0ac2
2021-01-29 14:33:57 -08:00
Bob Fournier
4a7d50ce56 For Supermicro BMCs set enable when changing boot device
When setting the boot settings on Supermicro BMCs, the
BootSourceOverrideEnabled must be set to the desired
value whenever the BootSourceOverrideTarget is set or
it will revert to the default value (Once).  This is
different than what is currently implemented for other
BMCs in which the BootSourceOverrideEnabled is not set
if it matches the current setting.

This change uses the vendor setting to determine if it's
a Supermicro BMC.

Story: 2008547
Task: 41652

Change-Id: I1b1a6baafd4cc4daa2fbdb82f69ded6253b1fcbf
2021-01-29 11:55:51 -05:00
Dmitry Tantsur
121b3348c8 Refactor vendor detection and add Redfish implementation
Get rid of the TODO in the code and prepare for more management
interfaces supporting detect_vendor(). Vendor detecting now runs
during transition to manageable and on power state sync (essentially
same as before but for all drivers not only IPMI).

Update the IPMI implementation to no longer hide exceptions since
they're not handled on the upper level. Simplify the regex and fix
the docstring.

Add the Redfish implementation as a foundation for future
vendor-specific changes.

Change-Id: Ie521cf2295613dde5842cbf9a053540a40be4b9c
2021-01-28 16:41:45 +01:00
Zuul
fd34d3c437 Merge "Add centralized secure boot documentation" 2021-01-27 13:36:39 +00:00
Dmitry Tantsur
4c4c7a869a Add a few words about UEFI user images
Change-Id: I37a686e6f48a422d38ac5921a188d894519b7530
2021-01-26 21:22:56 +01:00
Dmitry Tantsur
33d51f221f Redfish secure boot management
Story: #2008270
Task: #41137
Change-Id: Ied53f8dc5b93522ac9ffc25ec93ad2347a7d1c7c
2021-01-26 17:15:46 +01:00
Dmitry Tantsur
04400eea47 Add centralized secure boot documentation
Move the bits from iLO and iRMC, clean them up a bit.

Change-Id: I5b6da854ae0214141ae25a17b8ea3c7874636372
2021-01-26 17:00:50 +01:00
Zuul
2a7871d060 Merge "Common framework for configuring secure boot" 2021-01-26 12:25:26 +00:00
Zuul
8612c06d4a Merge "redfish-virtual-media: allow a link to raw configdrive image" 2021-01-25 21:22:50 +00:00
Zuul
4af9e49859 Merge "Update minversion of tox" 2021-01-24 15:02:18 +00:00
Lance Bragstad
72044aaa85 Pass context objects directly to policy enforcement
The oslo.policy Enforcer() object knows what to do with instances of
oslo.context RequestContext() if you pass it one.

This makes it easier for people to perform policy enforcement since they
don't need to map important authorization information from the context
object into a dictionary (historically called `creds`). This practiced
didn't guarantee any consistency in `creds` implementations.

You also don't need to call context.to_policy_values() anymore. The
oslo.policy library will do that for you under the hood and map context
values into a set of policy attributes it understands.

This commit updates the calls to enforcement to pass in the context
object where applicable.

Change-Id: Ife4ba098303088023e4341354a1e3bc9f378ce93
2021-01-23 04:36:54 +00:00
Dmitry Tantsur
bb318008b9 redfish-virtual-media: allow a link to raw configdrive image
For historical reasons we always base64+gzip configdrives, even
when accessing them via a URL. This change allows binary images
to work for the redfish-virtual-media case.

Change-Id: If19144de800b67275e3f8fb297f0a5c4a54b2981
2021-01-22 16:26:44 +01:00
Zuul
5640860c81 Merge "Follow-up for ramdisk deploy configdrive support" 2021-01-21 14:06:14 +00:00
Riccardo Pittau
5165edaf93 Update minversion of tox
The minimum version since when tox accepts having inline comments
in deps is 3.9.0

https://tox.readthedocs.io/en/latest/changelog.html#v3-9-0-2019-04-17

Change-Id: I4d2aaf97459344ca01e5ccd6e563474c094a4678
2021-01-21 14:57:07 +01:00
Zuul
d6661a2d4b Merge "Attempt to slim down protection test base class" 2021-01-21 11:33:13 +00:00
Zuul
ac70e24665 Merge "Write stub ACL test for every existing API call" 2021-01-21 11:29:03 +00:00
Zuul
cd89ea38e7 Merge "Update iDRAC doc with missing interfaces" 2021-01-21 08:53:21 +00:00
Lance Bragstad
c7f24e8ceb Attempt to slim down protection test base class
This commit builds on an attempt to introduce a base class that we can
use for protection testing. I'm removing the majority of the fake tokens
and context objects, ultimately removing anything that
keystonemiddleware_authtoken middleware does.

Doing this allows us to set authorization headers directly in the test,
which makes the test case really clear from an input and outcome
perspective because we can see who is make the request and the intended
outcome.

It also removes any need to mock or understand keystonemiddleware
implementation details to implement protection testing in ironic.

Change-Id: I9a3eb62bb41e0623df9aa5c34fde6f343053dd76
2021-01-20 06:41:47 -08:00
Steve Baker
36d819e2fb Write stub ACL test for every existing API call
This adds a skipped test for every documented path and method
to aid in getting test coverage of existing ACL behaviour, in
preparation for doing the same for secure-rbac.

When adding test coverage, the skip keys should be removed, and
specific test inputs and asserts should be added. The test can be
duplicated and renamed to get the required allow/deny test coverage.

Its possible we can delete some of these stubs as the path/method
shares a policy name with another path/method that has test coverage.

test_acl_existing.yaml was generated with the script
http://paste.openstack.org/show/801106/

Change-Id: Iee91d80cef3b9e6024507171352c6de9e89ce36e
2021-01-20 06:36:47 -08:00
Aija Jauntēva
b0df0960e2 Update iDRAC doc with missing interfaces
Change-Id: I691b76879ba00fb5535d7016c9d6fb53e9dde462
2021-01-20 09:25:19 -05:00
Steve Baker
71ccbf5955 Raw image size estimation improved
Adds the `[DEFAULT]raw_image_growth_factor` configuration option which
is a scale factor used for estimating the size of a raw image converted
from compact image formats such as QCOW2. By default this is set to 2.0.

When clearing the cache to make space for a converted raw image, the full
virtual size is attempted first, and if not enough space is available a
second attempt is made with the (smaller) estimated size.

Story: 1750515
Task: 9791
Change-Id: Id86e7641329a95f71ac005ee448b0ff4d7d0bbcd
2021-01-19 19:57:01 +00:00
Lance Bragstad
6ea73bdfbb Bump oslo.log requirement to 4.3.0
This ensures we have the WALLABY marker available for deprecation
noticies.

Change-Id: Ic91f34a90fdd0c22c06c669551eaf6f9a398956b
2021-01-19 10:59:48 -08:00
Dmitry Tantsur
b6f4587f0b Common framework for configuring secure boot
Two drivers already support turning secore boot on and off,
Redfish will follow soon. This patch adds ManagementInterface
calls to get and set the secure boot state.

Story: #2008270
Task: #41561
Change-Id: I96b2697163def52618b4c051a5c85adf7d1818a5
2021-01-19 17:56:06 +01:00
Zuul
7b0487df2e Merge "redfish-virtual-media: make fewer calls when preparing boot" 2021-01-19 06:47:21 +00:00