618 Commits

Author SHA1 Message Date
Zuul
3c91e55886 Merge "Software RAID: Add IPA deploy support" 2019-06-05 19:40:09 +00:00
Zuul
09d2db7c39 Merge "Software RAID: Create/delete configurations" 2019-06-05 07:34:01 +00:00
Arne Wiebalck
8fe390ecdd Software RAID: Add IPA deploy support
This patch proposes to extend the IPA to be able to deploy on
software RAID devices. The IPA needs to be able to detect an
md device, find the underlying holder disks and configure grub
on them.

Change-Id: Ieb2c95ff130b5cc1e643fcde500066d9458ddbec
Story: #2004581
Task: #29102
2019-06-04 12:43:09 +02:00
Arne Wiebalck
2db123d318 Software RAID: Create/delete configurations
This patch proposes to extend the IPA to be able to configure software
RAID devices. For this, the {create,delete}_configuration methods of
the GenericHardwareManager are implemented.

Change-Id: Id20302537f7994982c7584af546a7e7520e9612b
Story: #2004581
Task: #29101
2019-06-04 12:33:40 +02:00
Dmitry Tantsur
5c5328ccaa Supports fetching API endpoints from mDNS
This change enables IPA to receive API endpoints and configuration
via multicast DNS.

Story: #2005393
Task: #30382
Change-Id: Ibbf07052bea8f5c0305dda098b2879bcbc2fece5
2019-05-29 16:58:24 +02:00
Kaifeng Wang
a9cac52190 Relax checksum fields validation
In stein, ironic added the new os_hash_algo and os_hash_value checksum
fields provided by glance, but the checksum field is still mandatory,
which is inconvenient for standalone use case.

We could relax the checksum checking and proceed as long as there is at
least one of checksum mechanism available.

Change-Id: Ia90197416f76ada0422681044a16f1c07d7049a1
Story: 2005773
Task: 33490
2019-05-28 09:38:36 +08:00
Dongcan Ye
eb875cc9a2 Add more channel number for detecting BMC IPv6 address
This is a missing fix for patch.[1]

[1] https://review.opendev.org/#/c/656275/

Change-Id: I8443ae0c08487afea383e111b6c2b6d5941d4024
Task: 30654
Story: 2005528
2019-05-20 11:08:31 +00:00
Doug Hellmann
a2d25de639 show inspection callback url in error messages
We have seen issues in misconfigured systems where messages from IPA
to ironic-inspector fail with 404 or other standard HTTP error
codes. Although there is an info message reporting the URL, the error
messages do not include the URL of the service IPA is trying to talk
to, which makes debugging the configuration difficult. This change
adds the URL to the error already being reported to improve that
situation.

Change-Id: Ib092ac690d29c385c0564c086ad2fec3df0fb2e0
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2019-05-13 18:33:49 -04:00
Doug Hellmann
21393cb1c5 report the URL when heartbeats fail
We have seen issues in misconfigured systems where messages from IPA
to ironic fail with 404 or other standard HTTP error codes. The
messages do not include the URL of the service IPA is trying to talk
to, which makes debugging the configuration difficult. This change
adds the URLs to the end of a couple of errors already being reported
to improve that situation.

Change-Id: I6a4ea8768df016e58a073d78edbe4006e217d69e
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2019-05-13 10:25:10 -04:00
Dongcan Ye
66d3c8e453 Add more channel number for detecting BMC IP address
IPMI SPEC[1] assignment channel number 1-11 for vendors
implementation(We an search "Channel Numbers" in section 6.3).
We met an baremetal product, like H3C, use number 8 as LAN channel.

Current code limit channel number to 1~7,
I think this can increase to 1~11.

[1] https://www.intel.com/content/dam/www/public/us/en/documents/product-briefs/ipmi-second-gen-interface-spec-v2-rev1-1.pdf

Change-Id: I63a6ff91d702b990ef09da13e6e2e269e7274cce
Task: 30654
Story: 2005528
2019-05-05 10:29:58 +00:00
Derek Higgins
9c35f02792 Bind mount /run into chroot when installing grub
grub-mkconfig runs a lvs command that attempts to access /run/lvm
once for each block device, currently it times out after 10 seconds
for each device and moves on. Multiple 10 second delays become
a problem (causing IPA API timeouts) when multiple block devices
are present. Bind mounting in /run avoids the delay and the
timeouts.

Task: 30616
Story: 2005507

Change-Id: Iae8b7808a35bff121f64971aadd4bd36b5f5bb71
2019-04-25 23:53:19 +01:00
Zuul
e6eb4133b6 Merge "Ironic python agent does not extract correct available memory" 2019-04-16 15:16:05 +00:00
Uros Orozel
d8018b9f35 Ironic python agent does not extract correct available memory
This patch fixes parsing of memory class output of lshw output.
Task: 30201
Story: 2005308

Change-Id: I0e7ff9f86a266d2f798457136fed0100c78569e2
2019-04-15 10:59:42 +00:00
Zuul
55e3266c89 Merge "Discover IPv6 BMC address" 2019-04-10 03:49:03 +00:00
Kaifeng Wang
4cb2ac4ae4 Fix docs job failure due to malformated docstring
Change-Id: Ic3532e51481fd07e2f816aeacb07ded2d56791ee
2019-04-09 10:24:17 +08:00
Nikolay Fedotov
518c338ca3 Discover IPv6 BMC address
Change-Id: Ia7f24c1df782de4809a5195876bbf0309b481fd7
Story: #2005133
Task: #29817
2019-04-08 15:58:33 +03:00
Mark Goddard
b81b7c1c87 Add 'interface' to get_clean_steps docstring
Change-Id: Ice547f5eec849650828e1bd1aac48e7f041eeb34
2019-03-21 17:26:24 +00:00
Dmitry Tantsur
f821db3a54 Allow image checksum to be a URL
We allow image_source to be a URL, let us also support URLs for checksums.
This change copies handling of multi-file checksum files from metalsmith.

Change-Id: Ie4d7e5c79b76bdd72d50eeb384cf10519278a80c
Story: #2005061
Task: #29605
2019-02-25 14:28:09 +01:00
Zuul
e5da6d0007 Merge "Add secondary sorting by name when guessing root disk" 2019-02-13 07:41:41 +00:00
Arne Wiebalck
fb74b55606 Add secondary sorting by name when guessing root disk
As some BIOSes try to boot only from the "first" disk, Ironic
should order potential disks not only by size, but also by name.
This patch proposes to add secondary sorting by device name when
identifying the root disk.

Change-Id: I4017c839eeb9d00d2b4ad5b90e4e9b65b74296c7
Story: #2004976
Task: #29434
2019-02-11 17:53:47 +01:00
Riccardo Pittau
d525f8a07f Making ironic-python-agent able to stop with python 3.x
The agent stop function will write a byte string 'a' to the pipe
as a signal for the run function to end process.
The run function is expecting a literal string.
In python 2.x the byte string will automatically be converted to
literal, while python 3.x won't do the conversion, causing the
process to never stop.
This patch will fix that behavior, allowing the IPA to correctly
stop using python 3.x.

Story: 2004928
Task: 29308
Change-Id: Iad16e8bed2436d961dea8ddaec1c2724225b4097
2019-02-04 09:55:08 +01:00
Zuul
6be37298bd Merge "Replacing the HTTP protocal with HTTPS" 2019-01-11 16:22:40 +00:00
Zuul
c5f31db691 Merge "Attempt to read the partition table after writing an image" 2019-01-04 08:01:22 +00:00
zhangshj
d45650f086 Replacing the HTTP protocal with HTTPS
Change-Id: I9791b6781158c0ddebda636daeea39665b483dd6
2018-12-25 19:51:40 +08:00
Sam Betts
fc2dfcee60 Attempt to read the partition table after writing an image
This patch adds code that tries to read the partition table after we've
successfully written an image to make sure the image that we wrote has a
valid partition table so we can more easily guarantee that what we've
written is bootable and not just junk. Without a valid partition table
writing a config drive will fail for whole disk images.

Co-Authored-By: Dmitry Tantsur <dtantsur@redhat.com>
Change-Id: I5cfd8c433a4db3e0d2d5086250e629d16234b7a4
Story: 2001760
Task: 12159
2018-11-19 18:57:23 +01:00
Corey Wright
96961070ee Allow erasing metadata from disk partitions
Modify the metadata erasing call chain to retrieve a list of devices
that includes partitions in addition to disks so it can erase metadata
from all of them, otherwise incidentally recreating disk partitions
causes the Linux kernel to discover and automatically recreate some
types of storage entities (eg LVM PVs, VGs, & LVs, RAID members &
devices).

Change-Id: If8f47a083966051856439e3291a6872929b93e3b
Story: #2003673
Task: #26192
2018-11-13 00:36:18 -06:00
Zuul
3176ea483c Merge "Fix multi-device behavior" 2018-11-12 13:56:57 +00:00
Zuul
f63099ebb6 Merge "Allow streaming raw partition images" 2018-10-26 14:14:55 +00:00
Kaifeng Wang
6f42768756 Follow up to parallel disk erasure
Improve test to verify apply_async is called twice as expected.

Story: 1546949
Task: 11548

Change-Id: I41736dfb2932dd0036bbc4cbc51929bf61a16569
2018-10-24 10:51:43 +08:00
Zuul
67a516bdcf Merge "Parallel erase disk devices" 2018-10-22 20:31:34 +00:00
Zhenguo Niu
1ee42cc3ff Parallel erase disk devices
Currently we erase the disks one by one, which takes a long
time to finish, this patch adds support to the IPA so that
it can erase disks in parallel if told so.

Story: 1546949
Task: 11548
Co-Authored-By: yuan liang <leetpy2@gmail.com>
Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com>

Change-Id: If5cfb6ec000a654d07103c4b378d4c135249e238
2018-10-22 13:56:51 +08:00
Zuul
5ad766ebcb Merge "Enhanced checksum support" 2018-10-20 11:10:55 +00:00
Pierre Riteau
8eecadb0a8 Fix BMC IP address detection on CoreOS images
On CoreOS images, IPA runs in a Debian Jessie chroot which uses mawk as
its default awk implementation. However, mawk doesn't support POSIX
character classes such as [:space:], which means get_bmc_address() fails
to parse the BMC IP address from the output of ipmitool.

This patch replaces the use of [[:space:]] by [ \t] which is equivalent
for the purpose of parsing the output of ipmitool. Note that matching on
tab characters is not strictly required as the packaged version of
ipmitool only uses space characters, but is left in case tabs are used
in other versions.

Change-Id: I0e3306a4d4584ca28e03608e9f7270b770960a39
Story: #2004121
Task: #27571
2018-10-19 14:22:05 +01:00
Dmitry Tantsur
29136bf68d Allow streaming raw partition images
Currently we support streaming raw whole disk images, but not
partition ones. This change enables it.

Change-Id: Ie95102aa3f2054a6b429f3d3e0926e90923c5faf
Story: #2003809
Task: #26558
2018-10-17 11:16:04 +02:00
Kaifeng Wang
ec2bf8667d Enhanced checksum support
Adds enhanced checksum support to IPA, when os_hash_algo and os_hash_value
are passed in via image_info, it will be used to calculate image checksum
and verification.

In other cases, the old md5 checksum is used.

Change-Id: I1d2f33e7059910326b4ac3f7786543b333a93a5a
Story: 2003938
Task: 26846
2018-10-15 17:15:38 +08:00
Julia Kreger
c540731aee Fix multi-device behavior
ATARAID is functionally a version of software
RAID where the setup is managed by the controller
and the Operating System takes over managing the
RAID after boot. Most commonly this is found for
mirrored boot devices.

Prior to this patch, we were looking for non-dependent
items (i.e. base block devices), with a type of disk.

Now we will permit the "disk" to be added to the list
if lsblk indicates that it is a type containing "raid".
The lsblk results should not change as we explicitly
look for disk objects.

Change-Id: Ia4a03b33cc06ce42e1bc33026683c28b31901cb7
Story: #2003445
Task: #24647
2018-09-24 19:10:38 -07:00
Tao Li
8a70f12435 Replace assertEqual(True/False, expr) with assertTrue/assertFalse in tests
In some cases, If the result of expr is a boolen value, we shoud use
assertTrue/assertFalse to instead. Beacause it is clear and simple.

Change-Id: Ie61369f6335a90b09bb24192282d33da5272c13f
Story: #2003785
Task: #26490
2018-09-18 16:14:20 +08:00
Giulio Fidente
bc21b5b140 Clear GPT and MBR headers with dd to avoid sgdisk CRC errors
This change adds a dd before the existing sgdisk -Z command to
workaround CRC verification errors.

Change-Id: Ia1ac4e1c0faf14ad4bb11c2a1c796c93ca8cb5e3
Closes-Bug: #1737556
Story: 1737556
Task: 11496
2018-08-08 16:40:22 +00:00
Zuul
667589bb00 Merge "fall back to PARTUUID if UUID not found." 2018-08-03 15:40:51 +00:00
Zuul
d009263349 Merge "Collect IPv6 address during introspection" 2018-08-02 14:05:14 +00:00
Matthew Thode
7e0bcbbeee fall back to PARTUUID if UUID not found.
Change-Id: Icbf1fd8179658bd8bcd15f99aaaee796288dbf6f
Story: 2002052
Task: 19714
2018-07-31 13:01:33 -07:00
zhengyong
561be77303 Collect IPv6 address during introspection
This patch adds support to retrieve IPv6 address.

A new field ``ipv6_address`` is added to NetworkInterface
and store the assigned IPv6 address (if any).

Co-Authored-By: Kaifeng Wang <kaifeng.w@gmail.com>
Change-Id: Ia527a5aa48e3daf66d2be190e43935b38b3bd6f9
Closes-Bug: #1744064
Story: #1744064
Task: #11604
2018-07-24 09:22:40 +08:00
Michael Turek
b32750f5c4 Install grub to PReP partition when prep_boot_part_uuid is provided
Installs the grub bootloader to the PreP Boot partition when the
prep_boot_partition_uuid is provided. This is required when
booting a partition image locally on ppc64* systems.

This change also passes the cpu_arch along to work_on_disk so
that the PReP partition is created when partitioning disks for
local boot on ppc64* systems,

Change-Id: I70667d43af962b357e6eeccba258f4fa5a91a09e
Depends-On: I2bc9f13ec605de7b7b96d96a1a4edebee0af76dc
Story: #1749057
Task: #22999
2018-07-20 16:07:16 +00:00
Zuul
105e580310 Merge "Provide knob to disable ata secure erase" 2018-06-29 15:12:37 +00:00
Julia Kreger
d9f59d94f2 Provide knob to disable ata secure erase
We need to allow the operator to able to
explicitly disable secure erase, in case
it is problematic in their environment
or hardware.

Change-Id: I4c68efa65cdd7f88f54f8dd9a8bcbeee9e8124a8
Story: #2002546
Task: #22108
2018-06-28 22:20:15 +00:00
Julia Kreger
aef703b879 Refuse secure erase if ATA command does not work
Adds dependency upon smartmontools's binary smartctl to
query the block devices via ATA mode which fails on pass-thru
buses such as ATA over SCSI and ATA over USB, in an effort
to prevent the initiation of ATA secure erase with one
of these interfaces in place which may render the disk
unreachable after security options are enabled for
ATA Secure Erase or upon the Secure Erase command being
sent to the Hard Disk.

Change-Id: I7635a197eb000650e919fac386b38ac15ef17041
Story: #2002546
Task: #22109
Depends-On: Ibbfd168844524d91927bdd6e67d973e0bd519bf2
2018-06-28 18:38:10 +00:00
Julia Kreger
0f7b5a0896 Try to unlock failed device before proceeding
When a hard error has occured with secure erase,
we should attempt an unlock of the device becuase
the current mode can prevent disk IO. This may upset
some things like raid controllers even if they are
in a pass-through mode.

Change-Id: I32e1d962fbbb4a305d5dbebea92ac48ebd9b67ca
Story: #2002546
Task: #22107
2018-06-18 19:43:40 +00:00
Bill Dodd
3c30088c1e Add min/max values to integer config options
None of the existing ironic-python-agent integer config options included
min or max values. Added appropriate min/max values for the integer
config options.

Two of the integer options are for ports (listen_port and
advertise_port). These were changed to use the more appropriate
oslo_config cfg.PortOpt instead of cfg.IntOpt. PortOpt has the proper
min and max values built in.

Change-Id: I98709a45d099aea62c9973beb6817591cb445a9c
Story: 1731950
2018-05-23 12:08:42 -05:00
Zuul
cd96627760 Merge "write byte objects when using os.write" 2018-05-18 14:30:31 +00:00
Zuul
bcb4f3652d Merge "Fix for fatal error when GPT was used and only MBR was cleaned" 2018-05-18 14:30:29 +00:00