60 Commits

Author SHA1 Message Date
Riccardo Pittau
d525f8a07f Making ironic-python-agent able to stop with python 3.x
The agent stop function will write a byte string 'a' to the pipe
as a signal for the run function to end process.
The run function is expecting a literal string.
In python 2.x the byte string will automatically be converted to
literal, while python 3.x won't do the conversion, causing the
process to never stop.
This patch will fix that behavior, allowing the IPA to correctly
stop using python 3.x.

Story: 2004928
Task: 29308
Change-Id: Iad16e8bed2436d961dea8ddaec1c2724225b4097
2019-02-04 09:55:08 +01:00
Matthew Thode
a03661c4a8
write byte objects when using os.write
Change-Id: I184a9d0bf4a0ba0776d519b3a3b9ccd39151b4ae
Story: 2002052
Task: 19713
2018-05-17 11:11:55 -05:00
Shivanand Tendulker
f08636fe8b Follow-up patch for rescue extension for CoreOS
This patch addresses few minor comments in commit
a659306272542dd38420cb118cc7b04b1e8cf377

Change-Id: Id5b48e3cc96c8807c471c947da3e233cebdf687e
Related-Bug: #1526449
2018-01-30 19:00:13 +00:00
Zuul
893c63f24a Merge "Rescue extension for CoreOS with DHCP tenant networks" 2017-12-11 21:14:09 +00:00
Derek Higgins
214790d17e Ignore IPv6 link local addresses
Prevent IPA from picking up the IPv6 link-local address
as a callback_url in cases where it gets tried before other
addressing methods havn't complete yet. In this scenario IPA
sleeps for 10 seconds and then retries giving the nic a chance to
configure its routable IP address.

Change-Id: Ic53334c630180f0d77bb0231e548d2c44bfe55ca
Closes-Bug: #1732692
2017-11-21 10:11:21 +00:00
Mario Villaplana
a659306272 Rescue extension for CoreOS with DHCP tenant networks
This patch adds support for rescue mode with DHCP tenant networks in
CoreOS. Applying network config from a configdrive is not yet supported
but will be in a future patch.

Co-Authored-By: Jay Faulkner <jay@jvf.cc>
Co-Authored-By: Taku Izumi <izumi.taku@jp.fujitsu.com>
Co-Authored-By: Annie Lezil <annie.lezil@gmail.com>
Co-Authored-By: Aparna <aparnavtce@gmail.com>
Co-Authored-By: Shivanand Tendulker <stendulker@gmail.com>
Change-Id: I7898ff22800dedba73d7fbfb3801378867abe183
Partial-Bug: 1526449
2017-11-06 04:48:58 -05:00
John L. Villalovos
949f4f509e Use flake8-import-order
Use the flake8 plugin flake8-import-order to check import ordering. It
can do it automatically and don't need reviewers to check it.

Change-Id: I946457e9079ce0b54c7fe0ad554d024a1c61dce0
2017-02-16 09:46:21 -08:00
Jenkins
fd7f10b993 Merge "Configure and use SSL-related requests options" 2017-02-07 09:57:49 +00:00
Derek Higgins
b4e41e2dd2 Agent: Listen for connections on both IPv4 and IPv6 ports
Allow connections if deploying over a IPv6 network.

Change-Id: Ied2f6be4aa4d1a70524df1df3506e596f6926e5b
Closes-Bug: #1650539
2017-01-19 15:24:11 +00:00
Pavlo Shchelokovskyy
fdd11b54a5 Configure and use SSL-related requests options
This patch adds standard SSL options to IPA config and makes use of them
when making HTTP requests.

For now, a single set of certificates is used when needed.
In the future configuration can be expanded to allow per-service
certificates.

Besides, the 'insecure' option (defaults to False) can be overridden
through kernel command line parameter 'ipa-insecure'.
This will allow running IPA in CI-like environments with self-signed SSL
certificates.

Change-Id: I259d9b3caa9ba1dc3d7382f375b8e086a5348d80
Closes-Bug: #1642515
2017-01-13 11:33:44 +02:00
Derek Higgins
9f5f664080 Advertise the correct address when using IPv6
Parse the output of "ip route get $IP" taking
IPv6 into consideration. Also wrap the IP address
in square brackets if it is IPv6.

Change-Id: Ifc44e5aa3c5b814b6ceba04461bb68fe1d75c22b
Closes-Bug: #1650533
2017-01-11 11:00:56 +00:00
Yufei
dd9253f1b6 Skip API related work if no api url configured
Currently, if IPA is booted without an ironic api url, it will default
to localhost and fail to connect. Instead, we now explicitly fail and
print a log message if no api callback url is provided.

Change-Id: I0271be94ba7febc6abd5bf3343f6fa179bc1a6a4
Closes-Bug: #1643966
2016-12-07 17:04:05 +08:00
Pavlo Shchelokovskyy
b033bfd933 Remove old lookup/heartbeat from IPA
Lookup/Heartbeat via vendor passthru was deprecated in Newton.

This patch removes the corresponding functionality from IPA,
and also removes handling of 'ipa-driver-name' kernel parameter,
as it was only used in code related to old passthru.

Change-Id: I2c7989063ab3e4c0bae33f05d6d2ed857a2d9944
Closes-Bug: #1640533
2016-11-09 16:34:44 +00:00
John L. Villalovos
20d960ff98 Remove Python 2.6 format style
In Python 2.6 it was required to use {0}, {1}...{n} when using the
string format function. In Python 2.7 and Python 3 it it not required.

Change {N} to {} in code.

This brings the code in style alignment with other projects like
ironic and ironic-lib.

Change-Id: I81c4bb67b0974f73905f14b589b3dd0a7131650d
Depends-On: I8f0e5405f3e2d6e35418c73f610ac6b779dd75e5
2016-10-06 09:05:26 -07:00
John L. Villalovos
abf98ae84a Use namedtuple to improve code readability
Use the namedtuple class to improve code readability by creating a Host
class with namedtuple to store the 'hostname' and 'port'

Replace foo[0] with foo.hostname, and foo[1] with foo.port to make code
more readable.

Change-Id: Ie2b5f9cf89e7ccbbcf0a2573dab6f6c5d14c018b
2016-08-30 16:12:29 -07:00
Julia Kreger
95f31be108 Re-use API client for Heartbeat operations
Re-initialization of the API client causes the identification
of an older ironic API to be forgotten.

As such, we need to re-use the API client from the initial lookup
for the heartbeat.

Tested with current master branch and stable/mitaka ipa
in noauth mode, and was able to deploy without any issues.

Closes-Bug: #1616461
Change-Id: I3ae1e2ce83f614b5b936a3cbb1b9587c6c55a89a
2016-08-26 12:53:30 +00:00
Jenkins
dfaf92b611 Merge "Add metrics support to IPA" 2016-08-09 16:15:29 +00:00
Dmitry Tantsur
09265ba4b5 Use new agent API if available
Falls back to vendor passthru on receiving 404.

Also fixes logging around lookup: log traceback on unexpected
exceptions, log successful lookup and replace % with ,

Change-Id: I7160c99ca63585fc333482fa578fdf5e0962b2b6
Depends-On: I9080c07b03103cd7a323e2fc01be821733b07eea
Partial-Bug: #1570841
2016-08-05 12:02:41 +02:00
Josh Gachnang
fd874652e3 Add metrics support to IPA
This utilizes the new metrics support in ironic-lib to allow the agent to
report timing metrics for agent API methods as configured in ironic-lib.

Additionally, this adds developer docs on how to use metrics in IPA,
including some caveats specific to ironic-lib.metrics use in IPA.

Co-Authored-By: Jay Faulkner <jay@jvf.cc>
Co-Authored-By: Alex Weeks <alex.weeks@gmail.com>
Change-Id: Ic08d4ff78b6fb614b474b956a32eac352a14262a
Partial-bug: #1526219
2016-08-03 11:24:54 -07:00
vmud213
09db71d640 Wait for at least one interface before node lookup
During node look up sometimes IPA sends hardware inventory information before
the interfaces are up, resulting in empty interfaces information being sent
to the conductor. This causes conductor to throw exceptions and polluting
the log until the real interfaces information is populated. This change makes
IPA to wait for at least one interface to be up before trying for node lookup.

Change-Id: Ifdb91298eaa5c725f108fa722263ed925691ecda
Closes-Bug: #1562786
2016-06-14 12:36:52 +00:00
Jenkins
64e813ee67 Merge "Get root device hints from the node object" 2016-05-30 13:03:34 +00:00
Lucas Alvares Gomes
33535cd572 Get root device hints from the node object
In order to support a more complex syntax for root device hints (e.g
operators: greater than, less than, in, etc...) we need to stop relying
on the kernel command line for passing the root device hints. This patch
changes this approach by getting the root device hints from a cached
node object that was set in the hardware module.

Two new functions: "cache_node" and "get_cached_node" were added to the
hardware module. The idea is to facilitate the access to a node object
representation from the hardware extension methods without changing
method signatures, which would break compatibility with out-of-tree
hardware managers.

Note that the new "get_cached_node" is just a guard function to
facilitate the tests for the code.

The function parse_root_device_hints() and its tests were removed since
it's not used/needed anymore.

Partial-Bug: #1561137
Change-Id: I830fe7da1a59b46e348213b6f451c2ee55f6008c
2016-05-26 14:52:15 +01:00
Dmitry Tantsur
c09753bb60 Add a configurable sleep before IPA starts working
Some kernel modules take substantial time to initialize. For example,
with mpt2sas RAID driver inspection and deployment randomly fail
due to IPA starting before the driver finishes initialization.

As much as I hate it, the only way to guarantee that the hardware is
truely initalized is to wait for it. Apparently all hardware in Linux
is treated as hotplugged, so there is no such thing as "hardware
initialization is finished". Operators can add a sleep based on their
knowledge of their hardware.

The default behaviour remains the same.

Change-Id: I0446ae81d760dacaf31eea6ad9f9eaa098cf5e93
Partial-Bug: #1582797
2016-05-24 10:39:52 +02:00
Jay Faulkner
5a1a1ca61c Revert "Add hardware manager interface for hardware initialization"
I would've voted -1 on the patch in question had I reviewed it, and per
standard OpenStack/Ironic procedure, I'm reverting it for re-review and
discussion.

In this case; I don't think the new method in the HWM interface is
needed, and that evaluate_hardware_support() is intended to handle the
cases handled.

This reverts commit 0962cae1da69a1a2981d5950ad741d91115dac06.

Change-Id: Ic08e44bdf116403444b257ee9f4e5b906f5eac53
2016-05-23 17:41:29 +00:00
Dmitry Tantsur
0962cae1da Add hardware manager interface for hardware initialization
Some kernel modules take substantial time to initialize. For example,
with mpt2sas RAID driver inspection and deployment randomly fail
due to IPA starting before the driver finishes initialization.

Add a new hardware manager method initialize_hardware, which gets
run on start up before other hardware manager method invocations.

The generic implementation is to call udev settle and wait for
at least one suitable disk device to appear with the hardcoded
timeout of 15 seconds. Also preload the IPMI modules instead of
calling modprobe every time the inventory is requested.

Change-Id: If7758bb6e3faac7d05451baa3a26adb8ab9953d5
Partial-Bug: #1582797
2016-05-20 15:38:53 +02:00
Dmitry Tantsur
6829d34c15 Bind to interface routable to the ironic host, not a random one
Binding to the first interface that has an IP address is error-prone: there is
no guarantee that ironic can reach us via this inteface. It is much safer to
detect the interface facing ironic and bind to it.

Unused LookupAgentInterfaceError exception is deleted.

The TinyIPA build also requires iptables dependency at build time to insert the
required kernel modules.

Closes-Bug: #1558956
Change-Id: I9586805e6c7f52a50834bc03efeb72d1faa6cb65
2016-03-21 14:21:12 +00:00
Zhenguo Niu
d25d94b316 Change to use WARNING level for heartbeat conflict errors
It's normal that ironic returns 409 Conflict from time to time, so
it's a bit confusing that we report this with Exception level and
traceback.

Change-Id: I1627c61facc3fadd0f5d9d324150e7d2833c7fbc
Closes-Bug: #1533113
2016-03-06 17:13:02 +08:00
Zhenguo Niu
7fe40bb559 Replace all the 'self.log' calls with global LOG
Change-Id: Iae917e1139c8c95a1bad5466df8a607656b9de5c
2016-02-24 09:51:01 +08:00
John L. Villalovos
dcbba2b121 Enforce all flake8 rules except E129
Bring ironic-python-agent in line with the other ironic projects.

Stop ignoring all E12* errors except E129
Stop ignoring E711

Change-Id: Icb9bc198473d1b5e807c20869eb2af7f4d7ac360
2015-10-02 10:01:00 -07:00
Dmitry Tantsur
096830414b Add support for inspection using ironic-inspector
Adds a new module ironic_python_agent.inspector and new entry point
for extensions, which will allow vendor-specific inspection.

Inspection is run on service start up just before the lookup.
Due to this early start, and due to the fact we don't even know
MAC address of nodes on inspection (to say nothing about IP addresses),
exception handling is a bit different from other agent features:
we try hard not to error out until we send at least something to inspector.

Change-Id: I00932463d41819fd0a050782e2c88eddf6fc08c6
2015-09-07 18:22:54 +02:00
Jay Faulkner
09221f98cb Load Hardware Managers at runtime
Hardware managers should load at runtime. This will ensure the agent is
ready to respond to API calls before it begins heartbeating. Also, it
means in case of a syntax or other error in a HardwareManager, the agent
will crash before it heartbeats, which is better than it working until a
hardware manager method is needed.

Change-Id: I9403ce7bedc8d5af20b6d84371367253b26b74c2
Closes-bug: 1490008
2015-08-28 14:15:31 -07:00
Ghe Rivero
10a3a800e7 Use oslo_log lib
Module 'log' from oslo-incubator was removed after release of oslo_log
library.

Change-Id: Ia00e26071fc0cab4fd3f3334ca94860bc7b1c75c
2015-03-24 08:43:35 +01:00
Jay Faulkner
2bbec5770c Allow use of multiple simultaneous HW managers
Currently we pick the most specific manager and use it. Instead, call
each method on each hardware manager in priority order, and consider the
call successful if the method exists and doesn't throw
IncompatibleHardwareMethodError.

This is an API breaking change for anyone with out-of-tree
HardwareManagers.

Closes-bug: 1408469
Change-Id: I30c65c9259acd4f200cb554e7d688344b7486a58
2015-01-08 15:15:13 -08:00
Josh Gachnang
417bf086a5 Add standalone mode for IPA
This allows a developer to run IPA without an Ironic API. This can
be useful for testing (especially functional testing) or testing
integration of things like hardware managers.

Change-Id: I2dc49fbe306430bf5b05a36fe56de5275fc128b2
2014-12-16 17:59:29 -08:00
Jim Rollenhagen
082cf29cec Force heartbeat immediately after async command completes
This change passes the agent object into extensions, such that
they may call agent methods as needed. It also causes async
commands to force a heartbeat immediately after completing the
command. This allows Ironic to get a heartbeat and continue
work as quickly as possible, while also allowing deployers to
configure Ironic (agent) to heartbeat less often.

Change-Id: Ib3c3a43dfd0ed4ed51b7d52ac099f01181ca822f
2014-10-13 11:09:09 -07:00
Jim Rollenhagen
4f57590b2e Use poll instead of threading.Event in heartbeat thread
poll() does not rely on absolute time.
This fixes an issue where the system clock may be changed, and
the heartbeat thread would stop heartbeating.

This also gives us flexibility to force a heartbeat in a later
change.

Change-Id: Ifde7b53f59788addc43a5dac95c7b6519d87e834
Closes-Bug: 1355375
2014-10-13 10:50:08 -07:00
Jim Rollenhagen
de2b004654 Delete unused method get_agent_mac_address()
Does what it says.

Change-Id: I01e20e3a1e7317c17da6fc04ee0b0a672cf8247b
2014-09-22 14:32:27 +00:00
Jim Rollenhagen
69884b669d Add docstrings for agent.py
Does what it says.

Partial-Bug: 1367915

Change-Id: If2617f30c75c4a844c20a60afa41330daf84053a
2014-09-11 16:50:33 -07:00
Jim Rollenhagen
a9f2179761 Fix exception that is not properly raised
This commit fixes an exception that was not properly raised, and
also makes the exception more relevant.

This also fixes an outstanding bug where, if the agent
was not associated with a node, get_node_uuid() would fail in an
unexpected manner.

Change-Id: Ifca474a73dd50b5fd2242e5b7e938a5db04f27a8
2014-09-10 14:53:37 -07:00
Ellen Hui
b4f1a0b2d3 Tries to advertise valid default IP
During the first heartbeat, the heartbeater asks the agent to check
its advertised address; if the advertised IP is still the default
(None), the agent tries to replace it with the IP of the first network
interface it finds.  If it fails to find either a network interface or
an IP address, the agent raises an exception.

Change-Id: I6d435d39e99ed0ff5c8b4883b6aa0b356f6cb4ae
Closes-Bug: #1309110
2014-06-10 20:54:34 +00:00
Vladimir Kozhukalov
b306626e86 Flow extension uses extension manager from agent
Removed creating separate extension manager for flow extension.
Instead, have made flow extension using the same extension manager
instance which is initialized in agent. It fixes circular
extension loading in stevedore.

Closes-Bug: #1316145
Change-Id: Id339f1876168a41ca43ba7473f3ff6949a233ef3
2014-06-02 15:21:38 +04:00
Jenkins
d69bd2b054 Merge "Make encoding.serialize() more programmatical" 2014-05-29 21:58:50 +00:00
Russell Haering
cad3588071 Don't begin heartbeating until the API server is listening
As soon as the agent sends its first heartbeat, it may begin receiving
API calls. As such, we should wait until the server is actually
listening before starting the heartbeater.

Change-Id: Id2984f3f0f50cdb725b10913d38b990f99c48b0a
2014-05-20 14:45:39 -07:00
Alexander Gordeev
ed4460990e Make encoding.serialize() more programmatical
Introduce `serializable_fields` to express which class attributes
to be serialized.

Get rid of OrderedDict. Just replacing it with regular dict.

Change-Id: I3f7639dab171d3d62e92d0d1bb6d7b071cf963ad
2014-05-06 18:02:45 +04:00
Russell Haering
5ebd2e9797 Organize agent extensions
Move extensions under an ironic_python_agent.extensions module. This
change also moves the @async_command() decorator into the base
extension module.

Change-Id: I4021fcc33a30f3460a31bca44a4bf776cd53d488
2014-04-14 15:09:12 -07:00
Jim Rollenhagen
46e9776d42 Use configurable driver name in lookup URL
Allow configuration via command line arguments or kernel parameters.
Default to agent_ipmitool, the reference driver.

Depends on https://review.openstack.org/#/c/84795/12

Change-Id: I55c4a8713308d038002a6567471cd862bf89ec76
2014-04-14 11:32:16 -07:00
Jenkins
952ade37f6 Merge "Kill the build_agent function" 2014-04-14 18:12:45 +00:00
Russell Haering
8422118fa4 Kill the build_agent function
This used to actually do things, now it just wraps the constructor.
Lets kill it.

Change-Id: I320684f8f52d032a1755460eacc70f5aeee92d7a
2014-04-14 10:12:31 -07:00
Alexander Gordeev
5bc793bcb0 Resolve circular import, introduce ExecuteCommandMixin
Adding new methods to utils won't work as the circular import appears
between errors.py and utils.py.

Introduce ExecuteCommandMixin and use it for IronicPythonAgent and
FlowExtension (in future patch).

Also add tests for its.

Change-Id: Id95b31349292a7967d2ee66ec82c1662d8e5de94
2014-04-14 16:04:28 +04:00
Jim Rollenhagen
3c1d52cbb1 Use # instead of """ for copyright blocks
Reformats copyright messages to be comments rather than
docstring-style blocks.

Change-Id: I4d863f53b67bb49d03bda0952b9e6179b6d23c59
2014-04-10 07:14:06 -07:00