22766 Commits

Author SHA1 Message Date
Zuul
9426572b9e Merge "Add neutron-tempest-mariadb-full periodic job" 2019-12-08 03:25:27 +00:00
Zuul
a28fb51f50 Merge "[OVN] Import ovsdb related code" 2019-12-08 03:00:15 +00:00
Zuul
239c0f2683 Merge "Force arp_responder to True when DVR and tunneling enabled" 2019-12-07 16:25:51 +00:00
Zuul
3e6ba67407 Merge "[OVN] Move OVN commons to neutron tree" 2019-12-06 23:59:07 +00:00
Zuul
12852b3e68 Merge "Log exact error during getting value from file" 2019-12-06 15:05:01 +00:00
Zuul
84d568649d Merge "[OVN] Add revision number maintenance methods" 2019-12-06 08:42:49 +00:00
Zuul
0c7784a248 Merge "OpenFlow monitor" 2019-12-06 06:02:07 +00:00
Zuul
14b62b2dc0 Merge "Use threads insted of greethreads in IP monitor" 2019-12-06 06:02:03 +00:00
Zuul
23b138b2c3 Merge "Migrate from ofctl to native QoS DSCP" 2019-12-06 06:01:58 +00:00
Zuul
24891ab4c9 Merge "Move DevStack OVN module to Neutron" 2019-12-06 06:01:46 +00:00
Zuul
1845ee0814 Merge "Do not inherit from built-in "dict"" 2019-12-06 03:08:33 +00:00
David Shaughnessy
b81155ee27 Migrate from ofctl to native QoS DSCP
Neutron migrated from using ofctl from the command line in Pike.
This patch refactors DSCPs OvS backend to use native rather than
ofctl and deprecates the run_ofctl function in ovs_lib.OVSBridge.

Change-Id: Id9ab0eaf92b6ec6d5c9197bee60d324ffcb192a8
Related-Bug: #1853171
2019-12-05 12:08:21 +00:00
Lucas Alvares Gomes
96741866a7 Move DevStack OVN module to Neutron
This patch is moving the DevStack module to deploy OVN to the Neutron
DevStack plugin.

As a first step, this plugin will continue to use the
openstack/networking-ovn repository and, as we advance into the mergings
steps we will modify this plugin (just few lines) to use the code in
the Neutron repository itself.

TODO's where left in the code on top of all the bits that will change
once the networking-ovn code is merged into Neutron.

Below is a snippet of the what needs to be enabled in local.conf to get
OVN deployed:

[[local|localrc]]

enable_plugin neutron https://opendev.org/openstack/neutron

Q_AGENT=ovn
Q_ML2_PLUGIN_MECHANISM_DRIVERS=ovn,logger
Q_ML2_PLUGIN_TYPE_DRIVERS=local,flat,vlan,geneve
Q_ML2_TENANT_NETWORK_TYPE="geneve"

Partially-Implements: blueprint neutron-ovn-merge
Change-Id: I24ab11ab923339959eecfbaed79a3ceadc4a87f4
Signed-off-by: Lucas Alvares Gomes <lucasagomes@gmail.com>
2019-12-05 11:06:44 +00:00
Zuul
7ba82ebcdf Merge "[OVN] Add revision number methods" 2019-12-05 05:58:46 +00:00
Zuul
3a91398c14 Merge "Update comments messages in abandon_old_reviews script" 2019-12-05 02:09:03 +00:00
Zuul
392d93dc14 Merge "Remove grenade-py3 job from Neutron CI" 2019-12-04 19:56:30 +00:00
Rodolfo Alonso Hernandez
4b5cf9e5fb [OVN] Import ovsdb related code
This patch imports ovsdb related code from networking_ovn.

Previous paths in networking-ovn tree:
./networking_ovn/ovsdb/commands.py ->
  ./neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/commands.py
./networking_ovn/ovsdb/worker.py ->
  ./neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/worker.py
./networking_ovn/ovsdb/ovn_api.py ->
  ./neutron/plugins/ml2/drivers/ovn/mech_driver/ovsdb/api.py

ACL commands will be implemented when the related code is merged and
could be tested properly.

Related-Blueprint: neutron-ovn-merge

Co-Authored-By: Reedip <rbanerje@redhat.com>
Co-Authored-By: Numan Siddique <nusiddiq@redhat.com>
Co-Authored-By: Flavio Fernandes <flaviof@redhat.com>
Co-Authored-By: Terry Wilson <twilson@redhat.com>
Co-Authored-By: Daniel Alvarez <dalvarez@redhat.com>
Co-Authored-By: Changxun Zhou <zhoucx@dtdream.com>
Co-Authored-By: Gal Sagie <gal.sagie@huawei.com>
Co-Authored-By: Amitabha Biswas <abiswas@us.ibm.com>
Co-Authored-By: Richard Theis <rtheis@us.ibm.com>
Co-Authored-By: lzklibj <lzklibj@cn.ibm.com>
Co-Authored-By: zhufl <zhu.fanglei@zte.com.cn>
Co-Authored-By: Na <nazhu@cn.ibm.com>
Co-Authored-By: Chandra S Vejendla <csvejend@us.ibm.com>
Co-Authored-By: Gary Kotton <gkotton@vmware.com>
Co-Authored-By: Aaron Rosen <aaronorosen@gmail.com>
Co-Authored-By: Rodolfo Alonso Hernandez <ralonsoh@redhat.com>

Change-Id: I9fe64f954d227efaab5e96c6150df44f36a2530a
2019-12-04 13:18:50 +00:00
Maciej Józefczyk
65692127f6 [OVN] Move OVN commons to neutron tree
Move OVN related commons to neutron tree.

Previous paths in networking-ovn tree:
./networking_ovn/common/constants.py -> ./neutron/common/ovn/constants.py
./networking_ovn/common/exceptions.py -> ./neutron/common/ovn/exceptions.py
./networking_ovn/common/utils.py -> ./neutron/common/ovn/utils.py
./networking_ovn/common/hash_ring_manager.py -> neutron/common/ovn/hash_ring_manager.py
./networking_ovn/common/config.py -> ./neutron/conf/plugins/ml2/drivers/ovn/ovn_conf.py

Co-Authored-By: Gal Sagie <gal.sagie@huawei.com>
Co-Authored-By: Boden R <bodenvmw@gmail.com>
Co-Authored-By: Daniel Alvarez <dalvarez@redhat.com>
Co-Authored-By: Amitabha Biswas <abiswas@us.ibm.com>
Co-Authored-By: Chandra S Vejendla <csvejend@us.ibm.com>
Co-Authored-By: Babu Shanmugam <bschanmu@redhat.com>
Co-Authored-By: Lucas Alvares Gomes <lucasagomes@gmail.com>
Co-Authored-By: Terry Wilson <twilson@redhat.com>
Co-Authored-By: Ramu Ramamurthy <ramu.ramamurthy@us.ibm.com>
Co-Authored-By: Maciej Józefczyk <mjozefcz@redhat.com>
Co-Authored-By: Gary Kotton <gkotton@vmware.com>
Co-Authored-By: Andrew Austin <aaustin@redhat.com>
Co-Authored-By: Miguel Angel Ajo <majopela@redhat.com>
Co-Authored-By: Brian Haley <bhaley@redhat.com>
Co-Authored-By: Dong Jun <dongj@dtdream.com>
Co-Authored-By: xurong00037997 <xu.rong@zte.com.cn>
Co-Authored-By: Rodolfo Alonso Hernandez <ralonsoh@redhat.com>

Change-Id: Ib46bfdd14a150a324dbf28c6a50c839c5c824e35
Related-Blueprint: neutron-ovn-merge
2019-12-04 13:15:16 +00:00
Rodolfo Alonso Hernandez
f8cb7e849f [OVN] Add revision number maintenance methods
The OVNRevisionNumbers maintenance methods are used to check
inconsistences between the Neutron and the OVN databases. Two
methods are implemented:
- get_inconsistent_resources: to retrieve those registers with
  different revision number
- get_deleted_resources: to retrieve the list of OVNRevisionNumbers
  orphan registers (the resource register is deleted).

Previous paths in networking-ovn tree:
./networking_ovn/db/maintenance -> ./neutron/db/ovn_revision_numbers_db

Co-Authored-By: Lucas Alvares Gomes <lucasagomes@gmail.com>

Change-Id: I948af3d3b93731dbf0773615759b49bf6b4a8f5f
Partially-Implements: blueprint neutron-ovn-merge
2019-12-04 11:17:02 +00:00
Rodolfo Alonso Hernandez
04f693bbbb [OVN] Add revision number methods
OVNRevisionNumbers registers are used to check the synchronization
status of the objects handled by OVN, assigning a revision number
to those objects stored in the Neutron DB, also present in the OVN
one.

The methods implemented are:
- create_initial_revision
- delete_revision
- get_revision_row
- bump_revision

Partially-Implements: blueprint neutron-ovn-merge
Change-Id: I8a3bf9a915f6812db6e78a189127613a2fa204ae
2019-12-04 11:16:25 +00:00
Slawek Kaplonski
10bb7ecbb6 Update comments messages in abandon_old_reviews script
In comments given automatically by abandon_old_reviews.sh script
there was still Jenkins mentioned as CI tool.
We are using Zuul for very long time and "zuul" user is voting
now instead of "jenkins" so this patch updates those commit
messages to make this messages correct.

TrivialFix

Change-Id: Iad119108ea82b47ac2a66b9f6f8a8bb56b57e7b6
2019-12-04 10:38:14 +01:00
Zuul
97e495671f Merge "Set ignore_basepython_conflict to true in tox.ini" 2019-12-04 05:03:58 +00:00
Zuul
15f8c0e399 Merge "[OVN] Add hash ring methods" 2019-12-03 23:38:33 +00:00
Zuul
0496741071 Merge "Serialize subnet creating depending on the network ID" 2019-12-03 16:16:28 +00:00
Zuul
0efff05346 Merge "use objects common_types from neutron-lib" 2019-12-03 14:46:07 +00:00
Zuul
26526ba275 Merge "Add upgrade check for NIC Switch agent" 2019-12-03 12:06:10 +00:00
Slawek Kaplonski
b59fdaa9aa Log exact error during getting value from file
Before this patch function get_value_from_file() was only
logging something like "Unable to access {filename}" but it
would be useful sometimes to check what was exact reason of
IOError so lets log this info also.

TrivialFix

Related-bug: #1853652

Change-Id: I28b1753a8d5767babbb6f06a581ffe1fc4ad75d3
2019-12-03 10:10:27 +01:00
Rodolfo Alonso Hernandez
d63da24a13 [OVN] Add hash ring methods
OVNHashRing objects are used by OVN backend to register the nodes
controlled. The methods implemented are:
- add_node: including the group_name and optionally the node_uuid
- remove_nodes_from_host: remove all nodes maching a hostname and
  group_name
- touch_nodes_from_host: update a OVNHashRing register timestamp,
  filtering by hostname and group_name
- touch_node: update a OVNHashRing register timestamp, filtering by
  node_uuid
- get_active_nodes: retrieve any active node (filtering timestamp),
  group_name and hostname
  
Previous paths in networking-ovn tree:
./networking_ovn/db/hash_ring.py -> ./neutron/db/ovn_hash_ring_db.py
./networking_ovn/tests/unit/db/test_hash_ring.py -> 
./neutron/tests/unit/db/test_ovn_hash_ring_db.py


Co-Authored-By: Lucas Alvares Gomes <lucasagomes@gmail.com>
Change-Id: Ibb9c79771c48ccde564c39be9d781f3720802a2d
Partially-Implements: blueprint neutron-ovn-merge
2019-12-03 08:53:45 +00:00
Zuul
991126eb6e Merge "[OVS FW] Clean port rules if port not found in ovsdb" 2019-12-02 18:59:05 +00:00
Rodolfo Alonso Hernandez
397eb2a2fe Serialize subnet creating depending on the network ID
Add a new DB table "network_subnet_lock". The primary key will be the
network_id. When a subnet is created, inside the write context during
the "subnet" object creation, a register in the mentioned table is
created or updated. This will enforce the serialization of the "subnet"
registers belonging to the same network, due to the write lock in the
DB.

This will solve the problem of attending several "subnet" creation
requests, described in the related bug. If several subnets with the
same CIDR are processed in parallel, the implemented logic won't reject
them because any of them will not contain the information of each other.

This DB lock will also work in case of distributed servers because the
lock is not enforced in the server logic but in the DB backend.

Change-Id: Iecbb096e0b7e080a3e0299ea340f8b03e87ddfd2
Closes-Bug: #1852777
2019-12-02 17:19:03 +00:00
Zuul
b4ccac6773 Merge "[OVN] Add DB tables for OVN backend" 2019-12-02 16:38:54 +00:00
Slawek Kaplonski
b01e0c2aa9 [OVS FW] Clean port rules if port not found in ovsdb
During e.g. migration or shelve of VM it may happend that
port update event will be send to the ovs agent and in the almost
the same time, port will be removed from br-int.
In such case during update_port_filter method openvswitch firewall
driver will not find port in br-int, and it will do nothing with it.
That will lead to leftover rules for this port in br-int.

So this patch adds calling remove_port_filter() method if port was
not found in br-int. Just to be sure that there is no any leftovers
from the port in br-int anymore.

Change-Id: I06036ce5fe15d91aa440dc340a70dd27ae078c53
Closes-Bug: #1850557
2019-12-02 12:22:50 +01:00
Zuul
52890ebb77 Merge "Add more condition to check sg member exist" 2019-12-02 10:51:22 +00:00
Adrian Chiris
a8b5afa41e Add upgrade check for NIC Switch agent
This commit adds an upgrade check for NIC switch agent
alerting operators to ensure relevant hosts have kernel >= 3.13.

This check is introduced due to recent cleanup in NIC switch
agent code.

As of U release, Supported CentOS major was updated to 8 so it
is not expected that any deployment will have hosts running
kernel < 3.13. This check is added as an extra precaution.

This check should be removed in 1-2 cycles. A TODO was added to
reflect that.

Change-Id: I34003b3c2f0ac23185d036c9e47dc1c8662d6ce7
Related-bug: #1841067
2019-12-01 11:58:16 +02:00
Rodolfo Alonso Hernandez
4b3baeb15a Do not inherit from built-in "dict"
This is not recommended because some type methods are implemented not
in Python but in C [1][2] and should not be overridden. Subclassing
the built-in types directly, will yield non-obvious errors that are
hard to debug, and identify at first glance [3].

[1] http://www.kr41.net/2016/03-23-dont_inherit_python_builtin_dict_type.html
[2] https://treyhunner.com/2019/04/why-you-shouldnt-inherit-from-list-and-dict-in-python/
[3] https://medium.com/bynder-tech/using-collections-in-python-36129737b5a1

Closes-Bug: #1849980

Change-Id: I08c712ff1b093370cda2ce66b93e2a0709094fe1
2019-11-29 18:12:07 +00:00
Slawek Kaplonski
151dab39dc Remove grenade-py3 job from Neutron CI
We have run neutron-grenade-multinode in both check and gate queues
already. This job is almost the same as grenade-py3. The only difference
from Neutron pov is that neutron-grenade-multinode is multinode job and
grenade-py3 is single node job.
So we agreed on last Neutron CI meeting [1], that there is no need to
run both of those jobs.

As grenade-py3 is part of integrated-gate-networking template together
with tempest-integrated-networking job this patch removes also
integrated-gate-networking template from neutron's zuul config and adds
only tempest-integrated-networking to both check and gate queues.

[1] http://eavesdrop.openstack.org/meetings/neutron_ci/2019/neutron_ci.2019-11-19-16.00.log.html#l-179

Change-Id: Ic2e7ca04929dc89a2e408465eb34ef3feec2e328
2019-11-29 15:55:17 +01:00
Rico Lin
22d2bb94f7 Revert "Revise log when create port failed"
After this patch, during delete subnets with fixed ip, there are high
chances a subnet can't be found(because it just deleted) while
deleting another subnet and Raise BadRequest instead of Notfound.
The behavior change led to heat gate failure, since Heat expect to
see NotFound exception instead of BadRequest. 
Also the error message is really confusing while delete subnets
(you can check out more detail in
https://storyboard.openstack.org/#!/story/2006962)

hence, I think to revert this patch will make sense to maintain the
behavior before a proper solution comes out. 

Story: 2006962
Task: 37652
This reverts commit cae66a4d8d13deda74db806d733c2ac39ea2e849.

Change-Id: I9fc200de376668b662cd8c2bfba01483dc9bb677
2019-11-29 14:27:10 +00:00
Rodolfo Alonso Hernandez
f26b3620db [OVN] Add DB tables for OVN backend
Added "ovn_revision_numbers" and "ovn_hash_ring" DB tables and models.

Change-Id: I07237946af698816a4229efec16de8f8f8addee9
Partially-Implements: blueprint neutron-ovn-merge
2019-11-29 11:57:31 +00:00
Yang Li
5cb0ff418a Add more condition to check sg member exist
Only check sg object is not enough, we should also
check sg'ports is {} or not. Otherwise the old conjunction
will still exist.

Change-Id: I10588e73a9da7fdd43677f9247c176811dd68c62
Closes-Bug: #1854131
2019-11-29 11:13:20 +08:00
Zuul
8375f4cc1c Merge "[OVN] Disable ovn jobs temporary" 2019-11-29 01:55:19 +00:00
Zuul
c83aa9025c Merge "[OVN] Sync requirements with OVN" 2019-11-29 01:26:32 +00:00
Rodolfo Alonso Hernandez
1e5ce97bb9 [OVN] Disable ovn jobs temporary
During the OVN repository migration, the zuul jobs relaset to ovn
cannot be executed due to the DB redefinition. Once the OVN plugin is
totally migrated to Neutron, this job can be integrated as a in-tree
backend tempest job.

Change-Id: If1f0b0453d90eed27051fd516b95da32d4aa2afe
Partially-Implements: blueprint neutron-ovn-merge
2019-11-28 21:42:22 +01:00
Zuul
618d1b784b Merge "Rehome networking-ovn CI jobs to neutron repository" 2019-11-28 19:42:18 +00:00
Zuul
1728d60ffd Merge "Add upgrade check for networking-ovn db version" 2019-11-28 16:48:25 +00:00
Jens Harbott
d296188c2e Set ignore_basepython_conflict to true in tox.ini
Since dropping py2 support and adding a basepython setting to the global
tox testenv, tox complains about a basepython mismatch when the
installed python3 is python3.6 and the py37 environment is executed (or
vice versa). Setting ignore_basepython_conflict = True will avoid this
message and enforce the correct python version to be used.

Needs a bump in the minimum tox version supported.

See also [0].

[0] https://review.opendev.org/582392

Change-Id: I3d9beeaed104dcf12cfd97590b44c4003754b4e4
2019-11-28 12:47:19 +00:00
Slawek Kaplonski
9fa881f870 Add upgrade check for networking-ovn db version
When ovn code will be in neutron tree already, during the upgrade
we need to be sure that networking-ovn wasn't installed previously or
if it was installed, it's db version is the latest one.
Otherwise neutron's alembic migration script will not work properly as
it will check that ovn tables exists so it will skip creating them, even
if those tables aren't in proper shape (from latest networking-ovn
head).

Change-Id: I8c7b8df934c8b53ea5bc02dc8134db838f5e105a
Related-Blueprint: neutron-ovn-merge
2019-11-27 20:35:49 +01:00
Maciej Józefczyk
9267966628 [OVN] Sync requirements with OVN
This patch sync requirements from OVN requirements.

Change-Id: I988e506e9593e7c0856ef9e2955ec04b37f2ba3b
Related-Blueprint: neutron-ovn-merge
2019-11-27 12:23:56 +00:00
Slawek Kaplonski
9c6d3e073e Rehome networking-ovn CI jobs to neutron repository
As networking-ovn driver is going to be in neutron tree now, lets move
also networking-ovn jobs definitions to the neutron repo.

Change-Id: Ie1c66c773ee2d217b25bd915189e56254744a3d4
Related-Blueprint: neutron-ovn-merge
2019-11-27 10:38:04 +01:00
Zuul
f0db43df55 Merge "Dont schedule Network, respecting network_auto_schedule config" 2019-11-27 06:05:14 +00:00
Boden R
b4972e246d use objects common_types from neutron-lib
This patch consumes the neutron-lib OVO common_types module by
removing it from neutron and switching imports over to use neutron-lib
instead.

NeutronLibImpact

Change-Id: Ic9819a27e3e72038b819ab2fe845c26fc63b26d5
2019-11-26 15:34:18 +01:00