19911 Commits

Author SHA1 Message Date
Brian Haley
65a81623fc Process conntrack updates in worker threads
With a large number of instances and/or security group rules,
conntrack updates when ports are removed or rules are changed
can take a long time to process.  By enqueuing these to a set
or worker threads, the agent can continue with other work while
they are processed in the background.

This is a change in behavior in the agent since it could
program a new set of security group rules before all existing
conntrack entries are deleted, but since the iptables or OVSfw
NAT rules will have been removed, it should not pose a
security issue.

Change-Id: Ibf858c7fdf7a822a30e4a0c4722d70fd272741b6
Closes-bug: #1745468
2018-02-06 12:25:59 -05:00
Zuul
c7e144e9b4 Merge "Revert "[Fullstack] Mark test_bw_limit_qos_port_removed as unstable"" 2018-02-03 06:24:59 +00:00
Zuul
66ad2c4d87 Merge "fix same mechanism driver called twice bug" 2018-02-02 21:42:59 +00:00
Zuul
b354321399 Merge "Move Linuxbridge ARP spoofing to nat table PREROUTING chain" 2018-02-02 16:12:42 +00:00
Zuul
a120b4b3fc Merge "Updated from global requirements" 2018-02-02 12:48:08 +00:00
Zuul
ac24f25e78 Merge "Add notification for floatingip update/delete" 2018-02-02 03:41:22 +00:00
Zuul
52d78be6fb Merge "Zuul: Remove project name" 2018-02-01 21:26:56 +00:00
Zuul
00332d9b7c Merge "This patch changes the CT zone allocation range" 2018-02-01 21:19:07 +00:00
Manjeet Singh Bhatia
257de15310 Add notification for floatingip update/delete
As neutron now allows L3 Flavors, but when other backends use L3
flavors, the floating delete only commits db transaction therefore
removing fip from neutron db but backend still have that fip as active
because there's no notification for the subscribed precommit_delete
event.

Co-Authored-By: Isaku Yamahata <isaku.yamahata@intel.com>
Change-Id: I87dd124cc5bf17365ca197e4b35ffaa7825ca17c
Closes-Bug: #1742802
2018-02-01 11:55:49 -08:00
Sławek Kapłoński
68dcc4d3c8 Revert "[Fullstack] Mark test_bw_limit_qos_port_removed as unstable"
This reverts commit 7f7d65ea4c58b665c2b54fa9f2e8ec1f81668b48.

Now bug 1737892 is closed with patch
I3cf5c57c7f232deaa190ab6b0129e398fdabe592
so it should be safe to remove mark of this test as unstable.

Change-Id: Ie3f16dc113455062a9f404de045412d4097b3f8b
Related-Bug: #1737892
2018-02-01 14:34:37 +01:00
Bo Chi
892c1ec690 fix same mechanism driver called twice bug
when a mechinism driver calls context.continue_binding to
continue binding, it will be called again because
_check_driver_to_bind compares driver name with driver.

Closes-Bug: #1745572
Change-Id: I62b32c9b9d01dd929fe8cd3634c78dc0cbe325b6
2018-02-01 18:51:39 +08:00
OpenStack Proposal Bot
b8b043ada5 Updated from global requirements
Change-Id: I40151d3a87f40249b4393937631531bcec6d14eb
2018-02-01 07:17:59 +00:00
James E. Blair
163853a04c Zuul: Remove project name
Zuul no longer requires the project-name for in-repo configuration.
Omitting it makes forking or renaming projects easier.

Change-Id: Ia190b510f06d1140f95589543596d9949cfc4f88
2018-01-31 22:54:45 -08:00
Zuul
bc150cdbf8 Merge "Revert "Integration of (Distributed) Port Binding OVO"" 2018-02-01 06:41:48 +00:00
Brian Haley
08108c4199 Move Linuxbridge ARP spoofing to nat table PREROUTING chain
It was found that adding ebtables rules to the filter table
FORWARD chain could be vulnerable to a DoS attack.  Moving
to the nat table PREROUTING chain should mitigate this as
it is consulted prior to allowing the frame in.

In order to make this work with upgrades, had to make the code
detect and remove any old rules that might still exist in
the filter table.  That can be removed after a cycle.

Added some unit tests in addition to the existing functional
tests.

Change-Id: I87852b21db4404c58c83789cc267812030ac7d5f
Closes-bug: #1732294
2018-01-31 11:31:14 -05:00
Chandan Dutta Chowdhury
9a620f6ea5 This patch changes the CT zone allocation range
SG with hybrid-iptables driver uses per port conntrack zones.
FWaaS port security uses per network conntrack zones based on
local vlans assigned by ovs l2 agent.

In case both SG iptables-hybrid driver and FWaaS port security is enabled,
there is a posibility of iptables-hybrid  and OVS based FWaaS driver
allocating overlapping zone and creating security holes.

This patch changes the zone allocation range for iptables and
hybrid_iptables driver to  4097 - 65535. While OVS based
port security driver can use zones based on local vlan range 1 - 4096

Closes-Bug: #1745642
Change-Id: I4d51637ed1de8fe85b4982a03410d4a3f637ea3f
2018-01-31 15:11:45 +00:00
Zuul
f5e7cff702 Merge "ovsfw: Don't create rules if updated port doesn't exist" 2018-01-31 08:27:57 +00:00
Ihar Hrachyshka
906eda44d2 Revert "Integration of (Distributed) Port Binding OVO"
This reverts commit febeaf5d4016157d0cca41023e64b91e550453e8.

This patch broke postgres tempest jobs, as well as introduced potential
race conditions in database layer because of mixed usage of old and new
engine facades.

Related-Bug: #1744829
Change-Id: Ic142ae7faf4e5f10cbdf761d7e6f3d442e94a3eb
2018-01-27 18:19:20 -06:00
Zuul
03b1cca794 Merge "Log warnings when test fails because of them" 2018-01-27 17:05:20 +00:00
Zuul
a34b61f0c2 Merge "Fix race condition with enabling SG on many ports at once" 2018-01-27 10:36:37 +00:00
Zuul
6a18581467 Merge "Fix Port OVO filtering based on security groups" 2018-01-27 10:26:59 +00:00
Zuul
be87bcf60c Merge "Fix eventlet imports issue" 2018-01-27 01:36:50 +00:00
Zuul
c13511ae50 Merge "Minor fix to remove legacy OVSDB IDL connection" 2018-01-26 09:40:30 +00:00
Sławek Kapłoński
04efb3bca3 Fix eventlet imports issue
With Ubuntu python package 2.7.12-1ubuntu0~16.04.3 and
evetlet < 0.22.0 there is some issue with eventlet monkey_patching.
This commit adds workaround solution ported from eventlet patch
b756447bab

Change-Id: Ia381c98116765903bf2652acd175251233df4e54
Closes-Bug: #1745013
2018-01-25 16:52:06 +01:00
Tim Rozet
e35a0db136 Minor fix to remove legacy OVSDB IDL connection
Upon failure to create an OVSDB connection using IDL with the ovsdbapp
lib, a second attempt will be executed using legacy ovsdapp connection
api.  This patch removes the legacy connection attempt.

Change-Id: I07477ec92ba5e95794f91bf67cab97e9dca89dde
Signed-off-by: Tim Rozet <trozet@redhat.com>
2018-01-24 11:43:17 -05:00
Zuul
7e3d6a18fb Merge "[Fullstack] SG test use not too many fake hosts" 2018-01-23 20:45:35 +00:00
Zuul
0ad3f155d6 Merge "More efficiently clean up OVS ports" 2018-01-23 17:30:27 +00:00
Sławek Kapłoński
725df3e038 Fix race condition with enabling SG on many ports at once
When there are many calls to enable security groups on ports there
can be sometimes race condition between refresh recource_cache
with data get by "pull" call to neutron server and data received
with "push" rpc message from neutron server.
In such case when "push" message comes with information about
updated port (with enabled port_security), in local cache this port
is already updated so local AFTER_UPDATE call is not called for
such port and its rules in firewall are not updated.

It happend quite often in fullstack security groups test because
there are 4 ports created in this test and all 4 are updated to
apply SG to it one by one.
And here's what happen then in details:
1. port 1 was updated in neutron-server so it sends push notification
   to L2 agent to update security groups,
2. port 1 info was saved in resource cache on L2 agent's side and agent
   started to configure security groups for this port,
3. as one of steps L2 agent called
   SecurityGroupServerAPIShim._select_ips_for_remote_group() method;
   In that method RemoteResourceCache.get_resources() is called and this
   method asks neutron-server for details about ports from given
   security_group,
4. in the meantime neutron-server got port update call for second port
   (with same security group) so it sends to L2 agent informations about 2
   ports (as a reply to request sent from L2 agent in step 3),
5. resource cache updates informations about two ports in local cache,
   returns its data to
   SecurityGroupServerAPIShim._select_ips_for_remote_group() and all
   looks fine,
6. but now L2 agent receives push notification with info that port 2 is
   updated (changed security groups), so it checks info about this port
   in local cache,
7. in local cache info about port 2 is already WITH updated security
   group so RemoteResourceCache doesn't trigger local notification about
   port AFTER UPDATE and L2 agent doesn't know that security groups for this
   port should be changed

This patch fixes it by changing way how items are updated in
the resource_cache.
For now it is done with record_resource_update() method instead of
writing new values directly to resource_cache._type_cache dict.
Due to that if resource will be updated during "pull" call to neutron
server, local AFTER_UPDATE will still be triggered for such resource.

Change-Id: I5a62cc5731c5ba571506a3aa26303a1b0290d37b
Closes-Bug: #1742401
2018-01-23 12:19:43 +01:00
Zuul
d32ad6ee52 Merge "Log radvd config contents on reload when debug is on" 2018-01-23 03:58:56 +00:00
Zuul
8eb7d86f48 Merge "[Fullstack] Clean DSCP mark rule when port removed" 2018-01-23 02:32:55 +00:00
Zuul
1596fa0f85 Merge "Add log-tag to haproxy config file" 2018-01-22 22:10:08 +00:00
Sławek Kapłoński
2c1178c1ce [Fullstack] SG test use not too many fake hosts
In fullstack security groups test number of used fake hosts depends
on scenario.
For example for ovs-hybrid scenario only one such host is enough
because all ports are placed on first host always.
So there is no need to spawn another L2 and DHCP agents for such test.

Change-Id: Icfcf54bf9d7cc954aaa20b22d1f1746afc61179c
2018-01-22 21:11:32 +01:00
Terry Wilson
fef374131b More efficiently clean up OVS ports
Previously, running neutron_ovs_cleanup on an installation with
5000 ports would time out even after setting the timeout to 3
hours. The code would do a bridge and port "by name" lookup for
every port due to being based off the ovs-vsctl implementation
where names are how everything is looked up. With this change,
the same test runs in ~1.5 mins.

This implementation adds a new OVSDB command that just looks up
the bridge, iterates over its ports, and deletes the ones that
should be deleted in a single transaction per bridge.

Change-Id: I23c81813654596d61d8d930e0bfb0f016f91bc46
2018-01-22 14:07:42 -06:00
Sławek Kapłoński
e7c0ec17df Fix Port OVO filtering based on security groups
Filtering of port OVO based on ids of security groups which
are used by ports is now available.

Closes-Bug: 1744447

Change-Id: Ie5a3effe668db119d40728be5357f0851bdcebbe
2018-01-22 15:43:46 +01:00
Brian Haley
703ff85b82 Add log-tag to haproxy config file
By adding a log-tag line to the haproxy config file that contains
the network or router id, we will be able to differentiate which
proxy is logging what.  This should help with debugging.

Change-Id: I5bb57b7682c00645e20cce69847dbb3b72165aa8
Partial-bug: #1744359
2018-01-22 09:07:03 -05:00
gaofei
c46eb76394 Modify link address from http to https
Change-Id: Ieeeb5e1a199eb57c6d7b97d0f47d0e205197d82d
2018-01-22 17:47:45 +08:00
Zuul
fad50d424d Merge "l3_ha: only pass host into update_port when updating router port bindings" 2018-01-21 09:47:36 +00:00
Ihar Hrachyshka
b94b97617d Log warnings when test fails because of them
When the test case fails, it's useful to see which warnings triggered
the failure.

Change-Id: Ia4150a06989ba253ffed0c0995e1023ca1ed42a0
Related-Bug: #1744394
2018-01-19 14:19:39 -08:00
Sławek Kapłoński
8ee41a081a [Fullstack] Clean DSCP mark rule when port removed
DSCP marking rules are done by L2 agent as OpenFlow or iptables
rule.
This is new fullstack test to check if L2 agent don't left such
orphaned rules after delete of instance.

Change-Id: If45b20ea1d673eabd7713e461aa26a3618476967
2018-01-19 21:08:50 +01:00
Zuul
a12630ee85 Merge "Introduce rfe-confirmed and rfe-triaged tags" 2018-01-19 19:37:59 +00:00
Zuul
93f7c7707b Merge "Use constant 'IS_DEFAULT' from neutron-lib" 2018-01-19 14:37:34 +00:00
Zuul
9b46764d40 Merge "Fix _port_arg for security rules with icmp/ipv6-icmp aliases" 2018-01-19 14:37:19 +00:00
Zuul
b2da2ff24c Merge "Fixes minor typo in neutron" 2018-01-19 04:26:02 +00:00
Zuul
88aebd4a29 Merge "Update the gerrit dashboard" 2018-01-18 23:14:21 +00:00
Zuul
68ca126c9a Merge "Add retry decorator update_segment_host_mapping()" 2018-01-18 21:18:58 +00:00
Zuul
d10ef71e10 Merge "log-api: Don't enable logging for linuxbridge" 2018-01-18 19:43:31 +00:00
Zuul
edf85b906d Merge "Integration of (Distributed) Port Binding OVO" 2018-01-18 19:21:14 +00:00
Zuul
ff24fc0727 Merge "Fix ingress bw limit for OVS DPDK ports" 2018-01-18 18:07:39 +00:00
Zuul
fbea3e34f7 Merge "Use Router OVO in metering_db" 2018-01-18 16:56:06 +00:00
Jakub Libosvar
4db37df648 log-api: Don't enable logging for linuxbridge
We still run API tests agains logging API and the job has OVS enabled.
As all linuxbridge flavors contain same string, this patch changes when
logging service plugin is configured which is always but in linuxbridge
scenario job.

Change-Id: I66234f45ee74c070119d84830790df296ba8d5f7
Related-bug: #1743463
2018-01-18 13:52:09 +00:00