With a large number of instances and/or security group rules,
conntrack updates when ports are removed or rules are changed
can take a long time to process. By enqueuing these to a set
or worker threads, the agent can continue with other work while
they are processed in the background.
This is a change in behavior in the agent since it could
program a new set of security group rules before all existing
conntrack entries are deleted, but since the iptables or OVSfw
NAT rules will have been removed, it should not pose a
security issue.
Change-Id: Ibf858c7fdf7a822a30e4a0c4722d70fd272741b6
Closes-bug: #1745468
As neutron now allows L3 Flavors, but when other backends use L3
flavors, the floating delete only commits db transaction therefore
removing fip from neutron db but backend still have that fip as active
because there's no notification for the subscribed precommit_delete
event.
Co-Authored-By: Isaku Yamahata <isaku.yamahata@intel.com>
Change-Id: I87dd124cc5bf17365ca197e4b35ffaa7825ca17c
Closes-Bug: #1742802
This reverts commit 7f7d65ea4c58b665c2b54fa9f2e8ec1f81668b48.
Now bug 1737892 is closed with patch
I3cf5c57c7f232deaa190ab6b0129e398fdabe592
so it should be safe to remove mark of this test as unstable.
Change-Id: Ie3f16dc113455062a9f404de045412d4097b3f8b
Related-Bug: #1737892
when a mechinism driver calls context.continue_binding to
continue binding, it will be called again because
_check_driver_to_bind compares driver name with driver.
Closes-Bug: #1745572
Change-Id: I62b32c9b9d01dd929fe8cd3634c78dc0cbe325b6
Zuul no longer requires the project-name for in-repo configuration.
Omitting it makes forking or renaming projects easier.
Change-Id: Ia190b510f06d1140f95589543596d9949cfc4f88
It was found that adding ebtables rules to the filter table
FORWARD chain could be vulnerable to a DoS attack. Moving
to the nat table PREROUTING chain should mitigate this as
it is consulted prior to allowing the frame in.
In order to make this work with upgrades, had to make the code
detect and remove any old rules that might still exist in
the filter table. That can be removed after a cycle.
Added some unit tests in addition to the existing functional
tests.
Change-Id: I87852b21db4404c58c83789cc267812030ac7d5f
Closes-bug: #1732294
SG with hybrid-iptables driver uses per port conntrack zones.
FWaaS port security uses per network conntrack zones based on
local vlans assigned by ovs l2 agent.
In case both SG iptables-hybrid driver and FWaaS port security is enabled,
there is a posibility of iptables-hybrid and OVS based FWaaS driver
allocating overlapping zone and creating security holes.
This patch changes the zone allocation range for iptables and
hybrid_iptables driver to 4097 - 65535. While OVS based
port security driver can use zones based on local vlan range 1 - 4096
Closes-Bug: #1745642
Change-Id: I4d51637ed1de8fe85b4982a03410d4a3f637ea3f
This reverts commit febeaf5d4016157d0cca41023e64b91e550453e8.
This patch broke postgres tempest jobs, as well as introduced potential
race conditions in database layer because of mixed usage of old and new
engine facades.
Related-Bug: #1744829
Change-Id: Ic142ae7faf4e5f10cbdf761d7e6f3d442e94a3eb
With Ubuntu python package 2.7.12-1ubuntu0~16.04.3 and
evetlet < 0.22.0 there is some issue with eventlet monkey_patching.
This commit adds workaround solution ported from eventlet patch
b756447bab
Change-Id: Ia381c98116765903bf2652acd175251233df4e54
Closes-Bug: #1745013
Upon failure to create an OVSDB connection using IDL with the ovsdbapp
lib, a second attempt will be executed using legacy ovsdapp connection
api. This patch removes the legacy connection attempt.
Change-Id: I07477ec92ba5e95794f91bf67cab97e9dca89dde
Signed-off-by: Tim Rozet <trozet@redhat.com>
When there are many calls to enable security groups on ports there
can be sometimes race condition between refresh recource_cache
with data get by "pull" call to neutron server and data received
with "push" rpc message from neutron server.
In such case when "push" message comes with information about
updated port (with enabled port_security), in local cache this port
is already updated so local AFTER_UPDATE call is not called for
such port and its rules in firewall are not updated.
It happend quite often in fullstack security groups test because
there are 4 ports created in this test and all 4 are updated to
apply SG to it one by one.
And here's what happen then in details:
1. port 1 was updated in neutron-server so it sends push notification
to L2 agent to update security groups,
2. port 1 info was saved in resource cache on L2 agent's side and agent
started to configure security groups for this port,
3. as one of steps L2 agent called
SecurityGroupServerAPIShim._select_ips_for_remote_group() method;
In that method RemoteResourceCache.get_resources() is called and this
method asks neutron-server for details about ports from given
security_group,
4. in the meantime neutron-server got port update call for second port
(with same security group) so it sends to L2 agent informations about 2
ports (as a reply to request sent from L2 agent in step 3),
5. resource cache updates informations about two ports in local cache,
returns its data to
SecurityGroupServerAPIShim._select_ips_for_remote_group() and all
looks fine,
6. but now L2 agent receives push notification with info that port 2 is
updated (changed security groups), so it checks info about this port
in local cache,
7. in local cache info about port 2 is already WITH updated security
group so RemoteResourceCache doesn't trigger local notification about
port AFTER UPDATE and L2 agent doesn't know that security groups for this
port should be changed
This patch fixes it by changing way how items are updated in
the resource_cache.
For now it is done with record_resource_update() method instead of
writing new values directly to resource_cache._type_cache dict.
Due to that if resource will be updated during "pull" call to neutron
server, local AFTER_UPDATE will still be triggered for such resource.
Change-Id: I5a62cc5731c5ba571506a3aa26303a1b0290d37b
Closes-Bug: #1742401
In fullstack security groups test number of used fake hosts depends
on scenario.
For example for ovs-hybrid scenario only one such host is enough
because all ports are placed on first host always.
So there is no need to spawn another L2 and DHCP agents for such test.
Change-Id: Icfcf54bf9d7cc954aaa20b22d1f1746afc61179c
Previously, running neutron_ovs_cleanup on an installation with
5000 ports would time out even after setting the timeout to 3
hours. The code would do a bridge and port "by name" lookup for
every port due to being based off the ovs-vsctl implementation
where names are how everything is looked up. With this change,
the same test runs in ~1.5 mins.
This implementation adds a new OVSDB command that just looks up
the bridge, iterates over its ports, and deletes the ones that
should be deleted in a single transaction per bridge.
Change-Id: I23c81813654596d61d8d930e0bfb0f016f91bc46
Filtering of port OVO based on ids of security groups which
are used by ports is now available.
Closes-Bug: 1744447
Change-Id: Ie5a3effe668db119d40728be5357f0851bdcebbe
By adding a log-tag line to the haproxy config file that contains
the network or router id, we will be able to differentiate which
proxy is logging what. This should help with debugging.
Change-Id: I5bb57b7682c00645e20cce69847dbb3b72165aa8
Partial-bug: #1744359
When the test case fails, it's useful to see which warnings triggered
the failure.
Change-Id: Ia4150a06989ba253ffed0c0995e1023ca1ed42a0
Related-Bug: #1744394
DSCP marking rules are done by L2 agent as OpenFlow or iptables
rule.
This is new fullstack test to check if L2 agent don't left such
orphaned rules after delete of instance.
Change-Id: If45b20ea1d673eabd7713e461aa26a3618476967
We still run API tests agains logging API and the job has OVS enabled.
As all linuxbridge flavors contain same string, this patch changes when
logging service plugin is configured which is always but in linuxbridge
scenario job.
Change-Id: I66234f45ee74c070119d84830790df296ba8d5f7
Related-bug: #1743463