15279 Commits

Author SHA1 Message Date
Zuul
70d7a1bc86 Merge "Blacklist sphinx 2.1.0 (autodoc bug)" 2019-07-31 13:49:37 +00:00
pengyuesheng
e08a81100f Blacklist sphinx 2.1.0 (autodoc bug)
See https://github.com/sphinx-doc/sphinx/issues/6440 for upstream details

Change-Id: Ifdfe8dc074f7e3bb4ea0df4cd78e95842fc322ab
2019-07-30 16:53:08 +08:00
Zuul
4563f0d0db Merge "Disallow in-place update of Port MAC address" 2019-07-29 18:33:57 +00:00
Zuul
78b3ed5458 Merge "Fix intermittent error in test_decrypt_dict_invalid_key" 2019-07-29 04:26:41 +00:00
Zuul
3d4d8a7e22 Merge "Never pass 'value_specs' to Neutron" 2019-07-29 04:22:58 +00:00
Zuul
284ebac1ba Merge "Add periodic job template" 2019-07-26 16:59:17 +00:00
Zuul
d992b465ff Merge "Show an engine as down if service record is not updated twice" 2019-07-26 16:26:11 +00:00
Zane Bitter
d4b6f37ab5 Never pass 'value_specs' to Neutron
Due to a regression caused by d37cc18b1b7a531edbcf9658600fc467fbe24e2d
that was not caught in the unit tests because of a typo in
6c7e4e4b2f446f7b7678bfc71e5245d10dbf9fcf, we were passing a
'value_specs' field to Neutron when updating an OS::Neutron::Firewall,
OS::Neutron::Net, OS::Neutron::Port, or OS::Neutron::Subnet to remove
existing value_specs. This fixes that to ensure that the 'value_specs'
key is always removed before updating the resource.

Change-Id: I176ab2237c70a5ee78105007e60f841212fef439
Task: 34765
2019-07-26 10:43:31 -04:00
Zuul
98f08fe079 Merge "Don't send existing attributes in value_specs for neutron update" 2019-07-26 05:58:27 +00:00
Zuul
54005a2a9f Merge "Add dedicated auth endpoint config for servers" 2019-07-25 21:52:54 +00:00
Zuul
6ad9d3f43a Merge "Update install docs for Keystone v3" 2019-07-24 12:56:52 +00:00
Rabi Mishra
fd23308f6e Show an engine as down if service record is not updated twice
We use the same periodic_interval to update the service record
and compare it with current time when doing 'service list'. So, It's
possible there will be a small window where 'service list' would show
the engine as down. Tools that use service list for monitoring would
wrongly assume the service as down. Let's change service list to
report the service as down if it's not updated in 2*periodic_interval.

Change-Id: I0f6a30e06bb214bb673930b31a2db946600926b0
Task: 35946
2019-07-24 03:16:57 +00:00
Andreas Jaeger
208cdfea39 Update api-ref location
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.

Note that redirects will be set up as well but let's point now to the
new location.

For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html

Change-Id: I69a41aa850c87e92bf5c3e7fed32e0e961a3c6d3
2019-07-22 18:55:34 +02:00
Rabi Mishra
5a403d7090 Merge parameters and templates when resetting stack status
We keep the new template in the prev_raw_template_id. When setting
the stacks to FAILED we should also merge the templates for both
existing and backup stack.

Change-Id: Ic67a4833672d1c562980ee19fd8071f84dd9500a
Task: 35842
2019-07-17 07:30:03 +05:30
Georgina Shippey
5ba3b60874 Add dedicated auth endpoint config for servers
Added a new config option to specify the keystone authentication
endpoint to pass into cloud-init data.

Heat code currently has several different methods of retrieving the
keystone endpoint to embed into cloud-init data for created
servers. This data is currently read from several different parts
of the heat config file rather than the service catalog which results
in URLs being passed which are appropriate for the heat service rather
than the server. In particular there can be misconfiguration of
servers due to deployments which separate the internal and
external API endpoints.

This patch introduces a new config variable
server_keystone_endpoint_type which if set
reads the keystone endpoint directly from the service catalog,
if it is unset the original behavior is unchanged.

story: 2004808
task:  28967
story: 2004524
Change-Id: I5d8fc5977014b196c34f4a59a30a7525bc778359
2019-07-12 13:50:38 +00:00
Zuul
e42ef32b41 Merge "Unit tests: Fix wrong assert function name in port update" 2019-07-11 08:54:04 +00:00
Zuul
5ee25d3c45 Merge "Allow creating trusts with allow_redelegation" 2019-07-03 16:19:19 +00:00
Zuul
e8d91e8f16 Merge "Don't resolve properties for OS::Heat::None resource" 2019-07-03 05:14:05 +00:00
Rabi Mishra
7066bccc53 Don't resolve properties for OS::Heat::None resource
We don't store resource_properties_data for OS::Heat::None.
There is no point trying to resolve the properties for old resource
which would fail at times.

Change-Id: I4cf72ae6d11ffbedc20adedfe7b6d2a1d47e23ee
Task: 35661
Closes-Bug: #1834881
2019-07-02 23:39:34 +05:30
Zuul
4c8d77bb45 Merge "Log args during list_concat" 2019-07-02 15:43:54 +00:00
Elod Illes
8e784ff9c4 Add periodic job template
Periodic stable jobs don't run on stein, only up till rocky as
periodic-stable-jobs template is missing from master and stein
branches.

Change-Id: Ic7fa6a2bef3db1f6f2548d1c792e0cef29787586
2019-07-01 16:18:19 +02:00
Zuul
8aea1ca7b2 Merge "Add doc for multi-clouds support" 2019-06-28 15:16:02 +00:00
Zane Bitter
629d1042c8 Update install docs for Keystone v3
Since the Keystone v2 API was removed in Queens, the Keystone setup
documentation recommends using the same port (5000) for admin access as
for regular internal/public access.

Change-Id: Ic49acc5b57122fded11b5d17f8b51bf54dd29674
Task: 33508
2019-06-27 14:20:22 -04:00
Elod Illes
c49e0cbc3e Update tools/README.rst with bindep info
During local bindep.txt add [1] the old test-requires-* files were
removed, but the README was not updated.

[1] If9befe4115c64c2fda52321002ba5fe1124eaf7c

Change-Id: I43c93df9c0f7bc4f1c7a52907d58a82bf1393512
2019-06-27 18:25:55 +02:00
Zane Bitter
dd9bc3cd29 Unit tests: Fix wrong assert function name in port update
Due to a typo in an assert function name (assset_called_once_with()
instead of assert_called_once_with()), we weren't actually checking the
arguments passed to Neutron in UpdatePortTest. Fix the function name and
modify the test so it actually works.

Among the fixes required:

- Actually return data from create_port
- Call Resource.create() instead of handle_create(), to ensure that the
  resource is stored in the DB (and hence that it gets a name)
- Mock show_port(), which is called from check_create_complete()
- Handle later changes to the format of data passed (e.g. an empty
  allowed_address_pairs should be passed as an empty list instead of None)
- Actually test the case where the port name is initially specified, and
  then removed on update
- Fix the format of arguments to create_port and update_port

This patch contains a workaround for a regression introduced by
d37cc18b1b7a531edbcf9658600fc467fbe24e2d that causes a 'value_specs' key
to be passed to the update_port call. This will be fixed in a subsequent
patch.

Change-Id: I37948721a0a653a18049aae6502fafb88030479c
Co-Authored-By: zhufl <zhu.fanglei@zte.com.cn>
Task: 34764
2019-06-26 20:07:09 -04:00
Elod Illes
d116b21696 Add local bindep.txt
As it was announced [1] global bindep-fallback.txt was removed and now
projects need to have a local bindep.txt to be able to install binary
dependencies for testing.

In test jobs the script tools/test-setup.sh is called which requires
mysql and postgres servers and clients to be installed.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007272.html

Change-Id: If9befe4115c64c2fda52321002ba5fe1124eaf7c
2019-06-26 17:48:15 +02:00
ricolin
fdb5e892bf Add doc for multi-clouds support
Add doc for multi-clouds support in template guide.

Also remove redundant credential information in multi-clouds
integration test.

Change-Id: I76c6427b7bbdac2af3b7f01aff1b0541e56b3653
Story: #2002126
Task: #19808
2019-06-25 07:37:27 +00:00
Zane Bitter
d805e6b128 Ignore false positive Bandit test
Improvements[1] to the B105 hardcoded_password_string[2] test in Bandit
result in it now catching a false positive. Add a # nosec comment to
skip Bandit testing of that line.

[1] https://github.com/PyCQA/bandit/issues/386
[2] https://bandit.readthedocs.io/en/latest/plugins/b105_hardcoded_password_string.html

Change-Id: I822526a7dbdd9be51edefaf6b24011fcce6e4121
2019-06-18 17:00:03 -04:00
Zane Bitter
28975c7cd6 Disallow in-place update of Port MAC address
While Neutron may technically allow updating the requested MAC address
of a port (for admin users only), in practice this only appears to work
when the port is not in use. Use Heat's replace-on-update flow, which is
designed to handle resources that are in use, to deal with changes to
the requested MAC.

Change-Id: I278584ecfe59a338d3135416527d9d3332808d2a
Depends-On: https://review.opendev.org/665692
Task: 31012
2019-06-17 14:41:26 +00:00
Colleen Murphy
28dd8117ba Update keystone_authtoken config reference
The auth_uri parameter of keystonemiddleware was renamed to
'www_authenticate_uri'[1], so update the documentation accordingly.

[1] https://review.opendev.org/508522

Change-Id: Ie3967064493fafb68df8e56f3d3dc097fbd19cc4
2019-06-12 15:08:39 -07:00
Oleksiy Petrenko
e377658586 Allow creating trusts with allow_redelegation
If you set up heat with trusts enabled, heat fails to create remote
stack since by default it creates trusts with turned off redelegation.

This commit adds a new option `allow_trusts_redelegation`
(False by default) which, when enabled together with
`reauthentication_auth_method` set to `trusts` will make Heat to create
trusts with allow_redelegation=True, both for trusts used for deferred
auth and for long creating stacks.

Change-Id: I73e73455139a87fb798fd8a4651c075a91be75fd
Story: #2005062
Task: 29606
Task: 17266
2019-06-12 12:18:58 +03:00
Zuul
26f9c092f3 Merge "Update Python 3 test runtimes for Train" 2019-06-05 15:24:42 +00:00
Zuul
7472974179 Merge "Return None for attributes of sd with no actions" 2019-05-31 11:28:02 +00:00
Zane Bitter
d580565abf Fix regression with SW deployments when region not configured
The region name eventually has to pass through an os-collect-config.conf
file, the format of which is unable to distinguish between the JSON null
(equivalent to None in Python) and the string "null".

This means that Story 2002781 caused a regression for users who didn't
have the region_name_for_services config option explicitly set in
heat.conf.

To avoid this, only specify the region when we know what it is.

Change-Id: I23493b1c477d082c478f87167de2c1859ba5ace7
Story: #2005797
Task: 33527
Task: 33528
2019-05-29 15:45:32 -04:00
Rabi Mishra
ee06110347 Return None for attributes of sd with no actions
If the 'actions' property of a deployment is [], we don't
create a software deployment in heat and resource_id of the
software deployment is None. However, there is a possibility
that we access the attributes of the sd in the template and
that results in TypeError as we try to make an rpc call to
show the software deployment for None.

Change-Id: Iefd3cdd20bb51c63e7267ae0628e0f15544c0427
Task: 33516
2019-05-29 22:24:54 +05:30
Zuul
c4076e12c8 Merge "Ignore Not Found when deleting Keystone role assignment" 2019-05-28 15:21:16 +00:00
Zane Bitter
d50ded7395 Fix intermittent error in test_decrypt_dict_invalid_key
Sometimes decryption with the wrong key works but produces garbage data.
This is annoying because it occasionally fails the gate at random.

Change-Id: I1563962aca8efa30773f03792f7cfd6b7774443d
Task: 33482
2019-05-23 12:03:58 -04:00
Zuul
6323b173e8 Merge "Fix allowed address pair validation" 2019-05-23 15:38:15 +00:00
Zuul
1616c09b13 Merge "Zun: fix an issue on command property" 2019-05-23 15:38:09 +00:00
Tom Stappaerts
5e93b3e4cf Fix allowed address pair validation
Neutron requires the allowed address pair ip address to be
either an ip or a cidr.
https://review.opendev.org/#/c/575265/ made heat verify for
cidr only.

Change-Id: I2cc2785cb32cf8d788af6262992b1b76107c8292
Story: 2005674
Task: 30985
2019-05-15 13:06:36 +02:00
gao.hanxiang
011fa22c42 Blacklist bandit 1.6.0 and cap Sphinx on Python2
There's a regression[0] in bandit 1.6.0 which causes bandit to stop
respecting excluded directories, and our tests throw a bunch of
violations. Blacklist this version, but allow newer versions as there is
already a pull request[1] to fix it, and I expect it will be included in
the next release.

Also fix the requirements job which was broken by
https://review.opendev.org/657890 adding a cap on Sphinx on Python 2.

[0] https://github.com/PyCQA/bandit/issues/488
[1] https://github.com/PyCQA/bandit/pull/489

Change-Id: Ieabcd4e8c5e5354125a63e89b9b60931c760858a
2019-05-14 16:50:24 -04:00
Zane Bitter
8c67437378 Ignore Not Found when deleting Keystone role assignment
If the project has already been deleted, don't let that prevent role
assignments on it from being deleted.

Change-Id: I56aede8209e425ee6c2d762a44db8cda5416e69b
Task: 30955
2019-05-13 10:18:06 -04:00
ZhongShengping
f66dac5c63 Update Python 3 test runtimes for Train
This goal is to implement the process set out in the 2018-10-24 Python
Update Process TC resolution[1], for the Train cycle to ensure unit
testing is in place for all of the Tested Runtimes for Train[2].
In practice, this generally means adding unit tests for Python 3.7 and dropping
unit tests for Python 3.5. Using the Zuul template for Train will ensure that
all projects that support Python3 will be tested against the agreed runtime
versions, and make it easier to update them in future.

[1]https://governance.openstack.org/tc/resolutions/20181024-python-update-process.html
[2]https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I62abb218bb314345dd7da1cbf9133d10db9696ff
Depends-On: https://review.opendev.org/#/c/641878/
2019-05-09 17:35:06 +08:00
Zuul
563616967d Merge "Add special user options for domain user" 2019-05-08 14:44:06 +00:00
Zuul
38e43a0071 Merge "Dropping the py35 testing" 2019-05-04 16:01:58 +00:00
Zuul
c87fc9fb53 Merge "Retry on DB deadlock in event_create()" 2019-05-03 19:14:19 +00:00
Pavlo Shchelokovskyy
d695602397 Add special user options for domain user
those are automated users that are created by Heat and the should
not be subject to restrictions possibly configured in Keystone
for security compliance, as those may break automated nature of things.

Create domain users with several available user options that will
make Keystone ignore:
- password expiry
- requirement to change the password on first use
- lockout after failed auth attempts

There are more things that must be done to properly secure those users
from becoming non-working, but this will be proposed in the followup
patches.

Story: 2005210
Task: 29988

Change-Id: I3152ddb82426cf66f2bd8ed69f53c77c653142bf
2019-05-02 16:12:56 -06:00
Rabi Mishra
6d6d766520 Don't send existing attributes in value_specs for neutron update
When updating we merge the property keys in value_specs, but there
is no need to send attributes that have not changed to neutron
as part of update api call.

Change-Id: I6df86a8dc9c4e64e2b370d3c2744d44712cd2ce2
2019-04-24 08:52:05 +05:30
Rabi Mishra
86e41a8a8f Fix upper-constraints.txt url
Wrong link used in commit 203bce9cd7449ef09a8777a4761f71518da3ed72.

Change-Id: I90cf7584830afccdd0028b0ed7e5061e67430815
2019-04-24 08:52:03 +05:30
Zuul
a0c95d6f54 Merge "Switch to use opendev.org" 2019-04-23 03:25:29 +00:00