759 Commits

Author SHA1 Message Date
Ian Wienand
bf9186a37e Fallback to upstream for Fedora image
OpenDev infra only keep around the latest two Fedora releases in their
mirrors.  Probe for the image from the local test mirror, but if not
found, fallback to upstream.  This will be much less reliable, but can
avoid gate breakage until new images can be used.

Also, use endpoint_type when creating keystoneclient

Keystone admin endpoint has been removed from devstack with[1].
This would use the public endpoint by default.

Change-Id: I96ab14871ee8c5d5b83cc0cd4abc840ef0218ca8
2021-11-12 15:32:19 +05:30
Balazs Gibizer
ef731bca05 Fix timeout calc of nested resource groups
The timeout calculation of nested resource groups are using seconds but
then it passing the value in seconds to the stack resource that is
expected it in minutes. This leads to a rapidly increasing timeout value
in each nesting levels.

This patch converts the seconds to minutes before passing it forward.

Story: 2009237
Task: 43372
Change-Id: I3f132d4889723f7b4a3a416779ac5ee7663249b8
2021-09-21 14:21:16 +02:00
Balazs Gibizer
863d645d7e Reproduce bug story/2009237
Add a functional test that reproduces the bug that nested resource group
timeout value is increasing rapidly with every nesting level.

Story: 2009237
Task: 43373
Change-Id: I8e85434a5fb9cc6d0ef3240c8ae6085f5124174b
2021-09-21 14:20:45 +02:00
Takashi Kajinami
1828df32fa Use Block Storage API v3 instead of API v2
Block Storage API v2 was deprecated during Pike cycle and is being
removed during Xena cycle, and current v3 API should be used instead.
Unused volume_client and network_client in integration test code are
also removed by this change.

Note:
granade tests is made non-voting temporally until the same issue is
fixed in stable/wallaby.

Depends-on: https://review.opendev.org/802150/
Change-Id: I6a2b5afa13480791971bbd8bba1f43b9f2db8294
2021-08-02 14:39:30 +09:00
ricolin
ec0c79a155 Add explict dependency to avoid gate unstable
Gate job is extremely unstable because [1]
This patch propose to add explict dependency before we fix the issue.

Story: 2008936
Task: 42544

[1] https://storyboard.openstack.org/#!/story/2008936

Change-Id: I97e05c3d912030dbf77218b114e9e16fb25dca97
2021-06-07 17:12:51 +00:00
Matthias Runge
6a9c35d898 Move testing to Fedora 33
Now that the gate is unblocked, we can also test F33 for gating purposes
here.

Change-Id: Ie1c9075623d85b27aaf1ac67a3063e219726bc6f
2021-01-19 16:38:43 +00:00
Matthias Runge
61e53c4eca Replace Fedora test image with F32
Fedora 31 was retired and the image is gone from mirrors.
heat-cfntools have been dropped from fedora images, disable
the test till that's resolved.

Also makes grenade job non-voting, till this is backported
to stable/victoria.

Change-Id: Id869f83a46454897c2fe7a532eebfa2863befe5e
2020-12-04 12:18:59 +05:30
Sam Kumar
d2be2f9cfe Add test cases to check user{domain}
pattern.

Heat uses domain in user{domain} pattern to
find the user in the corresponding domain.

Change-Id: Ic01877e2524e094b087fcbddac7504356e395031
Story:2007867
Task:40234
2020-11-17 15:19:13 +00:00
Zuul
607d73ea6f Merge "Remove handling for client status races" 2020-11-16 09:20:26 +00:00
ricolin
6fe5230a34 Add more information for tests
Change-Id: Id0b1056398e1b186a336eec4244c446b5767e1a0
2020-11-13 00:48:04 +08:00
Zuul
cba6d9b03a Merge "Add template version 2021-04-16 for Wallaby" 2020-11-02 17:45:07 +00:00
Zane Bitter
674a62ae9b Add template version 2021-04-16 for Wallaby
Change-Id: I57047682cfa82ba6ca4affff54fab5216e9ba51c
Story: 2007388
2020-10-27 16:55:38 -04:00
Rabi Mishra
f18a676792 Use Fedora 31 for tests
Change-Id: Ia45e957bb24f5596a5ef16271f71ba8b77df9e42
2020-05-12 11:28:31 +05:30
Rabi Mishra
6d697ad2f0 Fix pep8 E741 errors
Change-Id: Ibb0da11fd89876602a6da62d08834937b81c2ce6
2020-05-12 11:28:31 +05:30
Zuul
c20124891e Merge "Remove six and python 2.7 full support" 2020-05-03 07:58:42 +00:00
Andreas Jaeger
37a83c1ede pep8: Enable E226
Enable
"E226 missing whitespace around arithmetic operator"

And fix problems encountered.

Change-Id: I90cbf6220f487a0808e13cd3998c8127be563498
2020-04-30 16:42:53 +02:00
Hervé Beraud
8c96a4d856 Remove six and python 2.7 full support
Six is in use to help us to keep support for python 2.7.
Since the ussuri cycle we decide to remove the python 2.7
support so we can go ahead and also remove six usage from
the python code.

Review process and help
-----------------------
Removing six introduce a lot of changes and an huge amount of modified files
To simplify reviews we decided to split changes into several patches to avoid
painful reviews and avoid mistakes.

To review this patch you can use the six documentation [1] to obtain help and
understand choices.

Additional informations
-----------------------
Changes related to 'six.b(data)' [2]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

six.b [2] encode the given datas in latin-1 in python3 so I did the same
things in this patch.

Latin-1 is equal to iso-8859-1 [3].

This encoding is the default encoding [4] of certain descriptive HTTP
headers.

I suggest to keep latin-1 for the moment and to move to another encoding
in a follow-up patch if needed to move to most powerful encoding (utf8).

HTML4 support utf8 charset and utf8 is the default charset for HTML5 [5].

Note that this commit message is autogenerated and not necesserly contains
changes related to 'six.b'

[1] https://six.readthedocs.io/
[2] https://six.readthedocs.io/#six.b
[3] https://docs.python.org/3/library/codecs.html#standard-encodings
[4] https://www.w3schools.com/charsets/ref_html_8859.asp
[5] https://www.w3schools.com/html/html_charset.asp

Patch 28 of a serie of 28 patches
six fully removed now!
Thank you six for the rendered services!

Change-Id: If44ee4b565cc9390fa0422fba4dda080b4f90b98
2020-04-23 14:49:12 +02:00
Pavlo Shchelokovskyy
31ea2fcde1 Fix genconfig tox env
we no longer keep configuration for in-repo integration tests in the
repo, relying on heat-tempest-plugin instead.

As a result, currently the `tox -egenconfig` is broken as it tries to
generate config sample calling non-existing config.py module.

Change-Id: I48685d7c791d98db83a7cbcf4ef926ee25ca62a4
2020-04-09 09:12:09 +00:00
ricolin
c61170e4e8 Update Fedora image ref for test jobs
Fedora-Cloud-Base-29-1.2.x86_64 is removed from openstack local mirror.
Update to use Fedora-Cloud-Base-30-1.2.x86_64 for test jobs.

Change-Id: Id3026a115e3a044b3dc00030fd6d82549dc189b6
2020-02-24 16:00:19 +00:00
Zuul
73f9dc295f Merge "Check task_state of instance before volume actions" 2020-02-20 04:42:20 +00:00
Zuul
8fa88c06dd Merge "Refactor test_conditions functional test" 2020-02-18 08:25:35 +00:00
Rabi Mishra
c923b9c64b Fix grenade job failure
Check if subnet already added to router before adding it.

Change-Id: I59903642aee125d3b5cfaa944d12cd3ed7b278b4
2020-02-07 18:37:55 +05:30
ricolin
c8d1a9f901 Migrate functional test jobs to zuul v3
Something are introduced in this patch:
* As devstack-gate/devstack-vm-gate-wrap.sh is not really zuul v3
native, we move all configs in to `devstack/lib/heat` and .zuul.yaml.

* Remove extra configs process in devstack. Like setup tempest(which
is well covered by tempest itself.) or overlapping heat test configs setup.
Use tempest config for all heat_integration tests. Also remove
heat_integrationtests/common/configs since they're no longer required.

* copy post.yaml for grenade jobs. As we migrate to zuul v3 for
functional tests but not grenade (not yet), the post.yaml should exists
under grenade dir. since it's only required by grenade jobs.

* Use post.yaml in functional tests for cleanup test environments.

Story: #2007056
Task: #37908

Depends-On: https://review.opendev.org/701105
Change-Id: I4f531161a7222e2c2a21f8d483f9c2a1d91dc38d
2020-01-07 09:38:04 +08:00
ricolin
40ca7e9e63 Check task_state of instance before volume actions
Check task_state before verify resize or before detachment.

We need to make sure the task_state moved out from resize_finish before
we trigger some action like attach/detach volume.

So when we update flvor and volume at the samethime, the resize action
will not affect volume attachment/detachment.

Depends-On: https://review.opendev.org/#/c/700512/

Change-Id: I64033d5a0a8fea5c4fd93b1deb111d3d8fba0cf7
Story: #2007042
Task: #37854
Task: #37855
Task: #37869
2019-12-26 15:29:05 +00:00
Zane Bitter
6b7d64988c Remove handling for client status races
Now that we guarantee that resources are UPDATE_IN_PROGRESS before the
stack update call returns for legacy stacks (this was already true for
convergence stacks), there is no need to have special handling to check the
updated_time either in StackResources or in functional tests.

Change-Id: I5bf7ed6cb9ba6c77a77dd36eb173e1927065c53e
Story: #1669608
Task: 23176
2019-12-23 18:43:11 +00:00
Zane Bitter
693b82fbd4 Refactor test_conditions functional test
Use the list_resources() method of the test base class.

Change-Id: I7835211c4420f9da44eb2ae2f1d0e1e4b868efc4
2019-12-23 18:27:12 +00:00
Zuul
496e8df76b Merge "Eliminate race in CancelUpdateTest" 2019-12-18 13:48:48 +00:00
Zuul
60ce510086 Merge "Pre-empt in-progress nested stack updates on new update" 2019-11-14 04:36:28 +00:00
Zuul
61048ef16b Merge "Add functional test for utf-8 decode issues" 2019-11-04 03:49:24 +00:00
Zane Bitter
8cd6a06736 Pre-empt in-progress nested stack updates on new update
If the parent resource of a nested stack is locked due to an IN_PROGRESS
update, cancel the nested stack update (which will result in the parent
resource being marked FAILED and releasing the lock so that the new
traversal can begin acting on it). This also cancels all descendants of
the nested stack.

This means that a concurrent update no longer gets blocked at a nested
stack boundary until the previous update has finished.

Change-Id: I5f14453ebab75d89672c6eea12de46d48a5147f3
Task: 17760
2019-10-29 23:15:31 -04:00
Zane Bitter
3b4e0cda28 Eliminate race in CancelUpdateTest
The purpose of this test is to ensure we can rollback an in-progress
Server resource with attached ports. However, by cancelling immediately
we create a race whereby the stack may be rolled back before we have
even started updating the stack, in which case we are not testing the
behaviour we actually care about. Inserting a delay gives Heat enough
time to process the Port and start updating the Server before we cancel
the update.

In some circumstances this could also trigger a race within Heat,
whereby the rollback traversal would not block, but the update traversal
would retrigger it anyway. This resulted in the stack rollback being
marked COMPLETE multiple times, which could also lead to errors in
deleting the old raw_template after it had already been removed. The
latter have been fixed by separate patches. The race itself should be
addressed separately.

Change-Id: I3f9db686c31659437ff4797555e6608986946572
Task: 37199
2019-10-16 21:46:52 -04:00
Zuul
89ab96176f Merge "Add regression tests for conditional outputs in nested stacks" 2019-10-14 07:53:02 +00:00
Rabi Mishra
e9e8b7407d Add functional test for utf-8 decode issues
Change-Id: Ifb5100ab2a933f112e76e131cd91ea711b4d1348
Depends-On: https://review.openstack.org/435667
Depends-On: https://review.openstack.org/643504
Story: 2005237
Task: 30025
2019-09-26 06:42:45 +00:00
ricolin
fdb5e892bf Add doc for multi-clouds support
Add doc for multi-clouds support in template guide.

Also remove redundant credential information in multi-clouds
integration test.

Change-Id: I76c6427b7bbdac2af3b7f01aff1b0541e56b3653
Story: #2002126
Task: #19808
2019-06-25 07:37:27 +00:00
Rabi Mishra
46f595b462 Fix grenade regression introduced by multicloud support
Fixes grenade job regression introduced by commit
6990331639a96a7073f66a63c99f49d8f6c576b7.

Change-Id: Ic2b782cd0a6e27c43d741ccf3864ecbcf543dc27
2019-04-10 09:11:40 +05:30
Zane Bitter
aa58fbcacf Load existing resources using correct environment
In convergence we were loading resources from the database using the
current environment. This is incorrect when a previous update has
failed, meaning the resources in the database were created with a
non-current template and environment. If an attempt was made to change
the type of a resource but that resource was never updated, this will
result in us loading a resource with the wrong type. If the type has
been removed then it can result in errors just trying to show the stack.

Note that the Resource.load() method used during a convergence traversal
already does the Right Thing - it only uses the new type if it is a
valid substitution for the old type, and UpdateReplace is later raised
in Resource.update_convergence() if the type does not match in that
specified in the new environment. So we don't see any problems with
stack updates, just with API calls.

Since we cannot change the signature of Resource.__new__() without also
modifying the signature of __init__() in every resource plugin that has
implemented it (many of which are out of tree), instead substitute the
stack definition for the duration of creating the Resource object. This
will result in stack.env returning the environment the resource was last
updated with.

Change-Id: I3fbd14324fc4681b26747ee7505000b8fc9439f1
Story: #2005090
Task: 29688
2019-04-01 14:21:44 +05:30
rabi
97f2636dc7 Use keystone session with heatclient in integration tests
Change-Id: I4c66927e3dd3e1a91cfab94f26a24dbe8932bde8
Related-Bug: #1695144
2019-03-15 09:13:17 +00:00
ricolin
6990331639 Support remote stack with another OpenStack provider
Allow OS::Heat::Stack to access remote stack from another OpenStack
provider. Also enable functional tests for multi-cloud.

Implement multi-cloud support as an extension to the existing multi-region
support. Allow operate a remote stack (from another OpenStack cloud) as a
resource in stack from local OpenStack cloud.

I propose we add multi cloud support into ``OS::Heat::Stack`` and change the
property schema for ``context``. Within context, we should adding
following properties:

* credential_secret_id: ID of Barbican Secret. Which stores authN
  information for remote cloud.

Service will use auth information from Barbican Secret to access
 Orchestration service in another OpenStack.
Must make sure you're able toget that secret from Barbican service when
provide `credential_secret_id` property.

Story: #2002126
Task: #26907
Depends-On: https://review.openstack.org/579750

Change-Id: I2f3de3e7c29cf7debb1474228c8a9a81725a72ed
2019-03-13 22:15:45 +08:00
Zuul
2e3091e5b6 Merge "Improve best existing resource selection" 2019-03-13 07:04:51 +00:00
Zane Bitter
ef8d50856f Remove VolumeBackupRestoreIntegrationTest from skip list
We cannot run this test in the gate because the volume backup feature is
enabled. However, with a change to the Heat tempest plugin to allow this
feature to be automatically skipped based on the tempest config, we no
longer need this test to be explicitly included in the skip list.

Change-Id: Iaefa5c272c4322498330307ea9bd6843d276ef1b
Depends-On: https://review.openstack.org/632989
2019-03-08 02:53:19 +00:00
Rabi Mishra
84c908ca9e Add disk size to the flavors
Nova has added a default policy
os_compute_api:servers:create:zero_disk_flavor to admin-only,
that will prevent non-admins from creating image-backed servers
with a flavor that has disk=0 which is potential a security
exposure.

Add appropriate disk size to flavors we use in tests.

Change-Id: I9c0af0b40fc5b5c33de7f0c690d156ed2bac2f15
2019-02-15 20:35:29 +05:30
Zane Bitter
97df8bb6ca Improve best existing resource selection
Rank all existing versions of a resource in a convergence stack to improve
the likelihood that we find the best one to update.

This allows us to roll back to the original version of a resource (or
even attempt an in-place update of it) when replacing it has failed.
Previously this only worked during automatic rollback; on subsequent
updates we would always work on the failed replacement (which inevitably
meant attempting another replacement in almost all cases).

Change-Id: Ia231fae85d1ddb9fc7b7de4e82cec0c0e0fd06b7
Story: #2003579
Task: 24881
2019-01-29 16:47:33 +13:00
Rabi Mishra
a2f74772e1 Add heat-agents as required project
We removed[1] these scripts from heat-templates.

[1] https://review.openstack.org/#/c/584777/

Change-Id: I6f0989a5cf4cf6253ff64a8c3c1ea023dd8b6db7
2019-01-03 18:16:52 +05:30
Rabi Mishra
8cbc8ccf6b Bump minimal_image_ref
Devstack bumped[1] the cirros image version. Changes
to source the old openrc file for grenade before configuring
tempest.

[1] https://review.openstack.org/#/c/624716/

Change-Id: I9481bbcb03609db6d2224998a4f83d3a8790e96c
2018-12-19 15:39:36 +05:30
Zane Bitter
d539fdcb2e Add regression tests for conditional outputs in nested stacks
The patch 85eff5fc3d03dce7d0d17d965f27246bd3306c53 had to be reverted
because it broke nested stacks in TripleO. To ensure that doesn't happen
again, add some functional tests for conditional outputs in a nested
stack.

Change-Id: I0f7a42c6d1754e340fb519bd228feaa625000ffc
Related-Bug: #1805589
2018-12-04 20:37:17 +13:00
Rabi Mishra
b5333aea94 Use fedora 29 image from nodepool mirror
It seems fedora 27 cloud images are not available.

Change-Id: I199afcbaa5050226a333779ac22aea3451f94b04
2018-11-28 13:05:31 +05:30
Zuul
3c878a16bf Merge "Remove unnecessary shebangs from service scripts" 2018-09-10 05:01:27 +00:00
rabi
8bb1cef48f Add functional test for legacy in-place update
in-place update of failed stacks with old resource references
deleted in last update.

Change-Id: I45fbca77c84744e1778c6f2494674233b9193921
Story: #2003612
Task: 25740
2018-09-03 09:57:39 +05:30
Zane Bitter
92101b18e2 Ignore conditions when reparsing ResourceDefinition
In the legacy path we reparse a resource definition to refer to a different
StackDefinition when copying a resource definition into the existing
template (as the resource is updated) or into the backup stack's template.
However, the resources 'condition' may reference conditionals that are not
defined in the template it is being copied into (e.g. during a stack update
that creates a new condition), and in this case the destination template
will become unusable.

Since the fact that we care about the resource definition at all indicates
that the condition was enabled, just ignore the condition after reparsing.
This is consistent with what we do for 'if' macros, which is to resolve the
condition part during reparsing.

Change-Id: I59a374435f6275badc8124efbd7b7db2e36e2de5
Story: #2003558
Task: 24847
2018-08-23 20:03:55 -04:00
rabi
9c421a6725 Remove unnecessary shebangs from service scripts
These are just python files.

Also removes internal .gitignore in heat_integrationtests

Change-Id: Ibea3253cd89f905e61b655861274c61ab48a7ea0
2018-08-01 10:53:51 +05:30