1001 Commits

Author SHA1 Message Date
Hongbin Lu
f99d1771ba Do not use pip 10 or higher
It looks pip 10 failed the uninstallation of distutils installed
packages. This patch temporarily cap the version of pip to work-around. 

Closes-Bug: #1763966
Change-Id: I8bf80efc04883cd754c19bea0303064080112c6e
2018-04-14 20:58:31 +00:00
Matt Riedemann
23df4da0d4 Use Queens UCA
This updates the UCA usage from Pike to Queens. As a result,
the various volume multiattach checks can also be removed
because the Queens UCA has libvirt 4.0.0.

Change-Id: Icb971831c8d4fe5f940d9e7993d53f1c3765e30f
2018-03-19 14:27:01 -04:00
Zuul
cc4f6717fb Merge "Break up fixup_stuff" 2018-03-13 07:35:05 +00:00
Dirk Mueller
4404f680f2 Add an openSUSE Tumbleweed devstack non-voting check job
This adds the necessary fixes to pass a devstack run
on openSUSE Tumbleweed. Also removes opensuse 42.2 as it
is EOL for some time already and no longer actively tested
in the OpenStack infra.

Depends-On: I1b68c08c07cf6653ea58506f738cbe0054b38f3a
Change-Id: I2894482deef063fd02b0818c695a2ddbf6767039
2018-03-11 14:04:36 +01:00
Zuul
8408edbdfd Merge "XenAPI: remove xen tools" 2018-02-25 15:29:39 +00:00
Jianghua Wang
e619603bda XenAPI: remove xen tools
The xen tools have been moved to the project of *os-xenapi* since
os-xenapi 0.3.0. We also did some refact work on these tools in
os-xenapi. This commit is to remove these tools from devstack. So
that os-xenapi will be the single place for xen tools.

Change-Id: I4fdbe6bce12dfedd0d1e975ab8dd624ee3740c11
2018-02-23 03:58:56 +00:00
Ian Wienand
83ecb97fec Add VERBOSE_NO_TIMESTAMP flag
As described in the documentation, this flag is intended for the case
where the console output is being captured by a tool that appends its
own timestamps.

In the gate this is the job-output.txt.  We want the console output as
people like to watch that scrolling by as part of the live console
log.  Although this gets saved to job-output.txt, we still want to
keep logging to the individual log files even though it's technically
a duplicate -- in the multinode case the job-output.txt gets
interleaved by all the running nodes; it's much easier to just look at
the individual log files.  Also, people are used to it where it is :)

Change-Id: I3486636f1c76139581f6cd9668426f507b7c621d
2018-02-23 08:26:34 +11:00
Ian Wienand
e033e1b80f py3 changes for outfilter.py
I started running this with dib where we have pure python3
environments and it failed.

You can't have unbuffered text i/o in python3 for ... reasons? [1]
Changing the file to binary mode works around this.  Python3 opens
sys.stdin in text mode, so we need to manually convert the unicode
strings to bytes before we write them to the binary file.

[1] http://bugs.python.org/issue17404

Change-Id: Iebb26f0d3c2347d262cbc10dfd0912840cd05878
2018-02-23 08:26:34 +11:00
IWAMOTO Toshihiro
4d835e33b6 Break up fixup_stuff
Neutron functional tests want to use ubuntu cloud archive but it's
not possible to source the fixup_stuff.sh from a neutron CI setup
script. Break it up so that only the UCA portion can be executed
from neutron.

Change-Id: Ie18833bfa30f1789e63cbe9c86f5ece3453f43fb
2018-02-16 05:04:16 +00:00
Matt Riedemann
ffe691ecd5 tempest: set compute-feature-enabled.volume_multiattach
This adds the ENABLE_VOLUME_MULTIATTACH flag and if True
configures Tempest to run volume multiattach tests.

Note that due to https://bugzilla.redhat.com/show_bug.cgi?id=1378242
we can't run multiattach tests with the Pike UCA packages since
those include qemu 2.10 and libvirt 3.6, and the valid versions for
multiattach support with libvirt is qemu<2.10 or libvirt>=3.10.

Depends-On: I80c20914c03d7371e798ca3567c37307a0d54aaa
Depends-On: I158c6f20e3e6a24bd2e5299abbeb3fc5208e5885

Part of nova blueprint multi-attach-volume

Change-Id: I46b7eabf6a28f230666f6933a087f73cb4408348
2018-01-18 18:00:04 +01:00
Hunt Xu
c396355bef Use "ip neigh" instead of "arp -n" in worlddump.py
Worlddumping on a system without net-tools package gets an error showing
"arp: not found". As iproute2 can also show arp tables, we use it
instead.

Change-Id: I0cd83e6d14959dc5a1147c487b11f27fb92aa20a
2018-01-08 16:18:32 +08:00
Sumit Jamgade
135ebe955e Revert "Resolve openSUSE devstack failures"
This reverts commit d325875508e7d35d6dd62302d852e83815be2278.
the issue mentioned in the comment is now resolved.

Change-Id: I2705daead3d3b95f6ad82261212f2a1f40a77fb5
2017-10-13 15:11:23 +02:00
Zuul
b3b6c102d9 Merge "Update to using pike cloud-archive" 2017-10-02 09:41:47 +00:00
Matt Riedemann
917ad0998b Update to using pike cloud-archive
This reverts commit ee22ca8373abd3b5a4c44a9c5c4da39c511195c8

Depends-On: Iae2962bb86100f03fd3ad9aac3767da876291e74

Change-Id: I4d5fa052bdc5eef1795f6507589e2eaf4e093e23
Related-Bug: #1718295
2017-09-20 14:46:48 +00:00
Jenkins
548198769f Merge "Revert "Update to using pike cloud-archive"" 2017-09-20 08:37:36 +00:00
Matt Riedemann
ee22ca8373 Revert "Update to using pike cloud-archive"
This reverts commit a7e9a5d447b3eeacfb52d7ddc94445058a8d6fd1.

The jobs that run live migration tests are failing at about
a rate of 50% since this merged. There are no recent changes
to nova in the last 24 hours that are related to live
migration, and this is failing on the master branch only,
so I suspect the failures are due to new qemu packages
getting pulled in from this change.

Change-Id: Ic8481539c6a0cc7af08a736a625b672979435908
Closes-Bug: #1718295
2017-09-20 00:29:36 +00:00
Jenkins
d9386eb0bd Merge "Update to using pike cloud-archive" 2017-09-19 14:18:20 +00:00
Jenkins
71e07c738d Merge "Replace pmap shellout with pure python implementation" 2017-09-13 17:55:21 +00:00
Dirk Mueller
02f9e8bbdd Replace pmap shellout with pure python implementation
Without this patch, the pmap -XX call fails on openSUSE Leap
distributions as those have a fairly ancient procps version that
does not support the -XX parameter. A pure python implementation
is more portable, faster and even shorter than the subprocess
call.

Closes-Bug: #1716066
Change-Id: I2fdb457e65359a1c9d40452c922cfdca0e6e74dc
2017-09-10 17:50:00 -06:00
Jens Harbott
a7e9a5d447 Update to using pike cloud-archive
Now that Pike has been released, switch to using the pike version of
UCA instead of ocata, too.

One reason to do so it that it adds python3-ceph packages, allow to have
progress with the python3 compatibility of the Ceph integration.

Change-Id: I7d95e53892b697c72af75ad0ce7ce2dec6d31fde
2017-09-06 10:58:04 +00:00
Monty Taylor
d8bb220606 Make list_images.sh emit the etcd3 tarball
We need this for every devstack run now, so downloading it from github
every time isn't the most awesome thing in the world.

Add an extra variable EXTRA_CACHE_URLS which will be appended to the
output of tools/image_list.sh.  This way, these files will be
downloaded during the daily nodepool build, but they will not be in
the IMAGE_LIST and hence be considered as images to upload.

Add a function get_extra_file which echos the path to a file given the
URL.  It will first check the cache at $FILES, and if not present
download it.

Update the documentation in image_list.sh to reflect what's happening.

Move the defaults for etcd variables into stackrc, since it is a base
service now.

Change-Id: I86104824a29d973a6288df1f24b7891feb86267c
2017-09-05 09:40:40 +10:00
Jenkins
c5aca3c996 Merge "Don't reinstall python-virtualenv on infra nodes" 2017-08-21 05:09:35 +00:00
Ian Wienand
2d57f93f68 Don't reinstall python-virtualenv on infra nodes
In the original change I said "for infra nodes, it shouldn't do
anything anyway ...".  Well that was pre-Fedora 26 :)

It seems that dnf > 2.0 now intentionally throws an error when trying
to explicitly install an ignored package.  Thus, as described in the
comment, take a simpler approach of skipping this on infra nodes.
pip-and-virtualenv in dib should have installed the latest pip,
virtualenv and setuptools, so we don't want to fiddle with that
anyway.

[1] https://review.openstack.org/#/c/338998/

Change-Id: Ib300b58377a0d0fe1bd7444c71acdb9a87dc033b
2017-08-10 16:42:45 +10:00
Vasyl Saienko
dea3083d98 Fix path to mlock_report
This patch fixes path to mlock_report.py. Also add python-psutil
to dstat depends as it is required by mlock_report.

Change-Id: Ia2b507a7b923f1e3393a9cb7746c66d39d6abfde
2017-08-01 08:15:07 +03:00
Dirk Mueller
6effdf370a Remove extra websockify dependency cleanup step
This started to fail due to a missing || : / --force option now
that the dependencies got fixed:

2017-07-30 19:38:37.260 | ++ tools/install_prereqs.sh:source:97       :   sudo rpm -e --nodeps python-cffi python-cryptography python-pyOpenSSL
2017-07-30 19:38:37.293 | error: package python-cffi is not installed
2017-07-30 19:38:37.293 | error: package python-cryptography is not installed
2017-07-30 19:38:37.293 | error: package python-pyOpenSSL is not installed

Change-Id: Ia59afb7ee564cf2044ebdb3c5ad3e54ee91d1222
2017-07-31 08:56:17 +02:00
Dirk Mueller
d325875508 Resolve openSUSE devstack failures
This update resolves multiple issues with python-cryptography
causing keystone server and nova deployment to fail.

This is a temporary workaround until I196f025dbf1a9ac297946b8165620676645f7210
has landed and the extraneous dependency on python-cryptography (the
package) has been removed.

Change-Id: Ifb29b9089197c0429a5fc1cd08a25d2095d481f1
2017-07-24 21:25:27 +02:00
Takashi NATSUME
fa0077707e Update URLs
Some URLs are broken, so fix them.
The others are redirect to new URLs,
so replace them with new ones.

The config options of nova serial console proxy
have been gathered in nova/conf/serial_console.py.
So the description in doc/source/guides/nova.rst
is fixed.

Change-Id: Ifd81cc09969341fbf8f135a913fc6003b94e0acc
2017-07-22 09:12:59 +09:00
Paul Belanger
331b3de88e
Reduce tools/image_list.sh down to 4 images
Now that mysql.qcow2 has been removed, we only have 4 images to worry
about. This fixes cache-devstack element for openstack-infra.

Change-Id: Ia06f0e0679e253a1a6614f7c38abf1f5cd13991b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-14 13:11:19 -04:00
Jenkins
2c0692c2c7 Merge "Remove temporary openSUSE-42.2 workaround" 2017-06-14 06:18:58 +00:00
Jenkins
9234316794 Merge "Make use of Ubuntu Cloud Archive switchable" 2017-06-09 01:55:36 +00:00
Dirk Mueller
bbf14db3a7 Remove temporary openSUSE-42.2 workaround
We required initially 42.2 test updates to be enabled as the
liberasurecode-devel update wasn't released. It is now released
so we can stop pulling that part in.

Change-Id: I4e514e317da8a95809593a49c6dce619bc4c021f
2017-06-06 23:29:41 +02:00
Dirk Mueller
e61e19ee82 Enable opensuse-42.2 as a tested distro
openSUSE 42.2 passes testing on the experimental gate and
in order to add it as continuosly tested target we need to
add it to the positive list of tested distributions.

Change-Id: I46f94cfad828534f324994c3d21bddff40e8f9a2
2017-06-05 19:28:30 +02:00
Amrith Kumar
bb436d3373 Make use of Ubuntu Cloud Archive switchable
The trove development environment is typically a linux VM within which
openstack is installed, and trove launches guest vm's within that
environment. To make it possible for these vm's to launch in one human
lifetime, one must set vt/x and enable nested hypervisors to use with
kvm; qemu emulation will take way too long.

The new libvirtd (v2.5.0) in Ubuntu Cloud Archive doesn't handle
nested hypervisors well and if you use it, you end up with a guest
hanging on the GRUB line.

To enable that use-case, we provide ENABLE_UBUNTU_CLOUD_ARCHIVE which
the trove developer can set (to False) before running devstack.

Change-Id: Ia0265c67bb7d2a438575a03c0ddbf2d9c53266ed
Closes-Bug: #1689370
2017-05-25 12:36:25 -04:00
melanie witt
fc572a5da0 Add --tcp option to dstat command for connection stats
This enables tcp stats (listen, established, syn, time_wait, close) in
dstat to allow us to get a high-level view of performance changes in
the system during gate runs.

Change-Id: Ifbffbed22446e7e6a3b825c18266b63d2f2e7718
2017-05-17 19:05:56 +00:00
Ian Wienand
71d20e6582 Always disable use of libvirt wheel with UCA
It's not only using our upstream caches that you might get an old
libvirt-python wheel that is incompatible with UCA.  Move the ignore
out of the mirror check to apply it globally.

This is an alternative to Iba301a8c80c9ed584f5fb5a816f3d2cf5f5f0e77

Change-Id: I588b1e8e49aa60f3ce976dc1b6c8013ba1d88079
2017-04-26 21:23:01 -04:00
Jenkins
f6c86749fd Merge "fix typo" 2017-04-19 12:32:59 +00:00
Jenkins
ee49497205 Merge "XenAPI: Remove final references to Integration bridge" 2017-04-18 18:08:09 +00:00
Paul Belanger
1f92d44544
Use apt_get_update after we setup UCA
It is possible some CI system are using an http_proxy. Use the helper
function to cover this use case.

Change-Id: Iee685147ca0244fc7de328a765f937602223de20
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-04-13 13:08:59 -04:00
Jenkins
42414520b7 Merge "Test using UCA for libvirt 2.5.0" 2017-04-12 15:15:45 +00:00
youri jeong
f68f6f2e33 fix typo
fix typo for tools/dstat.sh
retreive must be retrieve

Change-Id: I7a817ec02e7156c886d7d6abb28688bfe2ef5998
2017-04-12 19:24:14 +09:00
Clark Boylan
c9a9e415b9 Test using UCA for libvirt 2.5.0
We have had issues with libvirt 1.3.1 which is stock on Xenial. Try
using 2.5.0 from UCA instead.

Related-Bug: 1643911
Related-Bug: 1646779
Related-Bug: 1638982
Change-Id: Ia4434541c71f050fe1ffb54f4c4c1e302391d00b
2017-04-11 14:32:44 -07:00
Huan Xie
2a2db2efc5 XenAPI: Remove final references to Integration bridge
The change to remove references of XEN_INTEGRATION_BRIDGE
(If5886e3711765a97f40f20e478f958b988b5a620) unfortunately left some
code which should have been removed.  This remaining code caused an
error in some situations when deploying from scratch (which the CI
avoids for expediency)

Change-Id: Ia568462c9cca8cff8fcfada8148d185609d61a7d
2017-04-11 01:24:37 +00:00
Jenkins
ec60d050f5 Merge "Remove the EBTABLES_RACE_FIX added for Trusty" 2017-04-07 00:33:30 +00:00
Jenkins
9b6080d859 Merge "Use br-int when XenServer is hypervisor" 2017-04-06 23:30:03 +00:00
Jenkins
ee71ae03f6 Merge "Run mlock_report under python3" 2017-04-03 13:41:55 +00:00
Ian Wienand
bfcc760b96 Enable libvirt coredumps
This adds a flag and basic config for enabling coredumps for libvirt.

Partial-Bug: 1643911
Co-Authored-By: Matthew Booth <mbooth@redhat.com>

Change-Id: If7cd54e804a5a389a0d82a325b58f5b41b8ef0db
2017-03-30 17:29:29 +11:00
Ian Wienand
9573edb4eb Run mlock_report under python3
psutil is only installed under python3 for the 3.5 gate jobs.  Call
mlock_report.py with $PYTHON so we support both environments.

Updates to mlock_report.py for python3 compatability

Change-Id: If7926ce6a2996b766c49b010a7f6640ae624f860
2017-03-28 19:38:13 +11:00
Jordan Pittier
1298f1bacd Remove the EBTABLES_RACE_FIX added for Trusty
Now that we don't support Ubuntu Trusty anymore, we can remove
the ebtables race workaround.

Closes-Bug: #1675714
Change-Id: I70483f871e35fcaa933d1b7bac7dbb396aa22cef
2017-03-28 08:16:07 +11:00
Sean Dague
5ae945244e run memory_tracker as root
In order to get memlocked pages this needs to be run as root, just
start it as root so that we don't have issues with the inability to
run sudo later in the run.

Change-Id: I7adab8cbb6d89d4717e427aec22e316d27bea075
2017-03-23 12:45:29 -04:00
Huan Xie
9e64bad03a Use br-int when XenServer is hypervisor
Previously we use a specific integration bridge for neutron ovs agent
which is running in compute node, but this isn't necessary, this
patch is to remove the specific integration bridge for XenSever and
remove the custom integration bridge definition

Depends-On: I675565e1ea6c887d40d7a53f62968c4aa385ecca

Change-Id: If5886e3711765a97f40f20e478f958b988b5a620
2017-03-22 19:11:34 -07:00