990 Commits

Author SHA1 Message Date
Sumit Jamgade
135ebe955e Revert "Resolve openSUSE devstack failures"
This reverts commit d325875508e7d35d6dd62302d852e83815be2278.
the issue mentioned in the comment is now resolved.

Change-Id: I2705daead3d3b95f6ad82261212f2a1f40a77fb5
2017-10-13 15:11:23 +02:00
Zuul
b3b6c102d9 Merge "Update to using pike cloud-archive" 2017-10-02 09:41:47 +00:00
Matt Riedemann
917ad0998b Update to using pike cloud-archive
This reverts commit ee22ca8373abd3b5a4c44a9c5c4da39c511195c8

Depends-On: Iae2962bb86100f03fd3ad9aac3767da876291e74

Change-Id: I4d5fa052bdc5eef1795f6507589e2eaf4e093e23
Related-Bug: #1718295
2017-09-20 14:46:48 +00:00
Jenkins
548198769f Merge "Revert "Update to using pike cloud-archive"" 2017-09-20 08:37:36 +00:00
Matt Riedemann
ee22ca8373 Revert "Update to using pike cloud-archive"
This reverts commit a7e9a5d447b3eeacfb52d7ddc94445058a8d6fd1.

The jobs that run live migration tests are failing at about
a rate of 50% since this merged. There are no recent changes
to nova in the last 24 hours that are related to live
migration, and this is failing on the master branch only,
so I suspect the failures are due to new qemu packages
getting pulled in from this change.

Change-Id: Ic8481539c6a0cc7af08a736a625b672979435908
Closes-Bug: #1718295
2017-09-20 00:29:36 +00:00
Jenkins
d9386eb0bd Merge "Update to using pike cloud-archive" 2017-09-19 14:18:20 +00:00
Jenkins
71e07c738d Merge "Replace pmap shellout with pure python implementation" 2017-09-13 17:55:21 +00:00
Dirk Mueller
02f9e8bbdd Replace pmap shellout with pure python implementation
Without this patch, the pmap -XX call fails on openSUSE Leap
distributions as those have a fairly ancient procps version that
does not support the -XX parameter. A pure python implementation
is more portable, faster and even shorter than the subprocess
call.

Closes-Bug: #1716066
Change-Id: I2fdb457e65359a1c9d40452c922cfdca0e6e74dc
2017-09-10 17:50:00 -06:00
Jens Harbott
a7e9a5d447 Update to using pike cloud-archive
Now that Pike has been released, switch to using the pike version of
UCA instead of ocata, too.

One reason to do so it that it adds python3-ceph packages, allow to have
progress with the python3 compatibility of the Ceph integration.

Change-Id: I7d95e53892b697c72af75ad0ce7ce2dec6d31fde
2017-09-06 10:58:04 +00:00
Monty Taylor
d8bb220606 Make list_images.sh emit the etcd3 tarball
We need this for every devstack run now, so downloading it from github
every time isn't the most awesome thing in the world.

Add an extra variable EXTRA_CACHE_URLS which will be appended to the
output of tools/image_list.sh.  This way, these files will be
downloaded during the daily nodepool build, but they will not be in
the IMAGE_LIST and hence be considered as images to upload.

Add a function get_extra_file which echos the path to a file given the
URL.  It will first check the cache at $FILES, and if not present
download it.

Update the documentation in image_list.sh to reflect what's happening.

Move the defaults for etcd variables into stackrc, since it is a base
service now.

Change-Id: I86104824a29d973a6288df1f24b7891feb86267c
2017-09-05 09:40:40 +10:00
Jenkins
c5aca3c996 Merge "Don't reinstall python-virtualenv on infra nodes" 2017-08-21 05:09:35 +00:00
Ian Wienand
2d57f93f68 Don't reinstall python-virtualenv on infra nodes
In the original change I said "for infra nodes, it shouldn't do
anything anyway ...".  Well that was pre-Fedora 26 :)

It seems that dnf > 2.0 now intentionally throws an error when trying
to explicitly install an ignored package.  Thus, as described in the
comment, take a simpler approach of skipping this on infra nodes.
pip-and-virtualenv in dib should have installed the latest pip,
virtualenv and setuptools, so we don't want to fiddle with that
anyway.

[1] https://review.openstack.org/#/c/338998/

Change-Id: Ib300b58377a0d0fe1bd7444c71acdb9a87dc033b
2017-08-10 16:42:45 +10:00
Vasyl Saienko
dea3083d98 Fix path to mlock_report
This patch fixes path to mlock_report.py. Also add python-psutil
to dstat depends as it is required by mlock_report.

Change-Id: Ia2b507a7b923f1e3393a9cb7746c66d39d6abfde
2017-08-01 08:15:07 +03:00
Dirk Mueller
6effdf370a Remove extra websockify dependency cleanup step
This started to fail due to a missing || : / --force option now
that the dependencies got fixed:

2017-07-30 19:38:37.260 | ++ tools/install_prereqs.sh:source:97       :   sudo rpm -e --nodeps python-cffi python-cryptography python-pyOpenSSL
2017-07-30 19:38:37.293 | error: package python-cffi is not installed
2017-07-30 19:38:37.293 | error: package python-cryptography is not installed
2017-07-30 19:38:37.293 | error: package python-pyOpenSSL is not installed

Change-Id: Ia59afb7ee564cf2044ebdb3c5ad3e54ee91d1222
2017-07-31 08:56:17 +02:00
Dirk Mueller
d325875508 Resolve openSUSE devstack failures
This update resolves multiple issues with python-cryptography
causing keystone server and nova deployment to fail.

This is a temporary workaround until I196f025dbf1a9ac297946b8165620676645f7210
has landed and the extraneous dependency on python-cryptography (the
package) has been removed.

Change-Id: Ifb29b9089197c0429a5fc1cd08a25d2095d481f1
2017-07-24 21:25:27 +02:00
Takashi NATSUME
fa0077707e Update URLs
Some URLs are broken, so fix them.
The others are redirect to new URLs,
so replace them with new ones.

The config options of nova serial console proxy
have been gathered in nova/conf/serial_console.py.
So the description in doc/source/guides/nova.rst
is fixed.

Change-Id: Ifd81cc09969341fbf8f135a913fc6003b94e0acc
2017-07-22 09:12:59 +09:00
Paul Belanger
331b3de88e
Reduce tools/image_list.sh down to 4 images
Now that mysql.qcow2 has been removed, we only have 4 images to worry
about. This fixes cache-devstack element for openstack-infra.

Change-Id: Ia06f0e0679e253a1a6614f7c38abf1f5cd13991b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-14 13:11:19 -04:00
Jenkins
2c0692c2c7 Merge "Remove temporary openSUSE-42.2 workaround" 2017-06-14 06:18:58 +00:00
Jenkins
9234316794 Merge "Make use of Ubuntu Cloud Archive switchable" 2017-06-09 01:55:36 +00:00
Dirk Mueller
bbf14db3a7 Remove temporary openSUSE-42.2 workaround
We required initially 42.2 test updates to be enabled as the
liberasurecode-devel update wasn't released. It is now released
so we can stop pulling that part in.

Change-Id: I4e514e317da8a95809593a49c6dce619bc4c021f
2017-06-06 23:29:41 +02:00
Dirk Mueller
e61e19ee82 Enable opensuse-42.2 as a tested distro
openSUSE 42.2 passes testing on the experimental gate and
in order to add it as continuosly tested target we need to
add it to the positive list of tested distributions.

Change-Id: I46f94cfad828534f324994c3d21bddff40e8f9a2
2017-06-05 19:28:30 +02:00
Amrith Kumar
bb436d3373 Make use of Ubuntu Cloud Archive switchable
The trove development environment is typically a linux VM within which
openstack is installed, and trove launches guest vm's within that
environment. To make it possible for these vm's to launch in one human
lifetime, one must set vt/x and enable nested hypervisors to use with
kvm; qemu emulation will take way too long.

The new libvirtd (v2.5.0) in Ubuntu Cloud Archive doesn't handle
nested hypervisors well and if you use it, you end up with a guest
hanging on the GRUB line.

To enable that use-case, we provide ENABLE_UBUNTU_CLOUD_ARCHIVE which
the trove developer can set (to False) before running devstack.

Change-Id: Ia0265c67bb7d2a438575a03c0ddbf2d9c53266ed
Closes-Bug: #1689370
2017-05-25 12:36:25 -04:00
melanie witt
fc572a5da0 Add --tcp option to dstat command for connection stats
This enables tcp stats (listen, established, syn, time_wait, close) in
dstat to allow us to get a high-level view of performance changes in
the system during gate runs.

Change-Id: Ifbffbed22446e7e6a3b825c18266b63d2f2e7718
2017-05-17 19:05:56 +00:00
Ian Wienand
71d20e6582 Always disable use of libvirt wheel with UCA
It's not only using our upstream caches that you might get an old
libvirt-python wheel that is incompatible with UCA.  Move the ignore
out of the mirror check to apply it globally.

This is an alternative to Iba301a8c80c9ed584f5fb5a816f3d2cf5f5f0e77

Change-Id: I588b1e8e49aa60f3ce976dc1b6c8013ba1d88079
2017-04-26 21:23:01 -04:00
Jenkins
f6c86749fd Merge "fix typo" 2017-04-19 12:32:59 +00:00
Jenkins
ee49497205 Merge "XenAPI: Remove final references to Integration bridge" 2017-04-18 18:08:09 +00:00
Paul Belanger
1f92d44544
Use apt_get_update after we setup UCA
It is possible some CI system are using an http_proxy. Use the helper
function to cover this use case.

Change-Id: Iee685147ca0244fc7de328a765f937602223de20
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-04-13 13:08:59 -04:00
Jenkins
42414520b7 Merge "Test using UCA for libvirt 2.5.0" 2017-04-12 15:15:45 +00:00
youri jeong
f68f6f2e33 fix typo
fix typo for tools/dstat.sh
retreive must be retrieve

Change-Id: I7a817ec02e7156c886d7d6abb28688bfe2ef5998
2017-04-12 19:24:14 +09:00
Clark Boylan
c9a9e415b9 Test using UCA for libvirt 2.5.0
We have had issues with libvirt 1.3.1 which is stock on Xenial. Try
using 2.5.0 from UCA instead.

Related-Bug: 1643911
Related-Bug: 1646779
Related-Bug: 1638982
Change-Id: Ia4434541c71f050fe1ffb54f4c4c1e302391d00b
2017-04-11 14:32:44 -07:00
Huan Xie
2a2db2efc5 XenAPI: Remove final references to Integration bridge
The change to remove references of XEN_INTEGRATION_BRIDGE
(If5886e3711765a97f40f20e478f958b988b5a620) unfortunately left some
code which should have been removed.  This remaining code caused an
error in some situations when deploying from scratch (which the CI
avoids for expediency)

Change-Id: Ia568462c9cca8cff8fcfada8148d185609d61a7d
2017-04-11 01:24:37 +00:00
Jenkins
ec60d050f5 Merge "Remove the EBTABLES_RACE_FIX added for Trusty" 2017-04-07 00:33:30 +00:00
Jenkins
9b6080d859 Merge "Use br-int when XenServer is hypervisor" 2017-04-06 23:30:03 +00:00
Jenkins
ee71ae03f6 Merge "Run mlock_report under python3" 2017-04-03 13:41:55 +00:00
Ian Wienand
bfcc760b96 Enable libvirt coredumps
This adds a flag and basic config for enabling coredumps for libvirt.

Partial-Bug: 1643911
Co-Authored-By: Matthew Booth <mbooth@redhat.com>

Change-Id: If7cd54e804a5a389a0d82a325b58f5b41b8ef0db
2017-03-30 17:29:29 +11:00
Ian Wienand
9573edb4eb Run mlock_report under python3
psutil is only installed under python3 for the 3.5 gate jobs.  Call
mlock_report.py with $PYTHON so we support both environments.

Updates to mlock_report.py for python3 compatability

Change-Id: If7926ce6a2996b766c49b010a7f6640ae624f860
2017-03-28 19:38:13 +11:00
Jordan Pittier
1298f1bacd Remove the EBTABLES_RACE_FIX added for Trusty
Now that we don't support Ubuntu Trusty anymore, we can remove
the ebtables race workaround.

Closes-Bug: #1675714
Change-Id: I70483f871e35fcaa933d1b7bac7dbb396aa22cef
2017-03-28 08:16:07 +11:00
Sean Dague
5ae945244e run memory_tracker as root
In order to get memlocked pages this needs to be run as root, just
start it as root so that we don't have issues with the inability to
run sudo later in the run.

Change-Id: I7adab8cbb6d89d4717e427aec22e316d27bea075
2017-03-23 12:45:29 -04:00
Huan Xie
9e64bad03a Use br-int when XenServer is hypervisor
Previously we use a specific integration bridge for neutron ovs agent
which is running in compute node, but this isn't necessary, this
patch is to remove the specific integration bridge for XenSever and
remove the custom integration bridge definition

Depends-On: I675565e1ea6c887d40d7a53f62968c4aa385ecca

Change-Id: If5886e3711765a97f40f20e478f958b988b5a620
2017-03-22 19:11:34 -07:00
Jenkins
c292223f13 Merge "Xen: restrict devstack VM use upto 8 vCPUs." 2017-03-17 15:49:33 +00:00
jianghua
5f8bd0e452 Xen: restrict devstack VM use upto 8 vCPUs.
Change-Id: Iea6086a3feb03dff581d9c04560133c4f57a6f6c
Closes-Bug: 1672642
2017-03-16 08:44:44 +00:00
Ihar Hrachyshka
2b4735f1b3 Added list of mlock-using processes to peakmem_tracker output
The change makes peakmem_tracker list processes that lock memory pages
from swapping to disk. It may be helpful when debugging oom-killer job
failures in gate in case when dstat shows that swap is not fully used
when oom-killer is triggered.

The peakmem_tracker service was renamed into memory_tracker to reflect
its new broader scope.

Needed-By: I5862d92478397eac2e61b8a61ce3437b698678be
Change-Id: I1dca120448ee87930fe903fd81277b58efaefc92
2017-03-05 15:50:59 +00:00
Ian Wienand
1e7f738f28 Ensure we install setuptools from requirements
Use pip_install_gr so we get the version pinned by requirements.  The
depends-on is an example of where we're trying to pin to workaround
issues.

Depends-On: I9c57c08a150571c5bb62235d502839394d53a4c1
Change-Id: I780cca681b12a3e9d228dbf2fd9fa6e8ab1a82e1
2017-02-27 11:35:43 +11:00
Jenkins
6f23555c95 Merge "Fix error in 'ip netns' parsing" 2017-02-26 21:43:54 +00:00
Jenkins
9a21fe65a7 Merge "Xen: support Ubuntu Xenial for xen DevStack" 2017-02-26 03:41:50 +00:00
Jenkins
cffe4fea06 Merge "worlddump: Use __future__ print_function" 2017-02-24 12:00:27 +00:00
Jianghua Wang
6e49cab091 Xen: support Ubuntu Xenial for xen DevStack
Upgrade xen tool to install Ubuntu Xenial; change the upstart
task to systemd task to finish the OpenStack installation
by using devstack.

Change-Id: I8129923be3c41e7f60e9d32348a5ea8e07d4845b
2017-02-22 19:59:36 +08:00
Jenkins
5ed1e34802 Merge "Add top memory consuming process to dstat output" 2017-02-15 02:02:49 +00:00
John L. Villalovos
c6e6939e89 Fix error in 'ip netns' parsing
Sometimes when doing worlddump would see a command line like this:
 sudo ip netns exec (id: ip addr

This would cause an error to be seen in console.log:
  2017-02-07 00:03:03.659570 | /bin/sh: 1: Syntax error: "(" unexpected

This is caused by there sometimes being extra data returned from the
'ip netns' command [1]. For example it might look like:
    qrouter-0805fd7d-c493-4fa6-82ca-1c6c9b23cd9e (id: 1)
    qdhcp-bb2cc6ae-2ae8-474f-adda-a94059b872b5 (id: 0)

[1] https://lwn.net/Articles/629715/

Change-Id: Icece442023125ef55696b8d92a975d37e358b1b4
Closes-Bug: 1653969
2017-02-14 09:29:54 -08:00
John L. Villalovos
09949e0dc6 worlddump: Use __future__ print_function
Commit e7361775c112e32ea517eddc344641897d273d25 changed the code to
use Python 3 style print function, but when doing 'print()' in Python
2.7 it would print '()':
    >>> print()
    ()

Import the __future__ print function so that a blank line will be
printed as expected. This will now work the same in Python 2 & 3.

Change-Id: I61742e107278f2327c18c9ab0de52d1914f16c97
2017-02-06 13:55:52 -08:00