266 Commits

Author SHA1 Message Date
Dean Troyer
490430dbe3 Split functions-common: python functions
Move Python-related functions into inc/python

Should be transparent to all callers as it is sourced from functions-common

Change-Id: I88043830cef9211b4e0baa91bfcc7a92125afa9f
2015-01-30 14:59:48 -06:00
Sean Dague
537532931d Make changes such that -o nounset runs
This makes a bunch of variable cleanups that will let -o nounset
function, for the time being we hide nounset behind another setting
variable so that it's not on by default.

Because this is bash, and things are only executed on demand, this
probably only works in the config it was run in. Expect cleaning up
all the paths to be something that takes quite a while.

This also includes a new set of unit tests around the trueorfalse
function, because my change in how it worked, didn't. Tests are good
m'kay.

Change-Id: I71a896623ea9e1f042a73dc0678ce85acf0dc87d
2015-01-15 13:06:14 -05:00
Attila Fazekas
057d6ae225 wget less verbose
wget is too verbose in devstack logs [1] on image download.

Changing the progress bar style to giga, in order
to be less verbose.

http://logs.openstack.org/73/146573/2/check/
check-tempest-dsvm-full-juno/41ba988/logs/devstacklog.txt.gz#_2015-01-13_11_34_15_330

Change-Id: Ic5304893f4c97c50e7a2f29ad5cd77dba3d5a9dd
2015-01-13 14:01:26 +01:00
Jenkins
d09a1db5a4 Merge "add shebang lines to all lib files" 2014-12-10 20:32:15 +00:00
Sean Dague
e263c82e48 add shebang lines to all lib files
With gerrit 2.8, and the new change screen, this will trigger syntax
highlighting in gerrit. Thus making reviewing code a lot nicer.

Change-Id: Id238748417ffab53e02d59413dba66f61e724383
2014-12-10 11:28:05 -05:00
Sabari Kumar Murugesan
88cde0b3da VMware: add default image metadata for hw_vif_model
During glance image-create we are currently setting a null value
for the hw_vif_model, if we are unable to introspect from the vmdk
file.

This patch adds a default value if not found.

Change-Id: I6d0f551b18f03e2a799ae14c3347652a285693f0
2014-12-04 20:33:41 -08:00
Jenkins
e5dbec252a Merge "Fix docs build errors" 2014-10-08 22:14:51 +00:00
Dean Troyer
3324f19f5a Fix docs build errors
Fix shocco errors during docs generation

Closes-Bug: 1362691
Change-Id: I2b7fb008c89f0b4e7280b2d0a054320765e83e47
2014-10-02 15:30:50 -05:00
Joe Gordon
944b28280b Drop workaround for pip < 1.4
Now that we are on pip 1.5.6 lets drop the workaround to make pip 1.4
work. As this is a development/testing tool requiring a newer pip
shouldn't be an issue. Also stack.sh installs pip by default.

Work around introduced in https://github.com/pypa/pip/issues/709

Change-Id: I0e7aad1d21f4fce4c020ce36685bb56893c66bdc
2014-10-01 19:52:23 -07:00
Rob Crittenden
18d4778cf7 Configure endpoints to use SSL natively or via proxy
Configure nova, cinder, glance, swift and neutron to use SSL
on the endpoints using either SSL natively or via a TLS proxy
using stud.

To enable SSL via proxy, in local.conf add

ENABLED_SERVICES+=,tls-proxy

This will create a new test root CA, a subordinate CA and an SSL
server cert. It uses the value of hostname -f for the certificate
subject. The CA certicates are also added to the system CA bundle.

To enable SSL natively, in local.conf add:

USE_SSL=True

Native SSL by default will also use the devstack-generate root and
subordinate CA.

You can override this on a per-service basis by setting

<SERVICE>_SSL_CERT=/path/to/cert
<SERVICE>_SSL_KEY=/path/to/key
<SERVICE>_SSL_PATH=/path/to/ca

You should also set SERVICE_HOST to the FQDN of the host. This
value defaults to the host IP address.

Change-Id: I36fe56c063ca921131ad98439bd452cb135916ac
Closes-Bug: 1328226
2014-09-24 18:36:37 -04:00
Noboru Iwamatsu
b4495eb410 Use mod_version to clean-up apache version matching
This change uses mod_version (shipped by default on everything we care
about) to set-up version-specific config within apache rather than
within devstack scripts.

Clean up the horizon and keystone config file generation to use the
internal apache matching.

Since I6478db385fda2fa1c75ced12d3e886b2e1152852 the apache matching in
'functions' is actually duplicated.  just leave get_apache_version in
lib/apache as it is used for config-file name matching in there.

Change-Id: I6478db385fda2fa1c75ced12d3e886b2e1152852
2014-09-08 15:59:29 +10:00
Jenkins
a3ecab6618 Merge "Modify glance image-create commands to use openstackclient" 2014-08-15 00:50:41 +00:00
Alessandro Pilotti
ca8239449f Adds support for Hyper-V image formats
Adds support for VHD and VHDX image formats, including gz compression.

Change-Id: I9b143c9883b947397b435a671f13703f78019d23
Closes-bug: #1353726
2014-08-07 19:30:34 +03:00
Steve Martinelli
8d3ac2df58 Modify glance image-create commands to use openstackclient
Change `glance image-create` commands to use `openstack image create`,
in an effort to unify the CLI.

Change-Id: I96cef118daf931b648c0483525ac7d2287fec2e0
2014-08-02 23:47:15 -04:00
Jenkins
5acf159c72 Merge "Clean up local variable usage - upload_image()" 2014-08-02 21:49:29 +00:00
Dean Troyer
e9f7667050 Clean up local variable usage - upload_image()
Cleans up the rather coplex upload_image() function

Change-Id: Ie1ef200b905dd5d17ef887390d620817208f2bf5
2014-07-29 15:40:45 -05:00
Sébastien Han
36f2f024db Implement Ceph backend for Glance / Cinder / Nova
The new lib installs a full Ceph cluster. It can be managed
by the service init scripts. Ceph can also be installed in
standalone without any other components.
This implementation adds the auto-configuration for
the following services with Ceph:

* Glance
* Cinder
* Cinder backup
* Nova

To enable Ceph simply add: ENABLED_SERVICES+=,ceph to your localrc.
If you want to play with the Ceph replication, you can use the
CEPH_REPLICAS option and set a replica. This replica will be used for
every pools (Glance, Cinder, Cinder backup and Nova). The size of the
loopback disk used for Ceph can also be managed thanks to the
CEPH_LOOPBACK_DISK_SIZE option.

Going further pools, users and PGs are configurable as well. The
convention is <SERVICE_NAME_IN_CAPITAL>_CEPH_<OPTION> where services are
GLANCE, CINDER, NOVA, CINDER_BAK. Let's take the example of Cinder:

* CINDER_CEPH_POOL
* CINDER_CEPH_USER
* CINDER_CEPH_POOL_PG
* CINDER_CEPH_POOL_PGP

** Only works on Ubuntu Trusty, Fedora 19/20 or later **

Change-Id: Ifec850ba8e1e5263234ef428669150c76cfdb6ad
Implements: blueprint implement-ceph-backend
Signed-off-by: Sébastien Han <sebastien.han@enovance.com>
2014-07-23 16:13:45 +02:00
Jenkins
f3939bf077 Merge "Use correct path when extracting tar.gz image" 2014-06-19 22:47:39 +00:00
Greg Lucas
50e53925e1 Use correct path when extracting tar.gz image
Fix issue with file:/ image urls.

Change-Id: I8c5f5b27b5d6ebe5be84dc58d37752adbd12f89b
Closes-Bug: 1277609
2014-06-09 12:06:40 -04:00
Ian Wienand
54e3910f2d Add basic echo_* functions for grenade
Grenade, and possibly other components, re-use parts of devstack
libraries.  This provides basic echo_* functions to avoid missing
commands in several libraries.

Change-Id: I972d6208055a8c4ca4c9bfd5383ad3c7159d5cbe
2014-06-03 17:16:38 +10:00
Bob Ball
f1a2dbffe8 XenAPI: Cirros images must always boot as PV.
The default for VHD disk-types is PV, which is why booting from a
server works.  However, creating a volume from the image needs to
pass this parameter on to the volume.  Note that
Id673158442fde27e8d468ca412c9bd557a886e6b is also required to fix
bug 1294069

Change-Id: I7ea1d85d6082787ac4551f78300a04bf59074261
Partial-Bug: 1294069
2014-03-19 15:06:57 +00:00
Jenkins
911cc586ad Merge "Set correct default disk bus back to virtio on ppc64" 2014-03-17 23:41:39 +00:00
Rafael Folco
0b03e7acb8 Set correct default disk bus back to virtio on ppc64
virtio is supported and should be the default disk bus on Power to
take advantage of I/O performance drivers. This aligns with Nova
default bus values on PowerKVM. SCSI is the default for cdrom.

Change-Id: I5de08c90359b3a500c352c09c07b6b082ddb4325
2014-03-14 11:20:10 -03:00
Sreeram Yerrapragada
9c6d2840fd fix failing wget statements under -o errexit in vmdk upload routine
Fix the case when uploaded image has no descriptor.
Refactored the code a bit

Tested:
1. monithic Sparse
2. monolithic flat
    2.1 flat file name mentioned in descriptor file
    2.1 flat file name not mentioned in descriptor file
3. descriptor header not found in the file
    3.1 image file name is *-flat, download descriptor
    3.2 image file name does not end with *-flat
4. file name contains all image properties

Change-Id: I0df9be5c2a1b9ed53cdb22d5cd40b94e56c48f37
Closes-bug: #1289664
2014-03-12 10:35:19 -07:00
Ryan Hsu
99b622a936 Refactor vmdk upload code
A syntax error is hit when trying to upload a flat vmdk file that
is accompanied by a descriptor file. The code block that handles
this has some unneeded characters that cause the error. Also,
an else-block has been removed so that we can remove an extra
indent.

Change-Id: Iaf5c914e09da6831eeeec141228b39554a1e2216
Closes-bug: #1288471
2014-03-05 15:57:16 -08:00
Sreeram Yerrapragada
314af0a7a9 Fix upload function for vmdk files
Fix all grep statements failing under -o errexit.

Change-Id: I0591a2ba7351d598eb5b29d68a83ce6290600938
2014-03-03 21:56:09 -08:00
Jenkins
5894db323f Merge "Move some comments of variables to right place" 2014-03-01 13:11:14 +00:00
Masayuki Igawa
d20f632a70 Move some comments of variables to right place
setup_develop*() in functions has been moved to functions-common. But
some comments about the variables are still left.
This commit moves it to the right place.

Change-Id: Ic360454f1ee72f51c9979d0468dee0913e9b32e4
2014-02-28 10:35:49 +09:00
Ian Wienand
aee18c749b Enforce function declaration format in bash8
Check that function calls look like ^function foo {$ in bash8, and fix
all existing failures of that check.  Add a note to HACKING.rst

Change-Id: Ic19eecb39e0b20273d1bcd551a42fe400d54e938
2014-02-28 07:59:03 +11:00
Dean Troyer
af616d9341 Move setup_develop() to common
It's in the wrong place for current Grenade

Change-Id: Ia670198332af5945a56d708cd83d9239df0c2287
2014-02-24 10:38:18 -06:00
Dean Troyer
d8864feae9 Fix shocco errors
Clean up comments to fix errors seen while processing with shocco

Change-Id: I0e97ad27613313f03e47c107051ea93b115d4744
2014-02-17 11:18:12 -06:00
Dean Troyer
dff49a242e Split functions
Move shared and non-DevStack-specific functions to `functions-common`.  This is
a code move only with some updated comments.  The functions are now
sorted alphabetically within function groups, eg. all git-related functions
are grouped together.  The groups are listed at the top of the file.

'functions' sources 'functions-common' so no additional changes are required
for backward-compatability.

All functions shared with Grenade have also been moved.

functions-common was created from commit e0ed8ea038299952826b27a16753775472f108d8

Change-Id: I73bf7134fd6a60ec1ea44a5bfab08b0569b60ded
2014-02-12 12:10:11 -06:00
Dean Troyer
e4fa721322 Begin is_service_enabled() cleanup
This converts the special cases in the is_service_enabled() function to call
individual functions declared by the projects.  This allows projects that
are not in the DevStack repo and called via the extras.d plugin to handle
an equivalent service alias.

* Ceilometer
* Cinder
* Glance
* Neutron
* Nova
* Swift

TODO: remove the tests from is_service_enabled() after a transition period

Patch Set 2: Rebased

Change-Id: Ic78be433f93a9dd5f46be548bdbd4c984e0da6e7
2014-02-07 10:06:21 -06:00
Arnaud Legendre
8dad4bde88 upload_image.sh to support streamOptimized disks
The current version of the script will use "preallocated" as the
disk type of a stream optimized disk. This needs to be fixed by
introspecting the createType of the vmdk file.

Closes-Bug: #1275993

Change-Id: I98594acecf26dd1164870f43890254a19ef23fe9
2014-02-03 18:17:01 -08:00
Jenkins
bfbcef0887 Merge "Make sure not to revert local changes." 2014-01-26 02:43:31 +00:00
Jenkins
58cc03629f Merge "upload_image.sh should parse filenames correctly" 2014-01-26 02:33:34 +00:00
Arnaud Legendre
b93cd64343 upload_image.sh should parse filenames correctly
The upload_image script gives the ability to the user to provide
specific metadata using the filename:
file-adapter_type;disk_type;network_type.vmdk
Currently, the regex expects each of these types to be populated.
This patch fixes this issue by making the regex more flexible and
accepts only one of these metadata to be populated.

Change-Id: If74cb06cc640864e7e91fd88943cdb37e05935d6
Closes-Bug: #1272126
2014-01-23 18:22:34 -08:00
Dean Troyer
579af5d678 Kill process groups in screen_stop()
Previously only the top child process was killed, killing the process
group also takes all of the child processes with it.

Closes-bug: 1271889
Change-Id: If1864cc4f1944f417ea3473d81d8b6e8e40030c2
2014-01-23 11:32:25 -06:00
IWAMOTO Toshihiro
0b8f6e0fbb Make sure not to revert local changes.
"git diff --quiet" has a bug ignoring local changes if there's a
unchanged file with a newer timestamp.  This patch works around
the bug.

Change-Id: I0ddc24e0f7af21287c43c1e04dd166ebff6f2dca
Closes-Bug: 1264422
2014-01-23 12:02:34 +09:00
Dean Troyer
9fc8792b0a Robustify service shutdown
* Save PID when using screen in screen_it()
* Add screen_stop()
* Call out service stop_*() in unstack.sh functions so screen_stop()
  can do its thing

Closes-bug: 1183449
Change-Id: Iac84231cfda960c4197de5b6e8ba6eb19225169a
2014-01-11 11:46:19 -06:00
Isaku Yamahata
6681a4fae9 bash8: fix bash8 warning
This patch removes the following bash8 warnings.

> /devstack/ $ ./run_tests.sh
> Running bash8...
> E003: Indent not multiple of 4: '             wget -c $image_url -O $FILES/$IMAGE_FNAME'
>  - functions: L1367
> E003: Indent not multiple of 4: '             if [[ $? -ne 0 ]]; then'
>  - functions: L1368
> E003: Indent not multiple of 4: '                 echo "Not found: $image_url"'
>  - functions: L1369
> E003: Indent not multiple of 4: '                 return'
>  - functions: L1370
> E003: Indent not multiple of 4: '             fi'
>  - functions: L1371
> E003: Indent not multiple of 4: '                                 `"should use a descriptor-data pair."'
>  - functions: L1423
> E003: Indent not multiple of 4: '                         `" Attempt to retrieve the *-flat.vmdk: $flat_url"'
>  - functions: L1438
> E003: Indent not multiple of 4: '                             `" Attempt to retrieve the descriptor *.vmdk: $descriptor_url"'
>  - functions: L1477
> E003: Indent not multiple of 4: '                         warn $LINENO "Descriptor not found $descriptor_url"'
>  - functions: L1492
> E003: Indent not multiple of 4: '                         descriptor_found=false'
>  - functions: L1493
> E003: Indent not multiple of 4: '                 fi'
>  - functions: L1501
> E003: Indent not multiple of 4: '             fi'
>  - functions: L1502
> E003: Indent not multiple of 4: '             #TODO(alegendre): handle streamOptimized once supported by the VMware driver.'
>  - functions: L1503
> E003: Indent not multiple of 4: '             vmdk_disktype="preallocated"'
>  - functions: L1504
> 14 bash8 error(s) found

Change-Id: Icf2cddf283192a50253ccfa697c2d32eec75b4ba
Closes-Bug: #1267716
2014-01-10 15:39:12 +09:00
Ian Wienand
be2ff9a745 Match RHEL Beta release strings
RHEL7 (in beta) has "Beta" after the release number and before the
code-name.  Add a number-match to the regex so everything between that
match and the codename in parenthesis is considered the release.

Change-Id: I992f20eedcefc6aeed6bd3ad57fc4cf20c8ef15d
2013-12-17 16:26:21 +11:00
Jenkins
6ef165cd7a Merge "Make rejoin-stack.sh resume file logging if enabled" 2013-12-17 00:40:02 +00:00
Andrea Frittoli
cd7d956fbc Handle the case of pipe char in value for iniset
iniset did not handle the case of "|" in the value
to be injected. Fix this by replacing | with \000 (NULL).

Fixes bug #1258050

Change-Id: I8882c2f3f177ebdfa0c66270dbbc7fd50f30b065
2013-12-12 12:24:55 +00:00
Dean Troyer
2ac8b3f3c2 Fix a couple of INI whitespace bugs
* iniset() bails if no section or option (attribute) is supplied
* merge_config_file() properly skips lines with only whitespace

* Also split the ini-tests into their own script

Bug 1257954

Change-Id: Ie31c5bd0df8dfed129fbcf1e37228aaf25e9305d
2013-12-09 10:30:02 -06:00
Darragh O'Reilly
bf36e8e4cf Make rejoin-stack.sh resume file logging if enabled
This patch ensures that screen will resume logging to files
after rejoin-stack.sh when SCREEN_LOGDIR is set.

Change-Id: I4c3eae0df7755b700dd8acf4bf14b7e383372ca3
Closes-bug: 1192568
2013-12-09 13:34:51 +00:00
Jenkins
b20d4d124e Merge "Generate glance image with SCSI bus type for ppc arch" 2013-12-05 18:02:04 +00:00
Rafael Folco
ab77587a37 Generate glance image with SCSI bus type for ppc arch
This patch fixes wrong assumptions for bus types of disk and cdrom
on ppc64. Qemu driver assumes IDE bus type for cdrom device, which
is not supported on ppc arch.
Adds capability to add --property key-value to the glance
image-create command.
Using double brackets for portability reasons.

Change-Id: I9f55fa0b6a894a93926e4f8c3d0ea410b5283f9c
2013-12-04 16:52:31 -02:00
Arnaud Legendre
90bcd2ff4d Attempt to retrieve the vmdk descriptor data-pair
VMDK formats such as monolithicFlat and vmfs require two files to be
fully consumable by the Nova drivers (a descriptor-data pair: *.vmdk and
*-flat.vmdk).
On the upload of the descriptor (*.vmdk), upload_image.sh should attempt to
retrieve the *-flat.vmdk. The same way, the descriptor should be
retrieved when a flat disk is uploaded.
On success, the upload script will be able to use the flat disk as the image
content and the relevant descriptor settings as the image metadata.

Change-Id: I9214754029c46dd60b9e7d606d84d8819a498a8d
Closes-Bug: #1252443
2013-12-03 17:57:24 -08:00
Jenkins
1fe3fc0457 Merge "Update vsphere image filename pattern" 2013-11-22 16:19:01 +00:00