106 Commits

Author SHA1 Message Date
Ian Wienand
ada886dd43 Don't mix declaration and set of locals
Ia0957b47187c3dcadd46154b17022c4213781112 proposes to have bashate
find instances of setting a local value.  The issue is that "local"
always returns 0, thus hiding any failure in the commands running to
set the variable.

This is an automated replacement of such instances

Depends-On: I676c805e8f0401f75cc5367eee83b3d880cdef81
Change-Id: I9c8912a8fd596535589b207d7fc553b9d951d3fe
2015-10-07 17:03:32 +11:00
Sean Dague
346edcc532 check all possible services when configuring tempest
The previous approach assumed that devstack in tree service support
would always be a super set of tempest. That's not necessarily
true. Instead when configuring tempest we should look at all the
possible services that tempest could know about, which will let us
disable services we don't have support for.

Change-Id: I9c24705e494689f09a885eb0a640efd50db33fcf
2015-08-26 09:38:37 -04:00
Jamie Lennox
19c5e62a04 Configure glance swift communication with v3
With the aim of moving everything fully over to v3 authentication we
need to configure glance_store to use v3 when calling swift.

Requires glance_store 0.8.0

Change-Id: I61e8c5a4136404077f5505ebc2edfe49841c244f
Implements: bp keystonev3
2015-08-13 00:11:13 +00:00
Jenkins
aaae79b095 Merge "Use identity V3 API for endpoint creation" 2015-07-14 04:53:34 +00:00
Jenkins
410949801e Merge "Glance: move connection_url from DEFAULT section to DB section" 2015-07-09 03:32:39 +00:00
Jamie Lennox
b17ad7560d Use identity V3 API for endpoint creation
Always use the keystone V3 API when creating services and endpoints. The syntax
here is slightly different but we maintain the function interface.

Change-Id: Ib3a375918a45fd6e37d873a1a5c0c4b26bdbb5d8
Implements: bp keystonev3
2015-07-08 05:17:00 +10:00
Brian Haley
180f5eb652 Add IPv6 support to devstack infrastructure
By default, most Openstack services are bound to 0.0.0.0
and service endpoints are registered as IPv4 addresses.
With this change we introduce two new variables to control
this behavior:

SERVICE_IP_VERSION - can either be "4" or "6".

When set to "4" (default if not set) devstack will operate
as today - most services will open listen sockets on 0.0.0.0
and service endpoints will be registered using HOST_IP as the
address.

When set to "6" devstack services will open listen sockets on ::
and service endpoints will be registered using HOST_IPV6 as the
address.

There is no support for "4+6", more work is required for that.

HOST_IPV6 - if SERVICE_IP_VERSION=6 this must be an IPv6
address configured on the system.

Some existing services, like the Openvswitch agent, will continue
to use IPv4 addresses for things like tunnel endpoints.  This is
a current restriction in the code and can be updated at a later
time.  This change is just a first step to supporting IPv6-only
control and data planes in devstack.

This change is also partly based on two previous patches,
https://review.openstack.org/#/c/140519/ and
https://review.openstack.org/#/c/176898/

Change-Id: I5c0b775490ce54ab104fd5e89b20fb700212ae74
Co-Authored-By: Sean Collins <sean@coreitpro.com>
Co-Authored-By: Baodong Li <baoli@cisco.com>
Co-Authored-By: Sridhar Gaddam <sridhar.gaddam@enovance.com>
Co-Authored-By: Adam Kacmarsky <adam.kacmarsky@hp.com>
Co-Authored-By: Jeremy Alvis <jeremy.alvis@hp.com>
2015-07-02 15:16:34 -06:00
Jenkins
7a0f6833ad Merge "remove non RabbitMQ messaging" 2015-07-02 20:09:05 +00:00
Jenkins
7bc1f3c15f Merge "Use Keystone v3 API for user creation" 2015-06-30 17:15:58 +00:00
Jordan Pittier
ceca15db4d Glance: move connection_url from DEFAULT section to DB section
To get rid of Oslo_config deprecation warning.

Change-Id: I9de475f4dea4a4496cc8b5e93aa6928235d4cb29
2015-06-19 11:46:36 +02:00
Jamie Lennox
f4f01c6397 Use swift store config files in glance
Using the swift_store_auth_address, swift_store_user and swift_store_key are
marked as deprecated in glance in favour of using a standalone config file that
provides multiple auth options.

Create and use a standalone authentication file for communicating with swift.

Change-Id: I9b5361ce6e1771781d7ae7226974604a7f9e5d00
2015-06-19 11:10:23 +08:00
Sean Dague
37eca48970 remove non RabbitMQ messaging
Part of what was decided at summit is devstack needs to return to a
more opinionated stance, the following removes support for non
RabbitMQ messaging. RabbitMQ is used by over 95% of our community
(statistically all of it), so it's a pretty clear line to draw that
this shouldn't be in tree.

iniset_rpc_backend will be our stable hook for other projects that
want to implement this out of tree. The burden on creating those out
of tree plugins will be on those that wish to support those
alternative stacks.

Change-Id: I8073a895c03ec927a2598eff6c2f01e5c82606fc
2015-06-16 08:28:52 -04:00
Jamie Lennox
9d7e776b70 Use Keystone v3 API for user creation
This includes requiring a domain when creating a user. This will allow us to
control where users are created in a later patch.

Adding the token to the user creation call is required because of a bad
interaction between OpenStackClient, os-client-config and keystoneclient
when dealing with v2 authentication but v3 API calls. It will be cleaned
up when we switch to v3 credentials.

Change-Id: I6ef50fd384d423bc0f13ee1016a8bdbb0650ecd9
Implements: bp keystonev3
2015-06-16 10:06:39 +08:00
Rob Crittenden
6254d5fd0d Fix nova and glance discovery URLs when tls-proxy is enabled.
Retrieving the root page in the nova and glance APIs include
URLs for the various versions supported. These are by default
reported using unsecure URLs. Configure the services to report
a SSL-based URL instead.

Change-Id: I220757e53b94a5f6d19291371407220fdf54c645
2015-06-05 13:56:27 -04:00
Louis Taylor
cbe12eb72e glance: remove deprecated store options
glance_store has now been fully migrated, so we can remove these from the
config files.

Change-Id: I987ab6338b235f0beeed7c7fe74b0f5b6526f70d
2015-05-12 16:49:49 +00:00
Jenkins
c9197f6c72 Merge "Acknowledge API_WORKERS in glance-registry" 2015-04-24 20:25:23 +00:00
Wayne Okuma
dd62293591 Catalog Index Service - glance devstack
Implements: blueprint catalog-index-service

The changes to lib/glance incorporate the new g-search service.
The g-search service is optional.
To enable it add the following line to devstack/local.conf:
enable_service g-search

In addition to deploying g-search, the changes will also populate a
search type of keystone service and adds in appropriate endpoints.

Change-Id: I0272d56bc2e50e8174db78bd449f65f60f7f4000
2015-04-16 15:33:09 -07:00
Lianhao Lu
37b779c3bc Acknowledge API_WORKERS in glance-registry
Change-Id: Ifaf671439480719255c07673b54dc49c0c2ca4f6
Closes-Bug: #1444231
2015-04-15 10:27:06 +08:00
Sean Dague
ba1c56bf5e remove extraneous +x bits from lib files
2 files had execute bits set on them, fix as a cleanup. These files
should not be directly executed.

Change-Id: Ic0fdb85d77a3b47ef777524faf4fcdb0d8cedece
2015-04-01 07:40:10 -04:00
Jenkins
0e348aaeec Merge "Install Glance into its own venv" 2015-03-26 20:36:49 +00:00
Jenkins
743d03bf7d Merge "iniset_rpc_backend default section" 2015-03-22 02:41:03 +00:00
Dean Troyer
aed6079205 Install Glance into its own venv
Change-Id: Ib46b89dafa1fc81a2d0717150203b848b87ea323
2015-03-20 10:47:41 -05:00
Jenkins
05163d66ac Merge "Use install(1) where possible" 2015-03-20 14:30:06 +00:00
Dean Troyer
8421c2b9ab Use install(1) where possible
This eliminated a number of sudo calls by doing the copy/chown/chmod in
a single step and sets a common pattern.

Change-Id: I9c8f48854d5bc443cc187df0948c28b82c4d2838
2015-03-18 13:30:59 +11:00
Anand Shanmugam
1577663f4a Fix Traceback exceptions in g-api and g-reg
The log_context format specified in glance-api and glance-reistry
are causing tracebacks.  This fix changes the
project_id and user_id to tenant and user which are supported
in oslo-context.  This is the format used by other projects

Change-Id: Ifbf268e9765039a0085c9af930dabf8a5cc681b8
Closes-Bug: #1431784
2015-03-15 08:51:55 +00:00
Brant Knudson
2dd110ce86 iniset_rpc_backend default section
iniset_rpc_backend should know what section it needs to set the
config options in better than the callers. The config options
have actually been moved to different sections and the options
in the DEFAULT section are deprecated.

Change-Id: I0e07fe03c7812ef8df49e126bf71c57588635639
2015-03-14 17:53:54 +00:00
Louis Taylor
701276a800 Enable colorized logging for glance
Extensive QA has found this makes glance 100% more fabulous.

Change-Id: I1cf94096d6bd5033ef0f9e3ed2a77108677581f0
2015-02-11 19:37:57 +00:00
Jamie Lennox
9b215db569 Rename get_or_add_user_role
get_or_add_user_role is specific to adding a role on a project.
Rename it to get_or_add_user_project_role to allow room for adding a
domain specific role function.

Change-Id: I999308098d22be9800578ae67144a3b687fbc3be
2015-02-11 14:51:22 +11:00
Wayne Okuma
00d0da36af Adds elasticsearch support as a pre-req for the glance-index.
Add elastic search pkg installation and start into the glance install
and start paths.

Change-Id: I53fc37225dd606f627c9f967083007613eb1f1bb
Implements: blueprint catalog-index-service
2015-02-09 11:35:00 -08:00
Jamie Lennox
85ff5323ff Isolate creating service users
The code for creating service users is almost exactly the same. Abstract
this into a function that can be reused and standardized.

Change-Id: I3a4edbff0a928da7ef9b0097a5a8d508fdfab7ff
2015-02-09 14:36:20 +11:00
Jamie Lennox
18f39bfb1f Remove the default project from all users
The default project means that a user gains token scoping information
for a project if they don't specify another. This is something we want
to discourage for user creation. User's should specify there own
authentication scope when they authenticate.

Change-Id: I42c3060d59edfcd44d04cd166bad500419dd99bc
2015-01-28 13:38:32 +10:00
Ihar Hrachyshka
157c84b876 Enforce UTF-8 encoding when creating databases
We use InnoDB everywhere, so there should be no issues with long unicode
keys. Dropped charset parameter for recreate_database since it's not
needed anymore.

Change-Id: Ib768402a9337c918309030a92ab81da17269f4f6
2015-01-05 15:57:23 +01:00
Jenkins
d09a1db5a4 Merge "add shebang lines to all lib files" 2014-12-10 20:32:15 +00:00
Sean Dague
e263c82e48 add shebang lines to all lib files
With gerrit 2.8, and the new change screen, this will trigger syntax
highlighting in gerrit. Thus making reviewing code a lot nicer.

Change-Id: Id238748417ffab53e02d59413dba66f61e724383
2014-12-10 11:28:05 -05:00
Evgeny Antyshev
19354585e1 libvirt plugin "parallels" support
We're in the process of establishing the CI testing with
Parallels Cloud Server plugin for libvirt.
Currently we use local clone of devstack in our infrastructure,
but we want to switch to upstream

requires extra customization of nova.conf and glance-api.conf:
nova.conf should have "images_type" and "connection_uri"
defined, and glance-api.conf to have "ploop" in disk formats

Implements: blueprint pcs-support
Change-Id: I0b073359fda8b4531cae7b8946eb52a561c82857
2014-11-26 19:04:28 +04:00
Jenkins
e50db39baa Merge "Install bash completion for glanceclient" 2014-11-25 21:01:45 +00:00
Sean Dague
aecd189f80 add unit tests for GIT* definitions
This adds unit tests for all the GIT* definitions, ensuring that for
libraries we think should be defined, they are. It exposed a bug in
glance_store definitions in the process.

The GITDIR definition for python-openstackclient is moved to stackrc
for testability.

Change-Id: Ibd9ab665f0362a84f4f7e80e80da56a4737f584e
2014-11-20 08:32:26 -05:00
Louis Taylor
8df690c525 Install bash completion for glanceclient
This installs the new bash-complete file in the standard location,
alongside the rest of the completion files:

    /etc/bash_completion.d/glance.bash_completion

Change-Id: I2944c47d857f7125f370b2b1ee041d1282db09c5
2014-11-20 13:16:46 +00:00
Sean Dague
ee5ae7b923 further pypi libs conversions
glance_store and saharaclient were missed in the last round, make
them install from released clients in this patch.

Change-Id: I2e0ed2b7fb5994ae6abd92612a67ce5fd82b3f7e
2014-11-13 17:17:45 -05:00
Sean Dague
e08ab104e6 fix python-* lib from git installation
We're using all the magic variables based on python-fooclient, however
all the inline code was using fooclient for variables. So we had a
mismatch, which was kindly pointed out by some of the 3rd party ci
testers.

Change-Id: I27a56222c7e8e610fba8bf97672d2a42f5cf14ca
2014-11-13 17:17:38 -05:00
Sean Dague
5cb190697c support installing clients at released versions
expand the devstack support for libraries from released versions to
support python-* clients and tempest_lib.

Depends-On: I81b0d228e7769758c61e5b0323ecfce8c8886d39

Change-Id: I26fac0ccf8fd4818e24618d56bf04b32306f88f6
2014-11-04 10:18:58 +01:00
Rob Crittenden
18d4778cf7 Configure endpoints to use SSL natively or via proxy
Configure nova, cinder, glance, swift and neutron to use SSL
on the endpoints using either SSL natively or via a TLS proxy
using stud.

To enable SSL via proxy, in local.conf add

ENABLED_SERVICES+=,tls-proxy

This will create a new test root CA, a subordinate CA and an SSL
server cert. It uses the value of hostname -f for the certificate
subject. The CA certicates are also added to the system CA bundle.

To enable SSL natively, in local.conf add:

USE_SSL=True

Native SSL by default will also use the devstack-generate root and
subordinate CA.

You can override this on a per-service basis by setting

<SERVICE>_SSL_CERT=/path/to/cert
<SERVICE>_SSL_KEY=/path/to/key
<SERVICE>_SSL_PATH=/path/to/ca

You should also set SERVICE_HOST to the FQDN of the host. This
value defaults to the host IP address.

Change-Id: I36fe56c063ca921131ad98439bd452cb135916ac
Closes-Bug: 1328226
2014-09-24 18:36:37 -04:00
Brant Knudson
0595237e8a Function for auth_token middleware config
Each project was configuring the auth_token middleware using several
lines of inisets. Since all the projects should configure the
auth_token middleware in the same way create a function and call it.

Change-Id: I3b6727d5a3bdc0ca600d8faa23bc6db32bb32260
2014-09-21 11:18:01 -05:00
Dean Troyer
05bd7b803d Set default API_WORKERS
Set the API_WORKERS default to control memory usage. Maximum is nproc / 2 and
minimum is 2.

* Also updates https://review.openstack.org/#/c/117517/ to remove the
  conditional test as API_WORKERS should always be set.
* Update https://review.openstack.org/#/c/109058/ for ceilometer to use API_WORKERS

The following reviews can move forward either as-is or with minor tweaks:
* Keystone: https://review.openstack.org/#/c/121384/ - remove the if check
* Swift: https://review.openstack.org/#/c/121456/ - unabandon, the default
  to 1 is fine, or remove it to match the others.
* Trove: https://review.openstack.org/#/c/121438/ - remove the if check

https://etherpad.openstack.org/p/devstack-workers has the details

Change-Id: Id28d72ebf01c88b7df301edf7d1dd7ec23fcd0d6
2014-09-19 09:06:21 -05:00
Jenkins
efa18c73ab Merge "Replace screen_it() with run_process() throughout" 2014-09-13 12:38:34 +00:00
Jenkins
d577fdc794 Merge "Allow setting the number of workers to be used." 2014-09-13 07:18:05 +00:00
Jenkins
2b9acae9f2 Merge "Test against latest glance_store code" 2014-09-12 01:07:23 +00:00
Chris Dent
2f27a0ed3c Replace screen_it() with run_process() throughout
run_process will use screen if USE_SCREEN=True (the default),
otherwise it will simply start the requested service. Therefore
wherever screen_it used, run_process can be instead.

Where stop_screen was found it has been replaced with stop_process.

A tail_log function has been added which will tail a logfile in a
screen if USE_SCREEN is True.

lib/template has been updated to reflect the use of the new
functions.

When using sg the quoting in run_process gets very complicated.
To get around this run_process and the functions it calls accepts
an optional third argument. If set it is a group to be used with sg.

Change-Id: Ia3843818014f7c6c7526ef3aa9676bbddb8a85ca
2014-09-11 18:59:39 +01:00
Bob Ball
2f72050ace Allow setting the number of workers to be used.
Environments with large numbers of CPUs will create a large
number of workers which can have an unnecessarily large
impact on memory usage, particular where you know how many
workers are needed.

Change-Id: Ie4bb075310a61a0873c9e56e4974600dbb4794a1
2014-09-09 15:54:36 +01:00
Flavio Percoco
4f78f8f391 Test against latest glance_store code
Install glance_store from git so we can test Glance against master. This
is useful for both, glance and glance_store gates, to make sure nothing
is broken there.

Change-Id: I6c01165c4384c41f46f2c32d64475703b3178dab
2014-09-09 09:37:42 +02:00