192 Commits

Author SHA1 Message Date
Gregory Haynes
4b49e409f8 Use apache for tls-proxy ssl termination
Stud is now abandonware (see https://github.com/bumptech/stud) and is
not packaged in xenial. Lets use Apache for SSL termination since its
there already.

Change-Id: Ifcba410f5969521e8b3d30f02795541c1661f83a
2016-09-20 08:14:11 -07:00
Jenkins
5fc597289b Merge "Enable Swift Account Management" 2016-06-15 12:48:09 +00:00
Brant Knudson
e86b91ba78 Reference keystone URIs rather than building URI
lib/keystone sets KEYSTONE_AUTH_URI and KEYSTONE_SERVICE_URI that
other projects should use rather than building the URL themselves.
This will allow us to more easily drop the port altogether.

Change-Id: I7467aae680215f3045d32a088af2187e1eba8169
2016-05-19 21:44:43 +00:00
Brian Ober
958c169d79 Enable Swift Account Management
This patch enables account management by default in Swift.  This will be
leveraged by Tempest test cases validating account management APIs.

Depends-On: Id29f5ca48f92cd139535be7064107b8a61b02856
Change-Id: Ic01432939ed9b4cf0cbf20e3244d4d76847f539f
2016-05-16 23:00:43 +00:00
Matt Riedemann
ae4578bed9 Revert "Don't check for existing endpoints"
This reverts commit 7d1ec430046256ac3f536f4920a43936e28ef0c4.

This broke the sahara and layer4 dsvm jobs. The layer4 job
is voting on tempest changes so tempest is also broken.

Change-Id: Ide69f10cd85bf7ff0d86bc8cba56dedd26850362
Partial-Bug: #1573868
2016-04-23 01:45:40 +00:00
Sean Dague
7d1ec43004 Don't check for existing endpoints
We really should only have code that create endpoints once, making all
osc calls get_or_set adds 3 seconds per call for no really good
reason.

This also stops creating the internal endpoints in the service
catalog. It's a pattern that we're trying not to propogate, so lets
not have it in devstack any more.

Change-Id: Ia8cefe43753900d62117beae330db46deb6a9fc9
2016-04-22 10:44:10 -04:00
Jamie Lennox
cbcbd8f33c Allow putting service users in a seperate domain
Make it possible to construct the service users in their own seperate
domain. Changing this away from Default will not work for everyone yet,
though it does work for basic service interaction however enabling it
will allow us to start testing and hopefully gating that services aren't
relying on v2 only concepts.

Change-Id: I7e73df5dd1caabf355783da2bc0f3007ade92fba
2016-04-18 09:59:52 +10:00
Jenkins
19c7842e4f Merge "Add variable SWIFT_STORAGE_IPS" 2016-04-14 19:50:17 +00:00
Christian Schwede
91d2245d6e Add variable SWIFT_STORAGE_IPS
If the variable SWIFT_STORAGE_IPS contains a space-separated list of
IPs, we can use this to create consistent rings across all proxy and
storage nodes.

Change-Id: If9307196dc7e74e4a842c95503958ae2d7f7acc7
2016-04-14 14:16:55 +02:00
Sean Dague
19dfa2af32 convert tenant -> project for internal variables
This sets all the internal variables and service users that are set to
use project instead of tenant for clarity.

Change-Id: I4aa833bac2ee2281c5f2881f7ae1fd8e7c759f74
2016-04-05 11:40:25 -04:00
Sean Dague
260583e25e Switch over swift to using $(project_id) in service catalog
This removes one more place where we use tenant_id in our code.

Change-Id: I628fb7c1dd6f22ce499fc5cbe97b79458cce966f
2016-04-05 11:58:21 +00:00
Matthew Oliver
7b85723c04 Set the Swift max_file_size to 1/2 the loopback
Currently Devstack (and devstack gate) uses the default
max_file_size of for Swift (5GB). However the loopback file
is only 1 or 2 GB is size.

We are looking at setting a default FALLOCATE_RESERVE in swift
https://review.openstack.org/#/c/288011.
Because of this our max file size test fails due to the max_file_size
and devstacks loopback size being too small.

This patch sets this to a more sane size by default inside DevStack.
See the gerrit link above for more details.

Tempest uses a loopback of 6GB, so in this case the swift default of 5GB
should be fine.

Change-Id: If09eab7d16ae67bd252020e00e8812ff252f065b
2016-03-08 12:04:15 +11:00
Sean Dague
7580a0c3e3 Replace TENANT => PROJECT phase 1
This replaces the use of TENANT variables with PROJECT ones during the
initial setup. The openrc will still export a OS_TENANT_NAME because
many tools (cinderclient, glanceclient amoung them) will not function
without it. We warn when we do that.

Change-Id: I824b1121842eb5821034071874bf1bb2d7c3631e
2016-02-22 11:55:17 +11:00
Sean Dague
985e958dda remove keystone templated catalog
This is just another code path for little benefit in devstack which is
going to rot out. We should be opinionated here and only support the
dynamic catalog.

Change-Id: I4e5c7e86aefe72fc21c77d423033e9b169318fec
2016-02-10 15:39:31 -05:00
Dmitry Tantsur
f90c0b7d29 Stop referencing no longer existing SERVICE_ENDPOINT variable
It was probably finally removed by one of recent refactoring changes.
Now ironic gate fails due to Swift trying to access it to override OS_AUTH_URL.
This change drops this override and just uses OS_AUTH_URL.

Closes-Bug: #1535245
Change-Id: I145bec110c4299e61f2bce49df41dcd82e5d462d
2016-01-18 10:53:01 +01:00
Ian Wienand
523f488036 Namespace XTRACE commands
I noticed this when debugging some grenade issues failures.

An include of grenade/functions stores the current value of XTRACE
(on) and disables xtrace for the rest of the import.

We then include devstack's "functions" library, which now overwrites
the stored value of XTRACE the current state; i.e. disabled.

When it finishes it restores the prior state (disabled), and then
grenade restores the same value of XTRACE (disabled).

The result is that xtrace is incorrectly disabled until the next time
it just happens to be turned on.

The solution is to name-space the store of the current-value of xtrace
so when we finish sourcing a file, we always restore the tracing value
to what it was when we entered.

Some files had already discovered this.  In general there is
inconsistency around the setting of the variable, and a lot of obvious
copy-paste.  This brings consistency across all files by using
_XTRACE_* prefixes for the sotre/restore of tracing values.

Change-Id: Iba7739eada5711d9c269cb4127fa712e9f961695
2015-11-27 15:36:04 +11:00
Jenkins
fa8464bf34 Merge "Fix typos for stack.sh and lib of comments and message" 2015-11-18 02:20:45 +00:00
Atsushi SAKAI
fe7b56cdef Fix typos for stack.sh and lib of comments and message
Fix 10 comments and 1 message

stack.sh
  Certicate => Certificate (comment)
lib/stack
  Sentinal => Sentinel (comment)
lib/neutron-legacy
  overriden => overridden (comment)
  necesssary => necessary (comment)
  notifiy => notify (message)
  notifations => notifications (comment)
lib/rpc_backend
  orginal => original (comment)
  cofiguration => configuration (comment)
lib/stack
  confgured => configured (comment)
lib/swift
  additinal => additional (comment)
  calclution => calculation (comment)
  maximun => maximum (comment)

Change-Id: I3637388b67decb007cd49af9addecc654009559b
2015-11-13 17:28:05 +09:00
Steve Martinelli
a91d455e23 change swift post to openstackclient cli command
with the release of osc 1.8.0, modifying object store account
properties is now available. use this mechanism and avoid setting
environment variable that are only helpful for swift CLI.

Change-Id: Ie51e3e2bb86162763f23d0a6bed36208811f89fc
2015-11-03 20:07:58 +11:00
Jenkins
310cedf677 Merge "Use swift port variable in keystone and cinder" 2015-10-29 04:57:22 +00:00
Falk Reimann
22f747b8df Use swift port variable in keystone and cinder
This patch alows specifiying a deviation of the swift default port 8080 with
variable SWIFT_DEFAULT_BIND_PORT. The created endpoints in keystone for
object-store and the backup_swift_url in cinder.conf will use variable
SWIFT_DEFAULT_BIND_PORT instead of the fixed port 8080.

Change-Id: I47bbcf77368c430718fb8f29b7de1ff305e64422
Closes-Bug: #1489767
2015-10-23 09:19:32 +02:00
Jenkins
d111182f21 Merge "Don't mix declaration and set of locals" 2015-10-14 02:43:52 +00:00
Jenkins
17491f37d7 Merge "Revert "Revert "Convert identity defaults to keystone v3 api""" 2015-10-07 19:30:18 +00:00
Ian Wienand
ada886dd43 Don't mix declaration and set of locals
Ia0957b47187c3dcadd46154b17022c4213781112 proposes to have bashate
find instances of setting a local value.  The issue is that "local"
always returns 0, thus hiding any failure in the commands running to
set the variable.

This is an automated replacement of such instances

Depends-On: I676c805e8f0401f75cc5367eee83b3d880cdef81
Change-Id: I9c8912a8fd596535589b207d7fc553b9d951d3fe
2015-10-07 17:03:32 +11:00
Denis Afonso
dbe0868d11 Change the default bind_port for swift
The default bind_port (6011-6013) in the sample config files for swift
use port numbers that are in the range registered by X11 (6000-6063) and
can prevent swift from starting if the ports are in use.

We should use an unregistered range (6611-6613).

Change-Id: Ifd95b99004aead5ddc8ae1a8dd3ccd9c4f2abe91
Closes-Bug: #1254328
2015-10-02 23:51:41 -04:00
Steve Martinelli
050a0d5b30 Revert "Revert "Convert identity defaults to keystone v3 api""
This reverts commit f768787bdd6dddf2790f83a884618d29677ca77c.
And sets OS_AUTH_VERSION so swift CLI doesn't fall flat when
not using v2 keystone

Change-Id: If44a7e0d85e48020a3c90d8c5c027513129f0f3b
2015-09-29 19:52:28 +00:00
Christian Schwede
3e3212b52d Add trailing IDENTITY_API_VERSION to OS_AUTH_URL in swift_configure_tempurls
The python-keystoneclient requires a trailing /v<IDENTITY_API_VERSION> to
successfully authenticate, otherwise it fails with a 404 error due to a not
found resource.

This error showed up only when generating Swift tempurls, because the error was
raised when using python-swiftclient.

This change fixes this for python-swiftclient within devstack.

Change-Id: Ibe222d65162898db69acba076b5fe1cb3621fbc3
Closes-Bug: 1492216
2015-09-08 08:33:57 +02:00
Jenkins
f560df36cc Merge "check all possible services when configuring tempest" 2015-09-02 06:27:40 +00:00
Jenkins
46e6da4d0a Merge "Add region definition to swift3" 2015-09-01 18:19:31 +00:00
Sean Dague
346edcc532 check all possible services when configuring tempest
The previous approach assumed that devstack in tree service support
would always be a super set of tempest. That's not necessarily
true. Instead when configuring tempest we should look at all the
possible services that tempest could know about, which will let us
disable services we don't have support for.

Change-Id: I9c24705e494689f09a885eb0a640efd50db33fcf
2015-08-26 09:38:37 -04:00
Jenkins
c6646a650e Merge "Use standard get_or_create_role to find role" 2015-08-24 17:21:40 +00:00
Andrey Pavlov
9b21f98ce0 Add region definition to swift3
Swift3 should be in the same region as all other cloud.
By default it has regaion name 'US'. It's ok for requests
that signed by version 1 of signature because they haven't
region information in request.
But S3 signature of version 4 protocol sends region name to server
and swift3 plugin checks that input region equals to internal.
And because all cloud lives with 'RegionOne' then swift3 fails
request because it has 'US' region name by default.

Change-Id: Icd817183b1a040110372a8ae5d73fd2f0ec5559c
Related-Bug: #1411078
2015-08-20 23:37:04 +03:00
Marian Horban
ea21eb4f69 Remove non-ASCII characters
Change-Id: If1c68e5aab6990617519150d8aeb3f073df2ad17
2015-08-18 19:53:39 +00:00
Jamie Lennox
dbdee69870 Use standard get_or_create_role to find role
The openstack role list command doesn't include any identity API version
information and so will fail when running purely with v3. We could add
this information to the command however we already have a function that
does what swift requires so we should use it.

Change-Id: I5d5417eaed432760bfb97cf35bd76a0919c6004d
2015-08-14 12:22:18 +10:00
Rawlin Peters
92ad152512 Explicitly set bind_ip in Swift server config files
Currently, the Swift proxy, object, account, and container servers bind
to IPv4 address 0.0.0.0 by default. In the case of a user setting
SERVICE_IP_VERSION=6 in their local.conf file, these Swift servers still
listen on 0.0.0.0 instead of ::, which causes a ./stack.sh run to fail.

This change explicitly sets the bind_ip variable in the Swift server
config files so that the servers bind to either 0.0.0.0 (when
SERVICE_IP_VERSION != 6) or :: (when SERVICE_IP_VERSION == 6).

This patch is related to the following patch for devstack IPv6 support:
    https://review.openstack.org/#/c/192329

Change-Id: Ie268c6daf5374e67ef8710a731c3af50ffdb821e
2015-07-20 13:33:33 -06:00
Jenkins
aaae79b095 Merge "Use identity V3 API for endpoint creation" 2015-07-14 04:53:34 +00:00
Jenkins
ac3d3f336c Merge "Add IPv6 support to devstack infrastructure" 2015-07-08 12:44:51 +00:00
Jamie Lennox
b17ad7560d Use identity V3 API for endpoint creation
Always use the keystone V3 API when creating services and endpoints. The syntax
here is slightly different but we maintain the function interface.

Change-Id: Ib3a375918a45fd6e37d873a1a5c0c4b26bdbb5d8
Implements: bp keystonev3
2015-07-08 05:17:00 +10:00
Chris Dent
1848b837e6 Move ceilometermiddleware installation to lib/swift
lib/swift is where it is used so this makes the relationship more
clear and direct.

Change-Id: Ie6fc09e27a39295c92f0790856446edb7dedb995
2015-07-07 10:21:18 +10:00
Brian Haley
180f5eb652 Add IPv6 support to devstack infrastructure
By default, most Openstack services are bound to 0.0.0.0
and service endpoints are registered as IPv4 addresses.
With this change we introduce two new variables to control
this behavior:

SERVICE_IP_VERSION - can either be "4" or "6".

When set to "4" (default if not set) devstack will operate
as today - most services will open listen sockets on 0.0.0.0
and service endpoints will be registered using HOST_IP as the
address.

When set to "6" devstack services will open listen sockets on ::
and service endpoints will be registered using HOST_IPV6 as the
address.

There is no support for "4+6", more work is required for that.

HOST_IPV6 - if SERVICE_IP_VERSION=6 this must be an IPv6
address configured on the system.

Some existing services, like the Openvswitch agent, will continue
to use IPv4 addresses for things like tunnel endpoints.  This is
a current restriction in the code and can be updated at a later
time.  This change is just a first step to supporting IPv6-only
control and data planes in devstack.

This change is also partly based on two previous patches,
https://review.openstack.org/#/c/140519/ and
https://review.openstack.org/#/c/176898/

Change-Id: I5c0b775490ce54ab104fd5e89b20fb700212ae74
Co-Authored-By: Sean Collins <sean@coreitpro.com>
Co-Authored-By: Baodong Li <baoli@cisco.com>
Co-Authored-By: Sridhar Gaddam <sridhar.gaddam@enovance.com>
Co-Authored-By: Adam Kacmarsky <adam.kacmarsky@hp.com>
Co-Authored-By: Jeremy Alvis <jeremy.alvis@hp.com>
2015-07-02 15:16:34 -06:00
Attila Fazekas
f750a6fedb Already dead process tolerance in unstack.sh
The gate/updown.sh calls the unstack.sh with
-ex option. Normally we do not use -e with unstack.sh.

The unstack.sh can fail if the service already stopped,
and it also can have flaky failures on the gate.

For example the stop_swift function tries to kill swift in two
different ways, and if the first one succeeds before the 2th attempt
the pkill fails the whole unstack.sh.

This change accepts kill failure.
Normally the kill can fail if the process does not exits,
or when you do not have permission to the kill operation.

Since the permission issue is very unlikely in our case,
this change does not tries to distinguish the two operation.

The behavior of the unstack.sh wen you are not using -ex should
not be changed by this change.

Change-Id: I64bf3cbe1b60c96f5b271dcfb620c3d4b50de26b
2015-07-01 12:17:35 +02:00
Jamie Lennox
9d7e776b70 Use Keystone v3 API for user creation
This includes requiring a domain when creating a user. This will allow us to
control where users are created in a later patch.

Adding the token to the user creation call is required because of a bad
interaction between OpenStackClient, os-client-config and keystoneclient
when dealing with v2 authentication but v3 API calls. It will be cleaned
up when we switch to v3 credentials.

Change-Id: I6ef50fd384d423bc0f13ee1016a8bdbb0650ecd9
Implements: bp keystonev3
2015-06-16 10:06:39 +08:00
Jamie Lennox
b632c9ef81 Use keystone v3 API for projects
Always use the keystone v3 API for project creation. Make domain a
required argument. Whilst we could simply default this value within the
function I think it's better to make this explicit as these are things
deployers and services need to consider.

In future we will want to figure out how we want devstack to organize domains
however I don't believe that it belongs in this patch.

Change-Id: Ib9587193c5c8419dc4b5a608246709baaddd2a52
Implements: bp keystonev3
2015-06-16 08:38:09 +08:00
Cyril Roelandt
e8a2fa431b lib/swift: the s3_token middleware should be provided by keystonemiddleware
Recently, keystoneclient.middleware has been moved from keystoneclient to
keystonemiddleware. The latter should be used.

Change-Id: Ib9489a21b988b32fc17399c08eeb60862efae034
Closes-Bug: #1452315
2015-05-06 17:34:35 +02:00
Jenkins
d1549d624d Merge "Creates SWIFT_DATA_DIR if it does not exist" 2015-04-07 00:21:20 +00:00
Dean Troyer
dc97cb71e8 Mostly docs cleanups
Fix documentation build errors and RST formatting

Change-Id: Id93153400c5b069dd9d772381558c7085f64c207
2015-03-28 14:35:12 -05:00
Jenkins
4cc060e77c Merge "Increase Swift disk size up to 2GB if Glance is enabled" 2015-03-26 12:51:03 +00:00
Geronimo Orozco
2f6576bfaf Creates SWIFT_DATA_DIR if it does not exist
If SWIFT_DATA_DIR is set on local.conf we need to make sure to create
the directory with proper permissions

Change-Id: If29fa53f01b4c0c8a881ec3734383ecffac334ce
Closes-Bug: 1302893
2015-03-20 10:05:05 -06:00
Jenkins
05163d66ac Merge "Use install(1) where possible" 2015-03-20 14:30:06 +00:00
Jenkins
e64bd629b4 Merge "Swift use v3 auth_token credentials" 2015-03-20 14:29:36 +00:00