199 Commits

Author SHA1 Message Date
Jenkins
3d90db31e3 Merge "Bump python-swiftclient version" 2014-06-17 18:29:04 +00:00
Aaron Rosen
96169ab5a0 make uploading an image as public admin only by default
Uploading an image with --is-public=True should by default only be allowed
by an admin tenant. Allowing anyone to upload an image as is_public is likely
a security concern. This changes the previous default behavior glance had
of allowing anyone to upload an image as is_public previously by default.

DocImpact
Closes-bug: 1317314

Change-Id: I60d2257115e9207e09b50d9f950076b7fe8237d7
2014-06-03 05:47:24 -07:00
Thomas Leaman
c93a7a22c0 Bump python-swiftclient version
https://review.openstack.org/#/c/69187/ introduced SSL certificate checking
in python-swiftclient (released as v2.0). This patch ensures that the version
of swiftclient used will verify SSL certificates correctly.

This patch also documents the `swift_store_auth_insecure` configuration
option for bypassing the cert verification

DocImpact
SecurityImpact

Change-Id: I20ddab5c3359071daf7505268c72331e4c786987
2014-05-27 12:43:36 +00:00
Zhi Yan Liu
f4e48a8868 Update Glance configuration sample files for database options
Sample configuration files are out of date with respect to database
options. This change makes the options up-to-date.

Closes-bug: #1285617
Change-Id: I2694425a0b1610f37e77f1bc81aebed850a16348
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
2014-04-11 00:16:01 +08:00
Zhi Yan Liu
1a40831f17 Making DB sanity checking be optional for DB migration
Adding a deprecated option to skip the DB sanity check to db_sync
interface, to give deployer a chance to execute migration script.
Without this change all migration scripts will be blocked before get
execution.

DocImpact

Change-Id: If887ac6b814879a0140dc1065a060233aee7bc82
Partial-Bug: #1279000
Related-Id: I7f271d846141ac72dde3fb0d12159b125018eb2c
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
2014-04-02 00:01:16 +08:00
Erno Kuvaja
c6365cc4f5 Makes possible to enable Registry API v1 and v2
This change will make possible to enable/disable both v1 and v2 Registry APIs
either individually or together from glance-registry.conf.

DocImpact
Closes bug: #1290432
Change-Id: Ib2458b3312bdb552975b1e42485086a1c3aa6d3f
2014-03-14 13:12:04 +00:00
Flavio Percoco
1f6381a73f Don't enable all stores by default
Glance currently enables all stores by default. This patch changes that
by removing all stores that require manual configuration and leaving
those that work right out of the box.

Current behavior causes a lot of confusion to users since most of those
stores print errors when they're not configured correctly. All extra
stores should be enabled explicitly by users.

This fix makes tests use http locations. All other locations besides the
default ones should be tested in their own test suites.

DocImpact
Closes-bug: #1255556
Change-Id: I82073352641d3eb2ab3d6e9a6b64afc99a30dcc7
2014-03-10 10:40:31 +01:00
Arnaud Legendre
965e2c3594 Add the OVA container format
An OVA package is a tar archive usually containing an OVF directory
inside it. Nova needs to be able to differentiate OVF and OVA based on
the container format in order to extract the relevant information from
it.
This patch adds the OVA container format to the Glance configuration.

Closes-Bug: #1286375

Change-Id: I5034d2943907823a9296e7a5fb41a28f2d92ec5a
2014-03-03 19:44:21 -08:00
Jenkins
c5588da78a Merge "Add multifilesystem store to support NFS servers as backend" 2014-01-31 09:53:10 +00:00
Jenkins
84954a8522 Merge "Retry failed image download from Swift" 2014-01-31 05:50:59 +00:00
Arnaud Legendre
f9589bd010 VMware Datastore storage backend
Customers using a VMware environment with OpenStack should be able to
store their Glance images in VMware datastores. This is a first step to solve
the problem where Nova needs to copy the bits over the network
from Glance to the datastore when spawning an instance.
Also, this give the ability to provide some optimizations for specific
image formats in the future (fast cloning for example).

This patch contains a 'glance/store/vmware/' folder with the code
to manage the connection with vCenter or an ESX(i) host.
This code will go away as soon as it is merged to Olso:
see review https://review.openstack.org/#/c/65075/

The current implementation give this ability to specify the vCenter or
ESX(i) IP. In case of a vCenter IP, there is no optimization to reduce
the datapath (no host selected).
Consequently, it is recommended to specify an ESX IP if the ESX host
API endpoint is accessible from Glance.

docImpact
Implements bp vmware-datastore-storage-backend

Change-Id: I3837912e0d1614b9c31a689f71c2e34d453e2dc3
2014-01-30 18:14:15 -08:00
Aswad Rangnekar
7e2c16104d Add multifilesystem store to support NFS servers as backend
Currently to configure multiple nfs server as a backend
using filesystem store, you cannot mount all disks to a single directory.
Filesystem store allows administrator to configure only single directory with
filesystem_store_datadir parameter in the glance-api.conf.

With multifilesystem store, administrators can configure multiple directories
to store the glance image with help of filesystem_store_datadirs option.
Each directory can be coupled with its priority.

Format for filesystem_store_datadirs in glance-api.conf:
--------------------------------------------------------
filesystem_store_datadirs = </path/to/store>:<priority>
...
filesystem_store_datadirs = </path/to/store>:<priority>

Example:
filesystem_store_datadirs = /var/glance/store
filesystem_store_datadirs = /var/glance/store1:100
filesystem_store_datadirs = /var/glance/store2:200

Note:
-----
1. Either filesystem_store_datadir or filesystem_store_datadirs option
must be specified in glance-api.conf
2. Store with priority 200 has precedence over store with priority 100
3. If no priority is specified, default priority '0' is associated with it.
4. If two filesystem stores have same priority store with maximum free space
will be chosen to store the image.
5. If same store is specified multiple times then BadStoreConfiguration
exception will be raised.

blueprint: glance-multifilesystem-store

DocImpact: Multifilesystem support can be used by specifying multiple
directory paths to filesystem_store_datadirs param in glance-api.conf as
discussed above.

Change-Id: Ibb04ac14c472cd863c5e285b6dc6a08c69014fe8
2014-01-22 14:57:51 -08:00
Thomas Leaman
ae568d7858 Retry failed image download from Swift
If a request to Swift for an image fails before the full image is
downloaded, this will reconnect to Swift and download the remainder of
the image without having to start from scratch.

DocImpact

blueprint retry-swift-download

Change-Id: Ifde68415707506b56ffac1939c6750b216b87a87
2014-01-22 10:55:36 +00:00
Zhi Yan Liu
9567c2b6a0 Adding image location selection strategies
An image can be saved to more than one backend storage by multi-location
mechanism, Glance should have a capability to select the best backend
storage for improving image consuming performance. This change
implements two strategy modules: "location_order" and "store_type", and
allow Admin configure some strategy-specified options for Glance as
their deployment required.

As an end user, he can get VM or consuming image faster base on this
enhancement. This is not only for 'download' transport handling on the
API server side but also for client since they can obtain locations by
standard 'direct URL' interface.

docImpact
Implements bp: image-location-selection-strategy

Change-Id: I86f192aeae8e5f21a72f946552f6507654c25a6c
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
2014-01-16 15:17:34 +08:00
Florent Flament
df8be62e58 Allow users to customize max header size
Also increases the default value from 8192 to 16384

The former max header (8192) is exceeded in the following scenario
- Auth tokens built with a keystone v3 API catalog
- A catalog with approximately 8 or more endpoints defined

Equivalent fixes have been proposed in other projects

Change-Id: Ic714166440e2aa08b619ab1e5e5e0cf3cea3e309
Closes-Bug: 1190149
2014-01-13 14:05:14 +00:00
Jenkins
42c27ddbed Merge "Align glance-api.conf rbd option defaults with config" 2014-01-11 15:46:54 +00:00
Dirk Mueller
cf11e3579d Align glance-api.conf rbd option defaults with config
DocImpact: The example glance-api.conf set different defaults than
what was hardcoded in the code. Change code defaults to those
in the config.

Preparation for generating glance-api.conf with oslo.config generator.

Change-Id: I424e6c5984dbdf5082852c40d97ab1e1a04926ea
2014-01-10 15:02:57 +01:00
Fei Long Wang
5f98317684 Add config option to limit image locations
This patch adds the image_location_quota config option. This allows
a deployer to limit the number of image locations allowed on an image.
The default value is 10. Users will only be able to update an image
if the result of the transaction would be under this limit. This
behavior is intended to be similar to 'quota_metadata_items' in Nova.

We have implemented the same limit for image properties, members and tags,
see:
https://review.openstack.org/#/c/56981/
https://review.openstack.org/#/c/59563/
https://review.openstack.org/#/c/58056/

docImpact
Closes-Bug: #1259031

Change-Id: I0e0033a1d11be267144e8e10b730bb6331263d4c
2014-01-10 06:56:30 +08:00
Yanis Guenane
e69ec61aa6 Adding missing copy_from policy from policy.json
The current policy.json file is missing the copy_from policy in order
to be exhaustive.

DocImpact
Change-Id: Ibc0d1fa7626629c28c514514a985a6b89db2ac69
Closes-bug: #1258999
2013-12-13 12:22:15 +00:00
Jenkins
a3a29e686f Merge "Add documentation for some API parameters" 2013-12-05 08:35:59 +00:00
Jenkins
1ded23d6c3 Merge "Migrate to oslo.messaging" 2013-12-04 14:00:54 +00:00
Flavio Percoco
90d6ef8130 Migrate to oslo.messaging
Glance currently uses a custom notifier and it has been maintaining it
for a long time. In a hope of reducing duplicated code and improving
cross-project contributions, this patch replaces the old notifier with
the one, recently developed, in oslo.messaging.

The oslo.messaging project is a port of the old oslo-rpc code to a
standalone, more stable and improved project. It brings all the benefits
that oslo-rpc would've brought as well as an easier way to integrate
with other projects.

This patch also:

    - Reduces the code shipped along with Glance since all the code
      copied from oslo-incubator related to the notifier is not needed
      anymore.
    - Improves the stability of existing, broker based, notifications.
    - Brings HA support.
    - Keeps backward compatibility by translating the old
      `notifier_strategy` into oslo.messaging drivers.

Changes to the code:

    - It is now necessary to pass the request context to the
      notification call.
    - Notifier package is no longer necessary. A notifier module was
      added instead.
    - New, notifier related, configurations were added.
    - A lot of code was removed

Since there's still not an official release, requirements.txt points to
the latest tarball created. A release for oslo.messaging is planned for
Icehouse.

docImpact
Implements bp oslo-messaging

Change-Id: I8cd84772bc5867e06b2a50ed7e15b9e86f0b94ad
2013-12-04 09:05:51 +01:00
Alex Meade
b13e10b5e5 Add config option to limit image members
This patch adds the image_member_quota config option. This allows a deployer
to limit the number of image members allowed per image. The default value
is 128, to be consistent with other quota defaults. Users will only be able
to update an image if the result of the transaction would be under this limit.

This is for both Glance v1 and v2

Fixes bug 1252459
docImpact

Change-Id: I02f5e82ca4c4acf6cd7bc94f9b99086054a616c9
2013-12-03 16:05:49 +00:00
Alex Meade
4e7d9cdaf9 Add config option to limit image tags
This patch adds the image_tag_quota config option. This allows a deployer
to limit the number of image tags allowed on an image for v2. The default value
is 128. If an image is somehow over the limit, tags can still be removed but no
more may be added.

Fixes bug 1252337
docImpact

Change-Id: I2036e2a88601d7c5aa85fad32d90fe0ed30b84c8
2013-12-03 16:05:49 +00:00
Stuart McLaren
2425b93b49 Add documentation for some API parameters
Add documentation for some parameters which can
be present in glance-api.conf:

* use_user_token
* admin_user
* admin_password
* admin_tenant_name
* auth_url
* auth_region
* auth_strategy

Fix for bug 1256952.

DocImpact

Change-Id: Ia5b731cabaec961f3a4226d30abd3bca9bd87651
2013-12-02 16:27:03 +00:00
Fei Long Wang
d79861560b Adds domain level support for tasks
Includes:-
  - Domain Task class
  - Domain TaskFactory class
  - DB TaskRepo class
  - Necessary proxy classes for all of the above
  - Tasks related exceptions
  - Unit tests as applicable

Partially implement blueprint async-glance-workers

Change-Id: I619224bcc55b62303f3539454649528f2edc6e9d
2013-11-26 16:35:12 -05:00
iccha.sethi
d35bcee6ab Add upload policy for glance v2 api
Related to bug 1250918

There currently exists no check in set_data function for glance
api v2 in the policy layer. This patch adds upload_image policy.

DocImpact

Change-Id: Ibccc78f8bccff3baa5aca574cf17bd14c5403ca2
2013-11-25 12:42:44 +00:00
Jenkins
14d8fe408a Merge "Add config option to limit image properties" 2013-11-25 09:42:28 +00:00
Alex Meade
f63d2f67ed Add config option to limit image properties
This patch adds the image_property_quota config option. This allows a deployer
to limit the number of image properties allowed on an image. The default value
is 128, as is currently the limit enforced by nova. Users will only be able to
update an image if the result of the transaction would be under this limit.
This behavior is intended to be similar to 'quota_metadata_items' in nova.

This is for both Glance v1 and v2.

Fixes bug 1251518
docImpact

Change-Id: I4aa9504deae836404f11c9ada71a91f85caeba4c
2013-11-25 02:00:32 +00:00
Yanis Guenane
1407aa1818 Explicit listing of Glance policies in json file
Most OpenStack components do list their policies in their policy.json but not
Glance. It will let user directly know what options is available to them.
Previously non-listed policy do follow the default rule now.
List of available policies have been taken from Developer Guide page :
http://docs.openstack.org/developer/glance/policies.html#constructing-a-policy-configuration-file

Closes-Bug: #1254461
Change-Id: I833db8385c2d3e8962a9c19b77c94d962eac475e
2013-11-24 22:26:16 +00:00
Jenkins
a3b79e6bac Merge "Remove "image_cache_invalid_entry_grace_period" option" 2013-11-12 15:37:53 +00:00
Noboru Arai
3db3edabe6 Remove "image_cache_invalid_entry_grace_period" option
"image_cache_invalid_entry_grace_period" option is unused
in *.py, but the option is used in glance-cache.conf.

 I removed "image_cache_invalid_entry_grace_period" option
from glance-cache.conf.

Change-Id: Id235945f9cc89219118e4a3ec077f0e887a3b3ef
Closes-Bug: #1249949
2013-11-11 15:36:40 +09:00
Jon Bernard
d09e1447b2 Document default log location in config files
It's not immediately obvious where log output will go if the 'log_file' and
'use_syslog' directives are either missing or commented out.  This patch updates
the documentation in the config files to mention the stdout default.  This
should help remove any confusion when configuring logging for glance.

Change-Id: I36f2a9d30bdee1949e60f413b7f181631c564ed5
Closes-Bug: #1248292
2013-11-05 16:52:00 -05:00
Flavio Percoco
21ea5f6535 Replace qpid_host with qpid_hostname
This patch replaces `qpid_host` with `qpid_hostname` in both the
documentation and the config file sample. `qpid_host` has been shipped
as the parameter to specify qpid's hostname, although the correct one is
`qpid_hostname`.

DocImpact
Closes-bug 1242715

Change-Id: I25f2d0e505560e1762f7f059728c33c47a23439e
2013-10-28 15:35:33 +01:00
iccha.sethi
eb87f1fae8 Using policies for protected properties
This patch extends the way we can define rules for property protections.
It allows us to use the roles/rules defined in policy.json and leverage
the policy language for property protections as well.

DocImpact

Related to bp api-v2-property-protection

Change-Id: I4789ebb2645171280185d2c76138e78d954b5954
2013-10-17 01:55:00 +00:00
Stuart McLaren
8a85be099e Add swift_store_ssl_compression param
Add a new glance-api.conf parameter 'swift_store_ssl_compression',
which allows disabling SSL layer compression for https swift requests.

This can increase performance when transferring compressed images (eg
qcow2) to/from swift.

Addresses bug 1190666.

DocImpact.

Change-Id: Ic26dba9dce5ea7e5c09fdd531746b93cec54fc5c
2013-09-23 14:55:54 +00:00
iccha.sethi
e8440d1ee8 Rule parser for property protections
This patch introduces the way protected properties
will be configured and parsed.

Related to bp api-v2-property-protection
docImpact

Change-Id: I3d24cacccf3f51b07a4090b8a5db1f2451090762
2013-09-04 17:46:33 +00:00
Zhi Yan Liu
85075f4b11 Scrubber refactoring
* Adding multiple locations image support.
* Adding lock protection to prevent race condition between glance-api
and glance-scrubber service.
* Refactoring scrub queue code.

Implement bp: glance-scrubber-refactoring
docImpact

Change-Id: I050ff212d73ace8e84dcd800245b608210d6b29a
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
2013-09-04 12:36:25 +08:00
John Bresnahan
e3e4f4d927 Add a storage quota
This patch adds a storage quota that is applied against the sum total
of a users storage consumption against all configured storage systems.
A single quota is applied to all users via the configuration option
'total_storage_quota'.  Most of the patch is about enforcement so
when a separate service for quota management emerges in OpenStack the
per user value to enforce can be obtained from that service but the
enforcement code will remain the same.

blueprint glance-basic-quotas
docImpact
Change-Id: I251832f7372c70942be6f0c6aa12285145dd7c18
2013-08-28 17:12:34 -10:00
Jenkins
b0f5f041dc Merge "Make disk and container formats configurable" 2013-08-21 10:10:55 +00:00
Brian Waldon
830f27ba34 Make disk and container formats configurable
* Add disk_formats config attribute
* Add container_formats config attribute
* Implement bp configurable-formats

Change-Id: Ic52ffb46df9438c247ba063748cadd69b9c90bcd
2013-08-19 04:23:07 +00:00
lawrancejing
05739853ef Add space in etc/glance-cache.conf
Add space in "known_stores" block in etc/glance-cache.conf,
and keep glance code tidy and pleasing.

Change-Id: If112db41bfbce9f78013fe33229043f0f626f3f9
2013-08-17 21:24:13 +08:00
Jenkins
2f05e71f56 Merge "Turn off debug logging in sqlalchemy by default" 2013-08-02 19:21:57 +00:00
iccha.sethi
4b71309280 Include pipeline option for using identity headers
There have been patches in nova, glance client and glance
registry to give the user the option of not reauthenticating
in glance. Below are the related reviews. This patch
provides pipeline options the users will want to use for the same.

Nova: https://review.openstack.org/#/c/37070/
Glanceclient: https://review.openstack.org/#/c/37062/
Glance Registry: https://review.openstack.org/#/c/36563/

Change-Id: I904f8bafb7e5abceac5723ed8504c672a00c5ec7
2013-07-29 20:25:19 +00:00
Matthew Treinish
03bfc4e64a Turn off debug logging in sqlalchemy by default
This commit adds a new config option sqlalchemy_debug which is used
for enabling debug messages from sqlalchemy. Previously, this logging
was enabled whenever debug was enabled. The debug log level for
sqlalchemy prints out all SQL queries and results which is way too
verbose for most cases where just debug logging is useful. This
change disables this previous behavior by defaulting the new option
to false and decoupling it from the debug config option.

DocImpact

Change-Id: I298c40b71ed0b8772ff956f61f8d94217bce0e11
2013-07-23 13:53:38 -04:00
Alex Meade
3fa3891595 Glance api to pass identity headers to registry v1
This patch introduces the send_identity_headers
config option that allows glance-api to pass auth identity
headers when making calls to the registry v1.

docImpact
Fixes bug 1199990

Change-Id: Ie5f07ed6dfeaa8428de4f79c4d40d182328e6ab4
2013-07-22 22:25:51 +00:00
Jenkins
ddd5e34795 Merge "File system store can send metadata back with the location." 2013-07-18 23:21:12 +00:00
Zhi Yan Liu
d13493be80 Adding Cinder backend storage driver to Glance
This change allows Glance drive Cinder as a block storage backend to
store image data.
Before this we already use swift as an object storage backend to save
image.

Currently the patch is a partial implementation, after Cinder expose
'brick' library, 'host-volume-attaching' and 'multiple-attaching'
enhancement ready, the store will support ADD/GET/DELETE interface
finally.

blueprint: glance-cinder-driver

Change-Id: I4cdeccdb518972c0280e59c984ed6b001dafe243
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
2013-07-17 06:27:59 +08:00
John Bresnahan
3726418603 File system store can send metadata back with the location.
This patch allows the glance operator to associate metadata with
the file system store.  The metadata will be returned to v2 users
with each location that was created by the file system store.  This
information can give clients context into how to consume the data.

The new configuration option: file system_store_metadata_file is
added.  This is a path to a JSON file.  The JSON document can
contain lists, dicts and unicode values only.

blueprint: direct-url-meta-data
blueprint: multiple-image-locations
docImpact
Change-Id: I3f040374455feb876e970a1813099598301774b5
2013-07-16 10:18:27 -10:00
Flaper Fesp
de8eba091b DB Driver for the Registry Service
This patch adds a database driver that talks to a remote registry
service, which talks directly to a database back-end. The registry
service implements all the database API public functions that are
actually used from outside the API.

In order to test this patch correctly, it was necessary to add an
auth_token to each context used in the base tests and update that token
were needed.

NOTE: This patch adds tests for the above said driver but, it doesn't
add Visibility tests. The reason is that those tests are meant for
actual database back-ends. Given the "proxy" nature of this driver and
the fact that it doesn't do anything on the data returned by the remote
registry service, it isn't worth adding those tests. However, this
review does have support for DriverTests and implements each one of
them.

Implements blueprint registry-db-driver

Change-Id: I60f8e25c406c8e62b343267ba00f23299b19d4e5
2013-07-15 19:53:43 +02:00