From: http://docs.openstack.org/trunk/config-reference/content/section_compute-scheduler.html#imagepropertiesfilter
Filters hosts based on properties defined on the instance's image.
Updated descriptions to be provide better information.
Added baremetal "hypervisor_type" after getting info from
Nova core about what hypervisors to include.
Left out "architecture" for now, because horizon fails when
trying to update architecture property.
Change-Id: I9195825a0ab4b89e764983da06827173a2daa23c
Closes-bug: 1368965
The OS::Compute::HostCapabilities namespace should include more
information about how the properties are used by Nova.
Let admin's know that the ComputeCapabilitiesFilter needs to
be enabled.
Change-Id: Ib3232e560969462d3973606e138a96d1a5beb7de
Closes-bug: 1367860
The namespace had inconsistent naming when compared to
other namespaces.
OS::Glance:CommonImageProperties
should be
OS::Glance::CommonImageProperties
Change-Id: Iea9860be66eb06250f8a53b8c44ab787e9c8082a
Closes-Bug: 1367545
The namespace had incorrect capitalization. Also,
added additional text in description to make people
aware that the properties only apply to the VMware
compute driver.
Change-Id: Ic1c3c333a1c70782a0780ace0323b2f6728bdd09
Closes-bug: 1367548
A sample import script to successfully import image from http or https
location is introduced. This should work on a devstack installtion.
Also, the following changes are introduced:-
1. An interface for implementing any other type of executor.
2. Provides namespace for keeping Tasks scripts.
3. A config for choosing deployer specific executor.
4. An interface for writing Tasks scripts.
5. A module for common script related methods.
6. Logic for limiting number of simultaneous tasks execution on the
Glance API server.
partially implements bp async-glance-workers
DocImpact
Change-Id: I382472fffd0fdad43573e72b2e78a9a6ed1e1f1a
Implements: blueprint metadata-schema-catalog
A common API hosted by the Glance service for vendors, admins,
services, and users to meaningfully define available key / value
pair and tag metadata. The intent is to enable better metadata
collaboration across artifacts, services, and projects for
OpenStack users.
This is about the definition of the available metadata that can
be used on different types of resources (images, artifacts,
volumes, flavors, aggregates, etc). A definition includes the
properties type, its key, it's description, and it's constraints.
This catalogue will not store the values for specific instance
properties.
- REST API for CRUD on metadef namespace
- REST API for CRUD on metadef objects
- REST API for CRUD on metadef properites
- REST API for CRUD on metadef resource types
- REST API for JSON schemas on metadef API's
Change-Id: I8e6d88ffee9a9337bf82b1da85648ba638a154ab
DocImpact
Co-Authored-By: Lakshmi N Sampath <lakshmi.sampath@hp.com>
Co-Authored-By: Wayne Okuma <wayne.okuma@hp.com>
Co-Authored-By: Travis Tripp <travis.tripp@hp.com>
Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>
Co-Authored-By: Michal Jastrzebski <michal.jastrzebski@intel.com>
Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Implements: blueprint metadata-schema-catalog
A common API hosted by the Glance service for vendors, admins,
services, and users to meaningfully define available key / value
pair and tag metadata. The intent is to enable better metadata
collaboration across artifacts, services, and projects for
OpenStack users.
This is about the definition of the available metadata that can
be used on different types of resources (images, artifacts,
volumes, flavors, aggregates, etc). A definition includes the
properties type, its key, it's description, and it's constraints.
This catalogue will not store the values for specific instance
properties.
Change-Id: Ib1c1abf80879fb6dcd5ee30c7d2bc65b0ba720d5
DocImpact
Co-Authored-By: Lakshmi N Sampath <lakshmi.sampath@hp.com>
Co-Authored-By: Wayne Okuma <wayne.okuma@hp.com>
Co-Authored-By: Travis Tripp <travis.tripp@hp.com>
Co-Authored-By: Pawel Koniszewski <pawel.koniszewski@intel.com>
Co-Authored-By: Michal Jastrzebski <michal.jastrzebski@intel.com>
Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Put osprofiler middleware as a almost first middleware.
This allows us to:
1) Get more preciese duration. We are measuring duration of all middlewares.
2) authenticate middleware is requesting keystone, so if we init osprofiler
before this middleware, we will get trace infomration from keystone as well.
Change-Id: I11c5d21e6e6b0bd514dee295da410b208d7c798c
*) Add osprofiler wsgi middleware
This middleware is used for 2 things:
1) It checks that person who want to trace is trusted and knows
secret HMAC key.
2) It start tracing in case of proper trace headers
and add first wsgi trace point, with info about HTTP request
*) Add initialization of osprofiler at start of serivce
Set's olso.messaging notifer instance (to send notifications to
Ceilometer)
*) Add tracing support for sqlalchemy
*) Add profiler CONF group that has to options:
1) Enable or disable profiler
2) Enable or disable sqlalchemy tracing
(the reason why we put this in Conf options, is that usually
db requests create a lot of trace info)
*) Glance-registry wasn't setted properly to send notifications.
To test it you need 2 things:
1) Enable ceilometer in devstack:
http://paste.openstack.org/show/85724/
2) Initialize osprofiler and send out API request. You can use
one of follow approaches:
a. Run this script (pls provide proper credentials)
http://paste.openstack.org/show/85722/
b. Use the function https://review.openstack.org/#/c/111184/
added from glanceclient CLI directly.
DocImpact
Change-Id: I45a19f5eb5304c2b78a9e12cbc0744941a807304
Co-author: Zhi Yan Liu <zhiyanl@cn.ibm.com>#
Adding a status field to image's each location property, each location
status can be 'active', 'pending_delete' and 'deleted'.
Under location's status information Scrubber service can make cleanup
based on DB records also but not a dedicated queue-file for each image.
This is third part of this change which covered scrubber DB queue
enablement and double queue supporting.
Partially-Implements BP: image-location-status
Change-Id: I8ee9e9254c8371bbbce8b28d643e3d474b2361bc
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
We should set these defaults in glance-registry.conf
(they are actually the same as in glance-api.conf) to be able to
use oslo.messaging notification.api (in glance-registry)
Which is actually essential for osprofiler
Change-Id: I7768556aef17bb95123423f7e0cea54ecc21258b
The auth_token middleware in python-keystoneclient is now
deprecated and has been moved to keystonemiddleware.
Closes-Bug: #1342274
Change-Id: Ic8ba2d2f470ec052ba13c3b63b26d5e2270f7412
The config docs have historically recommended that the number of
glance workers should be set to the number of CPUs available for
best performance, so we should make this the default.
Commit 75c96a48fc7e5dfb59d8258142b01422f81b0253 did the same thing in
Nova in Icehouse and the plan is to do the same thing for Cinder
and Trove workers.
The config files are updated to match the help string in the code.
There is no upgrade impact since glance-api.conf previously
hard-coded the workers value to 1 so anyone upgrading to this
will still get whatever value was set in glance-api.conf prior
to this change.
DocImpact: glance workers will now be equal to the number of
CPUs available by default if not explicitly specified
in glance-api.conf and/or glance-registry.conf.
UpgradeImpact: There is no upgrade impact to glance-api workers
since glance-api.conf previously hard-coded the workers
value to 1 so anyone upgrading to tihs will still get
whatever value was set in glance-api.conf prior to
this change. There is an upgrade impact to the
glance-registry workers since glance-registry.conf
did not hard-code the workers value to 1 before change
I0cee0d284eef9ce5dcb26720499f2c4d31eaca0f, which is
overwritten here. So anyone upgrading to this change
that does not have workers specified in
glance-registry.conf will now be running multiple
workers by default when they restart the glance
registry service.
Closes-Bug: #1333325
Change-Id: I6795c6e22268bb3fb67331edc7af641aefa904cc
Currently the 'workers' option -- to increase the number of
service processes -- is only documented for the API.
This change adds the equivalent documentation for the registry.
DocImpact
Change-Id: I0cee0d284eef9ce5dcb26720499f2c4d31eaca0f
Closes-Bug: #1334711
Adding large images to glance now uses the S3 MultiPart Upload REST API.
Added a unit test to upload images of various sizes to trigger the usage
of the S3 Multipart Upload API and ensure that it is being exercised.
DocImpact: The following parameters are added in glance-api.conf
+# Size, in MB, should S3 start chunking image files
+# and do a multipart upload in S3. The default is 100MB.
+#s3_store_large_object_size = 100
+
+# Multipart upload part size, in MB, should S3 use when uploading
+# parts. The size must be greater than or equal to
+# 5MB. The default is 10MB.
+#s3_store_large_object_chunk_size = 10
+
+# The number of thread pools to perform a multipart upload
+# in S3. The default is 10.
+#s3_store_thread_pools = 10
Implements: blueprint s3-multi-part-upload
Change-Id: I36e28e7944e57cd7ac05e46afac461ec10245c68
The image locations table stores the swift url for images
which includes the user and key values. This if exposed,
can cause security risk. Hence this patch, santizies
that information out of the location before storing
and plugs it back in when it is required.
Introduced a new configuration file that supports
multiple swift account references. It has the credentials and
authurl for each store. It is specified using
'swift_store_config_file'.
In addition, this patch does the following things:
Differentiate user and system created swift locations
Currently we do not differentiate between user supplied
uri and system created locations that have the account
reference. This patch introduces new scheme:
'swift+config' for this purpose.
Image create in V1 should validate the uri in case where location isn't
specified.
This patch ensures that a store is not set while
creating an image or updating it.
Related to bp remove-sensitive-data-from-locations
Implements blueprint: support-multiple-swift-backends
Implements bp: v1-image-create-should-validate-the-location-uri
DocImpact
Co-authored by: sridevik <sridevi.koushik@rackspace.com>,
iccha-sethi <iccha.sethi@rackspace.com>,
amalabasha <amala.alungal@rackspace.com>
Change-Id: I75af34145521f533dcd6f5fd7690f5a68f3b44b3
The notifier_strategy option is deprecated since we merged commit
I8cd84772bc5867e06b2a50ed7e15b9e86f0b94ad during the Icehouse cycle.
Assuming the usual single cycle deprecation period, this can now be
removed.
DocImpact: the notifier_strategy has been removed in favor of the
notification_driver and related oslo.messaging options.
Change-Id: I3ab0133b697a820536e5da84e30b078915a29f6c
Uploading an image with --is-public=True should by default only be allowed
by an admin tenant. Allowing anyone to upload an image as is_public is likely
a security concern. This changes the previous default behavior glance had
of allowing anyone to upload an image as is_public previously by default.
DocImpact
Closes-bug: 1317314
Change-Id: I60d2257115e9207e09b50d9f950076b7fe8237d7
https://review.openstack.org/#/c/69187/ introduced SSL certificate checking
in python-swiftclient (released as v2.0). This patch ensures that the version
of swiftclient used will verify SSL certificates correctly.
This patch also documents the `swift_store_auth_insecure` configuration
option for bypassing the cert verification
DocImpact
SecurityImpact
Change-Id: I20ddab5c3359071daf7505268c72331e4c786987
Sample configuration files are out of date with respect to database
options. This change makes the options up-to-date.
Closes-bug: #1285617
Change-Id: I2694425a0b1610f37e77f1bc81aebed850a16348
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Adding a deprecated option to skip the DB sanity check to db_sync
interface, to give deployer a chance to execute migration script.
Without this change all migration scripts will be blocked before get
execution.
DocImpact
Change-Id: If887ac6b814879a0140dc1065a060233aee7bc82
Partial-Bug: #1279000
Related-Id: I7f271d846141ac72dde3fb0d12159b125018eb2c
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
This change will make possible to enable/disable both v1 and v2 Registry APIs
either individually or together from glance-registry.conf.
DocImpact
Closes bug: #1290432
Change-Id: Ib2458b3312bdb552975b1e42485086a1c3aa6d3f
Glance currently enables all stores by default. This patch changes that
by removing all stores that require manual configuration and leaving
those that work right out of the box.
Current behavior causes a lot of confusion to users since most of those
stores print errors when they're not configured correctly. All extra
stores should be enabled explicitly by users.
This fix makes tests use http locations. All other locations besides the
default ones should be tested in their own test suites.
DocImpact
Closes-bug: #1255556
Change-Id: I82073352641d3eb2ab3d6e9a6b64afc99a30dcc7
An OVA package is a tar archive usually containing an OVF directory
inside it. Nova needs to be able to differentiate OVF and OVA based on
the container format in order to extract the relevant information from
it.
This patch adds the OVA container format to the Glance configuration.
Closes-Bug: #1286375
Change-Id: I5034d2943907823a9296e7a5fb41a28f2d92ec5a
Customers using a VMware environment with OpenStack should be able to
store their Glance images in VMware datastores. This is a first step to solve
the problem where Nova needs to copy the bits over the network
from Glance to the datastore when spawning an instance.
Also, this give the ability to provide some optimizations for specific
image formats in the future (fast cloning for example).
This patch contains a 'glance/store/vmware/' folder with the code
to manage the connection with vCenter or an ESX(i) host.
This code will go away as soon as it is merged to Olso:
see review https://review.openstack.org/#/c/65075/
The current implementation give this ability to specify the vCenter or
ESX(i) IP. In case of a vCenter IP, there is no optimization to reduce
the datapath (no host selected).
Consequently, it is recommended to specify an ESX IP if the ESX host
API endpoint is accessible from Glance.
docImpact
Implements bp vmware-datastore-storage-backend
Change-Id: I3837912e0d1614b9c31a689f71c2e34d453e2dc3
Currently to configure multiple nfs server as a backend
using filesystem store, you cannot mount all disks to a single directory.
Filesystem store allows administrator to configure only single directory with
filesystem_store_datadir parameter in the glance-api.conf.
With multifilesystem store, administrators can configure multiple directories
to store the glance image with help of filesystem_store_datadirs option.
Each directory can be coupled with its priority.
Format for filesystem_store_datadirs in glance-api.conf:
--------------------------------------------------------
filesystem_store_datadirs = </path/to/store>:<priority>
...
filesystem_store_datadirs = </path/to/store>:<priority>
Example:
filesystem_store_datadirs = /var/glance/store
filesystem_store_datadirs = /var/glance/store1:100
filesystem_store_datadirs = /var/glance/store2:200
Note:
-----
1. Either filesystem_store_datadir or filesystem_store_datadirs option
must be specified in glance-api.conf
2. Store with priority 200 has precedence over store with priority 100
3. If no priority is specified, default priority '0' is associated with it.
4. If two filesystem stores have same priority store with maximum free space
will be chosen to store the image.
5. If same store is specified multiple times then BadStoreConfiguration
exception will be raised.
blueprint: glance-multifilesystem-store
DocImpact: Multifilesystem support can be used by specifying multiple
directory paths to filesystem_store_datadirs param in glance-api.conf as
discussed above.
Change-Id: Ibb04ac14c472cd863c5e285b6dc6a08c69014fe8
If a request to Swift for an image fails before the full image is
downloaded, this will reconnect to Swift and download the remainder of
the image without having to start from scratch.
DocImpact
blueprint retry-swift-download
Change-Id: Ifde68415707506b56ffac1939c6750b216b87a87
An image can be saved to more than one backend storage by multi-location
mechanism, Glance should have a capability to select the best backend
storage for improving image consuming performance. This change
implements two strategy modules: "location_order" and "store_type", and
allow Admin configure some strategy-specified options for Glance as
their deployment required.
As an end user, he can get VM or consuming image faster base on this
enhancement. This is not only for 'download' transport handling on the
API server side but also for client since they can obtain locations by
standard 'direct URL' interface.
docImpact
Implements bp: image-location-selection-strategy
Change-Id: I86f192aeae8e5f21a72f946552f6507654c25a6c
Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Also increases the default value from 8192 to 16384
The former max header (8192) is exceeded in the following scenario
- Auth tokens built with a keystone v3 API catalog
- A catalog with approximately 8 or more endpoints defined
Equivalent fixes have been proposed in other projects
Change-Id: Ic714166440e2aa08b619ab1e5e5e0cf3cea3e309
Closes-Bug: 1190149
DocImpact: The example glance-api.conf set different defaults than
what was hardcoded in the code. Change code defaults to those
in the config.
Preparation for generating glance-api.conf with oslo.config generator.
Change-Id: I424e6c5984dbdf5082852c40d97ab1e1a04926ea
This patch adds the image_location_quota config option. This allows
a deployer to limit the number of image locations allowed on an image.
The default value is 10. Users will only be able to update an image
if the result of the transaction would be under this limit. This
behavior is intended to be similar to 'quota_metadata_items' in Nova.
We have implemented the same limit for image properties, members and tags,
see:
https://review.openstack.org/#/c/56981/https://review.openstack.org/#/c/59563/https://review.openstack.org/#/c/58056/
docImpact
Closes-Bug: #1259031
Change-Id: I0e0033a1d11be267144e8e10b730bb6331263d4c
The current policy.json file is missing the copy_from policy in order
to be exhaustive.
DocImpact
Change-Id: Ibc0d1fa7626629c28c514514a985a6b89db2ac69
Closes-bug: #1258999
Glance currently uses a custom notifier and it has been maintaining it
for a long time. In a hope of reducing duplicated code and improving
cross-project contributions, this patch replaces the old notifier with
the one, recently developed, in oslo.messaging.
The oslo.messaging project is a port of the old oslo-rpc code to a
standalone, more stable and improved project. It brings all the benefits
that oslo-rpc would've brought as well as an easier way to integrate
with other projects.
This patch also:
- Reduces the code shipped along with Glance since all the code
copied from oslo-incubator related to the notifier is not needed
anymore.
- Improves the stability of existing, broker based, notifications.
- Brings HA support.
- Keeps backward compatibility by translating the old
`notifier_strategy` into oslo.messaging drivers.
Changes to the code:
- It is now necessary to pass the request context to the
notification call.
- Notifier package is no longer necessary. A notifier module was
added instead.
- New, notifier related, configurations were added.
- A lot of code was removed
Since there's still not an official release, requirements.txt points to
the latest tarball created. A release for oslo.messaging is planned for
Icehouse.
docImpact
Implements bp oslo-messaging
Change-Id: I8cd84772bc5867e06b2a50ed7e15b9e86f0b94ad
This patch adds the image_member_quota config option. This allows a deployer
to limit the number of image members allowed per image. The default value
is 128, to be consistent with other quota defaults. Users will only be able
to update an image if the result of the transaction would be under this limit.
This is for both Glance v1 and v2
Fixes bug 1252459
docImpact
Change-Id: I02f5e82ca4c4acf6cd7bc94f9b99086054a616c9
This patch adds the image_tag_quota config option. This allows a deployer
to limit the number of image tags allowed on an image for v2. The default value
is 128. If an image is somehow over the limit, tags can still be removed but no
more may be added.
Fixes bug 1252337
docImpact
Change-Id: I2036e2a88601d7c5aa85fad32d90fe0ed30b84c8
Add documentation for some parameters which can
be present in glance-api.conf:
* use_user_token
* admin_user
* admin_password
* admin_tenant_name
* auth_url
* auth_region
* auth_strategy
Fix for bug 1256952.
DocImpact
Change-Id: Ia5b731cabaec961f3a4226d30abd3bca9bd87651
Includes:-
- Domain Task class
- Domain TaskFactory class
- DB TaskRepo class
- Necessary proxy classes for all of the above
- Tasks related exceptions
- Unit tests as applicable
Partially implement blueprint async-glance-workers
Change-Id: I619224bcc55b62303f3539454649528f2edc6e9d
Related to bug 1250918
There currently exists no check in set_data function for glance
api v2 in the policy layer. This patch adds upload_image policy.
DocImpact
Change-Id: Ibccc78f8bccff3baa5aca574cf17bd14c5403ca2
