67 Commits

Author SHA1 Message Date
Jay Pipes
1ab63ff5c9 Fixes LP Bug#861650 - Glance client deps
This patch addresses the dependency proliferation in
the glance client stuff. It removes references to modules
that contain non-client-necessary libraries (like
eventlet, xattr, sqlalchemy-migrate and sqlalchemy)
by restructuring the modules slightly.

Note that the additional httplib2 dependency is because
that is used in the authentication strategy stuff in
glance.common.auth. This could be rewritten to use httplib
instead, further reducing the dependencies of the client lib.

IMPORTANT NOTE: This patch changes the default entrypoint
for the Images API router application, and therefore this
should be merged along with the packaging changes in this
branch:

https://code.launchpad.net/~jaypipes/glance/ubuntu/+merge/82318

Change-Id: I5dbc8584fb77e3e011fb6ff3532f792f5103e242
2011-11-30 09:41:22 -05:00
Paul Bourke
49d67bd505 Adds option to set custom data buffer dir
Implements blueprint custom-disk-buffer

When pushing an image large enough to be chunked into either a Swift
or S3 backend, the chunks are buffered to /tmp.

This change allows a user to define an alternate directory via the
configuration files.

Change-Id: I2660b6e5a76cbfd43834384b129550b66bb9afce
2011-11-22 17:08:32 +00:00
Jenkins
cc4e83bc89 Merge "Using Keystone's new port number 35357." 2011-11-11 18:17:19 +00:00
Rick Harris
94dcf3acd0 Using Keystone's new port number 35357.
Fixes bug 888753

Change-Id: I22894a31ee54edd6d00ac3458ea4a2f8842aa4a0
2011-11-10 23:33:45 +00:00
Rick Harris
0a394707be Adding admin_token to image-cache config.
Fixes bug 888802

Change-Id: If1fa7c4a6ba09b86ddba0ceff9a922aa6a9d27af
2011-11-10 23:22:46 +00:00
Mark McLoughlin
2c66c6972d Remove unused versions pipeline from PasteDeploy config
Since the version negotiation middleware was introduced, versions
pipeline app in the config is unused. Consequently, the versions
app factory is unused too.

Change-Id: I2d50432de215e51c1e52f8d68eaa0820e9b5bd0b
2011-11-09 22:56:19 +00:00
Jay Pipes
2a5a0e9a35 Fixes LP Bug#878411 - No docs for image cache
Adds documentation on managing the image cache

Adds a simple CLI program to queue an image for
caching.

Change-Id: I2b4c9708c521fdcfaf1c6b8f07505e130923c4b7
2011-11-09 13:40:49 -05:00
Stuart McLaren
5e6fb33b22 Adds option to encrypt 'location' metadata.
Implements blueprint swift-location-credentials

When the new option is enabled the location metadata
(which may include user credentials) is encrypted
before being sent to the registry server.

Change-Id: I072e3f5c81f187435b1c156627076d5fde087af5
2011-11-03 17:25:01 +00:00
Jay Pipes
39c8557434 Adds Driver Layer to Image Cache
Fixes LP Bug#879136 - keyerror: 'image' when doing nova image-list
Fixes LP Bug#819936 - New image cache breaks Glance on Windows

This patch refactors the image cache further by adding an
adaptable driver layer to the cache. The existing filesystem-based
driver that depended on python-xattr and conditional fstab support
has been moved to /glance/image_cache/drivers/xattr.py, and a new
default driver is now based on SQLite and has no special requirements.

The image cache now contains a simple interface for pruning the
cache. Instead of the logic being contained in
/glance/image_cache/pruner.py, now the prune logic is self-contained
within the ImageCache.prune() method, with pruning calling the
simple well-defined driver methods of get_least_recently_accessed()
and get_cache_size().

Adds a functional test case for the caching middleware and adds
documentation on how to configure the image cache drivers.

TODO: cache-manage middleware...
TODO: cache management docs

Change-Id: Id7ae73549d6bb39222eb7ac0427b0083fd1af3ec
2011-10-25 12:54:31 -04:00
Jay Pipes
ad9e9ca3f7 Overhauls the image cache to be truly optional
Fixes LP Bug#874580 - keyerror 'location' when fetch errors
Fixes LP Bug#817570 - Make new image cache a true extension
Fixes LP Bug#872372 - Image cache has virtually no unit test coverage

* Adds unit tests for the image cache (coverage goes from 26% to 100%)
* Removes caching logic from the images controller and places it into
  a removeable transparent caching middleware
* Adds a functional test case that verifies caching of an image
  and subsequent cache hits
* Removes the image_cache_enabled configuration variable, since it's
  now enabled by simply including the cache in the application
  pipeline
* Adds a singular glance-cache.conf to etc/ that replaces the
  multiple glance-pruner.conf, glance-reaper.conf and
  glance-prefetcher.conf files
* Adds documentation on enabling and configuring the image cache

TODO: Add documentation on the image cache utilities, like reaper,
      prefetcher, etc.

Change-Id: I58845871deee26f81ffabe1750adc472ce5b3797
2011-10-19 16:35:35 -04:00
Jenkins
d3a31b372e Merge "Adds SSL configuration params to the client" 2011-10-18 14:48:01 +00:00
Kevin L. Mitchell
5b27c663b9 Better document using Glance with Keystone.
Addresses bug 871803 by expounding on what configuration should
look like.  Also fixes the example config files, which generally
had the authentication setup completely wrong.

Change-Id: I44b2b8bd340ca95b5a2c2e9408797b0308000a65
2011-10-13 17:03:39 +00:00
Jay Pipes
eec5c1afa1 Adds SSL configuration params to the client
* Adds SSL configuration params to all client classes
* Adds unit test for bad SSL client configuration
* Refactors the registry methods to no longer need
  configuration options passed, and to configure the
  registry client once, on images.Controller.__init__
* Adds glance-api.conf options for SSL support in
  registry client connections.
* Adds SSL CA file socket wrappers via a client auth
  HTTPS connection class
* Adds server SSL support, but not functional tests
  for SSL support yet. Still need to research self-signed
  cert generation for tests...
* Updates documentation for bind and startup options
* Adds functional test case for secure communication
  with API server stood up with SSL supprt. Note it is
  not very DRY. There is some DRY cleanup to do in the
  future...

TODO: Integrate options with bin/glance CLI tool

Change-Id: Ie9fcd36337cc93fd5beeabb9186ad5e93ae2a0f0
2011-10-11 16:40:06 -04:00
Jenkins
fe1f1650a2 Merge "Refactoring registry api controllers" 2011-10-01 17:50:21 +00:00
Josh Durgin
42300518ea Add RBD store backend.
To use this you'll need the latest version of Ceph from git,
which will become version 0.37.

Change-Id: Ia2126395525759eb8c11df055bdc6f286dea97b9
2011-09-29 13:54:19 -07:00
Brian Waldon
7c4d935026 Refactoring registry api controllers
Break up images and members controllers in the registry api and restructure it to be more like the glance api. This is
in preparation for the future split.

Change-Id: I81e6c74815cf15adebb526b31614fd2e63982c3d
2011-09-29 10:01:46 -04:00
Jason Kölker
27b3df281f * Scrubber now uses registry client to communicate with registry
* glance-api writes out to a scrubber "queue" dir on delete
* Scrubber determines images to deleted from "queue" dir not db

Change-Id: Ia5574fc75f1a9c763bdef0f5773c2c182932b68a
2011-09-12 18:29:19 -05:00
Dan Prince
22d7864a41 Update paste config files with keystone examples.
see ticket: lp839559

Change-Id: Ie5d5011a601962bcc79cf69ef8db11029fcb866e
2011-09-02 09:32:54 -04:00
Jay Pipes
501b14bf94 Fixes LP Bug #827660 - Swift driver fail 5G upload
Fixes LP Bug #827660 - Swift driver fails to
upload image files greater than 5GB. The swift
store driver was not using the large object
manifest methods required to upload objects
greater than 5GB into Swift.

Adds configuration options to specify the
size in MB that an image file should be considered
for large object manifest, and a configurable
chunk size. We use a disk buffer of this size,
adding chunks of the image file into Swift and
then after all chunks are saved, add the manifest
to Swift.

Change-Id: I4b4421aa1bba584ab242016a041e864ef19d0214
2011-08-24 14:38:43 -04:00
Jenkins
4fa35070ce Merge "Add support for shared images" 2011-08-15 21:37:40 +00:00
Kevin L. Mitchell
1e4be06cb2 Add support for shared images
Change-Id: I3822a3841e1c10717c180052f929688b9f21a841
2011-08-15 16:20:49 -05:00
Johannes Erdfelt
091aae8a6d Add notifications for uploads, updates and deletes
Change-Id: I372f77fe2d1a575f2108c9b8d1f69301c0d5eb5e
2011-08-15 19:58:57 +00:00
Rick Harris
c451de83b9 Adds syslog support
Change-Id: I05ef04c77b68f0c68d86ae49a5fee68c2f86c80a
2011-08-11 20:15:33 +00:00
Jenkins
6cc1a157f9 Merge "breaking up MAX_ITEM_LIMIT and making the new values configurable" 2011-08-08 18:36:56 +00:00
Brian Waldon
faebdec11b breaking up MAX_ITEM_LIMIT and making the new values configurable
Change-Id: I6655c50a871c75e748106884aa8ab5f1ebe274a3
2011-08-08 13:59:14 -04:00
Rick Harris
09a67e4fb4 Adds swift_enable_snet config
Change-Id: I7b798e2b4256a1b6c4694f0d126d2fa4604c95aa
2011-08-05 11:54:52 -05:00
jaypipes@gmail.com
38cc809bb8 Merge latest trunk 2011-07-28 19:01:40 -04:00
Rick Harris
a4f4d6157c Adding back image_cache_enabled config option for glance-api 2011-07-27 19:50:04 -05:00
jaypipes@gmail.com
dfc71f44f2 Typo in error condition for create_bucket_on_put, make body seekable in req object, and remove +glance from docs and configs 2011-07-26 10:26:19 -04:00
Rick Harris
957e4d1831 Merging trunk 2011-07-25 12:53:01 -05:00
jaypipes@gmail.com
684d89117e Fixed review stuff from Brian 2011-07-25 13:45:35 -04:00
Jason Koelker
e4eaef7bb8 merge trunk 2011-07-22 19:07:38 -05:00
Rick Harris
631d707a35 Removing cache enabled flag from most confs 2011-07-22 17:11:18 -05:00
Rick Harris
35d9da1178 Removing imagecache from default WSGI pipeline 2011-07-22 16:03:03 -05:00
Rick Harris
11cd9dc8d1 Adding reap stalled images 2011-07-22 12:00:09 -05:00
Rick Harris
e54deefaff Adding reaper script 2011-07-22 00:53:43 -05:00
Kevin L. Mitchell
52064a6378 Add in security context information 2011-07-20 22:53:44 +00:00
Rick Harris
8094e74f5d Initial work on moving cached_images to WSGI middleware 2011-07-20 16:58:59 -05:00
Rick Harris
95c0acb9cb Adding prefetching of images 2011-07-13 23:00:52 +00:00
Rick Harris
7bc060eb31 Pruning invalid cache entries after grace period 2011-07-13 04:51:51 +00:00
Rick Harris
a4369c75b3 Create bin/glance-pruner 2011-07-12 02:55:45 -05:00
Rick Harris
4d4ef5482a Adding image_caching 2011-07-12 02:04:36 -05:00
Jason Kölker
112f6be82b default to no daemon 2011-06-15 10:47:38 -05:00
Jason Kölker
a517e1ab81 remove unnecessary option 2011-06-10 13:25:15 -05:00
Jason Kölker
0f88cfce25 pass in a wakeup_time for the default time between database hits 2011-06-09 16:42:34 -05:00
Jason Kölker
33c921a261 start making the server that will periodicly scrub 2011-06-09 16:37:31 -05:00
Jason Kölker
8cbfca5aa1 Config file for the scrubber. We make our own connection to the db here and
bypass using the registry client so we don't have to expose non-public
images over the http connection.
2011-06-09 15:40:00 -05:00
Jason Kölker
4cdc31d32d make the commits 2011-06-09 13:23:18 -05:00
Jason Kölker
f8d10a4bdf all delayed deletes will be going through a new service, if delayed_delete is False, then delete it right away, otherwise set it to pending_delete 2011-06-09 11:50:35 -05:00
Jason Kölker
24d17c1f13 add in delayed delete options 2011-06-08 13:45:31 -05:00