1195 Commits

Author SHA1 Message Date
Jenkins
70596b1d00 Merge "[Django] Allow to upload the image directly to Glance service" 2016-08-18 03:38:55 +00:00
Jenkins
2cb6970d28 Merge "Fix typo in keystone_data.py" 2016-08-18 03:33:13 +00:00
Jenkins
77a804ea4b Merge "Add more Nova API features for NG Instances" 2016-08-18 03:32:55 +00:00
Nguyen Hung Phuong
21b723ede3 Fix typo
TrivialFix

Change-Id: I94fba94aea07f697173cceacf55358f04777aa2f
2016-08-17 14:26:35 +07:00
zhangyanxian
d52631534b Fix typo in keystone_data.py
TrivialFix

Change-Id: I9cb576b47a645d96f57481ea3c4230efbd16b4bd
2016-08-17 02:58:47 +00:00
Jenkins
eaa194e497 Merge "Add ANGULAR_FEATURES setting dict" 2016-08-16 23:05:06 +00:00
Timur Sufiev
93af461e40 [Django] Allow to upload the image directly to Glance service
Since large Glance images even temporarily stored on dashboard side
tend to fill up Web Server filesystem, it is desirable to route image
payload directly to Glance service (which usually streams it to
storage backend, which in turn has plenty of space).

To make it possible we need to trick Django into thinking that a file
was selected inside FileInput, while its contents are not actually
transferred to Django server. Then, once image is created client-side
code needs to know the exact url the image payload needs to be
transferred to. Both tasks are solved via using ExternalFileField /
ExternalUploadMeta classes which allow to work around the usual Django
form processing workflow with minimal changes to CreateImage form
business logic.

The client-side code relies on CORS being enabled for Glance service
(otherwise browser would forbid the PUT request to a location
different from the one form content came from). In a Devstack setup
you'll need to edit [cors] section of glance-api.conf file, setting
`allowed_origin` setting to the full hostname of the web server (say,
http://<HOST_IP>/dashboard) and restart glance-api process.

A progress bar is implemented to track the progress of a file upload,
in case a really huge image is transferred.

The new machinery could be easily switched on/off with a single
setting `HORIZON_IMAGES_UPLOAD_MODE` set to 'direct' / 'legacy'.

Related-Bug: #1467890
Closes-Bug: #1403129
Implements blueprint: horizon-glance-large-image-upload
Change-Id: I01d02f75268186b43066df6fd966aa01c08e01d7
2016-08-16 14:30:38 +03:00
Luong Anh Tuan
ac404c4bcf Fix order of arguments in assertEqual
Some tests used incorrect order assertEqual(observed, expected).

The correct order expected by testtools is
assertEqual(expected, observed).

Change-Id: I83b7009f9b32f4abc88c23cafbe293b65dc25fb4
Partial-Bug: #1259292
2016-08-16 12:02:45 +07:00
Rob Cresswell
1050755d05 Add ANGULAR_FEATURES setting dict
We should unify the angular features (new panels, workflows, etc.) in a
single setting. This also makes it a little cleaner to read, simply
using the 'truthiness' of a key rather than string comparisons.

I haven't moved the 'swift_panel' setting, as that panel will be removed
in the O cycle anway, so it seems pointless to move a setting causing
potential issues for a single cycle.

Change-Id: Ia5702ff523355ae895e14cc3d49c895128478944
2016-08-15 21:01:53 +00:00
Matt Borland
93b7edd694 Add more Nova API features for NG Instances
This patch adds more Nova API features for instances.  It establishes
several new proxy APIs and provides the various JavaScript libraries for
accessing those APIs as well.

Change-Id: I016be1d0598faf78b1fb02f4d0768efdaf6cb7bf
Partially-Implements: blueprint angularize-instances-table
2016-08-12 14:43:37 -06:00
Jenkins
72ad1e3fd5 Merge "[NG] Support local file upload in Create Image workflow" 2016-08-08 23:01:16 +00:00
Timur Sufiev
0e1279d05c [NG] Support local file upload in Create Image workflow
First, now there are 2 '/api/glance/images/ API wrapper endpoints for
creating a new image - POST and PUT. The POST endpoint which existed
before now resides at PUT. This was done to support legacy
(i.e. proxied by web-server) file uploads in Angular Create Image,
because Django (which we need to use in that case) doesn't correctly
process PUT request. So, if local file binary payload is added to the
form contents, we send it using POST request.

Second, speaking of '/api/glance/images' PUT request previously known
as POST... Where before the POST call to Horizon REST API wrappers
returned the final image object that Glance just created, now there
are two possibilities for what happens after PUT is sent.

* When Create Image form Source Type is set to URL, then everything is
  going as before.
* When Source Type is set to File, then just a file name instead of an
  actual Blob is sent to '/api/glance/images', then the Glance Image
  is queued for creation and Horizon web-server responds with an Image
  object which dict() representation has 2 additional keys:
  'upload_url' and 'token_id'. The 'upload_url' tells the location for
  the subsequent CORS request, while 'token_id' is passed as a header
  in that request, so Keystone would let it in. CORS upload is started
  immediately as Image is queued for creation (first promise is
  resolved) and returns the second promise, which is resolved once the
  upload finishes. The modal form hangs until second promise resolves
  to indicate that upload is in progress. Upload progress notification
  is added in a follow-up patch.

DEPLOY NOTES

The client-side code relies on CORS being enabled for Glance service
(otherwise browser would forbid the PUT request to a location
different from the one form content came from). In a Devstack setup
you'll need to edit [cors] section of glance-api.conf file, setting
`allowed_origin` setting to the full hostname of the web server (say,
http://<HOST_IP>/dashboard).

Related-Bug: #1467890
Implements blueprint: horizon-glance-large-image-upload
Change-Id: I5d842d614c16d3250380ea1dc1c6e0289d206fb5
2016-08-08 20:02:22 +03:00
Matt Borland
20bc6e1516 Make 'switch' between legacy and Angular Images
This patch follows on the example that the Containers set, providing
a 'switch' in the panel-enablement file that currently defaults to
'legacy' (Python-based Images panel) and allows for 'angular' (Angular-
based Images panel).

To be clear, this does NOT enable Angular Images.  It's just setting the
stage to do so at some point, or to allow deployers/devs to easily switch
between the two.

A switch both for HORIZON_CONFIG and for integration tests is necessary
due to the way integration tests operate.

Co-Authored-By: Timur Sufiev <tsufiev@mirantis.com>
Change-Id: I12cd33552218ed1082d2d9a2ae8982639a217a6a
Partially-Implements: blueprint angularize-images-table
2016-08-05 08:17:21 -06:00
Luis Daniel Castellanos
989607e4fb Stop memoizing on request argument in neutron api
neutron api should memoize more persistanly on objects that do not
change when rendering each view, i.e. Supported extensions list

Implements blueprint: improve-horizon-caching

Change-Id: Iec38e865a704752309ba7035d19370dc0dbc31e0
2016-07-27 17:20:17 -05:00
Jenkins
224088a3f8 Merge "Embed support for external data sinks into api.glance" 2016-07-27 09:02:51 +00:00
Jenkins
4aa405bc08 Merge "Allow horizon to function without nova" 2016-07-26 20:59:55 +00:00
Steve McLellan
018e99d20e Allow horizon to function without nova
Adds conditional block to nova quotas to exclude them if nova is not
enabled; adds 'permission' checks to the project overview and
access_and_security panels to only enable them if compute is enabled;
adds permission checks on compute and image to the admin overview
and metadef panels; disables 'modify quota' and 'view usage' project
actions; disables 'update defaults' if there are no quotas available.

The 'access and security' panel still appears (under Compute) but
tabs other than the keystone endpoint and RC download tab are hidden.

Closes-Bug: #1580116
Change-Id: I1b2ddee0395ad9f55692111604b31618c4eaf69e
2016-07-26 09:15:48 -05:00
Timur Sufiev
7e65af5f13 Embed support for external data sinks into api.glance
In case 'data' image attribute is a base string (instead of in-memory
or on-disk file), api.glance sends back an image wrapper with a redirect
url and a token to its caller, so the caller could upload the file to
that url directly.

Provide a unit test for api.glance behavior when an external upload
location is used. That also requires to fix glance stub endpoint data
in keystone_data.py since it didn't reflect the reality.

Also document the new HORIZON_IMAGES_UPLOAD_MODE setting that will
govern direct images upload and the define approach to deprecating the
old HORIZON_IMAGES_ALLOW_UPLOAD setting. The old setting is deprecated
as of Newton release and planned to be removed in P. 'Removing' means
that it will no longer be used / referenced at all in code, not the
actual presence in settings.py (it is removed from settings.py in this
commit). What really matters is if the customized value of
HORIZON_IMAGES_ALLOW_UPLOAD in local_settings.py will be still
considered during the deprecation period.

Help text in Django Create Image form in case if local file upload was
enabled was wrong, fixed that.

Related-Bug: #1403129
Partially implements blueprint: horizon-glance-large-image-upload

Change-Id: I24ff55e0135514fae89c20175cf9c764e871969b
2016-07-26 13:38:58 +00:00
Jenkins
7ca5b3c0ff Merge "Choose a server group when booting a VM with NG launch instance" 2016-07-21 15:02:54 +00:00
Timur Sufiev
baca29144b Pay attention to Nova disabled quotas defined in a config file
Since Nova doesn't currently provide an API call to indicate whether
its quotas are disabled, we could use a parameter named
'enable_quotas' within 'OPENSTACK_HYPERVISOR_FEATURES' setting for
this purpose. This allows to avoid errors while trying to update
quotas which are disabled on service side. Also make disabled_quotas
collection to be a set instead of a list - this removes duplicate
fields that appear due to some possible quota overlaps between Nova
and Neutron.

Also, since we dropped out python2.6 support fancy set literals and
dict comprehensions can be used.

Co-Authored-By: Paul Karikh <pkarikh@mirantis.com>
Closes-Bug: #1286099
Change-Id: I10923f147e4c323aba8bbcc130d2016ad6725e86
2016-07-19 17:51:08 +00:00
Brad Pokorny
cf91124d0c Choose a server group when booting a VM with NG launch instance
Allow users to choose a server group when booting a VM.  Adds
an optional workflow step to the launch instance workflow that
shows the available server groups and details about each group.
The ability to choose a server groups already exists for the
legacy launch instance workflow as a dropdown list, but having
it as a separate step in the angular workflow provides the added
capability of seeing group details.

To test this patch, create a server group via the nova CLI. Example:
  nova server-group-create group1 affinity

And use the angular launch instance workflow to select a server
group. To validate the new instance was added to the server group,
use the nova CLI:
  nova server-group-get [ID of server group]

Change-Id: I651817850ef8a5afec047a9a481843a5eddbf5a9
Implements: blueprint nova-server-groups
2016-07-19 09:25:52 -07:00
Kevin Fox
31e5672905 Angular direct service access.
Angular based Horizon widgets and Horizon plugins will want to
contact OpenStack API's directly to enable much more responsive UI
and to place less load on the Horizon servers by proxying service
calls through horizon proxies.

Change-Id: I612c6bfefa8e157f7219938bb3e82896dde8fa09
2016-07-18 13:30:51 -07:00
Jenkins
c2fc95eb48 Merge "Revert "Prevent creation of subnet via RBAC during new network creation"" 2016-07-14 21:50:31 +00:00
David Lyle
31d6e2d4dc Revert "Prevent creation of subnet via RBAC during new network creation"
The policy rule to create subnets requires a target to make the policy check. This implementation does not provide a target, so is not a proper place to use the workflow policy checks. To do this, a more involved approach needs to be used that supplies a target. However, this will be challenging because the target is the network_id that has yet to be created.

This reverts commit 3befade1411783af088b0e72b08f8ff3701b02c8.

Change-Id: I6af1cbd559034bd5635dfb4360b525714225677d
2016-07-14 18:08:17 +00:00
Jenkins
107488f2f5 Merge "Pre-populate the Angular template cache and allow template overrides" 2016-07-14 01:06:01 +00:00
Jenkins
461f2edc26 Merge "Do not import local/enabled when to run test" 2016-07-13 23:34:53 +00:00
Tyr Johanson
fca46ab60f Pre-populate the Angular template cache and allow template overrides
This patch populates the Angular template cache from Django.
This eliminates the need for Angular to do an http get for every HTML
fragment.

In addition, now that we are filling the template cache, this patch
introduces the logic needed to override any Angular template HTML from
the current theme.

How it works:
A new template tag is created called "template_cache_preloads". This
tag is used in _scripts.html to generate a list of text/javascript
script tags, each one containing an Angular "run" method that loads
a template contents into the Angular template cache. The first time
any Horizon page is loaded after server start, the template cache
preloads are computed for the current theme.

The output of this tag is cached for 30 days in Django using the
"cache" tag. Further, that cached result is wrapped in a "compress js"
tag to collapse the individual <script> tags into 1 block of
javascript, and compress like all other javascript Horizon serves to
the client.

Finally, when using offline compression, the compressor evaluates the
nodelist (HTML content) of _scripts.html, notices the compress tag
and builds the template cache preloads for each possible theme. Later,
at runtime, when the preloads are generated for the current theme, the
compressor gets the result from the Django cache, and hashes the
contents to determine which manifest file to serve to the client.
Since the preloads generated at run-time are identical to those
generated off-line, the compressor hash matches an existing manifest
which is served to the client.

Notice that even though the template cache pre-loads are generated
off-line...the template_cache_preloads tag will be executed once
every 30 days anyway. However, since the result matches the off-line
compression, the existing manifest continues to be served to the client.

Finally, this patch ALSO watches for 'post_compress' signals. If it
detects that the angular template preloads have been re-compressed, it
clears the old version from the Django cache.

To test the template caching:
- Run horizon
- View page source
- Notice the new <script type="text/javascript"> tags contained in
  the body (only visible if COMPRESS_ENABLED=False
- Open the javascript inspector
- Load launch instance
- Notice there are no longer http calls to load each HTML fragment
  used by the Angular launch instance

To test the override:
- Set the DEFAULT_THEME='material'
- Create /horizon/openstack_dashboard/themes/material/\
static/templates/framework/widgets/help-panel/help-panel.html
- Set the content to <h1>TEST</h1>
- Run Horizon and open launch instance.
- The help content should contain "TEST"

To test the new template tag:
- set a breakpoint or print in angular.py:template_cache_preloads
  and observe when it is called during off-line or run-time use

Co-Authored-By: Diana Whitten <hurgleburgler@gmail.com>

Implements: blueprint angular-template-overrides
Change-Id: I0e4e2623be58abbc68c6e02b2e9c5d7cdaba8e4d
2016-07-13 15:38:07 -07:00
wangbo
f5c41b609f Do not import local/enabled when to run test
For horizon, it should only test openstack_dashboard exclude any plugin.

It throws error "can not find app searchlight_ui" When I run test
with enabling searchlight_ui plugin.

Change-Id: I203685fdafd577068769f2e362ecb2ef5385b207
Closes-Bug: #1554513
2016-07-01 18:31:36 +08:00
Richard Jones
086b33a978 Fix handling of swift object name == pseudo-folder name
Reinstate the proper path for swift pseudo-folders so
that they can be differentiated when adjacent to objects
with the same name.

In adding the test code for this patch the test data
also had to be modified to be more correctly mirroring
the API response data we get.

Change-Id: If0df0be612c0eeb999e2633eb6a00da80b2e0cf8
Closes-Bug: 1596568
2016-07-01 16:29:00 +10:00
Vivek Agrawal
a05883d88a Addition of metadata for snapshot in VolumeSnapshot object
When accessing the snapshot object in Horizon codebase, we do not have
access to the metadata of the snapshot. This is because the
VolumeSnapshot object does not have metadata as a member. So added
'metadata' to the VolumeSnapshot object. The fix should provide access
to metadata.
Added test data and test cases for snapshot object with metadata.

Closes-Bug: 1542468

Change-Id: I4ae2e7a74d3ddbff8493b42f8f86c7002229cf8c
2016-06-27 14:12:31 +00:00
Jenkins
89d0418a34 Merge " Fix some spelling mistakes in horizon intermittantly should be intermittently finctionality should be functionality funcionalities should be functionalities" 2016-06-24 18:07:52 +00:00
Jenkins
ff6ed5a0ae Merge "Adding Identity NGProjects API" 2016-06-24 03:08:42 +00:00
Jenkins
12d7d0bc72 Merge "Make sure we always get all limits" 2016-06-24 02:42:40 +00:00
Jenkins
d6b7b70c0d Merge "Improved table deletion efficiency" 2016-06-24 00:20:52 +00:00
Jenkins
a674fbeaff Merge "Fix dropdowns sometimes not opening in integration tests" 2016-06-23 17:26:50 +00:00
gecong1973
0854bb9962 Fix some spelling mistakes in horizon
intermittantly should be intermittently
  finctionality should be  functionality
  funcionalities should be functionalities

  Closes-Bug: #1595439

Change-Id: If04aa2170a0c27ebfa89a1e3fa1539ea0e52f7b6
2016-06-23 17:04:35 +08:00
Timur Sufiev
fe04af039f Fix dropdowns sometimes not opening in integration tests
Since most probable cause of dropdowns not opening intermittently is
Bootstrap initialization code not having enough time to complete
before test tries to open dropdown, make test wait a specified timeout
before clicking it (by means of setting a CSS class after that
timeout, which test in turn waits before clicking the dropdown).

Closes-Bug: #1594926
Change-Id: I32625caa4433f4af0de72c94b61e85ab4e16b1f9
2016-06-22 16:19:08 +03:00
jlopezgu
b450a03282 Improved table deletion efficiency
Volumes and instances integration tests deleted and validated entities
one row at a time, now they are able to delete and validate multiple
rows at a time.

Change-Id: I7f8fdc7e0ccefdd0f6d71f9f92c8a2aefeec8d54
Closes-bug: #1585209
2016-06-21 14:03:58 -05:00
Paulo Ewerton Gomes Fragoso
3945b24062 Adding Identity NGProjects API
This patch adds some projects-related endpoints to the Nova,
Cinder and Neutron REST/Angular APIs.

These changes are needed for the angular Identity Projects
actions that handle mainly project quota updates.

Partially-Implements: blueprint angularize-identity-projects

Change-Id: I98b8eb9e5e7bfb0d49a77c00115fe62d412abe21
2016-06-21 13:12:43 +00:00
Jenkins
a193f55172 Merge "python 3.4 support for urlparse" 2016-06-21 08:46:50 +00:00
Jenkins
b4fc2b0939 Merge "Prevent creation of subnet via RBAC during new network creation" 2016-06-17 21:32:32 +00:00
Jenkins
021daad8bd Merge "Stop memoizing on request arguments in nova api" 2016-06-17 08:40:28 +00:00
Rich Hagarty
b1a22463d6 Add support for volume consistency group snapshots
This extends the support for Cinder volume consistency groups.
This patch adds snapshot support for all volumes contained
within a consistency group.

Additional features includes the ability to create a new
consistency group from an existing consistency group
or consistency group snapshot. This will involve cloning
all volumes in the original CG.

To enable this feature, update "consistencygroup" items in
the cinder policy file (/etc/cinder/policy.json). The default
is that all consistencygroup actions are disabled.

Co-Authored-By: Brad Pokorny <brad_pokorny@symantec.com>
Change-Id: I2e2ae26369a23415bf58d6edac8d161d3a125dd6
Implements: blueprint cinder-consistency-groups
2016-06-15 22:57:33 +00:00
Jenkins
e9ade88fd0 Merge "Change default timeouts in integration tests" 2016-06-15 21:39:28 +00:00
Richard Jones
62b4e6f30a Escape angularjs templating in unsafe HTML
This code extends the unsafe (typically user-supplied) HTML escape
built into Django to also escape angularjs templating markers. Safe
HTML will be unaffected.

Closes-bug: 1567673
Change-Id: I0cbebfd0f814bdf1bf8c06833abf33cc2d4748e7
2016-06-15 11:03:01 -04:00
Timur Sufiev
eecb8bfe14 Change default timeouts in integration tests
First, increase page_timeout to one minute, which should help in case
of a first couple of tests waiting for Apache workers to "warm up".

Second, reduce explicit_wait to 90 seconds (from 5 minutes), because
if something in Horizon doesn't complete in 90 seconds, then almost
for sure it won't complete in 5 minutes as well and only will spend
time, which is limited for integration tests.

Change-Id: Icb0767146fdb2682dca237ee18bbc4c3c8d976d0
Closes-Bug: #1592349
2016-06-14 13:47:26 +03:00
Jenkins
5a7bef2ff1 Merge "admin permissions depends on OPENSTACK_KEYSTONE_ADMIN_ROLES" 2016-06-12 01:02:35 +00:00
Jenkins
8289a26725 Merge "Handle non ASCII tenant names gracefully when downloading RC file" 2016-06-12 00:55:19 +00:00
Yves-Gwenael Bourhis
56fce18378 Stop memoizing on request arguments in nova api
The request parameter is a django.http.HttpRequest instance object
which contains metadata about the request. This object is a new instance of
django.http.HttpRequest in every view, so memoizing with it memoizes during the
rendering of a single view only, thus preventing multiple api calls only
during the rendering of one view.

Some nova api memoized methods should memoize more persistantly with objects
which do not change after the rendering of each view, like the list of
extensions suported.

However, it is still a good idea to limit memoization of some api calls during
the rendering of a single view, if the query is subject to give different
results (i.e., the flavor_list can change when an administrator adds a flavor
so we need to retreive new data for each view in case a flavor gets added by a
cloud admin).

Implements blueprint: improve-horizon-caching
Co-Authored-By: Daniel Castellanos <luis.daniel.castellanos@intel.com>
Change-Id: I9f87df72cbf149e7b2d1763d7976cb845b1e98f9
2016-06-10 13:22:14 -05:00
Jenkins
fa6aa731c9 Merge "In integration tests prevent getting volume size too early" 2016-06-10 11:22:41 +00:00