56 Commits

Author SHA1 Message Date
Radoslav Gerganov
ca17b7bd39 Fix setting the SOAP headers for remote calls.
We use SOAP headers to set 'opID' for all service calls and
'vcSessionCookie' for PBM service calls. Commit 6a6336 introduced
regression in setting the vcSessionCookie. This patch fixes this by
setting all headers in one place.

Change-Id: If9630a14d6275f30275825f1822f2cf9af2959a7
2017-01-13 13:08:54 +02:00
Radoslav Gerganov
c9357e8677 Remove some useless log messages
'Waiting for function _invoke_api to return' doesn't give any
information and only makes log files hard to read.

Also reduce logging when polling NFC leases.

Change-Id: I08653a0f529a186405695cbcfb9fc9018c93f856
2016-12-21 17:40:49 +02:00
Jenkins
9b7e1deb2c Merge "Add operation ID for remote calls" 2016-12-21 08:44:07 +00:00
Radoslav Gerganov
6a63362919 Add operation ID for remote calls
The operation ID is a random string which can be added in the SOAP
headers of the remote request. vCenter and ESX will append this opID to
every log message associated with the remote request. This allows
administrators to correlate log messages for particular operation across
different systems (OpenStack -> vCenter -> ESX).

There is also addition keyword argument 'skip_op_id' which allows
callers to disable logging the opID if set to True.

Example for correlating logs for CreateVM_Task on devstack, vCenter and
ESX: http://paste.openstack.org/show/592614/

Change-Id: I75cb71c7c58b4ee9cd36fc977eb2a18a911f161f
2016-12-20 17:46:49 +02:00
Vipin Balachandran
a95530e97d Fix image meta-data update for Glance v2
This patch fixes the following issues with Glance v2 image meta-
data update:

* Glance v2 doesn't support updating image size. The size will
be set by the glance service and there is no need to update it.

* We need to convert the image version to string before update
because Glance v2 doesn't support integer values for properties.

This patch also deprecates the image version kwarg because it is
not used by VMware drivers.

Change-Id: I0da5958cbde8be2008ee00602e6e9561bf167bfd
Closes-bug: #1611288
2016-12-16 07:01:47 +00:00
Jenkins
e0fb3094f2 Merge "Files with no code must be left completely empty" 2016-11-29 15:08:41 +00:00
Cao Xuan Hoang
94fdb315c5 Files with no code must be left completely empty
As per OpenStack licensing guide lines [1]:
[H104] Files with no code shouldn't contain any license header nor
comments, and must be left completely empty.

[1] http://docs.openstack.org/developer/hacking/#openstack-licensing

Change-Id: Ic6e90fec4f93c3400c6a24437f7f7382dad5b5d3
2016-11-28 13:42:21 +07:00
int32bit
2c2956ed33 Add 'uncommitted' field to the 'Datastore' class
The 'uncommitted' field is in official API, but currently not in our
Datastore class, we need it to compute the amount of provisioned space:

provisionedSpace = Capacity - freeSpace - uncommitted

So a `uncommitted` keyword argument has to be added to the Datastore
class.

Change-Id: I156e2b4fb9a8355848947ec9dbb6abf014588fdf
Closes-Bug: #1643569
2016-11-25 21:53:29 +08:00
howardlee
061f5447da [TrivialFix] Replace 'assertTrue(a in b)' with 'assertIn(a, b)'
Change-Id: Ie7e037eaf2274cfb2f9f0700027f2c6f2071b16a
2016-10-21 16:29:02 +08:00
Radoslav Gerganov
53c7c370bc Set pool size for HTTPS connections
Setting the pool size was done only for the 'file' protocol which has no
effect when communicating with vCenter over HTTPS. This patch sets the
pool size for HTTPS connections as well.

Change-Id: I1bb72d27a6861b4a2cf327bfd6705d58a31d49c5
Closes-Bug: #1618477
2016-08-30 17:16:28 +03:00
Jenkins
6bf58463e0 Merge "Pass connection timeout so that invoke_api will not wait forever" 2016-08-26 05:24:43 +00:00
Haifeng.Yan
9c998c3b0c Fix TypeError:six.StringIO(resp.content) must be str or None, not bytes.
in py2, using requests, type(response.conten)=unicode,
six.StringIO(resp.content) is fine. But in py3,
type(response.conten)=byte, six.StringIO(resp.content) will raise
TypeError: initial_value must be str or None, not bytes.

This patch fix it by using six.BytesIO instead of six.StringIO, which is
actually the same as six.StringIO in py2.

Change-Id: If1b910104f36b583946f62b5a4a3c4fd7e16f6c2
Closes-Bug: 1610125
2016-08-17 02:48:22 +08:00
Vipin Balachandran
03faa1c24b Method to download file to VMware server
This patch adds a utility method in image_transfer to download
a file to VMware server.

Change-Id: Ic378d21d036f5296310697d73dbc6ee284fdd187
2016-08-02 22:25:37 +05:30
Aman Kumar
b32f627b81 Pass connection timeout so that invoke_api will not wait forever
invoke_api does not throw any exception when interface goes down and
application loses connectivity with vCenter because default timeout
is set to None which is equivalent to infinity.

This patch set allows any application to pass connection timeout so that
it will wait for the given timeout and then throw an exception.

Closes-Bug: #1564839

Change-Id: I69b099b7caaa4fb0e0067cb83b9468d4aa901cc7
Co-Authored-By: Raghuveer Shenoy <rshenoy@hp.com>
2016-08-02 08:35:52 +00:00
Sabari Kumar Murugesan
0258fe0c6a Add http_method to download_stream_optimized_data
download_stream_optimized_data method doesn't pass
http_method to VmdkWriteHandle. Added the kwarg.

Change-Id: I38755c0a693ee29189a79f48b9f61beb2e45de9a
2016-07-25 16:45:45 -07:00
Radoslav Gerganov
2f9af244de Refactor the image transfer
The image transfer is unnecessary complicated and buggy.
This patch replaces the complex usage of blocking queues and threads
with a simple read+write loop. It has the same performance and the
code is much cleaner. The NFC lease is updated with the
loopingcall utility.

Change-Id: I2b04173cd23c59162056360c03d419efbce77ba1
Closes-Bug: #1384840
2016-07-25 10:46:11 +03:00
Jenkins
ff60efbe74 Merge "Remove unnecessary properties from image-meta" 2016-06-29 19:28:45 +00:00
Vipin Balachandran
c202eff4e1 Remove unnecessary properties from image-meta
This patch removes unnecessary properties such as 'is_public',
'container_format' and 'status' from image-meta in the
image_tranfer.upload_image API. Removing these properties
does not affect backward compatibility.

Change-Id: I9f379f6ada91141da79fd75fbc4a1550a8a06db6
Partial-bug: #1596939
2016-06-28 19:00:51 +05:30
Jenkins
5fb1799d1c Merge "Support download of virtual disk in ova container" 2016-06-20 12:57:51 +00:00
Jenkins
1f8d09cbd0 Merge "Refactor VmdkWriteHandle and VmdkReadHandle" 2016-06-13 18:20:19 +00:00
Javeme
c19184765d Refactor VmdkWriteHandle and VmdkReadHandle
Add a class VmdkHandle as the base class of VmdkWriteHandle and
VmdkReadHandle, and add related methods into it such as:
 def update_progress()
 def _release_lease()

Change-Id: I29c48773842c1006b8499aad054e3901074d6195
Depends-On: Ib5b388f7eeb9f26de66c308a89f70c85ba6dc7a9
2016-05-31 10:44:33 +08:00
Vipin Balachandran
784ad53fc4 Support download of virtual disk in ova container
This patch updates download_stream_optimized_image method
in image_transfer to support streamOptimized disks in ova
container.

Change-Id: I693101f8b22eb416c5c0f92957ebb680d3b5bd8b
2016-05-19 22:45:40 -07:00
Jenkins
961869e061 Merge "Should not raise Exception before connection close" 2016-03-11 17:45:07 +00:00
Javeme
8ec8c09417 Should not raise Exception before connection close
Currently VmdkReadHandle.close() raises a VimException before
the read connection is closed, this patch let's add a `finally`
to close it.

Change-Id: Ib5b388f7eeb9f26de66c308a89f70c85ba6dc7a9
2016-03-11 11:40:58 +08:00
Eric Brown
90ca36466b Remove explicit use of asserts
The Bandit linter flagged code using the assert function. This
removes the assert and ensures an exception is thrown with
another call.

Change-Id: I582e0985e46414f97bd15d6d763bae132de7c12a
2016-03-07 12:59:32 -08:00
Javeme
0f9fc58d3a assertIsNone(val) instead of assertEqual(None,val)
use assertIsNone(val) instead of assertEqual(None, val).

Change-Id: I8312b91da916cee8bbafed3674d70c8371b34fac
2015-12-21 19:14:36 +08:00
Cedric Brandily
a5c6ed65d2 Define WithRetrieval context
WithRetrieval[1] is a context that handles and hides object retrieval
and retrieval cancel (when needed):

 with WithRetrieval(vim, retrieve_result) as objects:
   for obj in objects:
     do_something_including_breaking_for_loop(obj)

instead of:

 while retrieve_result:
   for obj in retrieve_result.objects:
      do_something_including_breaking_while_loop(obj)
   retrieve_result = continue_retrieval(vim, retrieve_result)
 if retrieve_result:
   cancel_retrieval(vim, retrieve_result)

[1] oslo_vmware.vim_util

Closes-Bug: #1284658
Change-Id: If29d45a25a81340e138eb9068a551fc1b172cf87
2015-09-29 11:38:24 +02:00
Jenkins
05dd89936f Merge "Add unit tests for FileHandle._create_connection" 2015-09-22 00:03:51 +00:00
Radoslav Gerganov
89e10095a7 Add support for using HTTP POST in VmdkWriteHandle
The NFC protocol requires HTTP POST with content type
'application/x-vnd.vmware-streamVmdk' in order to provision virtual
disks in thin format. This patch allows clients to specify the HTTP verb
when using VmdkWriteHandle and thus enables thin provisioning.

Change-Id: Ie11c2bfbbb1ca71e9d35c9407f871214b6aa0e9a
2015-09-17 13:22:38 +03:00
Vipin Balachandran
6cbffef1d7 Add unit tests for FileHandle._create_connection
Patch to increase unit test coverage for FileHandle._create_connection.

Change-Id: Ib495ff0c8fdfbefbb25d1c0b668df9a3afc9219b
2015-09-09 19:21:52 +05:30
Radoslav Gerganov
c47cfdcf6a Use SSL thumbprints for NFC transfer
There are cases when clients cannot build a trust chain in order to
verify ESX certificate. The SSL thumbprint is always available and it is
the preferred way for verifying the connection when doing NFC transfer.
This patch replace the usage of truststore with SSL thumbprints in
VmdkReadHandle and VmdkWriteHandle.

Change-Id: I9573bada15aef2bf8d25fa824bbbc2b421586733
2015-08-27 15:15:56 +03:00
ZHU ZHU
0fdb00522a Allow http connection pool size to be configured
With multiple concurrent VM spawning, the service time from compute
service flow to vcenter to trigger soap requests is getting slower.
As VIM adopt requests module to access the vcenter which have default
http pool size set to 10. This need to be configured to be larger
in scale environment.

Change-Id: Ic9e9a87de378151a66187222231eaafd7a402ac5
Closes-Bug: #1479183
2015-07-31 06:54:12 -05:00
Doug Hellmann
a1a6393990 Fix mock calls
Use the proper assert methods of the mock objects.

Squashed with Icc2c4095b78ef4dba091fc6792f0a38de1fda267 so
we can merge this change.

Change-Id: I8474b6a5cef1133fa6cb36bf08962113c6d908dc
2015-07-12 13:27:43 -04:00
Vipin Balachandran
d8ccf388f9 Mask/truncate vCenter credentials in suds logs
Suds logs will contain vCenter credentials if VIM Login,
SessionIsActive APIs fail. This patch adds a logging
filter to mask/truncate vCenter credentials in suds
logs.

SecImpact
Suds will log vCenter username, password if vCenter
Login API call fails.

Change-Id: Idf7276528f72a9e590f2d0134619d8039c3f5b7d
Co-Authored-By: Radoslav Gerganov<rgerganov@vmware.com>
Closes-Bug: #1471656
2015-07-09 17:15:39 +05:30
Davanum Srinivas
d388c8a67b Fix exception to string in py34
python 3.4 does not have a self.message, so we need to set one up
in our own base class so the conversion to string work.

Change-Id: I5d963125ccb5cc29fa0563313d229046a81ca3d0
2015-06-28 11:03:54 -04:00
Gary Kotton
c4680ba543 Add in support for ManagedObjectNotFound exception
Adds in support for propogating this exception to the user of the
oslo.vmware code.

Change-Id: I8b475a2e2e6df09554077b9bf3a5fb58bd2a5926
2015-06-22 00:22:52 -07:00
Vipin Balachandran
5cb43a9bb7 Raise VimFaultException for unknown faults
Currently VMwareDriverException is raised for unknown
VIM faults. Sometimes clients may need to handle such
faults. Therefore it is better if we throw
VimFaultException with the fault_list set to the relevant
VIM fault class name.

Change-Id: I97d93ceaa82514d029fff87f1c5ad942895d8940
2015-06-19 17:58:18 +05:30
Vipin Balachandran
084f547583 Exception hierarchy refactoring
There are two roots for the current exception hierarchy: VimException
and VMwareDriverException, which is not a good design. This patch
refactors the exception hierarchy to fix this problem.

Summary of changes:
* Change parent of VimException to VMwareDriverException
* Change parent of exceptions corresponding to known VIM faults
  to VimException
* Change parent of VimSessionOverLoadException, VimConnectionException,
  VimAttributeException and ImageTransferException to
  VMwareDriverException.

The guidelines followed for this refactoring are:
* The changes in existing driver code should be NIL.

* The changes in other parts of the library should be minimal.

* Any exception raised by invocation of vim APIs should be a VimException
  and any exception raised by pbm APIs should be a PbmException. But this
  will result in lot of changes in the library as well as existing clients.
  Therefore, we  define a single root for these two types of exceptions which
  is labeled as VimException.

* Any exception raised by the library should be a child of
  VMwareDriverException. Therefore, VimException, VimSessionOverLoadException,
  VimConnectionException, VimAttributeException and ImageTransferException
  are children of VMwareDriverException. Ideally, VMwareDriverException
  should be renamed as ServiceException, but it will result in backward
  incompatibility.

Change-Id: Ide1bf891be78766e8670f8446792e3dc9c7ec88f
2015-06-19 17:29:47 +05:30
Doug Hellmann
7ebc48cf9d Remove oslo namespace package
Blueprint remove-namespace-packages

Depends-on: Id86b74fb492e1b86d29197b4e127d8368affa3fd
for openstack/congress

Change-Id: I3db0b8da385cb4ca475129260725a66413b25854
2015-06-04 17:58:10 +00:00
Davanum Srinivas
f2dc3a9ade Port test from Nova
Add a test being dropped in nova, changeid is:
I28f29d3e5745c21b99dc0cf9eb0cfe4f95dffd17

Fixed a test race condition issue pointed out
by @lifeless in this review.

Change-Id: I55baa451ea2258fdcb821ec72a585007160690a3
2015-06-02 10:28:59 -04:00
Jenkins
e059a3e4e6 Merge "Revert "Raise VimFaultException for unknown faults"" 2015-05-28 09:31:11 +00:00
Matt Riedemann
8ec7f0fe04 Revert "Raise VimFaultException for unknown faults"
This reverts commit 1668fef9cabea6a23023aab6b97617b4368b14d6

This is a backwards incompatible change, as we see with the
nova driver code that is expecting a VMwareDriverException
but is now getting back a VimFaultException because the unit
tests in nova were passing an unknown fault class name.

Change-Id: I0f8042c6e47f6eb9802e770ffb7d09bdbe57013b
Closes-Bug: #1459021
2015-05-26 21:32:40 +00:00
Eric Brown
5df9daa5a3 Add ToolsUnavailable exception
Some API calls that depend on VMware Tools will return a
ToolsUnavailable fault such as ShutdownGuest. This patch adds the
new fault so that callers can catch this specific fault rather
than the general one.

Change-Id: Ibe132e54dd84e12d01e32528824e6db8ec785b52
2015-05-12 20:31:01 -07:00
Jenkins
0b3a3d9a6a Merge "Add utility function to get profiles by IDs" 2015-05-11 01:57:55 +00:00
Vipin Balachandran
1668fef9ca Raise VimFaultException for unknown faults
Currently VMwareDriverException is raised for unknown
VIM faults. Sometimes clients may need to handle such
faults. Therefore it is better if we throw
VimFaultException with the fault_list set to the relevant
VIM fault class name.

Change-Id: I34f84f3ea3f5f0d6878f1d4438cf25bf22f293fd
2015-04-30 12:10:29 +05:30
Vipin Balachandran
c338f19f84 Add NoDiskSpaceException
vSphere APIs such as ExtendVirtualDisk_Task, CopyVirtualDisk_Task
throw NoDiskSpace fault when there is insufficient datastore
space. Currently, VMwareDriverException is thrown by oslo.vmware in
response to NoDiskSpace VIM fault. This patch adds a new exception
class corresponding to NoDiskSpace fault.

Change-Id: I50c5a93aae7fa2b5ad29951eec4d5a07e4f02045
2015-04-27 14:53:20 +05:30
Vipin Balachandran
25ec49da50 Add utility function to get profiles by IDs
This patch adds a utility function to pbm module to
retrieve SPBM storage profiles by IDs.

Change-Id: I16eb4503707fdd1500151f41d9dad6b8d153531f
2015-04-27 14:49:11 +05:30
Gary Kotton
6d8594ae51 Revert "VMWare NSXv: Common components"
In the end we decided that the best place to store the NSXv interfaces
was in the vmware_nsx project. That will be imported by the projects
that require networking support.

So the networking support has been removed.

Change-Id: I45f871286415ae8bbc0a939390e9d72c8f148fd7
2015-03-24 07:30:29 -07:00
Vipin Balachandran
8445b499aa Move exception related tests to new module
Unit tests for exceptions module are wrongly placed in
test_api and test_vim. This patch moves such tests to
a new test module.

Change-Id: Ia5f0b046ca0fa7153a339b310d8559a6cca5f036
2015-03-20 03:35:11 -07:00
Davanum Srinivas
5de2c31e9d Switch to non-namespaced module imports
Looks like we missed a few

Change-Id: I803c09170c9dbc9439d71259f6b632c4cbb6037f
2015-03-12 11:38:50 -04:00