openstack/python-mistralclient
Code Issues Proposed changes

Moving Mistral Client code from main Mistral repository

Browse Source 
* Used revision baba497d9960d7cb127e1f627e789805272c8e84

Implements: blueprint move-python-client-to-the-separate-repository
Change-Id: Iaa5dfac742e93a882bf739f09ad83d5e83cabc67
This commit is contained in:
Renat Akhmerov 2014-01-16 15:01:41 -08:00
parent 732d165e69
commit 6120373a1b
35 changed files with 2244 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
.gitreview Normal file
View File

@ -0,0 +1,4 @@
[gerrit]
host=review.openstack.org
port=29418
project=stackforge/python-mistralclient.git

4
AUTHORS Normal file
View File

@ -0,0 +1,4 @@
Renat Akhmerov <rakhmerov@mirantis.com>
Nikolay Makhotkin <nmakhotkin@mirantis.com>
Alexander Kuznetsov <akuznetsov@mirantis.com>

175
LICENSE Normal file
View File

@ -0,0 +1,175 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

245
doc/source/conf.py Normal file
View File

@ -0,0 +1,245 @@
# -*- coding: utf-8 -*-
#
# Mistral documentation build configuration file, created by
# sphinx-quickstart on Fri Nov 1 02:06:28 2013.
#
# This file is execfile()d with the current directory set to its containing
# dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
# import sys, os
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#sys.path.insert(0, os.path.abspath('.'))
# -- General configuration ---------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = []
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
#source_encoding = 'utf-8-sig'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'Mistral Client'
copyright = u'2013, OpenStack Foundation'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '0.01'
# The full version, including alpha/beta/rc tags.
release = '0.01'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
#today = ''
# Else, today_fmt is used as the format for a strftime call.
#today_fmt = '%B %d, %Y'
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
exclude_patterns = []
# The reST default role (used for this markup: `text`) to use for all
# documents.
#default_role = None
# If true, '()' will be appended to :func: etc. cross-reference text.
#add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
#add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
#show_authors = False
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
#modindex_common_prefix = []
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
html_theme = 'default'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#html_theme_options = {}
# Add any paths that contain custom themes here, relative to this directory.
#html_theme_path = []
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
#html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
#html_short_title = None
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
#html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
#html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
#html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
#html_sidebars = {}
# Additional templates that should be rendered to pages, maps page names to
# template names.
#html_additional_pages = {}
# If false, no module index is generated.
#html_domain_indices = True
# If false, no index is generated.
#html_use_index = True
# If true, the index is split into individual pages for each letter.
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
# If true, "Created using Sphinx" is shown in the HTML footer. Default is True.
#html_show_sphinx = True
# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True.
#html_show_copyright = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
#html_use_opensearch = ''
# This is the file name suffix for HTML files (e.g. ".xhtml").
#html_file_suffix = None
# Output file base name for HTML help builder.
htmlhelp_basename = 'Mistraldoc'
# -- Options for LaTeX output ------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#'preamble': '',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author,
# documentclass [howto/manual]).
latex_documents = [
('index', 'MistralClient.tex', u'Mistral Client Documentation',
u'OpenStack Foundation', 'manual'),
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
#latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
#latex_use_parts = False
# If true, show page references after internal links.
#latex_show_pagerefs = False
# If true, show URL addresses after external links.
#latex_show_urls = False
# Documents to append as an appendix to all manuals.
#latex_appendices = []
# If false, no module index is generated.
#latex_domain_indices = True
# -- Options for manual page output ------------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
('index', 'mistral_client', u'Mistral Client Documentation',
[u'OpenStack Foundation'], 1)
]
# If true, show URL addresses after external links.
#man_show_urls = False
# -- Options for Texinfo output ----------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
('index', 'MistralClient', u'Mistral Client Documentation',
u'OpenStack Foundation', 'MistralClient',
'One line description of project.', 'Miscellaneous'),
]
# Documents to append as an appendix to all manuals.
#texinfo_appendices = []
# If false, no module index is generated.
#texinfo_domain_indices = True
# How to display URL addresses: 'footnote', 'no', or 'inline'.
#texinfo_show_urls = 'footnote'

22
doc/source/index.rst Normal file
View File

@ -0,0 +1,22 @@
.. Mistral Client documentation master file, created by
sphinx-quickstart on Fri Nov 1 02:06:28 2013.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to Mistral Client documentation!
===================================
Contents:
.. toctree::
:maxdepth: 2
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

0
mistralclient/__init__.py Normal file
View File

0
mistralclient/api/__init__.py Normal file
View File

153
mistralclient/api/base.py Normal file
View File

@ -0,0 +1,153 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
import logging
LOG = logging.getLogger(__name__)
class Resource(object):
resource_name = 'Something'
defaults = {}
def __init__(self, manager, data):
self.manager = manager
self._data = data
self._set_defaults()
self._set_attributes()
def _set_defaults(self):
for k, v in self.defaults.iteritems():
if k not in self._data:
self._data[k] = v
def _set_attributes(self):
for k, v in self._data.iteritems():
try:
setattr(self, k, v)
except AttributeError:
# In this case we already defined the attribute on the class
pass
def __str__(self):
vals = ", ".join(["%s='%s'" % (n, v)
for n, v in self._data.iteritems()])
return "%s [%s]" % (self.resource_name, vals)
def _check_items(obj, searches):
try:
return all(getattr(obj, attr) == value for (attr, value) in searches)
except AttributeError:
return False
def extract_json(response, response_key):
if response_key is not None:
return get_json(response)[response_key]
else:
return get_json(response)
class ResourceManager(object):
resource_class = None
def __init__(self, client):
self.client = client
def find(self, **kwargs):
return [i for i in self.list() if _check_items(i, kwargs.items())]
def _ensure_not_empty(self, **kwargs):
for name, value in kwargs.iteritems():
if value is None or (isinstance(value, str) and len(value) == 0):
raise APIException('%s is missing field "%s"' %
(self.resource_class.__name__, name))
def _copy_if_defined(self, data, **kwargs):
for name, value in kwargs.iteritems():
if value is not None:
data[name] = value
def _create(self, url, data, response_key=None, dump_json=True):
if dump_json:
data = json.dumps(data)
resp = self.client.http_client.post(url, data)
if resp.status_code != 201:
self._raise_api_exception(resp)
return self.resource_class(self, extract_json(resp, response_key))
def _update(self, url, data, response_key=None, dump_json=True):
if dump_json:
data = json.dumps(data)
resp = self.client.http_client.put(url, data)
if resp.status_code != 200:
self._raise_api_exception(resp)
return self.resource_class(self, extract_json(resp, response_key))
def _list(self, url, response_key=None):
resp = self.client.http_client.get(url)
if resp.status_code == 200:
return [self.resource_class(self, resource_data)
for resource_data in extract_json(resp, response_key)]
else:
self._raise_api_exception(resp)
def _get(self, url, response_key=None):
resp = self.client.http_client.get(url)
if resp.status_code == 200:
return self.resource_class(self, extract_json(resp, response_key))
else:
self._raise_api_exception(resp)
def _delete(self, url):
resp = self.client.http_client.delete(url)
if resp.status_code != 204:
self._raise_api_exception(resp)
def _plurify_resource_name(self):
return self.resource_class.resource_name + 's'
def _raise_api_exception(self, resp):
error_data = get_json(resp)
raise APIException(error_data["faultstring"])
def get_json(response):
"""This method provided backward compatibility with old versions
of requests library
"""
json_field_or_function = getattr(response, 'json', None)
if callable(json_field_or_function):
return response.json()
else:
return json.loads(response.content)
class APIException(Exception):
pass

98
mistralclient/api/client.py Normal file
View File

@ -0,0 +1,98 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import six
from keystoneclient.v3 import client as keystone_client
from mistralclient.api import httpclient
from mistralclient.api import workbooks
from mistralclient.api import executions
from mistralclient.api import tasks
from mistralclient.api import listeners
class Client(object):
def __init__(self, mistral_url=None, username=None, api_key=None,
project_name=None, auth_url=None, project_id=None,
endpoint_type='publicURL', service_type='workflow',
input_auth_token=None):
(mistral_url,
token,
project_id,
user_id) = self.authenticate(mistral_url, username,
api_key, project_name,
auth_url, project_id,
endpoint_type, service_type,
input_auth_token)
self.http_client = httpclient.HTTPClient(mistral_url,
token,
project_id,
user_id)
# Create all resource managers.
self.workbooks = workbooks.WorkbookManager(self)
self.executions = executions.ExecutionManager(self)
self.tasks = tasks.TaskManager(self)
self.listeners = listeners.ListenerManager(self)
def authenticate(self, mistral_url=None, username=None, api_key=None,
project_name=None, auth_url=None, project_id=None,
endpoint_type='publicURL', service_type='workflow',
input_auth_token=None):
if mistral_url and not isinstance(mistral_url, six.string_types):
raise RuntimeError('Mistral url should be string')
if (isinstance(project_name, six.string_types) or
isinstance(project_id, six.string_types)):
if project_name and project_id:
raise RuntimeError('Only project name or '
'project id should be set')
if "v2.0" in auth_url:
raise RuntimeError('Mistral supports only v3 '
'keystone API.')
keystone = keystone_client.Client(username=username,
password=api_key,
token=input_auth_token,
tenant_id=project_id,
tenant_name=project_name,
auth_url=auth_url)
keystone.authenticate()
token = keystone.auth_token
user_id = keystone.user_id
if project_name and not project_id:
if keystone.tenants.find(name=project_name):
project_id = str(keystone.tenants.find(
name=project_name).id)
else:
raise RuntimeError('Project name or project id should'
' not be empty and should be string')
if not mistral_url:
catalog = keystone.service_catalog.get_endpoints(service_type)
if service_type in catalog:
for e_type, endpoint in catalog.get[service_type][0].items():
if str(e_type).lower() == str(endpoint_type).lower():
mistral_url = endpoint
break
if not mistral_url:
mistral_url = "http://localhost:8989/v1"
return mistral_url, token, project_id, user_id

65
mistralclient/api/executions.py Normal file
View File

@ -0,0 +1,65 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from mistralclient.api import base
class Execution(base.Resource):
resource_name = 'Execution'
class ExecutionManager(base.ResourceManager):
resource_class = Execution
def create(self, workbook_name, target_task):
self._ensure_not_empty(workbook_name=workbook_name,
target_task=target_task)
data = {
'workbook_name': workbook_name,
'target_task': target_task
}
return self._create('/workbooks/%s/executions' % workbook_name, data)
def update(self, workbook_name, id, state):
self._ensure_not_empty(workbook_name=workbook_name, id=id,
state=state)
data = {
'workbook_name': workbook_name,
'id': id,
'state': state
}
return self._update('/workbooks/%s/executions/%s' %
(workbook_name, id), data)
def list(self, workbook_name):
self._ensure_not_empty(workbook_name=workbook_name)
return self._list('/workbooks/%s/executions' % workbook_name,
'executions')
def get(self, workbook_name, id):
self._ensure_not_empty(workbook_name=workbook_name, id=id)
return self._get('/workbooks/%s/executions/%s' % (workbook_name, id))
def delete(self, workbook_name, id):
self._ensure_not_empty(workbook_name=workbook_name, id=id)
self._delete('/workbooks/%s/executions/%s' % (workbook_name, id))

62
mistralclient/api/httpclient.py Normal file
View File

@ -0,0 +1,62 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import requests
class HTTPClient(object):
def __init__(self, base_url, token, project_id, user_id):
self.base_url = base_url
self.token = token
self.project_id = project_id
self.user_id = user_id
def get(self, url, headers=None):
headers = self._update_headers(headers)
return requests.get(self.base_url + url, headers=headers)
def post(self, url, body, headers=None):
headers = self._update_headers(headers)
content_type = headers.get('content-type', 'application/json')
headers['content-type'] = content_type
return requests.post(self.base_url + url, body, headers=headers)
def put(self, url, body, headers=None):
headers = self._update_headers(headers)
content_type = headers.get('content-type', 'application/json')
headers['content-type'] = content_type
return requests.put(self.base_url + url, body, headers=headers)
def delete(self, url, headers=None):
headers = self._update_headers(headers)
return requests.delete(self.base_url + url, headers=headers)
def _update_headers(self, headers):
if not headers:
headers = {}
token = headers.get('x-auth-token', self.token)
headers['x-auth-token'] = token
project_id = headers.get('X-Project-Id', self.project_id)
headers['X-Project-Id'] = project_id
user_id = headers.get('X-User-Id', self.user_id)
headers['X-User-Id'] = user_id
return headers

71
mistralclient/api/listeners.py Normal file
View File

@ -0,0 +1,71 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from mistralclient.api import base
class Listener(base.Resource):
resource_name = 'Listener'
class ListenerManager(base.ResourceManager):
resource_class = Listener
def create(self, workbook_name, webhook, description=None, events=None):
# TODO(rakhmerov): need to describe what events is (data type)
self._ensure_not_empty(workbook_name=workbook_name,
webhook=webhook)
data = {
'workbook_name': workbook_name,
'description': description,
'webhook': webhook,
'events': events
}
return self._create('/workbooks/%s/listeners' % workbook_name, data)
def update(self, workbook_name, id, webhook=None, description=None,
events=None):
#TODO: need to describe what events is
self._ensure_not_empty(workbook_name=workbook_name, id=id)
data = {
'id': id,
'workbook_name': workbook_name,
'description': description,
'webhook': webhook,
'events': events
}
return self._update('/workbooks/%s/listeners/%s' %
(workbook_name, id), data)
def list(self, workbook_name):
self._ensure_not_empty(workbook_name=workbook_name)
return self._list('/workbooks/%s/listeners' % workbook_name,
'listeners')
def get(self, workbook_name, id):
self._ensure_not_empty(workbook_name=workbook_name, id=id)
return self._get('/workbooks/%s/listeners/%s' % (workbook_name, id))
def delete(self, workbook_name, id):
self._ensure_not_empty(workbook_name=workbook_name, id=id)
self._delete('/workbooks/%s/listeners/%s' % (workbook_name, id))

57
mistralclient/api/tasks.py Normal file
View File

@ -0,0 +1,57 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from mistralclient.api import base
class Task(base.Resource):
resource_name = 'Task'
class TaskManager(base.ResourceManager):
resource_class = Task
def update(self, workbook_name, execution_id, id, state):
self._ensure_not_empty(workbook_name=workbook_name,
execution_id=execution_id,
id=id,
state=state)
data = {
'workbook_name': workbook_name,
'execution_id': execution_id,
'id': id,
'state': state
}
return self._update('/workbooks/%s/executions/%s/tasks/%s' %
(workbook_name, execution_id, id), data)
def list(self, workbook_name, execution_id):
self._ensure_not_empty(workbook_name=workbook_name,
execution_id=execution_id)
return self._list('/workbooks/%s/executions/%s/tasks' %
(workbook_name, execution_id),
'tasks')
def get(self, workbook_name, execution_id, id):
self._ensure_not_empty(workbook_name=workbook_name,
execution_id=execution_id,
id=id)
return self._get('/workbooks/%s/executions/%s/tasks/%s' %
(workbook_name, execution_id, id))

73
mistralclient/api/workbooks.py Normal file
View File

@ -0,0 +1,73 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from mistralclient.api import base
class Workbook(base.Resource):
resource_name = 'Workbook'
class WorkbookManager(base.ResourceManager):
resource_class = Workbook
def create(self, name, description=None, tags=None):
self._ensure_not_empty(name=name)
data = {
'name': name,
'description': description,
'tags': tags,
}
return self._create('/workbooks', data)
def update(self, name, description=None, tags=None):
self._ensure_not_empty(name=name)
data = {
'name': name,
'description': description,
'tags': tags,
}
return self._update('/workbooks', data)
def list(self):
return self._list('/workbooks', 'workbooks')
def get(self, name):
self._ensure_not_empty(name=name)
return self._get('/workbooks/%s' % name)
def delete(self, name):
self._ensure_not_empty(name=name)
self._delete('/workbooks/%s' % name)
def upload_definition(self, name, text):
self._ensure_not_empty(name=name)
self.client.http_client.put('/workbooks/%s/definition' % name,
text,
headers={'content-type': 'text/plain'})
def get_definition(self, name):
self._ensure_not_empty(name=name)
return self.client.http_client.get('/workbooks/%s/definition'
% name).content

0
mistralclient/tests/__init__.py Normal file
View File

61
mistralclient/tests/base.py Normal file
View File

@ -0,0 +1,61 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import unittest2
import mock
from mistralclient.api import client
class FakeResponse(object):
"""Fake response for testing Mistral Client."""
def __init__(self, status_code, json_values={}, content=None):
self.status_code = status_code
self.json_values = json_values
self.content = content
def json(self):
return self.json_values
class BaseClientTest(unittest2.TestCase):
@mock.patch('keystoneclient.v3.client.Client')
def setUp(self, keystone):
keystone.return_value = mock.Mock()
self._client = client.Client(project_name="test",
auth_url="v3.0",
mistral_url="test")
self.workbooks = self._client.workbooks
self.executions = self._client.executions
self.tasks = self._client.tasks
self.listeners = self._client.listeners
def mock_http_get(self, json, status_code=200):
self._client.http_client.get = \
mock.MagicMock(return_value=FakeResponse(status_code, json))
def mock_http_post(self, json, status_code=201):
self._client.http_client.post = \
mock.MagicMock(return_value=FakeResponse(status_code, json))
def mock_http_put(self, json, status_code=200):
self._client.http_client.put = \
mock.MagicMock(return_value=FakeResponse(status_code, json))
def mock_http_delete(self, status_code=204):
self._client.http_client.delete = \
mock.MagicMock(return_value=FakeResponse(status_code))

86
mistralclient/tests/test_executions.py Normal file
View File

@ -0,0 +1,86 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from mistralclient.tests import base
# TODO: later we need additional tests verifying all the errors etc.
EXECS = [
{
'id': "123",
'workbook_name': "my_workbook",
'target_task': 'my_task',
'state': 'RUNNING'
}
]
class TestExecutions(base.BaseClientTest):
def test_create(self):
self.mock_http_post(json=EXECS[0])
wb = self.executions.create(EXECS[0]['workbook_name'],
EXECS[0]['target_task'])
self.assertIsNotNone(wb)
self.assertEqual(EXECS[0]['id'], wb.id)
self.assertEqual(EXECS[0]['workbook_name'], wb.workbook_name)
self.assertEqual(EXECS[0]['target_task'], wb.target_task)
self.assertEqual(EXECS[0]['state'], wb.state)
def test_update(self):
self.mock_http_put(json=EXECS[0])
ex = self.executions.update(EXECS[0]['workbook_name'],
EXECS[0]['id'],
EXECS[0]['state'])
self.assertIsNotNone(ex)
self.assertEqual(EXECS[0]['id'], ex.id)
self.assertEqual(EXECS[0]['workbook_name'], ex.workbook_name)
self.assertEqual(EXECS[0]['target_task'], ex.target_task)
self.assertEqual(EXECS[0]['state'], ex.state)
def test_list(self):
self.mock_http_get(json={'executions': EXECS})
executions = self.executions.list(EXECS[0]['workbook_name'])
self.assertEqual(1, len(executions))
ex = executions[0]
self.assertEqual(EXECS[0]['id'], ex.id)
self.assertEqual(EXECS[0]['workbook_name'], ex.workbook_name)
self.assertEqual(EXECS[0]['target_task'], ex.target_task)
self.assertEqual(EXECS[0]['state'], ex.state)
def test_get(self):
self.mock_http_get(json=EXECS[0])
ex = self.executions.get(EXECS[0]['workbook_name'], EXECS[0]['id'])
self.assertEqual(EXECS[0]['id'], ex.id)
self.assertEqual(EXECS[0]['workbook_name'], ex.workbook_name)
self.assertEqual(EXECS[0]['target_task'], ex.target_task)
self.assertEqual(EXECS[0]['state'], ex.state)
def test_delete(self):
self.mock_http_delete(status_code=204)
# Just make sure it doesn't throw any exceptions.
self.executions.delete(EXECS[0]['workbook_name'], EXECS[0]['id'])

88
mistralclient/tests/test_listeners.py Normal file
View File

@ -0,0 +1,88 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from mistralclient.tests import base
# TODO: later we need additional tests verifying all the errors etc.
LISTENERS = [
{
'id': "1",
'workbook_name': "my_workbook",
'description': "My cool Mistral workbook",
'webhook': "http://my.website.org"
}
]
class TestListeners(base.BaseClientTest):
def test_create(self):
self.mock_http_post(json=LISTENERS[0])
lsnr = self.listeners.create(LISTENERS[0]['workbook_name'],
LISTENERS[0]['webhook'],
LISTENERS[0]['description'])
self.assertIsNotNone(lsnr)
self.assertEqual(LISTENERS[0]['id'], lsnr.id)
self.assertEqual(LISTENERS[0]['workbook_name'], lsnr.workbook_name)
self.assertEqual(LISTENERS[0]['webhook'], lsnr.webhook)
self.assertEqual(LISTENERS[0]['description'], lsnr.description)
def test_update(self):
self.mock_http_put(json=LISTENERS[0])
lsnr = self.listeners.update(LISTENERS[0]['workbook_name'],
LISTENERS[0]['webhook'],
LISTENERS[0]['description'])
self.assertIsNotNone(lsnr)
self.assertEqual(LISTENERS[0]['id'], lsnr.id)
self.assertEqual(LISTENERS[0]['workbook_name'], lsnr.workbook_name)
self.assertEqual(LISTENERS[0]['webhook'], lsnr.webhook)
self.assertEqual(LISTENERS[0]['description'], lsnr.description)
def test_list(self):
self.mock_http_get(json={'listeners': LISTENERS})
listeners = self.listeners.list(LISTENERS[0]['workbook_name'])
self.assertEqual(1, len(listeners))
lsnr = listeners[0]
self.assertEqual(LISTENERS[0]['id'], lsnr.id)
self.assertEqual(LISTENERS[0]['workbook_name'], lsnr.workbook_name)
self.assertEqual(LISTENERS[0]['webhook'], lsnr.webhook)
self.assertEqual(LISTENERS[0]['description'], lsnr.description)
def test_get(self):
self.mock_http_get(json=LISTENERS[0])
lsnr = self.listeners.get(LISTENERS[0]['workbook_name'],
LISTENERS[0]['id'])
self.assertEqual(LISTENERS[0]['id'], lsnr.id)
self.assertEqual(LISTENERS[0]['workbook_name'], lsnr.workbook_name)
self.assertEqual(LISTENERS[0]['webhook'], lsnr.webhook)
self.assertEqual(LISTENERS[0]['description'], lsnr.description)
def test_delete(self):
self.mock_http_delete(status_code=204)
# Just make sure it doesn't throw any exceptions.
self.listeners.delete(LISTENERS[0]['workbook_name'],
LISTENERS[0]['id'])

84
mistralclient/tests/test_tasks.py Normal file
View File

@ -0,0 +1,84 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from mistralclient.tests import base
# TODO: later we need additional tests verifying all the errors etc.
TASKS = [
{
'id': "1",
'workbook_name': "my_workbook",
'execution_id': '123',
'name': 'my_task',
'description': 'My cool task',
'action': 'my_action',
'state': 'RUNNING',
'tags': ['deployment', 'demo']
}
]
class TestTasks(base.BaseClientTest):
def test_update(self):
self.mock_http_put(json=TASKS[0])
task = self.tasks.update(TASKS[0]['workbook_name'],
TASKS[0]['execution_id'],
TASKS[0]['id'],
TASKS[0]['state'])
self.assertIsNotNone(task)
self.assertEqual(TASKS[0]['id'], task.id)
self.assertEqual(TASKS[0]['workbook_name'], task.workbook_name)
self.assertEqual(TASKS[0]['execution_id'], task.execution_id)
self.assertEqual(TASKS[0]['description'], task.description)
self.assertEqual(TASKS[0]['action'], task.action)
self.assertEqual(TASKS[0]['state'], task.state)
self.assertEqual(TASKS[0]['tags'], task.tags)
def test_list(self):
self.mock_http_get(json={'tasks': TASKS})
tasks = self.tasks.list(TASKS[0]['workbook_name'],
TASKS[0]['execution_id'])
self.assertEqual(1, len(tasks))
task = tasks[0]
self.assertEqual(TASKS[0]['id'], task.id)
self.assertEqual(TASKS[0]['workbook_name'], task.workbook_name)
self.assertEqual(TASKS[0]['execution_id'], task.execution_id)
self.assertEqual(TASKS[0]['description'], task.description)
self.assertEqual(TASKS[0]['action'], task.action)
self.assertEqual(TASKS[0]['state'], task.state)
self.assertEqual(TASKS[0]['tags'], task.tags)
def test_get(self):
self.mock_http_get(json=TASKS[0])
task = self.tasks.get(TASKS[0]['workbook_name'],
TASKS[0]['execution_id'],
TASKS[0]['id'])
self.assertEqual(TASKS[0]['id'], task.id)
self.assertEqual(TASKS[0]['workbook_name'], task.workbook_name)
self.assertEqual(TASKS[0]['execution_id'], task.execution_id)
self.assertEqual(TASKS[0]['description'], task.description)
self.assertEqual(TASKS[0]['action'], task.action)
self.assertEqual(TASKS[0]['state'], task.state)
self.assertEqual(TASKS[0]['tags'], task.tags)

122
mistralclient/tests/test_workbooks.py Normal file
View File

@ -0,0 +1,122 @@
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import mock
from mistralclient.tests import base
# TODO: later we need additional tests verifying all the errors etc.
WORKBOOKS = [
{
'name': "my_workbook",
'description': "My cool Mistral workbook",
'tags': ['deployment', 'demo']
}
]
WB_DEF = """
Service:
name: my_service
type: REST
parameters:
baseUrl: http://my.service.org
actions:
action1:
parameters:
url: servers
method: POST
task-parameters:
param1:
optional: false
param2:
optional: false
Workflow:
tasks:
task1:
action: my_service:create-vm
parameters:
param1: 1234
param2: 42
"""
class TestWorkbooks(base.BaseClientTest):
def test_create(self):
self.mock_http_post(json=WORKBOOKS[0])
wb = self.workbooks.create(WORKBOOKS[0]['name'],
WORKBOOKS[0]['description'],
WORKBOOKS[0]['tags'])
self.assertIsNotNone(wb)
self.assertEqual(WORKBOOKS[0]['name'], wb.name)
self.assertEqual(WORKBOOKS[0]['description'], wb.description)
self.assertEqual(WORKBOOKS[0]['tags'], wb.tags)
def test_update(self):
self.mock_http_put(json=WORKBOOKS[0])
wb = self.workbooks.update(WORKBOOKS[0]['name'],
WORKBOOKS[0]['description'],
WORKBOOKS[0]['tags'])
self.assertIsNotNone(wb)
self.assertEqual(WORKBOOKS[0]['name'], wb.name)
self.assertEqual(WORKBOOKS[0]['description'], wb.description)
self.assertEqual(WORKBOOKS[0]['tags'], wb.tags)
def test_list(self):
self.mock_http_get(json={'workbooks': WORKBOOKS})
workbooks = self.workbooks.list()
self.assertEqual(1, len(workbooks))
wb = workbooks[0]
self.assertEqual(WORKBOOKS[0]['name'], wb.name)
self.assertEqual(WORKBOOKS[0]['description'], wb.description)
self.assertEqual(WORKBOOKS[0]['tags'], wb.tags)
def test_get(self):
self.mock_http_get(json=WORKBOOKS[0])
wb = self.workbooks.get(WORKBOOKS[0]['name'])
self.assertIsNotNone(wb)
self.assertEqual(WORKBOOKS[0]['name'], wb.name)
self.assertEqual(WORKBOOKS[0]['description'], wb.description)
self.assertEqual(WORKBOOKS[0]['tags'], wb.tags)
def test_delete(self):
self.mock_http_delete(status_code=204)
# Just make sure it doesn't throw any exceptions.
self.workbooks.delete(WORKBOOKS[0]['name'])
def test_upload_definition(self):
self.mock_http_put(None, status_code=200)
# Just make sure it doesn't throw any exceptions.
self.workbooks.upload_definition("my_workbook", WB_DEF)
def test_get_definition(self):
self._client.http_client.get =\
mock.MagicMock(return_value=base.FakeResponse(200, None, WB_DEF))
text = self.workbooks.get_definition("my_workbook")
self.assertEqual(WB_DEF, text)

7
openstack-common.conf Normal file
View File

@ -0,0 +1,7 @@
[DEFAULT]
# The list of modules to copy from oslo-incubator.git
# TODO(rakhmerov): We'll need to use apiclient later.
# The base module to hold the copy of openstack.common
base=mistralclient

3
requirements.txt Normal file
View File

@ -0,0 +1,3 @@
pbr>=0.5.21,<1.0
requests
python-keystoneclient>=0.3.2

27
setup.cfg Normal file
View File

@ -0,0 +1,27 @@
[metadata]
name = python-mistralclient
version = 0.01
summary = Mistral Client Library
description-file = README.rd
#license = Apache Software License
classifiers =
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Environment :: OpenStack
Intended Audience :: Information Technology
Intended Audience :: System Administrators
#License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
author = OpenStack
author-email = openstack-dev@lists.openstack.org
[files]
packages =
mistralclient
[nosetests]
cover-package = mistralclient
[extract_messages]
keywords = _ gettext ngettext l_ lazy_gettext

22
setup.py Normal file
View File

@ -0,0 +1,22 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Copyright 2013 - Mirantis, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT
import setuptools
setuptools.setup(
name="python-mistralclient")

11
test-requirements.txt Normal file
View File

@ -0,0 +1,11 @@
pep8==1.4.5
pyflakes>=0.7.2,<0.7.4
flake8==2.0
pylint==0.25.2
sphinx>=1.1.2
unittest2
fixtures>=0.3.14
mock>=1.0
nose
testtools>=0.9.32
lockfile>=0.9.1

99
tools/config/generate_sample.sh Executable file
View File

@ -0,0 +1,99 @@
#!/usr/bin/env bash
print_hint() {
echo "Try \`${0##*/} --help' for more information." >&2
}
PARSED_OPTIONS=$(getopt -n "${0##*/}" -o hb:p:o: \
--long help,base-dir:,package-name:,output-dir: -- "$@")
if [ $? != 0 ] ; then print_hint ; exit 1 ; fi
eval set -- "$PARSED_OPTIONS"
while true; do
case "$1" in
-h|--help)
echo "${0##*/} [options]"
echo ""
echo "options:"
echo "-h, --help show brief help"
echo "-b, --base-dir=DIR project base directory"
echo "-p, --package-name=NAME project package name"
echo "-o, --output-dir=DIR file output directory"
exit 0
;;
-b|--base-dir)
shift
BASEDIR=`echo $1 | sed -e 's/\/*$//g'`
shift
;;
-p|--package-name)
shift
PACKAGENAME=`echo $1`
shift
;;
-o|--output-dir)
shift
OUTPUTDIR=`echo $1 | sed -e 's/\/*$//g'`
shift
;;
--)
break
;;
esac
done
BASEDIR=${BASEDIR:-`pwd`}
if ! [ -d $BASEDIR ]
then
echo "${0##*/}: missing project base directory" >&2 ; print_hint ; exit 1
elif [[ $BASEDIR != /* ]]
then
BASEDIR=$(cd "$BASEDIR" && pwd)
fi
PACKAGENAME=${PACKAGENAME:-${BASEDIR##*/}}
TARGETDIR=$BASEDIR/$PACKAGENAME
if ! [ -d $TARGETDIR ]
then
echo "${0##*/}: invalid project package name" >&2 ; print_hint ; exit 1
fi
OUTPUTDIR=${OUTPUTDIR:-$BASEDIR/etc}
# NOTE(bnemec): Some projects put their sample config in etc/,
# some in etc/$PACKAGENAME/
if [ -d $OUTPUTDIR/$PACKAGENAME ]
then
OUTPUTDIR=$OUTPUTDIR/$PACKAGENAME
elif ! [ -d $OUTPUTDIR ]
then
echo "${0##*/}: cannot access \`$OUTPUTDIR': No such file or directory" >&2
exit 1
fi
BASEDIRESC=`echo $BASEDIR | sed -e 's/\//\\\\\//g'`
find $TARGETDIR -type f -name "*.pyc" -delete
FILES=$(find $TARGETDIR -type f -name "*.py" ! -path "*/tests/*" \
-exec grep -l "Opt(" {} + | sed -e "s/^$BASEDIRESC\///g" | sort -u)
EXTRA_MODULES_FILE="`dirname $0`/oslo.config.generator.rc"
if test -r "$EXTRA_MODULES_FILE"
then
source "$EXTRA_MODULES_FILE"
fi
export EVENTLET_NO_GREENDNS=yes
OS_VARS=$(set | sed -n '/^OS_/s/=[^=]*$//gp' | xargs)
[ "$OS_VARS" ] && eval "unset \$OS_VARS"
DEFAULT_MODULEPATH=mistral.openstack.common.config.generator
MODULEPATH=${MODULEPATH:-$DEFAULT_MODULEPATH}
OUTPUTFILE=$OUTPUTDIR/$PACKAGENAME.conf.sample
python -m $MODULEPATH $FILES > $OUTPUTFILE
# Hook to allow projects to append custom config file snippets
CONCAT_FILES=$(ls $BASEDIR/tools/config/*.conf.sample 2>/dev/null)
for CONCAT_FILE in $CONCAT_FILES; do
cat $CONCAT_FILE >> $OUTPUTFILE
done

3
tools/install_venv Executable file
View File

@ -0,0 +1,3 @@
#!/bin/sh
tox -vvv -evenv -- python --version

77
tools/install_venv.py Normal file
View File

@ -0,0 +1,77 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2010 United States Government as represented by the
# Administrator of the National Aeronautics and Space Administration.
# All Rights Reserved.
#
# Copyright 2010 OpenStack Foundation
# Copyright 2013 IBM Corp.
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import ConfigParser
import os
import sys
import install_venv_common as install_venv # flake8: noqa
def print_help(project, venv, root):
help = """
%(project)s development environment setup is complete.
%(project)s development uses virtualenv to track and manage Python
dependencies while in development and testing.
To activate the %(project)s virtualenv for the extent of your current
shell session you can run:
$ source %(venv)s/bin/activate
Or, if you prefer, you can run commands in the virtualenv on a case by
case basis by running:
$ %(root)s/tools/with_venv.sh <your command>
"""
print help % dict(project=project, venv=venv, root=root)
def main(argv):
root = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
if os.environ.get('tools_path'):
root = os.environ['tools_path']
venv = os.path.join(root, '.venv')
if os.environ.get('venv'):
venv = os.environ['venv']
pip_requires = os.path.join(root, 'requirements.txt')
test_requires = os.path.join(root, 'test-requirements.txt')
py_version = "python%s.%s" % (sys.version_info[0], sys.version_info[1])
setup_cfg = ConfigParser.ConfigParser()
setup_cfg.read('setup.cfg')
project = setup_cfg.get('metadata', 'name')
install = install_venv.InstallVenv(
root, venv, pip_requires, test_requires, py_version, project)
options = install.parse_args(argv)
install.check_python_version()
install.check_dependencies()
install.create_virtualenv(no_site_packages=options.no_site_packages)
install.install_dependencies()
install.post_process()
print_help(project, venv, root)
if __name__ == '__main__':
main(sys.argv)

212
tools/install_venv_common.py Normal file
View File

@ -0,0 +1,212 @@
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright 2013 OpenStack Foundation
# Copyright 2013 IBM Corp.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
"""Provides methods needed by installation script for OpenStack development
virtual environments.
Since this script is used to bootstrap a virtualenv from the system's Python
environment, it should be kept strictly compatible with Python 2.6.
Synced in from openstack-common
"""
from __future__ import print_function
import optparse
import os
import subprocess
import sys
class InstallVenv(object):
def __init__(self, root, venv, requirements,
test_requirements, py_version,
project):
self.root = root
self.venv = venv
self.requirements = requirements
self.test_requirements = test_requirements
self.py_version = py_version
self.project = project
def die(self, message, *args):
print(message % args, file=sys.stderr)
sys.exit(1)
def check_python_version(self):
if sys.version_info < (2, 6):
self.die("Need Python Version >= 2.6")
def run_command_with_code(self, cmd, redirect_output=True,
check_exit_code=True):
"""Runs a command in an out-of-process shell.
Returns the output of that command. Working directory is self.root.
"""
if redirect_output:
stdout = subprocess.PIPE
else:
stdout = None
proc = subprocess.Popen(cmd, cwd=self.root, stdout=stdout)
output = proc.communicate()[0]
if check_exit_code and proc.returncode != 0:
self.die('Command "%s" failed.\n%s', ' '.join(cmd), output)
return (output, proc.returncode)
def run_command(self, cmd, redirect_output=True, check_exit_code=True):
return self.run_command_with_code(cmd, redirect_output,
check_exit_code)[0]
def get_distro(self):
if (os.path.exists('/etc/fedora-release') or
os.path.exists('/etc/redhat-release')):
return Fedora(
self.root, self.venv, self.requirements,
self.test_requirements, self.py_version, self.project)
else:
return Distro(
self.root, self.venv, self.requirements,
self.test_requirements, self.py_version, self.project)
def check_dependencies(self):
self.get_distro().install_virtualenv()
def create_virtualenv(self, no_site_packages=True):
"""Creates the virtual environment and installs PIP.
Creates the virtual environment and installs PIP only into the
virtual environment.
"""
if not os.path.isdir(self.venv):
print('Creating venv...', end=' ')
if no_site_packages:
self.run_command(['virtualenv', '-q', '--no-site-packages',
self.venv])
else:
self.run_command(['virtualenv', '-q', self.venv])
print('done.')
else:
print("venv already exists...")
pass
def pip_install(self, *args):
self.run_command(['tools/with_venv.sh',
'pip', 'install', '--upgrade'] + list(args),
redirect_output=False)
def install_dependencies(self):
print('Installing dependencies with pip (this can take a while)...')
# First things first, make sure our venv has the latest pip and
# setuptools.
self.pip_install('pip>=1.3')
self.pip_install('setuptools')
self.pip_install('-r', self.requirements)
self.pip_install('-r', self.test_requirements)
def post_process(self):
self.get_distro().post_process()
def parse_args(self, argv):
"""Parses command-line arguments."""
parser = optparse.OptionParser()
parser.add_option('-n', '--no-site-packages',
action='store_true',
help="Do not inherit packages from global Python "
"install")
return parser.parse_args(argv[1:])[0]
class Distro(InstallVenv):
def check_cmd(self, cmd):
return bool(self.run_command(['which', cmd],
check_exit_code=False).strip())
def install_virtualenv(self):
if self.check_cmd('virtualenv'):
return
if self.check_cmd('easy_install'):
print('Installing virtualenv via easy_install...', end=' ')
if self.run_command(['easy_install', 'virtualenv']):
print('Succeeded')
return
else:
print('Failed')
self.die('ERROR: virtualenv not found.\n\n%s development'
' requires virtualenv, please install it using your'
' favorite package management tool' % self.project)
def post_process(self):
"""Any distribution-specific post-processing gets done here.
In particular, this is useful for applying patches to code inside
the venv.
"""
pass
class Fedora(Distro):
"""This covers all Fedora-based distributions.
Includes: Fedora, RHEL, CentOS, Scientific Linux
"""
def check_pkg(self, pkg):
return self.run_command_with_code(['rpm', '-q', pkg],
check_exit_code=False)[1] == 0
def apply_patch(self, originalfile, patchfile):
self.run_command(['patch', '-N', originalfile, patchfile],
check_exit_code=False)
def install_virtualenv(self):
if self.check_cmd('virtualenv'):
return
if not self.check_pkg('python-virtualenv'):
self.die("Please install 'python-virtualenv'.")
super(Fedora, self).install_virtualenv()
def post_process(self):
"""Workaround for a bug in eventlet.
This currently affects RHEL6.1, but the fix can safely be
applied to all RHEL and Fedora distributions.
This can be removed when the fix is applied upstream.
Nova: https://bugs.launchpad.net/nova/+bug/884915
Upstream: https://bitbucket.org/eventlet/eventlet/issue/89
RHEL: https://bugzilla.redhat.com/958868
"""
# Install "patch" program if it's not there
if not self.check_pkg('patch'):
self.die("Please install 'patch'.")
# Apply the eventlet patch
self.apply_patch(os.path.join(self.venv, 'lib', self.py_version,
'site-packages',
'eventlet/green/subprocess.py'),
'contrib/redhat-eventlet.patch')

200
tools/lintstack.py Executable file
View File

@ -0,0 +1,200 @@
#!/usr/bin/env python
# vim: tabstop=4 shiftwidth=4 softtabstop=4
# Copyright (c) 2012, AT&T Labs, Yun Mao <yunmao@gmail.com>
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Stolen from OpenStack Nova
"""pylint error checking."""
import cStringIO as StringIO
import json
import re
import sys
from pylint import lint
from pylint.reporters import text
# Note(maoy): E1103 is error code related to partial type inference
ignore_codes = ["E1103"]
# Note(maoy): the error message is the pattern of E0202. It should be ignored
# for savanna.tests modules
ignore_messages = ["An attribute affected in savanna.tests"]
# We ignore all errors in openstack.common because it should be checked
# elsewhere.
ignore_modules = ["savanna/openstack/common/"]
KNOWN_PYLINT_EXCEPTIONS_FILE = "tools/pylint_exceptions"
class LintOutput(object):
_cached_filename = None
_cached_content = None
def __init__(self, filename, lineno, line_content, code, message,
lintoutput):
self.filename = filename
self.lineno = lineno
self.line_content = line_content
self.code = code
self.message = message
self.lintoutput = lintoutput
@classmethod
def from_line(cls, line):
m = re.search(r"(\S+):(\d+): \[(\S+)(, \S+)?] (.*)", line)
matched = m.groups()
filename, lineno, code, message = (matched[0], int(matched[1]),
matched[2], matched[-1])
if cls._cached_filename != filename:
with open(filename) as f:
cls._cached_content = list(f.readlines())
cls._cached_filename = filename
line_content = cls._cached_content[lineno - 1].rstrip()
return cls(filename, lineno, line_content, code, message,
line.rstrip())
@classmethod
def from_msg_to_dict(cls, msg):
"""From the output of pylint msg, to a dict, where each key
is a unique error identifier, value is a list of LintOutput
"""
result = {}
for line in msg.splitlines():
obj = cls.from_line(line)
if obj.is_ignored():
continue
key = obj.key()
if key not in result:
result[key] = []
result[key].append(obj)
return result
def is_ignored(self):
if self.code in ignore_codes:
return True
if any(self.filename.startswith(name) for name in ignore_modules):
return True
if any(msg in self.message for msg in ignore_messages):
return True
return False
def key(self):
if self.code in ["E1101", "E1103"]:
# These two types of errors are like Foo class has no member bar.
# We discard the source code so that the error will be ignored
# next time another Foo.bar is encountered.
return self.message, ""
return self.message, self.line_content.strip()
def json(self):
return json.dumps(self.__dict__)
def review_str(self):
return ("File %(filename)s\nLine %(lineno)d:%(line_content)s\n"
"%(code)s: %(message)s" % self.__dict__)
class ErrorKeys(object):
@classmethod
def print_json(cls, errors, output=sys.stdout):
print >>output, "# automatically generated by tools/lintstack.py"
for i in sorted(errors.keys()):
print >>output, json.dumps(i)
@classmethod
def from_file(cls, filename):
keys = set()
for line in open(filename):
if line and line[0] != "#":
d = json.loads(line)
keys.add(tuple(d))
return keys
def run_pylint():
buff = StringIO.StringIO()
reporter = text.ParseableTextReporter(output=buff)
args = ["--include-ids=y", "-E", "savanna"]
lint.Run(args, reporter=reporter, exit=False)
val = buff.getvalue()
buff.close()
return val
def generate_error_keys(msg=None):
print "Generating", KNOWN_PYLINT_EXCEPTIONS_FILE
if msg is None:
msg = run_pylint()
errors = LintOutput.from_msg_to_dict(msg)
with open(KNOWN_PYLINT_EXCEPTIONS_FILE, "w") as f:
ErrorKeys.print_json(errors, output=f)
def validate(newmsg=None):
print "Loading", KNOWN_PYLINT_EXCEPTIONS_FILE
known = ErrorKeys.from_file(KNOWN_PYLINT_EXCEPTIONS_FILE)
if newmsg is None:
print "Running pylint. Be patient..."
newmsg = run_pylint()
errors = LintOutput.from_msg_to_dict(newmsg)
print "Unique errors reported by pylint: was %d, now %d." \
% (len(known), len(errors))
passed = True
for err_key, err_list in errors.items():
for err in err_list:
if err_key not in known:
print err.lintoutput
print
passed = False
if passed:
print "Congrats! pylint check passed."
redundant = known - set(errors.keys())
if redundant:
print "Extra credit: some known pylint exceptions disappeared."
for i in sorted(redundant):
print json.dumps(i)
print "Consider regenerating the exception file if you will."
else:
print ("Please fix the errors above. If you believe they are false"
" positives, run 'tools/lintstack.py generate' to overwrite.")
sys.exit(1)
def usage():
print """Usage: tools/lintstack.py [generate|validate]
To generate pylint_exceptions file: tools/lintstack.py generate
To validate the current commit: tools/lintstack.py
"""
def main():
option = "validate"
if len(sys.argv) > 1:
option = sys.argv[1]
if option == "generate":
generate_error_keys()
elif option == "validate":
validate()
else:
usage()
if __name__ == "__main__":
main()

61
tools/lintstack.sh Executable file
View File

@ -0,0 +1,61 @@
#!/usr/bin/env bash
# Copyright (c) 2012-2013, AT&T Labs, Yun Mao <yunmao@gmail.com>
# All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
# Stolen from OpenStack Nova
# Use lintstack.py to compare pylint errors.
# We run pylint twice, once on HEAD, once on the code before the latest
# commit for review.
set -e
TOOLS_DIR=$(cd $(dirname "$0") && pwd)
# Get the current branch name.
GITHEAD=`git rev-parse --abbrev-ref HEAD`
if [[ "$GITHEAD" == "HEAD" ]]; then
# In detached head mode, get revision number instead
GITHEAD=`git rev-parse HEAD`
echo "Currently we are at commit $GITHEAD"
else
echo "Currently we are at branch $GITHEAD"
fi
cp -f $TOOLS_DIR/lintstack.py $TOOLS_DIR/lintstack.head.py
if git rev-parse HEAD^2 2>/dev/null; then
# The HEAD is a Merge commit. Here, the patch to review is
# HEAD^2, the master branch is at HEAD^1, and the patch was
# written based on HEAD^2~1.
PREV_COMMIT=`git rev-parse HEAD^2~1`
git checkout HEAD~1
# The git merge is necessary for reviews with a series of patches.
# If not, this is a no-op so won't hurt either.
git merge $PREV_COMMIT
else
# The HEAD is not a merge commit. This won't happen on gerrit.
# Most likely you are running against your own patch locally.
# We assume the patch to examine is HEAD, and we compare it against
# HEAD~1
git checkout HEAD~1
fi
# First generate tools/pylint_exceptions from HEAD~1
$TOOLS_DIR/lintstack.head.py generate
# Then use that as a reference to compare against HEAD
git checkout $GITHEAD
$TOOLS_DIR/lintstack.head.py
echo "Check passed. FYI: the pylint exceptions are:"
cat $TOOLS_DIR/pylint_exceptions

3
tools/run_pep8 Executable file
View File

@ -0,0 +1,3 @@
#!/bin/sh
tox -epep8

3
tools/run_pylint Executable file
View File

@ -0,0 +1,3 @@
#!/bin/sh
tox -epylint

4
tools/with_venv.sh Executable file
View File

@ -0,0 +1,4 @@
#!/bin/bash
TOOLS=`dirname $0`
VENV=$TOOLS/../.venv
source $VENV/bin/activate && $@

42
tox.ini Normal file
View File

@ -0,0 +1,42 @@
[tox]
envlist = py26,py27,py33,pep8
minversion = 1.6
skipsdist = True
[testenv]
usedevelop = True
install_command = pip install -U {opts} {packages}
setenv =
VIRTUAL_ENV={envdir}
NOSE_WITH_OPENSTACK=1
NOSE_OPENSTACK_COLOR=1
NOSE_OPENSTACK_RED=0.05
NOSE_OPENSTACK_YELLOW=0.025
NOSE_OPENSTACK_SHOW_ELAPSED=1
NOSE_OPENSTACK_STDOUT=1
NOSE_XUNIT=1
deps =
-r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
commands = nosetests
[testenv:pep8]
commands = flake8 {posargs}
[testenv:venv]
commands = {posargs}
[testenv:docs]
commands =
rm -rf doc/html doc/build
rm -rf doc/source/apidoc doc/source/api
python setup.py build_sphinx
[testenv:pylint]
setenv = VIRTUAL_ENV={envdir}
commands = bash tools/lintstack.sh
[flake8]
show-source = true
builtins = _
exclude=.venv,.git,.tox,dist,doc,*openstack/common*,*lib/python*,*egg,tools