airship/tempest-plugin
Code Issues Proposed changes

Cleanup clients, add documenation, and fix 2 tests

Browse Source 
This commit does the following:

- Cleanup the service clients to not hard code resource values
  used in API endpoints
- Add documentation to the README
- Fix the following tests that were not previously working:
  - `test_get_action_validation`
  - `test_invoke_action_control`

Change-Id: I8636f3b704871ad98c346b4a19c7f7f41c705e13
This commit is contained in:
Rick Bartra 2018-08-23 12:06:07 -04:00
parent 9f473f288a
commit 9ede7e5237
7 changed files with 86 additions and 31 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

45
README.rst
View File

@ -2,13 +2,58 @@
Tempest Integration of airship-tempest-plugin
===============================================
Purpose:
--------
The purpose of this plugin is to provide automated tests
for all OpenStack Airship components.
DISCALIMER:
-----------
This initial implementation is just to meet the first use case which is RBAC
testing. For RBAC testing, we only need to hit the API endpoint and check
role permission to the API being tested. Some of the REST clients will need to be
rewritten if functional testing is desired. Those that need to be rewritten
are documented in each service client code.
Environment Information:
------------------------
Testing can be done in a airship-in-a-bottle environment. Please refer to [0] and [1].
Tempest and Tempest plugin installation can be done in a Python virtual environment.
FAQ:
----
- Where do the REST clients exist?
https://github.com/att-comdev/airship-tempest-plugin/tree/master/airship_tempest_plugin/services
- Where do the tests exists? [3]
https://github.com/att-comdev/airship-tempest-plugin/tree/master/airship_tempest_plugin/tests/api
- Example of where/how the REST clients are instantiated.
https://github.com/att-comdev/airship-tempest-plugin/blob/master/airship_tempest_plugin/tests/api/shipyard/base.py
- Where do we define expected results (requirements)?
https://github.com/att-comdev/airship-tempest-plugin/blob/master/airship_tempest_plugin/tests/api/common/rbac_roles.yaml
- Where do we add configuration to support another Airship component?
https://github.com/att-comdev/airship-tempest-plugin/blob/master/airship_tempest_plugin/config.py
- Where do we run the test from?
After the plugin is installed, run it from the tempest directory
- Example of how to run all the RBAC tests for Shipyard:
'tempest run --regex airship_tempest_plugin.tests.api.shipyard.rbac'
- What is Patrole?
https://github.com/openstack/patrole/blob/master/README.rst
- What is a Tempest plugin? [8]
https://docs.openstack.org/tempest/latest/plugin.html
Patrole Supporting Documentation:
---------------------------------
Patrole documentation for requirements driven approach that is used: https://github.com/openstack/patrole/blob/master/doc/source/framework/requirements_authority.rst
Patrole role-overriding: https://github.com/openstack/patrole/blob/master/doc/source/framework/rbac_utils.rst#role-overriding
Patrole under-permission exception: https://github.com/openstack/patrole/blob/master/patrole_tempest_plugin/rbac_exceptions.py#L51
Patrole over-permission exception: https://github.com/openstack/patrole/blob/master/patrole_tempest_plugin/rbac_exceptions.py#L44
Future Considerations:
---------------------
Will the airship-tempest-plugin continue to live here: https://github.com/att-comdev/airship-tempest-plugin or will it be moved under OpenStack?
Will there exist a RBAC gate for all Airship projects?
Referenced Links:
-----------------
[0] https://github.com/openstack/airship-in-a-bottle
[1] https://www.airshipit.org/

22
airship_tempest_plugin/services/shipyard/json/actions_client.py
View File

@ -24,7 +24,7 @@ from tempest.lib.common import rest_client
# NOTE(rb560u): The following will need to be rewritten in the future if
# functional testing is desired:
# - 'def post_actions`
# - 'def create_action`
# This initial implementation is just to meet the first use case which is RBAC
# testing. For RBAC testing, we only need to hit the API endpoint and check
# role permission to that API.
@ -40,33 +40,35 @@ class ActionsClient(rest_client.RestClient):
return rest_client.ResponseBody(resp, body)
def create_action(self):
url = "actions"
url = 'actions'
# Update post_body if functional testing is desired
post_body = json.dumps({})
resp, body = self.post(url, post_body)
self.expected_success(201, resp.status)
body = json.loads(body)
return rest_client.ResponseBody(resp, body)
def get_action(self):
resp, body = self.get('actions/1')
def get_action(self, action_id=None):
resp, body = self.get('actions/%s' % action_id)
self.expected_success(200, resp.status)
body = json.loads(body)
return rest_client.ResponseBody(resp, body)
def get_action_validation(self):
resp, body = self.get('actions/1/validationdetails/1')
def get_action_validation(self, action_id=None, validation_id=None):
resp, body = \
self.get('actions/%s/validations/%s' % (action_id, validation_id))
self.expected_success(200, resp.status)
body = json.loads(body)
return rest_client.ResponseBody(resp, body)
def get_action_step(self):
resp, body = self.get('actions/1/steps/1')
def get_action_step(self, action_id=None, step_id=None):
resp, body = self.get('actions/%s/steps/%s' % (action_id, step_id))
self.expected_success(200, resp.status)
body = json.loads(body)
return rest_client.ResponseBody(resp, body)
def invoke_action_control(self):
url = "actions/1/pause"
def invoke_action_control(self, action_id=None, control_verb=None):
url = 'actions/%s/control/%s' % (action_id, control_verb)
post_body = json.dumps({})
resp, body = self.post(url, post_body)
self.expected_success(202, resp.status)

4
airship_tempest_plugin/services/shipyard/json/airflow_monitoring_client.py
View File

@ -32,8 +32,8 @@ class AirflowMonitoringClient(rest_client.RestClient):
body = json.loads(body)
return rest_client.ResponseBody(resp, body)
def get_workflow(self):
resp, body = self.get('workflows/1')
def get_workflow(self, workflow_id=None):
resp, body = self.get('workflows/%s' % workflow_id)
self.expected_success(200, resp.status)
body = json.loads(body)
return rest_client.ResponseBody(resp, body)

17
airship_tempest_plugin/services/shipyard/json/document_staging_client.py
View File

@ -24,9 +24,7 @@ from tempest.lib.common import rest_client
# NOTE(rb560u): The following will need to be rewritten in the future if
# functional testing is desired:
# - 'def post_configdocs`
# - `def get_configdocs_within_collection`
# - 'def post_commitconfigdocs'
# - 'def create_configdocs`
# This initial implementation is just to meet the first use case which is RBAC
# testing. For RBAC testing, we only need to hit the API endpoint and check
# role permission to that API.
@ -41,16 +39,17 @@ class DocumentStagingClient(rest_client.RestClient):
body = json.loads(body)
return rest_client.ResponseBody(resp, body)
def create_configdocs(self):
url = "configdocs/1"
def create_configdocs(self, collection_id=None):
url = "configdocs/%s" % collection_id
# Update post_body if functional testing is desired
post_body = json.dumps({})
resp, body = self.post(url, post_body)
self.expected_success(201, resp.status)
body = json.loads(body)
return rest_client.ResponseBody(resp, body)
def get_configdocs(self):
resp, body = self.get('configdocs/1')
def get_configdocs(self, collection_id=None):
resp, body = self.get('configdocs/%s' % collection_id)
self.expected_success(200, resp.status)
body = json.loads(body)
return rest_client.ResponseBody(resp, body)
@ -61,8 +60,8 @@ class DocumentStagingClient(rest_client.RestClient):
body = json.loads(body)
return rest_client.ResponseBody(resp, body)
def commit_configdocs(self):
post_body = json.dumps({})
def commit_configdocs(self, force=False, dryrun=False):
post_body = json.dumps({"force": force, "dryrun": dryrun})
resp, body = self.post("commitconfigdocs", post_body)
self.expected_success(200, resp.status)
body = json.loads(body)

5
airship_tempest_plugin/services/shipyard/json/log_retrieval_client.py
View File

@ -26,8 +26,9 @@ from tempest.lib.common import rest_client
class LogRetrievalClient(rest_client.RestClient):
api_version = "v1.0"
def get_action_step_logs(self):
resp, body = self.get('actions/1/steps/1/logs')
def get_action_step_logs(self, action_id=None, step_id=None):
resp, body = \
self.get('actions/%s/steps/%s/logs' % (action_id, step_id))
self.expected_success(200, resp.status)
body = json.loads(body)
return rest_client.ResponseBody(resp, body)

1
airship_tempest_plugin/tests/api/common/rbac_roles.yaml
View File

@ -43,7 +43,6 @@ shipyard:
workflow_orchestrator:commit_configdocs:
- admin
- admin_ucp
- admin_ucp_viewer
workflow_orchestrator:list_workflows:
- admin
- admin_ucp

23
airship_tempest_plugin/tests/api/shipyard/rbac/test_actions_rbac.py
View File

@ -41,9 +41,10 @@ class ActionsRbacTest(rbac_base.BaseShipyardRbacTest):
# As this is a RBAC test, we only care about whether the role has
# permission or not. Role permission is checked prior to validating
# the post body, therefore we will ignore a BadRequest exception
# and NotFound exception
try:
self.shipyard_actions_client.create_action()
except exceptions.BadRequest:
except (exceptions.BadRequest, exceptions.NotFound):
pass
@rbac_rule_validation.action(
@ -60,15 +61,19 @@ class ActionsRbacTest(rbac_base.BaseShipyardRbacTest):
except exceptions.NotFound:
pass
''' NEEDS REWORK AS SHIPYARD NOT DOING POLICY ENFORCEMENT FIRST
@rbac_rule_validation.action(
service="shipyard",
rules=["workflow_orchestrator:get_action_validation"])
@decorators.idempotent_id('a5156dcd-2674-4295-aa6a-d8db1bd4cf4b')
def test_get_action_validation(self):
with self.rbac_utils.override_role(self):
self.shipyard_actions_client.get_action_validation()
'''
# As this is a RBAC test, we only care about whether the role has
# permission or not. Role permission is checked prior to validating
# the post body, therefore we will ignore a NotFound exception
try:
self.shipyard_actions_client.get_action_validation()
except exceptions.NotFound:
pass
@rbac_rule_validation.action(
service="shipyard",
@ -84,12 +89,16 @@ class ActionsRbacTest(rbac_base.BaseShipyardRbacTest):
except exceptions.NotFound:
pass
''' NEEDS REWORK AS SHIPYARD NOT DOING POLICY ENFORCEMENT FIRST
@rbac_rule_validation.action(
service="shipyard",
rules=["workflow_orchestrator:invoke_action_control"])
@decorators.idempotent_id('4f6b6564-ff1d-463a-aee8-ed2d51e2a286')
def test_invoke_action_control(self):
with self.rbac_utils.override_role(self):
self.shipyard_actions_client.invoke_action_control()
'''
# As this is a RBAC test, we only care about whether the role has
# permission or not. Role permission is checked prior to validating
# the post body, therefore we will ignore a NotFound exception
try:
self.shipyard_actions_client.invoke_action_control()
except exceptions.NotFound:
pass