93 Commits

Author SHA1 Message Date
Devananda van der Veen
c7da7a6925 Add keystone policy support to Ironic
Implements more fine-grained policy support within our API service,
following the oslo policy-in-code spec, while maintaining compatibility
with the previous default policy.json file. An empty policy.json file is
included, along with a sample file listig all supported policy settings
and their default values.

A new tox target "genpolicy" has been added to ease automation of
sample policy file generation.

All calls to policy.enforce() have been replaced with with
policy.authorize() to avoid silent failures when a rule is undefined,
because enforce() does not raise() if the target rule does not exist.

  NOTE: policy.enforce() is not removed by this patch, but a deprecation
  warning will be logged if it this method is invoked.

Updates unit test coverage for the new authorize() method, as well as
more general unit test updates for some of the new rules.

Partial-bug: #1526752
Change-Id: Ie4398f840601d027e2fe209c17d854421687c7b7
2016-08-04 12:43:20 +00:00
Jay Faulkner
dac5aee05d Add py35 to tox environments
Since we're running python 3.5 tests now, we should also be
running thme when running all tests with just `tox`.

Change-Id: I6ec077878906fd48e9e4dc654c63da3dbc83173b
2016-07-15 20:58:47 +00:00
Lucas Alvares Gomes
7abfe28618 Add debug environment to tox
The oslo_debug_helper enables developers to use pdb for debugging the
unittests.

Ref: http://docs.openstack.org/developer/oslotest/features.html

Change-Id: I08d4c0fcb2e8015c02f5e9b9d4e9dff7f58d21ea
2016-06-03 17:09:22 +01:00
Lin Tan
773aa20a4a Make use of oslo-config-generator
oslo_config provide a utility for generating sample config files,
which provide more detail about opts, like Minimum/Maximum value
and Allowed values.
So drop Ironic's "generate_sample.sh" which was copied from
oslo-incubator long time ago.

Add a new entry point "ironic" under oslo.config.opts namespace to
explore config options to oslo-config-generator.

After this patch, new config options of Ironic code should register
with ironic/conf/opts.py. New external libraries should
register with tools/config/ironic-config-generator.conf

There is a bug #1554657 with oslo-config about deprecated_group.
This bug have impact of some configs from keystonemiddleware
and oslo.messaging in ironic.conf.sample
So currently, deprecated option should always add the deprecated_group
even it didn't alter the group, otherwise the deprecated group value will
be 'DEFAULT'.

Update etc/ironic/ironic.conf.sample via running 'tox -egenconfig'.

Closes-Bug: #1564195
Change-Id: If7721e98e69b6f54f1ee04a07477396b86583371
2016-05-25 11:59:13 +08:00
Devananda van der Veen
6e82b2118a Update the api-ref documentation for Root and Nodes
This updates the API root documentation, which was non-existent,
and the Node documentation significantly, replacing many of the samples
and adding more verbiage to each property.

It's not perfect, but this is vastly more correct than the current docs.

Change-Id: Iaeb1c80ac0655fc0d73fa495ed43e3c02b345495
2016-05-20 08:35:29 -07:00
Jim Rollenhagen
6fbf5c5516 Eliminate warnings about rm in api-ref build
This currently gives a warning about the 'rm' command not being
installed in the venv. Whitelist bash and run rm under bash (like we do
with other external commands) to squash this warning.

Closes-Bug: #1581778
Change-Id: Id26926ad7a75b94f0f57ea3c2083312c66663b34
2016-05-16 21:51:48 +00:00
Jim Rollenhagen
fa098f69e4 Migrate api-ref into our tree
Per this email[0], it's time to migrate our API docs into our tree.

This copies the sphinx extension from nova into our tree as well, which
will be later split into its own python module. At that time we can cut
over to the module and drop it from our tree.

The conf.py and the tox environment are also cribbed from nova.

[0] http://lists.openstack.org/pipermail/openstack-dev/2016-May/093765.html

Change-Id: I683cc4d6459d77c42951a06247b6f9ff8f8456bd
2016-05-03 21:39:42 +00:00
Alexey Galkin
76cacceb60 Fix tox cover command
Add testr-args and coverage in tox cover command

Change-Id: I69192c464a36b42d97559534d2df19f737d58407
Closes-bug: #1571366
2016-04-29 16:26:48 +03:00
Jim Rollenhagen
e27e7ed4a9 Make tox respect upper-constraints.txt
This will force pip install to use the upper-constraints.txt specified
version of pip modules. When you don't do this, you are out on the
bleeding edge and become unstable everytime some python library in the
wide world changes in a way that you don't expect.

Post jobs are not yet compatible with the upper-constraints file, so
override the install_command there to skip upper-constraints.

Blatantly cribbed from the equivalent trove patch:
https://review.openstack.org/#/c/298850

Change-Id: I73aa8cf37adc36db2c56a8f29ea165775a83dc7f
Closes-Bug: #1563038
2016-04-01 05:07:19 -07:00
Lucas Alvares Gomes
d85b9970ce Fix doc8's "duplicated target names" (D000) error
This patch is enabling the D000 error (duplicated target names) in doc8
and fixing the occurrences of this error in the docs.

Change-Id: Iac048d213e0571b99696ec708f5c07d6e1bebb19
2016-02-03 15:22:35 +00:00
Lucas Alvares Gomes
fc72b4d904 Enable doc8 style checker for *.rst files
This patch is enabling the doc8 style checker for the *.rst files in
Ironic.

Since this patch is just the first stab at enabling this new check two
errors are currently being ignored by doc8 because of the number of
occurrences in our docs:

* D000 - Duplicated target names. We do have many links with generic
         names scattered around the docs (e.g "see", "link")

* D001 - Line's too long

These errors will be fixed in a following patch.

Change-Id: I32fb5a3104b9773c55a80a84dd81e49c567c76ac
2016-02-02 12:38:20 +00:00
Jenkins
357da052b4 Merge "Remove bashate from envlist" 2016-01-05 19:18:37 +00:00
Zhenguo Niu
29b37065d7 tox: make it possible to run pep8 on current patch only
This makes tox use a simple wrapper around flake8 like
Nova, which can be told to restrict the check to only
files changed in the current command. This can be invoked
in a simple manner with 'tox -epep8 -- -HEAD'. Since most
commits only touch a handful of files, this will usually
be far faster than checking all source files.

To check an entire branch for bisectability it can be
automated via

  git rebase -i master -x 'tox -epep8 -- -HEAD'

Change-Id: I75b1cfe3198b1217f8b25877714a4af47ae7069f
2016-01-05 09:55:48 +08:00
Ruby Loo
a7908dc5c0 Remove bashate from envlist
This removes 'bashate' from the envlist in tox.ini because it doesn't
cause bashate to be run. Rather, it causes 'tox' to run the unit
tests :-(

'tox pep8' will invoke bashate.

Change-Id: I235ed7b20b45f2052f0ca0e9e4d96e379667b567
2016-01-04 15:48:39 +00:00
Ruby Loo
def448e386 Remove */openstack/common* in tox
Ironic remove oslo incubator (ironic/openstack/*), so removes
them in flake8 too.

A follow-patch on d7741ea1eebde2fb9966dd53372eb8655bdb12ac
Co-Authored-By: Tan Lin <lin.tan@intel.com>
Change-Id: I9d8fd28f53e8b4bc289a1015e464585b8bdeebd9
2015-12-28 14:09:50 +08:00
Lucas Alvares Gomes
b51f7d4d8c Run flake8 against the python scripts under tools/ and devstack/tools
This patch allows flake8 to run against the scripts under tools/ and
devstack/tools directories.

It also fix the current pep8 errors where it's needed.

Change-Id: I3f447564dd528c6efdee10a4a13f1ff1d7e1c3b1
2015-12-24 01:56:39 +00:00
Lucas Alvares Gomes
d7c41ebf9b Fix "No closing quotation" error when building with tox
Apparently # is treated as a comment inside the string and as
consequence the rest of the command gets stripped. This patch drops
those comments from the bashate command.

This patch also moves the bashate command to its own script under
tools/.

Change-Id: Ic44bb64419ca1465f69c91fd579e8c84d69f84b8
Closes-Bug: #1527562
2015-12-21 11:53:41 +00:00
Jim Rollenhagen
95fcd6433b Run bashate as part of the pep8 command
In order to save gate resources we could just run bashate as part of the
the pep8 command.

The tox bashate command is removed and bashate is added as a dependency
in the test-requirements.txt file.

Change-Id: I75df4103c0154c5424b8781c1db3480336a5e1d2
2015-12-17 17:45:30 +00:00
Lucas Alvares Gomes
49e273b1a5 Fix syntax errors in the shell scripts under devstack/tools
This patch is fixing the syntax errors/warnings in the shell scripts
under devstack/tools and add that directory to the tox bashate command.

Change-Id: I957a88f5b7009fe92078eb3d53e225943d28a539
Partial-Bug: #1526518
2015-12-17 17:26:48 +00:00
Lucas Alvares Gomes
e95a8b9ae9 Add bashate tox command
Now that we copied the devstack code to our tree we need to ensure that
the bash scripts we have are correct. This patch is adding a tox command
(bashate) that will check the syntax of the shell scripts under
the devstack/ directory.

The scripts under devstack/tools/ are not being checked, that's because
they weren't check when the code was in devstack as well and there's
loads to fix. It will be fixed in a following patch.

Change-Id: I8f66492081e0fd10e1de057febb9324a81411c7a
Partial-Bug: #1526518
2015-12-15 20:03:27 +00:00
Ondřej Nový
5ea3d08e14 Deprecated tox -downloadcache option removed
Caching is enabled by default from pip version 6.0

More info:
https://testrun.org/tox/latest/config.html#confval-downloadcache=path
https://pip.pypa.io/en/stable/reference/pip_install/#caching

Change-Id: Ib8580ff9614b9e846d2b442e886d408b55469471
2015-12-11 21:17:54 +01:00
Naohiro Tamura
1b39899c45 force releasenotes warnings to be treated as errors
This patch applies the instruction in [1] to ensure that the release
notes render properly and without warnings by adding "-W" to the
sphinx-build line in tox.ini so that Sphinx can treat warnings as
errors.

  [1] [openstack-dev] [release] better linting of release notes
  http://lists.openstack.org/pipermail/openstack-dev/2015-November/080521.html

Change-Id: I64c05014e88a48b6336ce92c87da47dbe34b20d8
2015-11-27 10:56:00 +09:00
Jim Rollenhagen
aae12e5109 Add reno for release notes management
Mostly cargo-culted from I9549e6b491b26232f4dac900b0bf05f10eb0e30a.

Note that reno automatically inserts version numbers; liberty.rst is
sufficient for intermediate release models.

Change-Id: Ic700d719242b17532b0b43a68b7eff507cb3a43c
2015-11-13 03:22:32 -08:00
John L. Villalovos
3e22cc4ea0 Python 3.4 unit tests fail with LANG=C
After the move to os-testr the 'tox -epy34' unit tests will fail when
LANG=C is set in the environment.

Set LC_ALL=en_US.UTF-8 in the testenv to solve the issue. This is
in-line with our curent method of running the unit tests in US English
so that logging, messages, and exceptions can be validated by unit
tests.

Closes-Bug: #1501415
Change-Id: Ifefdd67c9e5edf22f3b7a1fce56f21c788ac9ead
2015-10-01 09:52:39 -07:00
John L. Villalovos
5ac855e25d Add 'whitelist_externals = bash' for two testenvs
Two of the testenvs are using bash. So we need to have
'whitelist_externals = bash' for them.

Closes-Bug: #1501119
Change-Id: I62cd35e87f33cb97a2d7cf4a6ebbc34b996a8def
2015-09-30 01:00:30 +00:00
Matthew Treinish
9b20bf883f Switch tox unit test command to use ostestr
This commit switches the tox command to use ostestr instead of calling
testr through setup.py. The primary advantage here is that it uses the
subunit-trace output filter. ostestr primarily exists as a replacement
for pretty_tox.sh bash scripts which spread like a plague through
OpenStack projects after Nova copied it from Tempest. (although ostestr
also provides some other useful features) Granted ironic wasn't using
that, but this commit makes the switch to enable getting the
subunit-trace output which is generally useful.

Note that this dramatically increases output while running tests.
However, test failures are still at the bottom of the output, so it
shouldn't cause much pain, if any.

Change-Id: I1acc2deadc01421baec256352b35a646676445e5
2015-09-29 14:05:54 +00:00
John L. Villalovos
f350d87af8 Prepare for functional testing
Prepare for functional testing by creating two new directories:
  * ironic/tests/unit
  * ironic/tests/functional

Move all files currently in ironic/tests/* to ironic/tests/unit/

Update imports from 'ironic.tests.YYY' to 'ironic.tests.unit.YYY'

Fix any PEP8 issues due to longer lines

Partial-Bug: #1491670
Change-Id: I1166f20adf8c84b2042b62e348926502a4851e08
2015-09-28 21:27:01 +00:00
John L. Villalovos
3f6e443232 Add constraint target to tox.ini
This adds a pip install command to tox.ini that is only used when the
tox env is passed with the 'constraints' factor appended onto it.
As such this will not effect developer workflows or current unit tests.

The initial use of this will be in a non-voting job, to verify that the
constrained checks with tox are stable.  DevStack is already running
constrained jobs, as such problems are not expected.

To run a tox with pip using constraints on a developer system a
developer should run the desired tox environment with -constraints.
For example: $(tox -epy27-constraints)
Pip will pull the current version of the upper-constraints.txt file down
from the git.openstack.org, however this method can be overriden to use
a local file setting the environment variable "UPPER_CONSTRAINTS_FILE"
to the local path or a different URL, it is passed directly to pip.

This is currently not enabled in the default tox run, however it is
possible to enable it as a default by adding it to 'envlist' in tox.ini

Backport of Nova commit ad38ce1fc98692ee59ef061de1a35f355d219aa1

Change-Id: Ia3cfb30fd1d8529bb4fbcf8873036149ea6f483c
Implements Blueprint: Requirements-Management
2015-09-17 11:42:58 -07:00
Thiago Paiva
bcdacbaf9a Put py34 first in the env order of tox
To solve the problem of "db type could not be determined" on py34
we have to run first the py34 env to, then, run py27. This patch
puts py34 first on the tox.ini list of envs to avoid this problem
to happen.

Change-Id: Ib09cbc71149ffcc1f36495faf1c6586c4e0a0a86
Closes-bug: #1489059
2015-08-26 12:58:56 -03:00
Lucas Alvares Gomes
7f1aebfc2b Enforce flake8 E711
This patch enforces the rules E711 in the Ironic code base.

Change-Id: Ic9395f5f6869212e4076a045ced83b57d0c16324
2015-08-13 16:36:15 +01:00
Jim Rollenhagen
6f8ad61cbd Remove requirements.txt from tox.ini deps
This is redundant, per lifeless email:
http://lists.openstack.org/pipermail/openstack-dev/2015-July/069663.html

Change-Id: Ie78d0e5b2fc8274867f2c6013b04ae9e0796f81f
2015-07-15 18:31:29 -07:00
Yuriy Zveryanskyy
8e0838536f Remove db connection string env variable from tox.ini
It is not needed with oslo.db 1.12.0.

Change-Id: I49dc450d6bb84e5a75ef9ecdf248eac567c3442c
2015-07-02 14:37:10 +03:00
Jeremy Stanley
25f2c5e079 Switch from MySQL-python to PyMySQL
As discussed in the Liberty Design Summit "Moving apps to Python 3"
cross-project workshop, the way forward in the near future is to
switch to the pure-python PyMySQL library as a default.

https://etherpad.openstack.org/p/liberty-cross-project-python3

Also set the OS_TEST_DBAPI_ADMIN_CONNECTION override variable so
that oslo.db opportunistic detection will know to use PyMySQL until
I12b32dc097a121bd43991bc38dd4d289b65e86c1 makes it the default
behavior.

Change-Id: Icd91a065d3c4f62791ba0dca99a822e3a1a0ad44
Co-Authored-By: Victor Sergeyev <vsergeyev@mirantis.com>
2015-06-22 15:51:10 +03:00
Jenkins
b6e447a1aa Merge "Enable flake8 checking of ironic/nova/*" 2015-06-18 18:19:47 +00:00
Yuiko Takada
c924c3f1a4 Pass environment variables of proxy to tox
When a development environment is under a proxy, tox is failed even if
environment variables of the proxy are set.

This patch fix this problem.

Change-Id: I650019ae86e223d5e9c2d4a5b51b92377f3c471e
2015-06-15 10:00:23 +09:00
John L. Villalovos
9508d86dd0 Enable flake8 checking of ironic/nova/*
The directory ironic/nova/* was being excluded from the flake8 check.

Make some minor whitespace and comment changes to
ironic/nova/compute/manager.py to make it pass the flake8 checks.

Change-Id: I5a76550db3b28a87964d1505441246cdb0354ee7
2015-06-12 11:48:20 -07:00
Jenkins
309707bab0 Merge "Enforce flake8 E123/6/7/8 in ironic" 2015-06-04 13:49:28 +00:00
Sam Betts
8c07c4fda3 Enforce flake8 E123/6/7/8 in ironic
This patch enforces the rules E123, E126, E127, and E128 in the ironic
code base:

E123 - closing bracket does not match indentation of opening
bracket’s line
E126 - continuation line over-indented for hanging indent
E127 - continuation line over-indented for visual indent
E128 - continuation line under-indented for visual indent

This fixes any parts of the current code which fails these rules and
removes these rules from the tox.ini flake8 ignore list.

Change-Id: Ia96582b5e9abc088d6c1694afc93c59be4a4065c
Closes-Bug: 1421522
2015-06-04 12:56:51 +01:00
Joshua Harlow
3681315012 Use svg as it looks better/scales better than png
Change-Id: I2a17b5ddbe90fe8ea1ead53497e4dfcac5bf5647
2015-06-03 16:56:45 -07:00
John L. Villalovos
1acab160cc Revert "Add simplegeneric to py34 requirements"
This reverts commit fe2f445418ad7cb7741a7d237fd2ad69c27833a4.

WSME has been updated to 0.7 which fixes the underlying issue with
'simplegeneric' not being pulled in as a dependency.

Change-Id: Ia224eafb2c2220c970367580a8818746f9479f9d
Depends-On: I28861178fac89e6f3fabd34b048f53b6506eda5e
2015-06-02 15:01:01 +00:00
Andreas Jaeger
6befe83d87 Rename gendocs tox environment
All other openstack projects use docs environment, use the same
name for ironic as well.

Change-Id: I46fd50da89c3247bffefc9c971f8eda309bbe9bd
2015-05-21 08:06:06 +02:00
Victor Sergeyev
fe2f445418 Add simplegeneric to py34 requirements
simplegeneric suppoosed to be installed by WSME, but for a some reasons
it was missed in py3 env. We can't add this library to requirements.txt,
because it missed in global requirements, so I've added simplegeneric
to the `deps` section in py34 environment.

This if a fast-and-dirty way to unblock python34 gates for ironic.

Change-Id: Ib0cddf7541defbdfe1b17448b52becf8bd4ef3de
2015-05-20 16:42:34 +03:00
Victor Sergeyev
70062322a2 Run tests in py34 environment
A lot of fixes to be compatible with python 3:
- fix encoding/decoding errors
- fix issues with comparison
- use `reload`, `reraise`, ext. modules from six
- use items() instead of iteritems()
- add a new file with py3 specific test requirements
- drop passing the arbitrary arguments to object.__new__ method.
  See bug [1] for more details.
- add a workaround to bug in `mock` library
- add py33 and py34 test environment to tox.ini

[1] http://bugs.python.org/issue1683368

Change-Id: I90936cb6b6eaaf4b5e1ce67732caec3c8bdc1cc2
2015-05-06 11:51:44 +03:00
Joshua Harlow
4c7a7bdf64 Add a venv that can generate/write/update the states diagram
Instead of requiring manual understanding of how to generate
the states diagram, add an explicit virtualenv that can be
used to create and generate that docs diagram (and replace the
existing file with it).

Change-Id: I1738da9bf08bfda9b9be01489e8456569a31709e
2015-04-27 11:07:01 -07:00
Jay Faulkner
021462beed Set hash seed to 0 in gendocs environment
This was done in the venv environment to fix docs builds in the gate,
but they still fail randonly locally when using the tox environment to
build them. This change ensures the gendocs environment works for
generating documentation locally.

Change-Id: I41e4af9edaaa76b837a6c92a296b5738c85b632b
2015-04-07 13:32:53 -07:00
John L. Villalovos
826cd21a7d Remove setting language to en_US for 'venv'
Remove setting LANGUAGE=en_US for 'venv' virtualenv.

Since the 'venv' environment is not used for unittests and users may
want to see messages in their native language in the 'venv' virtualenv.

Also makes it consistent with python-ironicclient

Change-Id: I1902d162cc7a5a1c8ee2e6a7412096d645277cc0
2015-04-01 13:35:35 -07:00
John L. Villalovos
bf6d966756 Force LANGUAGE=en_US in test runs
Basically back-porting what was done for Nova in this patch:
https://review.openstack.org/#/c/160892/

If your native locale is not en_US and you run tests, some could fail
due to the message translation that happens since the assertion messages
are in en_US (not translated, just hard-coded in the tests).

Note that setting LANG=en_US had no effect.

Change-Id: I16fab554612a718871625f688dd5501a69fb3417
2015-03-13 08:42:51 -07:00
John L. Villalovos
45b525d341 Fix PEP8 E124 & E125 errors
Fix PEP8 E124 & E125 errors
    E124: closing bracket does not match visual indentation
    E125: continuation line with same indent as next logical line

Remove E124 & E125 from the ignore list for flake8

Change-Id: Ia9aee2fb7e0bb6771aad11d3523119944eeb4170
2015-02-20 12:24:28 -08:00
John L. Villalovos
c1c4c5a41b Fix PEP8 E121 & E122 errors
Fix PEP8 E121 & E122 errors
 E121: continuation line under-indented for hanging indent
 E122: continuation line missing indentation or outdented

Remove E121 & E122 from the ignore list for flake8

Change-Id: If63dd80ee85fc97596492e0b2cb6db9e10458c54
2015-02-19 08:29:35 -08:00
John L. Villalovos
bb6a0b29c9 For flake8 check, make the 'E12' ignore be more granular
ignore = E12 means ignore E12* errors.

Make it more granular by converting to E121,E122,...,E129

This will make it easier in the future to remove a single ignore and
update the code to resolve the issues.

Change-Id: Iddff6a826bc7c9ff07643b7408c51f1bd129530c
2015-02-18 14:39:30 -08:00