824 Commits

Author SHA1 Message Date
Zuul
8b8680a52f Merge "Use consistent arguments for invocation_flow method" 2020-09-10 11:31:06 +00:00
Zuul
45b38871e1 Merge "Add more informative debug logs to agent processes" 2020-09-10 11:31:05 +00:00
Takashi Kajinami
e31497dd19 Remove unnecessory -it from docker run command
We expect that storlets docker containers run in background,
so We don't need -i and -t to start them.
This patch removes these unnecessory options, and also fixes
some outdated comments in restart_docker_container script.

Change-Id: I1f3d207a75767811e8c22a114b7f4c6edfccb1dd
2020-09-06 15:27:07 +00:00
Takashi Kajinami
be9e28dc67 Add more informative debug logs to agent processes
This patch adds some more debug logs into agent processes, so that
we can observe their behavior more in detail.
Also, this patch removes some detailed informations like pids from
responses returned to gateway/runtime, because gateway/runtime doesn't
require the details of agent behavior.

Change-Id: I2fd9998966120c46414e607b94ce03bee884e11f
2020-09-06 20:03:25 +09:00
Takashi Kajinami
9b8fb2d745 Use consistent arguments for invocation_flow method
Trivial-Fix
Change-Id: I6481147a722b2f366300f2ac7952ebd94f1a2ada
2020-09-06 13:03:38 +09:00
Ghanshyam Mann
f0b74e1a76 [goal] Migrate to victoria testing runtime
As per victoria cycle testing runtime and community goal[1]
we need to migrate upstream CI/CD to Ubuntu Focal(20.04).

updated bindeps fpr py3 deps.

Story: #2007865
Task: #40220

[1] https://governance.openstack.org/tc/goals/selected/victoria/migrate-ci-cd-jobs-to-ubuntu-focal.html
Change-Id: I3e95701c5f533f17d368fd9cd9276ab62ca41876
2020-08-17 00:53:58 +00:00
Kota Tsuyuzaki
c92df8ee53 Try to add mock explicitly
the version is same as swift upstream.

Change-Id: I8cddda3d16a18274e9e46e9e946ebc75dc8faac5
2020-08-13 11:03:55 +00:00
Zuul
8462e4e688 Merge "Small cleanups" 2020-07-03 10:47:59 +00:00
Andreas Jaeger
a3ec6967fc Fix functional job: Install tox
Recently, the OpenDev images did not install tox anymore, add
ensure-tox role to functional job pre-run playbook so that tox is
installed.

Change-Id: I56057f2120dafa31e24688f66cc8016f1a19681e
2020-07-03 09:57:07 +02:00
Andreas Jaeger
46a90854f6 Small cleanups
* Remove obsolete translation entries from setup.cfg
* Remove obsolete files babel.cfg and openstack-common.conf
* Remove install_command from tox.ini, the default is fine.
* Replace obsolete UPPER_CONSTRAINTS_FILE with current
  TOX_CONSTRAINTS_FILE.

Change-Id: Ib837a4dc71b8d758a81d453c26d251a16aa72394
2020-06-09 12:02:03 +02:00
Andreas Jaeger
05941f13eb Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.1 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering problems

Update Sphinx version as well.

Disable openstackdocs_auto_version to not auto-version the documents.

Disable openstackdocs_auto_name to use 'project' variable as name.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

Use doc/requirements for doc requirements following PTI.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I1c0aeb590b1ce0f5ee43f9bcea51eeef1c98f057
2020-06-02 15:09:20 +02:00
Zuul
0cbb51abe9 Merge "typo: storlet, not storelt" 2020-06-01 17:32:06 +00:00
Takashi Kajinami
e7678fc454 Make py2 functional tests job non-voting
Currently py2 functional tests job is broken because of incorrect
constraints used in devstack.
This patch is a temporal workaround to unblock our gate, while we are
working to fix in devstack.

Change-Id: I76d054fd976e022109acd44ca0cf3740fb73f945
2020-05-30 00:15:32 +09:00
Tim Burke
405ea64398 typo: storlet, not storelt
Change-Id: Ic5b17ab46435eba7695b0eec700887eb3969ef4f
2020-05-15 19:45:20 -07:00
Zuul
4c6181f453 Merge "Fix hacking min version to 3.0.1" 2020-05-13 12:42:41 +00:00
Zuul
0c3eea51cc Merge "Add py38 package metadata" 2020-05-13 05:09:07 +00:00
Ghanshyam Mann
08b99dd247 Fix hacking min version to 3.0.1
flake8 new release 3.8.0 added new checks and gate pep8
job start failing. hacking 3.0.1 fix the pinning of flake8 to
avoid bringing in a new version with new checks.

Though it is fixed in latest hacking but 2.0 and 3.0 has cap for
flake8 as <4.0.0 which mean flake8 new version 3.9.0 can also
break the pep8 job if new check are added.

To avoid similar gate break in future, we need to bump the hacking min
version.

- http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014828.html

Change-Id: If8001b28d913a6076d85a4fca278f8d42e13d781
2020-05-12 21:15:47 -05:00
Sean McGinnis
6d63f55723 Add py38 package metadata
Now that we are running the Victoria tests that include a
voting py38, we can now add the Python 3.8 metadata to the
package information to reflect that support.

Change-Id: I37357354e70df2b030b97a6870077e2644c3f2ee
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-10 23:23:40 +00:00
Sean McGinnis
6231cd1ea3 Use local py2 upper-constraints
The global upper constraints repo will no longer track py2 packages,
meaning upper constraints for the py2.7 runtime will need to be handled
locally.

This adds an in-repo file for upper constraints enforcement using the
last py27 constraints from openstack/requirements to validate
functionality.

See Swift change I624f914c6155178cbe474f1eae2871e2aef721eb for
equivalent update done there.

Change-Id: Ic97ec7d3c78414cce227d75d7fcba4fd7a39c3e4
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-10 23:23:26 +00:00
Takashi Kajinami
9e0b6d962d Fix functional job failure in gate
Currently functional tests are failing because of the following error
during installation kicked by tox.
  error: [Errno 13] Permission denied: 'storlets.egg-info'

This patch makes sure that egg-info directory is removed after engine
installation, so that tox won't fail with the permission error.

Change-Id: I3842a758748450da98f9b4e320eb4a6f27c98bca
2020-05-07 12:41:36 +00:00
fa53a62cf7 Add Python3 victoria unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for victoria.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: Iefb0807262dd17c16847549364210e6d06e0d3c2
2020-04-26 13:38:59 +00:00
b14ffe2fc3 Update master for stable/ussuri
Add file to the reno documentation build to show release notes for
stable/ussuri.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.

Change-Id: I249ca2fb187e44a2967aec85d167e3eb3981dddb
Sem-Ver: feature
2020-04-26 22:38:34 +09:00
Zuul
3e8f98f4b6 Merge "Add release note for 5.0.0" ussuri-em 5.0.0 5.0.0.0rc1 2020-04-23 03:05:35 +00:00
Takashi Kajinami
7a5542d156 Add release note for 5.0.0
Change-Id: I9b9799e0a8759e4a2279bedaf0e425a190413786
2020-04-23 11:22:07 +09:00
Takashi Kajinami
8a0156692d Revert "[ussuri][goal] Drop python 2.7 support and testing"
This reverts commit 2f5d0300cc5c9ca5ffc8599baa3f9f86d3b8f746.

Change-Id: I015cb1969c6024a374f2a54a5df47a853dc000bc
2020-04-21 11:30:12 +09:00
Takashi Kajinami
b9b56196ce Use python-swiftclient 3.8.1 to run func tests in Python2.7
... because python-swiftclient=3.9.0 causes connection leak in
Pyton2.7, which causes failure in functional tests.
Currently adding constraint in test-requirments.txt is not working
as expected because of global-requirements.txt, so reinstall
pyhton-swiftclient during devstack installation.

Change-Id: If863bc966a1a5d25db40b812daafcf6a95f5dda5
2020-04-21 11:30:12 +09:00
Takashi Kajinami
a090737cf9 Use stable/train keystone to run Python 2.7 job
Change-Id: Idda17c542afc87bcf3bd30e8fcad8bb1bb2bb936
2020-04-17 16:51:41 +09:00
Takashi Kajinami
5250bc85b0 Fix cap for hacking in python 2.7
... so that we can keep Python 2.7 support during Ussuri.

Change-Id: Ic6cafc21efd6f7eacc43fcd59ccfb61ec765d251
2020-04-16 11:53:41 +09:00
Takashi Kajinami
271dae8106 Revert "Cleanup py27 support"
This partially reverts commit 60c1454e953f5dda8c0101562051adfecab1c91d.

It only reverts the py27 changes, not the docs change since that is
not relevant for py27.

Because swift still has some remaining known issue with Py3,
we still need to keep py2 compatibility during Ussuri cycle.

Change-Id: I3d6e1363a7f001618b594ded8bdaefc3164544f6
2020-04-16 11:48:13 +09:00
Zuul
a8f1fce69a Merge "Cleanup py27 support" 2020-04-15 01:53:38 +00:00
Zuul
ddf82cf59c Merge "Use SBusClient for execute command" 2020-04-15 01:38:44 +00:00
Takashi Kajinami
826be15b6a Use SBusClient for execute command
This patch is the final refactoring work to have client library
for sbus communication, and port current implementaion about
execute command from runtime to SBusClient.

This patch makes execute command rely on the same communication
as the other commands, which happens on SERVICE_OUT_FD, and remove
TASK_ID_OUT_FD which is used only for returning task id.
This allows us to avoid special handling for execute command.

Change-Id: I00dc7b3004512624f9aaf120420f910e451cac89
2020-04-13 16:54:01 +09:00
Andreas Jaeger
60c1454e95 Cleanup py27 support
Make a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
  know about the requirement
- Remove obsolete sections from setup.cfg
- Update classifiers
- Switch to using sphinx-build, fix doc problems

Change-Id: I2db738820744e1931949e4618299cdb2e7832d2a
2020-04-07 17:29:35 +02:00
Andreas Jaeger
ceda1ba934 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found.

Change-Id: I2695f5a2371babfbf2a16565ff2ce5205a45fd2c
2020-03-31 06:49:06 +00:00
Takashi Kajinami
82c1595c12 Install test-requirements.txt during installation
... because we need python-swiftclient packages listead in
test-requirements.txt to complete deployment.

Change-Id: Ia575839a291b26d4e02d8d5e661f468c4a657f78
2020-03-30 23:00:04 +09:00
Takashi Kajinami
d30f5fcd46 Remove compatibility codes for swift < 2.8.0
Change-Id: I774ce8d54da45f47650213cca33c0efe0e959740
2020-02-09 15:02:20 +09:00
Ghanshyam Mann
2f5d0300cc [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in ussuri cycle.

storlets is ready with python 3 and ok to drop the
python 2.7 support.

Complete discussion & schedule can be found in
- http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html
Depends-On: https://review.opendev.org/#/c/693631/

Change-Id: I62239e865d2938f0fc7dc2755156206b29095524
2020-01-21 20:12:37 -06:00
Takashi Kajinami
821570021b Raise timeout value for zuul job execution.
We recently observe that functional tests often fail because of
timeout in gate.
This patch raises timeout from 30 min to 40 min to avoild
timeouts and unnecessory rechecks.

Change-Id: Id148885190bfca694e1fa00439c8a20eff3e4da4
2020-01-17 20:40:44 +09:00
Zuul
2af9e9608d Merge "Use Python 3 by default to run storlet applications" 2020-01-17 10:04:38 +00:00
Zuul
417cf25e53 Merge "Py3: Encode log messages in StorletLogger" 2020-01-16 23:12:46 +00:00
Takashi Kajinami
62cf0e55c7 Use https://repo.maven.apache.org/maven2/ to retrieve jar files
The Central Repository has migrated to new HTTPS version[1].
Update urls to retrive jar packages accordingly.

[1] https://support.sonatype.com/hc/en-us/articles/360041287334

Change-Id: Iba582c226f91456e95bae1aaf2a7b388f6e1388e
2020-01-16 23:19:51 +09:00
Takashi Kajinami
55d9c28253 Use Python 3 by default to run storlet applications
Use Python 3 to run storlet application when runtime version is not
specified.
Also fix py3 compatibility for some python storlet applications used
for functional tests.

Change-Id: Ic33bb339339e9ff446fd308663c692db834f2a81
2020-01-07 10:12:50 +09:00
Takashi Kajinami
fe165a2047 Py3: Encode log messages in StorletLogger
Change-Id: I936935d405da7f86e7828e528d0bfae76e50df00
2020-01-06 23:07:53 +09:00
Takashi Kajinami
f5815f065a Use the same python version to run storlet agents
This patch refactors package installations in storlets devstack
plugin, so that we can run storlets agents with the same python
version as the one we have in host to run storlets middleware.

This patch also bumps up openjdk version from 8 to 11, and python3
version from 3.5 to 3.6, so that we use the latest versin provided
in distro.

Change-Id: I6cb87976ff4fa6caaa58cf555685f24d95f214a8
5.0.0.0b1
2019-12-23 12:44:54 +09:00
Zuul
88b0ae777c Merge "Do not bind python2 specific directory into container" 2019-12-18 11:30:57 +00:00
Takashi Kajinami
7ab7b1029f Do not bind python2 specific directory into container
Use more generic path to place storlets python codes, so that
we can use the same path for python2 and python3.

Change-Id: Ia4ee0790ea58a42173f58bed2f59249c0b7edfd9
2019-12-17 21:22:16 +09:00
Zuul
2c8c270242 Merge "Change storlets-functional-py3 as voting gate" 2019-11-29 23:13:03 +00:00
Zuul
9d9cbb6068 Merge "PY3: Fix ipython extension as py3 compatible" 2019-11-29 23:13:02 +00:00
Zuul
d4d8ea5c73 Merge "Remove unused PUT path in test_half_storlet" 2019-11-29 08:46:06 +00:00
Kota Tsuyuzaki
148c626f2a Change storlets-functional-py3 as voting gate
Since we've passed all py3 functional gate, let's remove
vating: False flag from .zuul.yaml

Change-Id: I939e817e29085b29e9b993af0b502c9020928f74
2019-11-28 14:19:30 +09:00