openstack/swift
Code Issues Proposed changes

Drop py2 support

Browse Source 
* Remove py2 gate jobs.
* Build non-universal, py3-only wheels.
* Specify minimum python version in package metadata.
* Clean up requirements/constraints/bindep (a little, anyway).

Change-Id: I53153c4fde043e964e1daa7bbf2089e0471dede2
This commit is contained in:
Tim Burke 2022-08-17 23:09:27 -07:00
parent 3944630007
commit 7367907c58
14 changed files with 46 additions and 266 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

87
.zuul.yaml
View File

@ -12,22 +12,6 @@
tox_environment:
TMPDIR: '{{ ansible_env.HOME }}/xfstmp'
- job:
name: swift-tox-py27
parent: swift-tox-base
nodeset: ubuntu-bionic
description: |
Run unit-tests for swift under cPython version 2.7.
Uses tox with the ``py27`` environment.
It sets TMPDIR to an XFS mount point created via
tools/test-setup.sh.
vars:
tox_envlist: py27
bindep_profile: test py27
python_version: 2.7
post-run: tools/playbooks/common/cover-post.yaml
- job:
name: swift-tox-py36
parent: swift-tox-base
@ -157,21 +141,6 @@
python_use_pyenv: True
python_version: '3.12'
- job:
name: swift-tox-func-py27
parent: swift-tox-base
nodeset: ubuntu-bionic
description: |
Run functional tests for swift under cPython version 2.7.
Uses tox with the ``func`` environment.
It sets TMPDIR to an XFS mount point created via
tools/test-setup.sh.
vars:
tox_envlist: func
bindep_profile: test py27
python_version: 2.7
- job:
name: swift-tox-func-py312
parent: swift-tox-base
@ -183,7 +152,7 @@
It sets TMPDIR to an XFS mount point created via
tools/test-setup.sh.
vars:
tox_envlist: func-py3
tox_envlist: func
bindep_profile: test py312
python_version: 3.12
@ -199,13 +168,13 @@
name: swift-tox-func-encryption-py39-centos-9-stream
parent: swift-tox-func-py39-centos-9-stream
vars:
tox_envlist: func-encryption-py3
tox_envlist: func-encryption
- job:
name: swift-tox-func-ec-py39-centos-9-stream
parent: swift-tox-func-py39-centos-9-stream
vars:
tox_envlist: func-ec-py3
tox_envlist: func-ec
- job:
name: swift-tox-func-encryption-py312
@ -217,7 +186,7 @@
It sets TMPDIR to an XFS mount point created via
tools/test-setup.sh.
vars:
tox_envlist: func-encryption-py3
tox_envlist: func-encryption
- job:
name: swift-tox-func-encryption-py312-arm64
@ -254,30 +223,6 @@
Uses tox with the ``func-ec-py3`` environment.
It sets TMPDIR to an XFS mount point created via
tools/test-setup.sh.
vars:
tox_envlist: func-ec-py3
- job:
name: swift-tox-func-encryption-py27
parent: swift-tox-func-py27
description: |
Run functional tests for swift under cPython version 2.7.
Uses tox with the ``func-encryption`` environment.
It sets TMPDIR to an XFS mount point created via
tools/test-setup.sh.
vars:
tox_envlist: func-encryption
- job:
name: swift-tox-func-ec-py27
parent: swift-tox-func-py27
description: |
Run functional tests for swift under cPython version 2.7.
Uses tox with the ``func-ec`` environment.
It sets TMPDIR to an XFS mount point created via
tools/test-setup.sh.
vars:
tox_envlist: func-ec
@ -294,7 +239,7 @@
vars:
tox_constraints_file: '{{ ansible_user_dir }}/src/opendev.org/openstack/requirements/upper-constraints.txt'
# This tox env get run twice; once for Keystone and once for tempauth
tox_envlist: func-py3
tox_envlist: func
devstack_localrc:
SWIFT_HASH: changeme
# We don't need multiple replicas to run purely functional tests.
@ -515,10 +460,10 @@
name: swift-tox-lower-constraints
parent: openstack-tox-lower-constraints
# This seems defensible for a l-c job
nodeset: ubuntu-jammy
nodeset: ubuntu-bionic
vars:
bindep_profile: test py27
python_version: 2.7
bindep_profile: test py36
python_version: 3.6
tox_environment:
TMPDIR: '{{ ansible_env.HOME }}/xfstmp'
@ -655,26 +600,16 @@
irrelevant-files: *docker-irrelevant-files
# Unit tests
- swift-tox-py27:
- swift-tox-py36:
irrelevant-files: &unittest-irrelevant-files
- ^(api-ref|doc|releasenotes)/.*$
- ^test/(cors|functional|probe|s3api)/.*$
- swift-tox-py36:
irrelevant-files: *unittest-irrelevant-files
- swift-tox-py39:
irrelevant-files: *unittest-irrelevant-files
- swift-tox-py312:
irrelevant-files: *unittest-irrelevant-files
# Functional tests
- swift-tox-func-py27:
irrelevant-files: *functest-irrelevant-files
- swift-tox-func-encryption-py27:
irrelevant-files: *functest-irrelevant-files
- swift-tox-func-ec-py27:
irrelevant-files: *functest-irrelevant-files
# py3 functional tests
- swift-tox-func-py312:
irrelevant-files: *functest-irrelevant-files
- swift-tox-func-encryption-py312:
@ -750,13 +685,9 @@
irrelevant-files: *docker-irrelevant-files
- swift-upload-image-py3:
irrelevant-files: *docker-irrelevant-files
- swift-tox-py27
- swift-tox-py36
- swift-tox-py39
- swift-tox-py312
- swift-tox-func-py27
- swift-tox-func-encryption-py27
- swift-tox-func-ec-py27
- swift-tox-func-py312
- swift-tox-func-encryption-py312
- swift-tox-func-ec-py312

2
CONTRIBUTING.rst
View File

@ -156,7 +156,7 @@ run tox and then point your browser to swift/cover/index.html:
.. code-block:: console
tox -e py27 -- test.unit.common.middleware.test_healthcheck:TestHealthCheck.test_healthcheck
tox -e py3 -- test.unit.common.middleware.test_healthcheck:TestHealthCheck.test_healthcheck
Swift's unit tests are designed to test small parts of the code in
isolation. The functional tests validate that the entire system is

10
bindep.txt
View File

@ -17,17 +17,13 @@ libxslt-devel [platform:rpm]
libxslt1-dev [platform:dpkg]
libxslt-dev [platform:apk]
memcached
python-dev [py27 platform:dpkg platform:apk !platform:ubuntu-jammy]
python-devel [py27 platform:rpm]
python3-dev [platform:dpkg platform:apk test]
python3-devel [platform:rpm !py27 test]
python3-devel [platform:rpm test]
# python3-devel does not pull in the python3 package on openSUSE so
# we need to be explicit. The python3 package contains the XML module
# which is required by a python3 virtualenv. Similarly, in python2,
# the XML module is located in python-xml which is not pulled in
# by python-devel as well. See https://bugzilla.suse.com/show_bug.cgi?id=1046990
# which is required by a python3 virtualenv.
# See https://bugzilla.suse.com/show_bug.cgi?id=1046990
python3 [platform:suse platform:apk test]
python-xml [platform:suse]
rsync
xfsprogs
libssl-dev [platform:dpkg]

2
doc/requirements.txt
View File

@ -6,5 +6,5 @@ sphinx>=2.0.0,!=2.1.0 # BSD
openstackdocstheme>=2.2.1 # Apache-2.0
reno>=3.1.0 # Apache-2.0
os-api-ref>=1.0.0 # Apache-2.0
python-keystoneclient!=2.1.0,>=2.0.0 # Apache-2.0
python-keystoneclient>=3.19.0 # Apache-2.0
sphinxcontrib-svg2pdfconverter>=0.1.0 # BSD

10
doc/source/development_guidelines.rst
View File

@ -81,18 +81,18 @@ To run a selected subset of unit tests with ``pytest``:
* By default ``tox`` will run **all of the unit test** and pep8 checks listed in
the ``tox.ini`` file ``envlist`` option. A subset of the test environments
can be specified on the ``tox`` command line or by setting the ``TOXENV``
environment variable. For example, to run only the pep8 checks and python2.7
environment variable. For example, to run only the pep8 checks and python3
unit tests use::
tox -e pep8,py27
tox -e pep8,py3
or::
TOXENV=py27,pep8 tox
TOXENV=py3,pep8 tox
To run unit tests with python3.8::
To run unit tests with python3.12 specifically::
tox -e py38
tox -e py312
.. note::
As of ``tox`` version 2.0.0, most environment variables are not automatically

2
doc/source/getting_started.rst
View File

@ -11,7 +11,7 @@ most Linux platforms.
Swift is written in Python and has these dependencies:
* Python (2.7 or 3.6-3.10)
* Python (3.6-3.12)
* rsync 3.x
* `liberasurecode <https://opendev.org/openstack/liberasurecode/>`__
* The Python packages listed in `the requirements file <https://github.com/openstack/swift/blob/master/requirements.txt>`__

1
docker/install_scripts/99_apk_uninstall_dev.sh
View File

@ -13,6 +13,5 @@ apk del g++
apk del libffi-dev
apk del libxslt-dev
apk del libxml2-dev
apk del python2-dev || :
apk del python3-dev
rm -rf /var/cache/apk/*

15
lower-constraints.txt
View File

@ -12,11 +12,11 @@ cffi==1.11.5
chardet==3.0.4
cliff==2.11.0
cmd2==0.8.1
coverage==5.0.4
coverage==5.2.1
cryptography==2.0.2
debtcollector==1.19.0
dnspython==1.15.0
docutils==0.11
docutils==0.15
dulwich==0.19.0
enum-compat==0.0.2
eventlet==0.25.0
@ -25,11 +25,10 @@ fixtures==3.0.0
future==0.16.0
gitdb2==2.0.3
GitPython==2.1.8
greenlet==0.3.2
greenlet==0.3.3
idna==2.6
imagesize==1.0.0
iso8601==0.1.12
ipaddress==1.0.16
Jinja2==2.10
keystoneauth1==3.4.0
keystonemiddleware==4.17.0
@ -40,8 +39,8 @@ mock==3.0
monotonic==1.4
msgpack==0.5.6
netaddr==0.7.19
netifaces==0.8
oslo.config==4.0.0
netifaces==0.10.4
oslo.config==5.2.0
oslo.i18n==3.20.0
oslo.log==3.22.0
oslo.serialization==2.25.0
@ -57,7 +56,7 @@ pyparsing==2.2.0
pyperclip==1.6.0
pytest==4.6.11
pytest-cov==2.12.1
python-keystoneclient==2.0.0
python-keystoneclient==3.19.0
python-mimeparse==1.6.0
python-subunit==1.2.0
python-swiftclient==3.2.0
@ -77,6 +76,6 @@ unittest2==1.1.0
urllib3==1.22
voluptuous==0.11.1
wrapt==1.10.11
xattr==0.4
xattr==0.7.2
pycadf===2.10.0
python-dateutil===2.4.2

85
py2-constraints.txt
View File

@ -1,85 +0,0 @@
voluptuous===0.11.7
chardet===3.0.4
enum-compat===0.0.3
os-api-ref===1.6.2
alabaster===0.7.12
pbr===5.4.5
oslo.i18n===3.25.1
fixtures===3.0.0
sphinxcontrib-websupport===1.1.2
ipaddress===1.0.23
debtcollector===1.22.0
MarkupSafe===1.1.1
netaddr===0.7.19
prettytable===0.7.2
traceback2===1.4.0
eventlet===0.25.2
extras===1.0.0
reno===2.11.3
imagesize===1.2.0
mock===3.0.5
cryptography===2.9
requests-mock===1.7.0
unittest2===1.1.0
Pygments===2.5.2
snowballstemmer===2.0.0
Jinja2===2.11.1
cliff===2.18.0
castellan===1.4.0
coverage===5.0.4
oslo.log===3.45.2
docutils===0.15.2
boto3===1.12.39
stestr===2.6.0
oslo.serialization===2.29.2
testtools===2.4.0
keystonemiddleware===9.0.0
iso8601===0.1.12
linecache2===1.0.0
idna===2.9
msgpack===0.6.2
Sphinx===1.8.5
oslo.config===7.0.0
openstackdocstheme===1.31.2
stevedore===1.32.0
botocore===1.15.39
cmd2===0.8.9
xattr===0.9.7
six===1.14.0
dulwich===0.19.15
GitPython===2.1.11
wrapt===1.12.1
rfc3986===1.4.0
future===0.18.2
boto===2.49.0
monotonic===1.5
netifaces===0.10.9
keystoneauth1===4.0.0
cffi===1.14.0
Babel===2.8.0
greenlet===0.4.15
oslo.utils===3.42.1
gitdb===0.6.4
gitdb2===2.0.6
pathlib2==2.3.6
# python-dateutil seems likely to bring back support, but 2.9.0 is known-broken
# see https://github.com/dateutil/dateutil/issues/1344
python-dateutil===2.8.2
# Projects that are known to have had a final py2-supporting release
bandit===1.6.2
python-keystoneclient===3.22.0
dnspython===1.16.0
setuptools===44.1.1
pycadf===2.10.0
smmap===3.0.5
PasteDeploy==2.1.1
pytest==4.6.11
pytest-cov==2.12.1
attrs==21.4.0
lxml===5.0.2
certifi===2020.4.5.1
PyYAML===5.4.1
requests===2.27.1
urllib3===1.26.19

14
requirements.txt
View File

@ -3,20 +3,12 @@
# process, which may cause wedges in the gate later.
eventlet>=0.25.0,!=0.34.3 # MIT
greenlet>=0.3.2
greenlet>=0.3.3
PasteDeploy>=2.0.0
lxml>=4.2.3
requests>=2.14.2 # Apache-2.0
six>=1.10.0
xattr>=0.4;sys_platform!='win32' # MIT
xattr>=0.7.2;sys_platform!='win32' # MIT
PyECLib>=1.3.1,!=1.6.2,!=1.6.3 # BSD
cryptography>=2.0.2 # BSD/Apache-2.0
# For python 2.7, the following requirements are needed; they are not
# included since the requirments-check check will fail otherwise since
# global requirements do not support these anymore.
# Fortunately, these packages come in as dependencies from others and
# thus the py27 jobs still work.
#
# dnspython>=1.15.0;python_version=='2.7' # http://www.dnspython.org/LICENSE
# ipaddress>=1.0.16;python_version<'3.3' # PSF
dnspython>=1.15.0 # http://www.dnspython.org/LICENSE

7
setup.cfg
View File

@ -7,6 +7,7 @@ long_description_content_type = text/x-rst
author = OpenStack
author_email = openstack-discuss@lists.openstack.org
url = https://docs.openstack.org/swift/latest/
python_requires = >=3.6
project_urls =
Documentation = https://docs.openstack.org/swift/latest/
Bug Tracker = https://bugs.launchpad.net/swift
@ -20,8 +21,6 @@ classifier =
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 2.7
Programming Language :: Python :: 3
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
@ -42,7 +41,7 @@ packages =
[extras]
kms_keymaster =
oslo.config>=4.0.0,!=4.3.0,!=4.4.0 # Apache-2.0
oslo.config>=5.2.0 # Apache-2.0
castellan>=0.13.0 # Apache-2.0
kmip_keymaster =
@ -148,5 +147,3 @@ swift.diskfile =
swift.object_audit_watcher =
dark_data = swift.obj.watchers.dark_data:DarkDataWatcher
[bdist_wheel]
universal = 1

11
setup.py
View File

@ -1,4 +1,4 @@
#!/usr/bin/env python
#!/usr/bin/env python3
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
@ -14,17 +14,8 @@
# See the License for the specific language governing permissions and
# limitations under the License.
# THIS FILE IS MANAGED BY THE GLOBAL REQUIREMENTS REPO - DO NOT EDIT
import setuptools
# In python < 2.7.4, a lazy loading of package `pbr` will break
# setuptools if some other modules registered functions in `atexit`.
# solution from: http://bugs.python.org/issue15881#msg170215
try:
import multiprocessing # noqa
except ImportError:
pass
setuptools.setup(
setup_requires=['pbr'],
pbr=True)

6
test-requirements.txt
View File

@ -4,13 +4,13 @@
# Hacking already pins down pep8, pyflakes and flake8
hacking>=2.0,<7.1.0 # Apache-2.0
coverage>=5.0.4 # Apache-2.0
coverage>=5.2.1 # Apache-2.0
pytest>=4.6.11 # MIT
pytest-cov>=2.12.1 # MIT
stestr>=2.0.0 # Apache-2.0
mock>=3.0 # BSD
python-swiftclient>=3.2.0
python-keystoneclient!=2.1.0,>=2.0.0 # Apache-2.0
python-keystoneclient>=3.19.0 # Apache-2.0
boto>=2.32.1
boto3>=1.9
botocore>=1.12
@ -20,4 +20,4 @@ keystonemiddleware>=4.17.0 # Apache-2.0
# Security checks
bandit>=1.1.0 # Apache-2.0
docutils>=0.11 # OSI-Approved Open Source, Public Domain
docutils>=0.15 # OSI-Approved Open Source, Public Domain

60
tox.ini
View File

@ -1,5 +1,5 @@
[tox]
envlist = py37,py27,pep8
envlist = py3,pep8
minversion = 3.18.0
requires =
# required to support py27/py36 envs
@ -36,12 +36,6 @@ passenv = SWIFT_*
skipsdist = True
skip_install = True
[testenv:py27]
deps =
-c{toxinidir}/py2-constraints.txt
-r{toxinidir}/requirements.txt
-r{toxinidir}/test-requirements.txt
[testenv:py36]
deps =
-c{toxinidir}/py3-constraints.txt
@ -66,58 +60,30 @@ commands =
bandit -c bandit.yaml -r swift -n 5
./.manpages {posargs}
[testenv:func-py3]
basepython = python3
allowlist_externals =
./.functests
commands = ./.functests {posargs}
[testenv:func-ec-py3]
basepython = python3
allowlist_externals =
./.functests
commands = ./.functests {posargs}
setenv = SWIFT_TEST_IN_PROCESS=1
SWIFT_TEST_IN_PROCESS_CONF_LOADER=ec
[testenv:func-encryption-py3]
basepython = python3
allowlist_externals =
./.functests
commands = ./.functests {posargs}
setenv = SWIFT_TEST_IN_PROCESS=1
SWIFT_TEST_IN_PROCESS_CONF_LOADER=encryption
[testenv:func]
basepython = python2.7
deps = {[testenv:py27]deps}
allowlist_externals =
./.functests
commands = ./.functests {posargs}
[testenv:func-encryption]
basepython = python2.7
deps = {[testenv:py27]deps}
allowlist_externals =
./.functests
commands = ./.functests {posargs}
setenv = SWIFT_TEST_IN_PROCESS=1
SWIFT_TEST_IN_PROCESS_CONF_LOADER=encryption
[testenv:func-ec]
basepython = python2.7
deps = {[testenv:py27]deps}
allowlist_externals =
./.functests
commands = ./.functests {posargs}
setenv = SWIFT_TEST_IN_PROCESS=1
SWIFT_TEST_IN_PROCESS_CONF_LOADER=ec
[testenv:func-encryption]
deps = {[testenv:py36]deps}
allowlist_externals =
./.functests
commands = ./.functests {posargs}
setenv = SWIFT_TEST_IN_PROCESS=1
SWIFT_TEST_IN_PROCESS_CONF_LOADER=encryption
[testenv:venv]
commands = {posargs}
[testenv:docs]
basepython = python3
deps =
-c{env:TOX_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master}
-r{toxinidir}/doc/requirements.txt
@ -126,7 +92,6 @@ commands = sphinx-build -W -b html doc/source doc/build/html
[testenv:api-ref]
# This environment is called from CI scripts to test and publish
# the API Ref to docs.openstack.org.
basepython = python3
deps = {[testenv:docs]deps}
allowlist_externals =
rm
@ -168,22 +133,17 @@ deps = bindep
commands = bindep test doc
[testenv:releasenotes]
basepython = python3
deps = {[testenv:docs]deps}
commands = sphinx-build -a -W -E -d releasenotes/build/doctrees -b html releasenotes/source releasenotes/build/html
[testenv:lower-constraints]
# FIXME(dhellmann): We'll want to uncomment this
# when the full test suite works with python 3.
# basepython = python3
basepython = python2.7
deps =
-c{toxinidir}/lower-constraints.txt
-r{toxinidir}/test-requirements.txt
-r{toxinidir}/requirements.txt
basepython = python3.6
[testenv:pdf-docs]
basepython = python3
deps = {[testenv:docs]deps}
allowlist_externals =
make