2838 Commits

Author SHA1 Message Date
shenjiatong
26e8bb4cc9 Catch ConnectionForced Exception
Occasionally I saw large number of connectionforced exceptions.
such exceptions cannot be healed automatically and requires a reboot.
catch it and ensure connection established.

Change-Id: I5c468d10230f05438df780e045f1e92662edf070
Closes-Bug: #1883038
2020-07-19 07:41:17 +08:00
Zuul
f988751c5c Merge "Drop a python 2 exception management" 2020-07-17 16:28:14 +00:00
Stephen Finucane
afb035d971 tests: Resolves issues with kombu > 4.6.8
The 'kombu.connection.Connection.ensure_connection' method has changed
from calling 'retry_over_time' on 'self.connect' to calling it on
'self._connection_factory' [1], meaning our mocks are outdated. Address
this change.

[1] https://github.com/celery/kombu/pull/1193/commits/398aa5b8cd1fe1fc

Change-Id: Ibbcf21a57ab1e3f90c21901296e5c088b645127c
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Closes-Bug: #1885923
2020-07-02 10:37:09 +01:00
Andreas Jaeger
7e406c312a Simplify tools/test-setup.sh
test-setup.sh is called after bindep is run in CI, and tox is not setup.

Simplify the script so that it only does what's needed - install
qdrouterd for Ubuntu from qpid/testing.

Install the other packages using bindep, add test to amqp1 profile.

Add amqp1 to bindep_profiles where needed for scenario tests, previously
test-setup.sh added it everywhere.

Some tests are skipped due to changes in recent versions of kombu. These
will be unskipped when we come up with a proper fix.

Change-Id: Ic3a9e2c873619670edfbf71022d593f3cb5f70f2
Related-Bug: #1885923
2020-07-01 14:40:24 +01:00
Zuul
30a4a3d9bd Merge "Align contributing doc with oslo's policy" 2020-06-30 16:08:02 +00:00
Hervé Beraud
661bdd7f41 Drop a python 2 exception management
Previously (when python 2 was supported) we introduced some specific
exception management to detect connections issues when a rabbitmq
cluster node disappearing.

The original issue was that a ConnectionRefusedError is thrown and not
managed by oslo.messaging to detect heartbeat issue and call ensure_connection
for switching the connection destination (cluster node).

`ConnectionRefusedError` is only a python 3 exception [2].

Now that we only support python 3 we don't need to continue to
wrap/emulate this kind of exceptions (ConnectionRefusedError) [2] and
so we can drop the python 2 compatibility code to only support python 3 code.

[1] 9d8b1430e5
[2] https://docs.python.org/3/library/exceptions.html#ConnectionRefusedError

Change-Id: I4c459d8c947dac213a1866c0d37e8f3d547aa82e
2020-06-18 13:25:24 +02:00
melissaml
c2074e4760 Fix pygments style
New theme of docs respects pygments_style.

more info: http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I287fd718f9283e5d7d6951b9e55d5a6b48e07e3c
2020-06-04 15:27:58 +08:00
Stephen Finucane
8d29ab3d2c bindep: Add 'librdkafka-dev' dependency
We're seeing the following in our build logs:

    In file included from confluent_kafka/src/confluent_kafka.c:17:0:
    confluent_kafka/src/confluent_kafka.h:22:10: fatal error: librdkafka/rdkafka.h: No such file or directory
     #include <librdkafka/rdkafka.h>
              ^~~~~~~~~~~~~~~~~~~~~~
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Resolve this by installing the development headers.

Change-Id: Idda79dc87bcd0e3367a6abd1b52104c000ad1dcd
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2020-06-02 16:07:00 +00:00
Zuul
4fc9e70aa1 Merge "Print warning message when connection running out" 2020-05-29 11:38:38 +00:00
Zuul
03f0d63bdf Merge "Fix hacking min version to 3.0.1" 2020-05-26 15:57:59 +00:00
Zuul
fe0698e614 Merge "Remove the unused coding style modules" 2020-05-26 13:50:47 +00:00
zhangboye
28d0f432ef Fix hacking min version to 3.0.1
Change-Id: I3282e4962e92bdd5a401c5b1c61fced68e8d0c8a
2020-05-22 10:20:59 +08:00
Andreas Jaeger
9cc3f52ac3 Switch to newer openstackdocstheme and reno versions
Switch to openstackdocstheme 2.2.0 and reno 3.1.0 versions. Using
these versions will allow especially:
* Linking from HTML to PDF document
* Allow parallel building of documents
* Fix some rendering

Update Sphinx version as well.

Remove docs requirements from lower-constraints, they are not needed
during install or test but only for docs building.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

Depends-On: https://review.opendev.org/728938
Change-Id: I70c7edf8b95cde890e6263195be1de6bb826e700
2020-05-18 20:55:51 +02:00
jacky06
67defab90e Remove the unused coding style modules
Python modules related to coding style checks (listed in blacklist.txt in
openstack/requirements repo) are dropped from lower-constraints.txt
as they are not actually used in tests (other than pep8).

more info: https://github.com/openstack/requirements/blob/master/blacklist.txt

Change-Id: Iac09dcd4c97950717c72f2f6a669b2f11e5f6ee6
2020-05-15 17:16:12 +00:00
shenjiatong
96300a3221 Print warning message when connection running out
I believe when connection pool running out of available
resources, there will be all kinds of weird consequences
following. So let's print warning logs when we are going
to waiting indefinitely for the lock

Change-Id: I5ca930cada523bc77c644703a0f02b9160816231
Related-bug: #1871813
2020-05-13 04:10:11 +00:00
Andreas Jaeger
e44c988306 Remove six usage
Remove six, the python 2/3 compatibility library. It's not needed
anymore since the repo is python3 only.

Remove a now unneeded hacking test.

Change-Id: I40522c4accb4aaf8115d11fee8b081e2d991cb4d
2020-05-11 10:21:58 +02:00
Sean McGinnis
c1768401f7
Remove monotonic usage
The monotonic package was needed for monotonic time operations when
running under Python runtimes older than 3.3. Since we now only support
versions higher than this, this third party package requirement can now
be removed.

Change-Id: I598530b3f417964ff697b48e681b135bd119ae81
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-05-04 17:02:44 -05:00
Hervé Beraud
8a6f84a529 Align contributing doc with oslo's policy
Related to:
- https://review.opendev.org/#/c/723044/
- https://governance.openstack.org/tc/goals/selected/ussuri/project-ptl-and-contrib-docs.html

Change-Id: I006d5ca0f9ae492dcd53b1d8799ac3e375bf9127
2020-05-04 18:25:31 +02:00
Zuul
c7272ed465 Merge "Bump default tox env from py37 to py38" 2020-04-29 13:35:00 +00:00
Sean McGinnis
a7b9c70f5c
Bump default tox env from py37 to py38
Python 3.8 is now our highest level supported python runtime.
This updates the default tox target environments to swap out
py37 for py38 to make sure local development testing is
covering this version.

This does not impact zuul jobs in any way, nor prevent local
tests against py37. It just changes the default if none is
explicitly provided.

Change-Id: I1992bad2ee6bffade2f937cc0d8dc6ae800f4159
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-24 10:25:49 -05:00
Sean McGinnis
243c6a7eda
Add py38 package metadata
Now that we are running the Victoria tests that include a
voting py38, we can now add the Python 3.8 metadata to the
package information to reflect that support.

Change-Id: Ibe733cb482501544644133fed494f672ef566b7a
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2020-04-24 08:23:12 -05:00
Zuul
0bcc443af7 Merge "Add release notes links to doc index" 2020-04-23 14:38:42 +00:00
Hervé Beraud
7f4dafc327 Add release notes links to doc index
Change-Id: I93ae9330509b89629d8c360437065267db637f49
2020-04-16 15:30:22 +02:00
OpenStack Proposal Bot
d011c7f262 Imported Translations from Zanata
For more information about this automatic import see:
https://docs.openstack.org/i18n/latest/reviewing-translation-import.html

Change-Id: I792154ba794512046b265718c2f01db624413f30
2020-04-16 08:13:17 +00:00
Zuul
cf23a0d37a Merge "Fix some typos" 2020-04-15 22:02:17 +00:00
9a66fe2abd Add Python3 victoria unit tests
This is an automatically generated patch to ensure unit testing
is in place for all the of the tested runtimes for victoria.

See also the PTI in governance [1].

[1]: https://governance.openstack.org/tc/reference/project-testing-interface.html

Change-Id: I76094aecc710a44f43b6d4052851b87a0f596932
2020-04-14 10:15:32 +00:00
434ec937f9 Update master for stable/ussuri
Add file to the reno documentation build to show release notes for
stable/ussuri.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.

Change-Id: I74595d2712ed45ce5b9336e17152f50caab838a0
Sem-Ver: feature
2020-04-14 10:15:30 +00:00
ushen
2c2779a974 Fix some typos
Change-Id: Ic0a311d290682a923ec4f8cbe36f875d560cd41a
2020-04-05 20:55:04 +08:00
Andreas Jaeger
7f638bb493 Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Update local hacking check for new flake8 version.

Blacklist:
W504 line break after binary operator

Fix:
E741 ambiguous variable name
E117 over-indented
E305 expected 2 blank lines after class or function definition, found 1
F841 local variable 'e' is assigned to but never used
W605 invalid escape sequence '\.'

Change-Id: I99d574ca6569f1f177d2c5ce1011f269f4343619
2020-03-30 13:49:29 +00:00
Zuul
d87618b24f Merge "Setup backend scenarios for functional tests" 2020-02-21 15:52:26 +00:00
Hervé Beraud
fed48aea44 Remove the deprecated blocking executor
The blocking executor has been deprecated in Pike and marked for removal
in Rocky, but some user like Mistral asked us to wait before. We decided
to remove this executor for Train or next cycle, now we are in the
Ussuri and after some researchs on usage I think we can go ahead.

This patch drop the deprecation warnings, related unit tests and
set the server with the threading executor is the default executor.

Change-Id: If07bab61ee2b148658b88be98b12f8539f274efe
Closes-Bug: #1715141
2020-02-18 15:49:27 +01:00
Hervé Beraud
3359c520d3 remove outdated header
Change-Id: I1ac92354fea2a70e5fe72a0e3b48d4c435b5286d
2020-02-10 18:33:15 +01:00
Hervé Beraud
b55da38368 reword releasenote for py27 support dropping
Change-Id: Ic65842371eac32cec3a7b66cb6aa0f3ef52b8637
2020-02-06 11:03:41 +01:00
Andy Smith
ab78c8e3dd Setup backend scenarios for functional tests
This patchset introduces scenarios for the functional tests to unify the
setup and configuration of alternate messaging backends for RPC and
Notifications. The scenarios are defined to reduce redundancy in
the testing of backends and to demonstrate functional correctness
across driver combinations.

Current driver support:
rabbit - RPC, Notify
amqp   - RPC
kafka  - Notify

                  RPC       Notify
               ---------  ----------
scenario01       rabbit     rabbit
scenario02       rabbit     kafka
scenario03        amqp      rabbit
scenario04        amqp      kafka

It is anticipated that additional scenarios will be defined as new
drivers are introduced and/or new messaging intermediarites are supported.

Note: The removal of python2 jobs are also included patch

Change-Id: I0f86416623a0b718516147f0660b4df2b74cf867
2020-02-05 08:00:56 -05:00
Hervé Beraud
32a1b6e948 [ussuri][goal] Drop python 2.7 support and testing
OpenStack is dropping the py2.7 support in Ussuri cycle.

Complete discussion & schedule can be found in
-
http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010142.html
- https://etherpad.openstack.org/p/drop-python2-support

Ussuri Communtiy-wide goal:
https://governance.openstack.org/tc/goals/selected/ussuri/drop-py27.html

Change-Id: I15cc79159b603e232e442e202c78d6c56dc73252
2020-02-03 18:35:08 +01:00
Zuul
3ad0a72cdd Merge "Add support for kafka SSL autentication" 2020-01-30 13:14:32 +00:00
Oleg Bondarev
d7eb89eac8 Don't log NoSuchMethod for special non-existing methods
Such non-existing methods are used as health probes.
Please see bug for details.

Change-Id: I8f6b4a669ec11721f76cf03abcd7d802c3c11eb5
Closes-Bug: #1857319
2020-01-23 15:37:31 +04:00
Michal Arbet
5a43d4548a Add support for kafka SSL autentication
Change-Id: Idef066a2e3b4923789a6b081d5442e931aba4507
2020-01-16 23:26:53 +01:00
Zuul
04b2b5d451 Merge "Adding debug logs on AMQPListener poll" 2020-01-15 13:15:32 +00:00
Hervé Beraud
e4a8900aa7 Adding debug logs on AMQPListener poll
Adding some debug logs to track execution on the AMQPListener poll
function to observe if workers incoming messages are dispatched to
workers and if listener is stuck.

Change-Id: Ia9b9ef62b0b59c78283c6a8cc5b7ce42282e73b5
2020-01-14 11:57:10 +00:00
Hervé Beraud
cc6cf12c3d tox: Trivial cleanup
Move 'basepython' to the top-level 'testenv'.

Change-Id: I13e72ec6cbd006ca6eccfe21f2a0584e86a52669
2020-01-08 19:59:34 +01:00
Zuul
41a4423043 Merge "Ignore releasenote cache within git untracked files" 2019-12-19 19:19:22 +00:00
Zuul
4fc0e6241b Merge "Remove telemetry checks" 2019-12-19 12:40:28 +00:00
Zuul
4e49a286e7 Merge "Do not use threading.Event" 2019-12-18 23:58:11 +00:00
Hervé Beraud
2a4cea9e85 Ignore releasenote cache within git untracked files
Change-Id: If616ac023f578f073d9ae8596bc8ce870096ef0b
2019-12-18 17:03:10 +01:00
Zuul
524f72261c Merge "Removed unused variable pools" 2019-12-18 15:33:35 +00:00
John Eckersberg
d873c0d8f5 Do not use threading.Event
Waiting on a threading.Event with eventlet can cause busy looping via
epoll_wait, see related bug for more details.

Change-Id: I007613058a2d21d1712c02fa6d1602b63705c1ab
Related-bug: #1518430
2019-12-18 13:11:41 +00:00
Chenjun Shen
2b0613b208 Removed unused variable pools
Change-Id: I8f3a22d2af3f34b368bae3465bf8433fe2d3e22b
2019-12-18 11:06:17 +00:00
Kenneth Giusti
859e0d4eaa Revert "Add RPC incoming and reply log"
This broke non-rabbitmq message bus drivers. See:

https://bugs.launchpad.net/oslo.messaging/+bug/1855775

This reverts commit b104f254ab43d5e2589e845eff84b1bd096cb42d.

Change-Id: I17a448a768b544482b375b0076889db989e03e8c
2019-12-09 16:22:06 -05:00
Andy Smith
1057cfc60f Remove telemetry checks
The telemetry tests are not actively maintained and have been failing
checks for quite some time. Remove the non-voting tests for now and
plan to introduce new tests in the future for better coverage of
notifications.

Change-Id: I6c30c28ee4c8be4677ccecf8822af15b060cf804
2019-12-02 07:54:34 -05:00