2552 Commits

Author SHA1 Message Date
pengyuesheng
33dc529c37 Bump the openstackdocstheme extension to 1.20
Some options are now automatically configured by the version 1.20:
- project
- html_last_updated_fmt
- latex_engine
- latex_elements
- version
- release.

Change-Id: Ib5e22f6a5374f05e576bbc00a209209fdb09acad
2019-07-19 11:24:08 +08:00
Zuul
6cdd4cb007 Merge "Implement the transport options" 2019-06-26 13:09:53 +00:00
Gabriele
e804874c50
Implement the transport options
With this feature, it is possible to specialize the parameters to send.
`options = oslo_messaging.TransportOptions(at_least_once=True)`

TransportOptions is used in every single driver,
for example in RabbitMQ driver is used to handle the mandatory flag.

Notes:
  - The idea of creating a new  class TransportOptions is because I'd like
    to have an abstract class not related only to the RPCClient
  - at_least_once is the first parameter, when needed we can add the
    others.

Implements: blueprint transport-options (second point)

The blueprint link is [1]
To test it you can use [2]

1- https://blueprints.launchpad.net/oslo.messaging/+spec/transport-options
2- https://github.com/Gsantomaggio/rabbitmq-utils/
   tree/master/openstack/mandatory_test

Change-Id: I1858e4a990507d3c2bac2ef7fbef75d8c2dbfce2
2019-06-24 16:50:35 +02:00
Kenneth Giusti
73c0c0071e Add the "transport_options" parameter to the amqp1 and kafka drivers.
Change-Id: I5ca6ec2cb30b8d7f18e1770f80024957bc029bf3
2019-06-20 16:12:39 -04:00
Zuul
b8c3a1ea59 Merge "fix typos" 2019-06-18 17:59:06 +00:00
Hervé Beraud
54356899da fix typos
Change-Id: Id11db4113c9b1c3add602192c1e915218704ef27
2019-06-18 14:00:14 +02:00
Zuul
03ec779cdf Merge "Download kafka from archive.apache.org" 2019-06-18 04:37:26 +00:00
Zuul
e7420afa2f Merge "Add transport_options parameter" 2019-06-17 16:35:55 +00:00
Gabriele
83266cc6ea
Add transport_options parameter
With this new parameter is possible to pass other parameters
from the client to the drivers.
So it is possible to tune the driver behavior.

For example can be used to send the mandatory flag in RabbitMQ

Note:
  - The transport_options parameter is not actually used (yet).
  - This part of blueprint transport-options  (first part)

Implements: blueprint transport-options
The blueprint link is
https://blueprints.launchpad.net/oslo.messaging/+spec/transport-options

Change-Id: Iff23a9344c2c87259cf36b0d14c0a9fc075a2a72
2019-06-14 15:47:39 +02:00
Ben Nemec
70b03bcf83 Download kafka from archive.apache.org
It seems that versions are deleted from www.apache.org pretty quickly.
They stick around longer on archive.apache.org so we won't have to
be constantly chasing the latest version in our functional tests.

Change-Id: I047edac67699dd598f8dfd0f859b3772f6068bd3
2019-06-10 15:35:43 +00:00
Zuul
e45dfa7ea2 Merge "Add help msg to payload for CLI notifier" 9.7.2 2019-06-10 13:40:49 +00:00
Gabriele
da6caaef07 Add thread name to the RabbitMQ heartbeat thread
With this information it is easier to debug the application.

Change-Id: Icafae894ede54d8e802aef556e4239408cc23bcd
9.7.1
2019-05-24 22:03:40 +02:00
Hervé Beraud
1a860467bd Add help msg to payload for CLI notifier
Change-Id: I3d712ecfcea00ece0180108c585c858795fd2bce
2019-05-23 18:52:19 +02:00
Zuul
fe0ac3195e Merge "Fix switch connection destination when a rabbitmq cluster node disappear" 9.7.0 2019-05-20 21:05:57 +00:00
Kenneth Giusti
4f467fbd73 Cap Bandit below 1.6.0 and update Sphinx requirement
Bandit 1.6.0 accidentally changed how the exclusion list option is
handled and breaks our use of it. Cap to the previous version until
Bandit has fixed the problem.

Sphinx 2.0 no longer works on python 2.7, so we need to start
capping it there as well.

Change-Id: Ie6b379f2c99862c37891ac03c52464e07bc2b2cc
2019-05-14 10:46:32 +02:00
Hervé Beraud
9d8b1430e5 Fix switch connection destination when a rabbitmq cluster node disappear
In a clustered rabbitmq when a node disappears, we get a
ConnectionRefusedError because the socket get disconnected.

The socket access yields a OSError because the heartbeat
tries to reach an unreachable host (No route to host).

Catch these exceptions to ensure that we call ensure_connection for switching
the connection destination.

POC is available at github.com:4383/rabbitmq-oslo_messging-error-poc

Example:
    $ git clone git@github.com:4383/rabbitmq-oslo_messging-error-poc
    $ cd rabbitmq-oslo_messging-error-poc
    $ python -m virtualenv .
    $ source bin/activate
    $ pip install -r requirements.txt
    $ sudo podman run -d --hostname my-rabbit --name rabbit rabbitmq:3
    $ python poc.py $(sudo podman inspect rabbit | niet '.[0].NetworkSettings.IPAddress')

And in parallele in an another shell|tmux
    $ podman stop rabbit
    $ # observe the output of the poc.py script we now call ensure_connection

Now you can observe some output relative to the connection who is
modified and not catched before these changes.

Related to: https://bugzilla.redhat.com/show_bug.cgi?id=1665399

Closes-Bug: #1828841

Change-Id: I9dc1644cac0e39eb11bf05f57bde77dcf6d42ed3
2019-05-13 17:09:04 +02:00
Zuul
40c25c2bde Merge "Remove log translation and i18n" 2019-05-04 22:31:13 +00:00
Zuul
55e8fd009c Merge "Replace git.openstack.org URLs with opendev.org URLs" 2019-05-03 18:42:20 +00:00
Zuul
558fc5f4e8 Merge "Consider the topic parameter as an array in client-notify" 2019-04-29 22:36:42 +00:00
jacky06
9b7bcbda2f Replace git.openstack.org URLs with opendev.org URLs
Change-Id: Ib4a523adf3275a8727d5a4f2a149e163a81be3cc
2019-04-26 10:31:26 +08:00
Hervé Beraud
4f385720d7 Remove log translation and i18n
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.

Change-Id: I9ddb6595fc52e46ed8844e39d2fa71029c90b65c
Closes-Bug: #1674567
2019-04-24 11:55:38 +02:00
Zuul
433d34ed41 Merge "Retry to declare a queue after internal error" 2019-04-22 22:23:07 +00:00
Zuul
78777a92b8 Merge "Handle unexpected failures during call monitor heartbeat" 2019-04-22 22:11:48 +00:00
OpenDev Sysadmins
815ea9ac98 OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
9.6.0
2019-04-19 19:31:51 +00:00
Zuul
d4873f983b Merge "Dropping the py35 testing" 2019-04-18 01:14:36 +00:00
Zuul
81e9d81770 Merge "Update messaging intermediaries for amqp1 tests" 2019-04-17 15:14:01 +00:00
Ghanshyam Mann
0d4f1d64b6 Dropping the py35 testing
All the integration testing has been moved to
Bionic now[1] and py3.5 is not tested runtime for
Train or stable/stein[2].

As per below ML thread, we are good to drop the py35
testing now:
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/005097.html

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004647.html
[2]
https://governance.openstack.org/tc/reference/runtimes/stein.html
https://governance.openstack.org/tc/reference/runtimes/train.html

Change-Id: I1874f96f78cb403e6f3a56a49cb83df40d531f8d
2019-04-15 01:14:11 +00:00
Zuul
56ac3a27b3 Merge "Handle collections.abc deprecations" 2019-04-11 15:21:12 +00:00
Gabriele
0ad4a204b3 Consider the topic parameter as an array in client-notify
Without this fix, the simulator splits the "topic" value in different
strings, in notify-client case
Instead of having one queue there are multi-queues.
See [1] for more details

Closes-Bug: #1824339

1- https://bugs.launchpad.net/oslo.messaging/+bug/1824339

Change-Id: I438c7279d584bdb00da18477a83d5e2a53c588c4
2019-04-11 13:55:21 +02:00
Ian Wienand
e99af8ca57 Merge "Replace openstack.org git:// URLs with https://" 2019-04-10 01:32:56 +00:00
Sean McGinnis
4d48b33a41 Handle collections.abc deprecations
The use of ABC classes directly from collections has been deprecated in
3.x versions of Python. The direction is to use the classes defined in
collections.abc. Python 2.7 does not have this, but Python 3.8 will be
dropping the backwards compatibility to use the old location.

Six also does not have support for this yet, so in the mean time to make
sure we don't run into issues as folks try to move to 3.8, and to get
rid of deprecation warnings in logs, this handles importing from the
preferred location and falls back if it not available.

Change-Id: If67133813634f41d89ccdf0f6d6d5ffa66c97dd8
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
2019-04-09 16:43:45 -05:00
Gabriele
4d2787227b Retry to declare a queue after internal error
Without this commit, the client can lose the messages, because the
client does not handler the 'AMQP internal error 541',
read here [2] for details.
The fix retries to create the queue after a delay.
When the virtual-host is ready the declare does not fail.
This is a rare condiction, please read the bug [1] for details.

Closes-Bug: #1822778

[1] https://bugs.launchpad.net/oslo.messaging/+bug/1822778
[2] https://www.rabbitmq.com/amqp-0-9-1-reference.html

Change-Id: I7ab1f9d21ebb807285bf1422bc14cc6e07dcd32a
2019-04-05 15:08:12 +02:00
Pierre Riteau
44bd971bfb Unmark RabbitMQ heartbeat as experimental
Commit 287a4f56f45ed9cd40116a9e7b6e529f3382a925 marked RabbitMQ
heartbeat as experimental in 2015, due to issues with the underlying
amqp library. Later, commit 168f6cc2bb72591012d54b15e1c2e4a0332fb5c9
enabled heartbeat by default, but the EXPERIMENTAL string was left.

Change-Id: I78838df2a0db9836f98d8f79624d688c3ed351ae
2019-03-26 10:50:33 +00:00
Zuul
51fa1b81e3 Merge "Explain why Listener connections cannot be pooled" 2019-03-26 09:48:03 +00:00
Zuul
cee6aec5dc Merge "Clarify the documentation for pooled Notification Listeners" 2019-03-25 16:47:11 +00:00
Ian Wienand
a4d087cc1b Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: Iec59af4e909e307df789589d9e40d019d2545088
2019-03-24 20:35:28 +00:00
Kenneth Giusti
83a84d3f98 Explain why Listener connections cannot be pooled
Restoring an old comment to the code that explains why the RabbitMQ
driver provides a dedicated connection for each listener.

Change-Id: If833d41d78444cc26535cb165ea821dce21b768b
2019-03-18 12:26:47 -04:00
d143a06b48 Update master for stable/stein
Add file to the reno documentation build to show release notes for
stable/stein.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/stein.

Change-Id: Icb3ca5bb875df1dda9f3d86bb164b6299059664e
Sem-Ver: feature
2019-03-18 14:45:21 +00:00
Kenneth Giusti
855a85fa2d Clarify the documentation for pooled Notification Listeners
Change-Id: I184b045edaa8e61f075df576992426e94bf7386a
Closes-Bug: #1814544
2019-03-11 09:31:38 -04:00
Andy Smith
8897e1584f Update messaging intermediaries for amqp1 tests
This patch messaging intermediaries used for the amqp1 driver
test. The combination of the apache qpid-dispatch-router (qdrouterd)
with an attached apache artemis broker is used in place of the qpidd
broker. The qdrouterd will directly message oslo.messaging rpc
communications and will link route oslo.messaging notify
communications to the artemis broker (e.g. for persistence).

This patch:
* updates the dependencies
* updates the test environment setup
* uses the pifpaf drivers to create servers

Change-Id: I9fb56450346fca84bc1573fa5e9b6fe81c14e33a
2019-03-11 08:47:46 -04:00
Zuul
472d483942 Merge "Change python3.5 job to python3.7 job on Stein+" 2019-02-22 01:23:52 +00:00
Zuul
d1db4509ac Merge "Kafka driver deployment guide" 2019-02-21 18:59:53 +00:00
Zuul
28e213c506 Merge "Bump amqp requirement version to >=2.4.1" 2019-02-20 22:39:25 +00:00
Kenneth Giusti
f948e24294 Handle unexpected failures during call monitor heartbeat
Change-Id: Iec04c18ac3565a3610377d94caf128c6704a89eb
Closes-Bug: #1816816
2019-02-20 15:07:31 -05:00
Zuul
7102eddb1f Merge "Update hacking version" 2019-02-20 18:59:03 +00:00
Zuul
b2439756a0 Merge "add python 3.7 unit test job" 2019-02-20 04:13:42 +00:00
ZhongShengping
76101e32fe add python 3.7 unit test job
This is a mechanically generated patch to add a unit test job running
under Python 3.7.

See ML discussion here [1] for context.

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135626.html
Story: #2004073
Task: #27440

Change-Id: I2ee368245286ca5fb6091a5ea5ebbed5261c54c0
2019-02-19 13:56:43 +08:00
Andy Smith
f8d0dcf04c Mark telemetry tests nv and remove from gate
Change-Id: I3a49e7397df9cd0b2d7eba5747795c3d771dcbea
2019-02-18 15:19:15 -05:00
mb
64f74cd2cc Bump amqp requirement version to >=2.4.1
Bumping the version of amqp in requirements.txt and
lower-constraints.txt from 2.4.0 to 2.4.1 to mitigate issues with
MessagingTimeouts when using TLS due to a amqp bug that was fixed in
2.4.1

Change-Id: I0b094f32dec24c70f95ccd509164a14a71fcfc7d
Closes-Bug: #1800957
2019-02-18 18:29:26 +01:00
Charles Short
fa2a3e4fcc Change python3.5 job to python3.7 job on Stein+
python3.5 was the only supported python3 version on Xenial, now that we have
Bionic Beaver nodes that support python3.7, lets switch to testing with
python3.7 in addition with python3.6 in Stein and beyond.
See ML discussion here [1] for context.

[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135632.html

Change-Id: I3335ccb01667d22a181d99d4d53d7356005d72ad
Signed-off-by: Charles Short <chucks@redhat.com>
Story: #2004073
Task: #27440
2019-02-14 10:28:20 +00:00