2212 Commits

Author SHA1 Message Date
Mehdi Abaakouk
58b026a2aa drivers: use common.ConfigOptsProxy everywhere
ConfigOptsProxy have been implemented only pika driver while
the oslo.messaging allow to pass the query string for all drivers.

This change fixes that.
Closes-Bug: #1666903
Closes-Bug: #1607889

Next step is to validate the query with ConfigOptsProxy, to
raise appropriate exception in case of mis-configuration.

Change-Id: I573334e774ccf33ecd27a85067045f3c6489ee89
2017-02-27 13:10:31 +01:00
Brant Knudson
41fa1ac649 Stop using oslotest.mockpatch
This module has been deprecated in favor of fixtures.MockPatch.

Change-Id: Id60c1a35cc1ec832da1a459b19c43020987d0769
2017-02-26 00:35:29 +00:00
Mehdi Abaakouk
f9cba9bd09 tests: don't run functional tests in parallel
kafka doesn't play well with //, each times a new partition/topic is
created, kafka rebalances things across brokers and messages are not
delivered at the expected time.

So this change disable the parallelizism for functional tests for kafka
only.

Change-Id: Ia497334efcd4685850ac1529a7fc38073dae7cd1
2017-02-23 10:59:00 +01:00
Mehdi Abaakouk
7e71ac821f rabbit: make ack/requeue thread-safe
ack/requeue messages are currently done in the
MessageHandlingServer._process_incoming().

But _process_incoming() in run by a futurist Executor.
That can be a threading or an eventlet executor.

With eventlet, we don't really share the socket between threads.

But with threading executor and expecialy ssl, this can't work, if you
write data with two different threads to the socket.

This change moves back the message ack/requeue to the polling threads
that handle the connection, instead of the threads we spawn for the
application.

Oslo Messaging now always use a connection in the same thread.

Change-Id: I5c0e6def6b34f4d195fb1f8dbb26eda0f21ff34e
2017-02-22 18:39:53 +01:00
Hanxi Liu
4cf269afee Fix releasenotes
Fix the unreleased page and remove reno pages extra symbols.

Change-Id: I291b18560a11a318577b61c15a3462dabb0eb366
2017-02-22 14:08:19 +08:00
Brant Knudson
7728afe175 Remove mox3 from test-requirements.txt
oslo.messaging does not import mox3 directly, it uses mox3 through
oslotest.

Change-Id: Iaae0b03d2c0511d12f1ab2c8e93622dfaa6c27ce
2017-02-21 13:49:07 -06:00
Jenkins
296d93d586 Merge "Fix the typo" 2017-02-20 20:09:48 +00:00
OpenStack Proposal Bot
3e727ea715 Updated from global requirements
Change-Id: I1a5882d3fe0189ab3e651155681a1410c37d8a82
2017-02-16 23:17:36 +00:00
Jenkins
a916fce706 Merge "support kombu4" 2017-02-16 02:26:08 +00:00
Jenkins
706cfde8d3 Merge "Test:Use unittest.mock on Python 3" 2017-02-16 01:37:24 +00:00
Jenkins
07486845eb Merge "Fix type of the kafka_consumer_timeout option" 2017-02-15 12:39:35 +00:00
Jenkins
bf966f6f1f Merge "[zmq] Dynamic connections send failure" 2017-02-14 17:41:20 +00:00
ozamiatin
6c7094a921 [zmq] Update configurations documentation
In this change new configurations appeared in Ocata release
like dynamic connections and other types of proxied deployments
being described.

Change-Id: Id6e9b062101d8916323edc143ea5379585192581
2017-02-14 12:41:21 +02:00
Pierre Riteau
aab5ea2dfd Fix type of the kafka_consumer_timeout option
Change-Id: Ibe7a72cef031e84221522183521f65c238a2d8e5
2017-02-13 09:45:02 +00:00
ozamiatin
4a1679450d [zmq] Dynamic connections send failure
For dynamic connections it is crucial to close connection
and not to have hanging sockets either we have sent message
successfully or not.

eventlet.green.zmq by default blocks the calling thread on sending message
when connection was not established yet (which is correct DEALER
socket behavior though), but socket cannot be closed when we hang on
sending forever (if we never get the valid host to connect).

eventlet also shields EAGAIN exception in default (blocking) sending mode
so we need to use async zmq.NOBLOCK flag to receive this exception
and hanlde it in our own way to not block forever.

Change-Id: Ib561e061c4b20644213c059a8e8d0efd225edea1
Closes-Bug: #1658913
Closes-Bug: #1663459
2017-02-11 05:25:53 +02:00
Jenkins
337f499c58 Merge "[zmq] Properly analyse use_dynamic_connections option" 2017-02-10 19:13:08 +00:00
gord chung
5bacea1f42 support kombu4
- kombu4 wraps recoverable errors as OperationalErrors rather than
raising amqp errors
- also, raise a recoverable error and redeclare if for some reason a
message is double acknowledged... previously, this was hidden.
- ensure socket is not none
- use connect method to ensure connection

Depends-On: I9f980b51901ac31599b9651633956ad2eea6a1ac
Change-Id: I73958c8057353a2eefe1baaa7a41148193d507f7
2017-02-10 13:21:22 +00:00
ChangBo Guo(gcb)
4f229832c0 Test:Use unittest.mock on Python 3
The mock module has been added to Python 3.3 as unittest.mock.
The third party mock module doesn't seem to be maintained anymore.
This is follow up of 72c501454ea33b603918de35565a66dda8abd66a and
add hacking rule to avoid regression issue.

oslotest prepares mock for six in oslotest/__init__.py as follow:
six.add_move(six.MovedModule('mock', 'mock', 'unittest.mock')) and
oslo_messaging/tests/__init__.py imports oslotest before importing
test submodules to setup six.moves for mock, then "from six.moves
import mock" works well.

Note: unittest.mock also detects wrong usage of method
assert_called_once_with.

Change-Id: I3d09733789cfa2550cf47c24f2553357d36bcc0d
2017-02-08 19:15:17 +08:00
Jenkins
f3cc165dba Merge "[zmq] Dummy add value aging mechanism" 2017-02-07 03:13:57 +00:00
Jenkins
3b5fb1968b Merge "pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)" 2017-02-07 02:45:16 +00:00
chenxing
7e767256c3 Fix the typo
Change-Id: Ieebb6d30a0b9e237eac930fb19d40a686d972aa1
2017-02-07 02:39:25 +00:00
Jenkins
eca7e934a8 Merge "Update reno for stable/ocata" 2017-02-06 21:26:32 +00:00
Davanum Srinivas
f33bb02c82 pbr.version.VersionInfo needs package name (oslo.xyz and not oslo_xyz)
We need to pass into VersionInfo what one would expect from running:
 setup.py --name

 Right now we pass in say oslo_context and pbr fails if there is no .git in
  the python source tree

  Closes-Bug: #1662266

Change-Id: I45d2edddfd516f65e1589e9b7816f1256e3f0d24
2017-02-06 12:49:23 -05:00
ozamiatin
35744c88d3 [zmq] Properly analyse use_dynamic_connections option
Option use_dynamic_connections has to be counted when direct
publisher is being created along with proxy publisher (ZmqClientMixDirectPubSub).

Change-Id: If0d5674d7015b164095a437464d3c780db892e19
Closes-Bug: #1662134
2017-02-06 12:06:59 +02:00
lihong7313
407544a040 [zmq] Dummy add value aging mechanism
Dummy does not have value aging mechanisms and value
filtering mechanisms. Like redis, we should use these
two mechanisms

Change-Id: I58d73e6b15a601e9e95274d47e822ae54ef5f5ba
Closes-Bug: #1658940
2017-02-06 02:30:26 +00:00
Jenkins
35bf674364 Merge "kafka: skip multiple servers tests" 2017-01-31 20:22:09 +00:00
Jenkins
4ec746dfba Merge "kafka: ensure topics are created" 2017-01-31 20:21:55 +00:00
Jenkins
3b91f3d1cc Merge "kafka: fix python3 exception" 2017-01-28 11:06:33 +00:00
Jenkins
32758817fe Merge "kafka: Remove testing hack for kafka" 2017-01-28 10:38:36 +00:00
Mehdi Abaakouk
fe8d61345e kafka: skip multiple servers tests
I can't figure out why it doesn't pass.

Change-Id: I091c0ffd71e9a2164354cebc248af6cd3675fe07
2017-01-26 08:00:40 +01:00
Mehdi Abaakouk
578a186515 kafka: ensure topics are created
Change-Id: I3fb5ee1c134ca6ea220b204af8f6325eeadf9465
2017-01-26 08:00:40 +01:00
Mehdi Abaakouk
c7cdf2d9b7 kafka: fix python3 exception
Change-Id: I7244515f274719eb8e9189677554637e20917274
2017-01-26 08:00:40 +01:00
Mehdi Abaakouk
61c9e03b6b kafka: Remove testing hack for kafka
Change-Id: Ie304f1ed34156bfb60b9bc9acd4ad0aa5bad4657
2017-01-26 08:00:39 +01:00
ozamiatin
08b6472dbf [zmq] Failure of dynamic connections fanout
Change-Id: I740ac926141ff788e75be788ebaadb71e440c9f4
Closes-Bug: #1658601
2017-01-23 09:36:27 +02:00
4fbfd95fca Update reno for stable/ocata
Change-Id: I4d365377e6c01e040f705bf70ed86c9f1d97234e
2017-01-20 19:15:37 +00:00
Andrew Smith
58f20a8812 Return list of addresses for IPV4 and IPV6
Closes-Bug: 1657485
Change-Id: Ifd0c338442b97b2ad9476e2856af120d7080f025
2017-01-19 09:08:13 -05:00
ozamiatin
8424dbc7cb [zmq] Dynamic connections failover
ZeroMQ manages failover automatically when socket is
being connected to multiple hosts. Dynamic connections
mode assumes to connect once per message so we connect
only to a single host. However we cannot say for sure
if this host is still alive. For failover reasons
we can connect to a primary host and then add some
more hosts as failover and let ZeroMQ to decide where
it finally sends the message.

Change-Id: I19cbb75aaea8a0b725dd6a8ff665392738e164a1
2017-01-16 20:09:26 +00:00
Jenkins
f07652ae2d Merge "[zmq] Fix py35 gate" 2017-01-16 19:53:11 +00:00
Jenkins
79df3f3646 Merge "[AMQP 1.0] Fix SSL client authentication" 2017-01-16 17:45:23 +00:00
ozamiatin
07b1933531 [zmq] Fix py35 gate
Change-Id: I786908e35b2a886e9aba52b359e954d51ac9b297
2017-01-16 08:14:12 +00:00
ozamiatin
fa4054a51f [zmq] Use more stable configuration in voting job
Use static direct connections as voting job for zmq

Change-Id: I921317987dfccbd03b8c8c68fc1a169b6a8100c3
2017-01-16 10:13:17 +02:00
Eric Brown
ed2c1e3767 Remove references to Python 3.4
Now that there exists only a gate job for Python 3.5 and not 3.4,
we should remove those references to the 3.4 that is untested.

Change-Id: I2a00ead626d9eced96487ee82b5d7857126d8355
2017-01-13 18:13:32 +00:00
Kenneth Giusti
53ead5c11d [AMQP 1.0] Fix SSL client authentication
The driver incorrectly set up SSL client authentication.  This patch
fixes the configuration. Unit tests added.

Change-Id: I64b1736f5a708c70013d9fedba73da4fa8d9ccfb
Closes-Bug: #1606985
2017-01-13 11:52:06 -05:00
Jenkins
e90792de85 Merge "[zmq] Support py35 testenv" 2017-01-13 10:08:51 +00:00
Jenkins
fbe856575a Merge "[AMQP 1.0] Resend messages that are released or modified" 2017-01-12 17:45:40 +00:00
Jenkins
268e2c5103 Merge "[zmq] Redis TTL for values" 2017-01-12 15:49:06 +00:00
Jenkins
be9d3f4182 Merge "[zmq] Refactor make zmq_address.target_to_key a universal method" 2017-01-12 15:48:56 +00:00
Jenkins
9f2b59c957 Merge "tests: cleanup monkey path" 2017-01-12 15:48:40 +00:00
ozamiatin
8c5e2bd4e1 [zmq] Support py35 testenv
Change-Id: If33583d1f89e6068b232fe09f1483c979b0ef84c
2017-01-11 14:04:35 +02:00
ozamiatin
492ffe9774 [zmq] Distinguish Round-Robin/Fanout socket sending mode
For zmq.DEALER socket there are two round-robin modes exist,
controlled by zmq.IMMEDIATE option. Immediate True means sending
only to running servers and False means to send to all connected
servers (may be not up and running at the moment).

If we do all messaging over DEALER socket as we do for direct
connections, immediate=True better suits for direct CALL/CAST
message types and immediate=False is better for fanout.

This patch fixes things for dynamic connections as for static
we already used the approach.

Change-Id: I70c7aeb3c7a2c63c128bec394827577cab580def
2017-01-11 13:26:16 +02:00