2212 Commits

Author SHA1 Message Date
Jenkins
fe540187d5 Merge "gate: Remove useless files" 2017-01-10 19:26:47 +00:00
Mehdi Abaakouk
d64d05f3e0 tests: cleanup monkey path
This change makes the monkeypatching the first thing to do.
It removes the try/except since the tests requires eventlet.

Change-Id: Ic6bbe303d7f4a606212b46add2fa51ce14c569ae
2017-01-10 18:27:48 +01:00
Kenneth Giusti
9fab0bdbc8 [AMQP 1.0] Resend messages that are released or modified
A message ack status of 'RELEASED' or 'MODIFIED' indicates that the
message was not accepted by the destination due to some temporary
issue.  These status are used to indicate to the sender that the
message can be safely re-transmitted without risk of duplication
(i.e. the delivery is not 'in-doubt').  For example this may happen
during a message bus topology update if a message is sent before the
topology stabilizes.

This change implements re-send for these cases.

Closes-Bug: #1646586
Change-Id: I419e23b59e3eb90fda3f1c0e7ddf54ef98870e4b
2017-01-09 09:30:57 -05:00
Mehdi Abaakouk
5f4935dbc7 gate: Remove useless files
Change-Id: I368c28cbad0973831d1b92b627e5dd9138c3937b
2017-01-09 13:35:05 +01:00
Jenkins
bbf77bfcca Merge "tox: use already installed kafka if present" 2017-01-06 15:23:23 +00:00
Jenkins
fad55f603f Merge "tests: make rabbit failover failure more helpful" 2017-01-06 14:40:52 +00:00
Jenkins
a6820de99c Merge "kafka: remove no really implemented feature" 2017-01-06 09:08:16 +00:00
Jenkins
234e920b6f Merge "kafka: return to poller when timeout is reach" 2017-01-06 08:01:16 +00:00
Jenkins
a4c28cc577 Merge "eventlet is no more a hard dependency" 2017-01-06 00:38:50 +00:00
Jenkins
3db6c77675 Merge "[AMQP 1.0] Propagate authentication errors to caller" 2017-01-05 16:04:39 +00:00
Jenkins
a1fecbec58 Merge "ensure we set channel in lock" 2017-01-05 02:46:44 +00:00
Jenkins
6809dbe044 Merge "kafka: Don't hide unpack/unserialize exception" 2017-01-04 13:11:35 +00:00
Jenkins
8ac9558c6e Merge "kafka: timeout is in milliseconds" 2017-01-04 13:11:29 +00:00
Jenkins
42d1fa4aac Merge "kafka: disable batch for functional tests" 2017-01-04 13:11:23 +00:00
Jenkins
654be38629 Merge "kafka: Remove Producer singleton" 2017-01-04 13:11:21 +00:00
Jenkins
74f5dd5c1c Merge "Moving driver to new kafka-python version" 2017-01-04 13:11:06 +00:00
ozamiatin
e39be47b06 [zmq] Redis TTL for values
Redis as a key-value storage has limitations on values
expiration by ttl, only keys can expire. Regarding this
we need to introduce some redundancy and register hostnames
as keys along with general target keys to make them expire
on ttl timeout.

Change-Id: Ic0b06d8ec27b63ca49afe941a32020b5e532598c
2017-01-04 15:09:16 +02:00
Mehdi Abaakouk
31db9516c2 eventlet is no more a hard dependency
Change-Id: Ie386dea5cc4b9ff02dc2b638d2d6908c3cdad883
2017-01-04 08:10:56 +01:00
Kenneth Giusti
9972158baa [AMQP 1.0] Propagate authentication errors to caller
When the connection fails due to an authentication issue raise a
MessageDeliveryFailure on all pending send requests. Include the
authentication error message in the exception.

Change-Id: I06b40c6c480a4d082dce64801736b3d12f696c26
Closes-Bug: 1508512
2017-01-03 09:48:55 -05:00
gord chung
1dc575ce5b ensure we set channel in lock
_set_current_channel must be called  with a lock

Change-Id: I151564670946b3399158352c51d1670b9e7291fb
2017-01-03 14:39:21 +00:00
Mehdi Abaakouk
925081b2d6 tox: use already installed kafka if present
Change-Id: Ie1539035625693ec21a6ca279d59bb818343e618
2017-01-02 11:46:48 +01:00
Mehdi Abaakouk
1ee3d7001a kafka: remove no really implemented feature
Change-Id: I6f2693c48d5d0ac1af68b3d4bb5ff361facef977
2017-01-02 11:46:48 +01:00
Mehdi Abaakouk
488594936a kafka: return to poller when timeout is reach
consume() must return only if user timeout is reached and not
when driver consumer_timeout is reached.

Change-Id: I6b2b2a28038a194224e79fa37285436ca6787a0a
2017-01-02 11:46:48 +01:00
Mehdi Abaakouk
a7044799ad kafka: Don't hide unpack/unserialize exception
These exception are software bugs, don't convert them into
log for deployer.

Change-Id: I10e9112b53e5c754f38247679896d1d24dd454bf
2017-01-02 11:46:48 +01:00
Mehdi Abaakouk
c8880b6f11 kafka: timeout is in milliseconds
Change-Id: I791c5dd8e7f4fb46a0aea3e825b9b392686a2abd
2017-01-02 11:46:48 +01:00
Mehdi Abaakouk
799cd6fa8f kafka: disable batch for functional tests
Change-Id: I09a3049ca5f4647d0f6b002b3732a4c0edd43986
2017-01-02 11:46:48 +01:00
Mehdi Abaakouk
a76a51a78c kafka: Remove Producer singleton
This producer singleton works only if one transport use kafka.

If multiple transports with kafka are used, their overrides each other
the producer on each send.

This change creates only one producer per connection.

A later optimisation can be one producer per driver instance.

Change-Id: I429f7c5efcb41690dd1b17f856bda2425c788e53
2017-01-02 11:46:48 +01:00
Ilya Tyaptin
f139eb258d Moving driver to new kafka-python version
Currently Kafka driver for an oslo.messaging uses kafka-python==0.9.5
and mostly broken. This package version supports only low level Kafka
producer and consumer API which are marked as deprecated now [1]. Using
of these interfaces bring a big concern to the message processing,
because current KafkaConsumer has not any consuming coordination. This
fact causes a message duplication for the several consumers of one
topic. This behavior is specific to Ceilometer and services which read
and process notifications from other services.

New version of kafka-python allows to use async thread safe message
producers and coordinated consumers [1].

[1] http://kafka-python.readthedocs.io/en/master/changelog.html#feb-15-2016

The driver is currently experimental, python-kafka<1.0.0 API have major
issue described above that can't make the oslo.messaging driver works,
so we prefer having a working driver with a non-synced dependencies, that the
reverse.

Co-Authored-By: Mehdi Abaakouk <sileht@redhat.com>
Change-Id: I29862ed7bf56b9d8878fa8e9fb1cbd9d643908a4
2017-01-02 11:46:48 +01:00
Mehdi Abaakouk
8d2331a3b7 tox: rename zeromq target
The current zeromq target name are too long and reach the limitation
of 127 chars of the sheband of the virtualenv binary.

A target name must not be more than 22 chars to run in gate with
the tox job template.

This renames them to a shorter one.

It also creates aliases for the legacy jobs that we will remove
when the tox jobs are ready.

Change-Id: I5cbc368e7ea252f8c0c949c8e7ee3a70a560db22
2017-01-02 11:34:02 +01:00
Mehdi Abaakouk
301a7d945a tests: make rabbit failover failure more helpful
Change-Id: Ic50b9996e8b58d5b174b9e188d70a384240499ea
2017-01-02 11:33:55 +01:00
ozamiatin
96d506377f [zmq] Refactor make zmq_address.target_to_key a universal method
Reduce conditional logic everywhere analysing fanout/round-robin
target to key conversion. Provide single conversion method for all
cases.

Change-Id: I3d02d25fe806dabe0022de2dc964d23b1aaff0e7
2016-12-31 01:02:52 +02:00
Jenkins
8fefbc064d Merge "Updated from global requirements" 2016-12-30 04:30:47 +00:00
Jenkins
d265b49ac3 Merge "[zmq] Restore static direct connections" 2016-12-29 22:54:01 +00:00
OpenStack Proposal Bot
64914790f4 Updated from global requirements
Change-Id: I6375d299c56fb3789c9d37276dc0a6adb6ec4de8
2016-12-28 09:15:07 +00:00
Jenkins
1242564e4d Merge "reject when skipping failed messages" 2016-12-26 07:39:49 +00:00
Jenkins
99789d2381 Merge "test_rabbitmq: remove dead code" 2016-12-25 21:44:32 +00:00
ozamiatin
fe6cf0f8a2 [zmq] Restore static direct connections
Restore static direct connections which may be useful
for services running RPC on controllers only and talking
to agents via other means (ssh for example).

Change-Id: Icbe45978cb4a8ba5db74e5593ea7be23cba3f44e
2016-12-24 04:26:19 +02:00
Jenkins
b0507a6bdb Merge "[AMQP 1.0] Setup the amqp1 test environment on ubuntu" 2016-12-23 15:02:36 +00:00
gord chung
85b05a7f2f reject when skipping failed messages
reject is a more accurate action when we are tossing away messages
we cannot handle.

Change-Id: I7f40ac8809f93afb8e518f236d0ddf64a3b241eb
2016-12-21 16:03:24 +00:00
jolie
f1efaf1b6e fix one typo
Change-Id: Ie4ce6e4a8bad8610b40050c94bb3d0796f9f4d2f
2016-12-21 17:52:14 +08:00
Kenneth Giusti
2a9094af5d [AMQP 1.0] Setup the amqp1 test environment on ubuntu
Fixes the amqp1 functional tests so they can be run on Xenial.

Closes-Bug: 1643644
Change-Id: I3e3ee6779a60774ebffc1f45ae5564db934f8eaf
2016-12-19 14:47:04 -05:00
Julien Danjou
fd9f5073c7 test_rabbitmq: remove dead code
This method does not seem to be used by anything.

Change-Id: I71a0a6b12b45a4c7e2c4f301abda5a9cde3e777f
2016-12-19 10:30:51 +01:00
OpenStack Proposal Bot
ea61a81014 Updated from global requirements
Change-Id: I35e0f9e78c31a7c68746401dcf289f8648ef7e6a
2016-12-17 20:48:42 +00:00
Jenkins
d16e4883fe Merge "Replace mox with mock" 2016-12-17 18:42:26 +00:00
howardlee
de88370982 Replace mox with mock
Replace mox with mock in some files for testing.

See:
https://blueprints.launchpad.net/nova/+spec/remove-mox-ocata

Change-Id: Ic2a3c760d03d19c5a07d71796f24580393102163
2016-12-16 11:24:13 +01:00
Mehdi Abaakouk
04f9be69e5 tests: fix test-setup.sh
tox doesn't have quiet option, so we have to create the virtualenv first
and run bindep manually.

Change-Id: I3ae76278391baa5cd80202f32ad88b1368b0f16d
2016-12-15 09:41:29 +01:00
Jenkins
c805618d9a Merge "[AMQP 1.0] Simplify the I/O event loop code" 2016-12-09 17:26:34 +00:00
Jenkins
a88637253e Merge "tests: remove useless debug" 2016-12-09 16:57:59 +00:00
Mehdi Abaakouk
2f1ee2f71b tests: remove useless debug
It's useless to print the 250 events

Change-Id: Ib1e05bc20869a0892dfae4b4e45832ebce6373d8
2016-12-09 13:36:21 +01:00
Dmitry Mescheryakov
04f89a76f0 [rabbit] Log correct topic on cast/call
Change-Id: Icc06f7f0f99728fcf4abbdce0bf85af88713494a
2016-12-09 14:24:52 +04:00