* The restart loop for rabbitmq-server can trigger socket activation
of epmd without rabbitmq-server running. This can lead to 'systemctl
status' reporting 'State: degraded' with no simple way to reset to
'State: running'.
* It's important to note that this socket activation failure is benign
and is not an indicator of system failure.
Change-Id: Iede4f5ebeffb59644dee4a17b6331b3cdd04d146
Signed-off-by: Jan Gutter <jan.gutter@netronome.com>
This adds the necessary fixes to pass a devstack run
on openSUSE Tumbleweed. Also removes opensuse 42.2 as it
is EOL for some time already and no longer actively tested
in the OpenStack infra.
Depends-On: I1b68c08c07cf6653ea58506f738cbe0054b38f3a
Change-Id: I2894482deef063fd02b0818c695a2ddbf6767039
Leaving the default user enabled is a security issue, as it can be used
without credentials. It also may mask issues like seen in [1].
[1] https://bugs.launchpad.net/bugs/1651576
Change-Id: I75b4e5696c0f8017b869127a10f3c14e2f8bd121
If a project manually configures the oslo.messaging transport url for
notifications it should use 'get_notification_url', not
'get_transport_url'. get_transport_url should only be used to obtain
the RPC transport address.
Change-Id: I77772dfa9f30a3db2db6d0387260dfe3452a26ef
Closes-Bug: #1708754
This defines a new function get_notification_url, which returns the URL
of RabbitMQ when you want connect to it, and uses in
ceilometermiddleware. This fixes an issue when we try to use AMQP for
RPC, but not for notifications.
Change-Id: I14450b2440806a17a90e5ddefc243868fdbe4f2c
The creation of the cellsv1 rpc vhost was buried in the restart function,
which makes it hard to extend. This breaks it out into a helper method
and moves the conditional logic into the nova module itself.
Change-Id: Ib0e377aabe45c27bb6ce59ca275ce73085e8b9d2
This moves setting of RABBIT_HOST from stack.sh to lib/rpc_backend
so it may be used in grenade runs, which don't have the defaulted
value from stack.sh. The RABBIT_HOST is needed in order to call
get_transport_url in lib/rpc_backend.
Change-Id: I504f7fac7bb9a8c158e20046dbd1dd2d507db02b
Closes-Bug: #1649586
Depends-On: I3d4d7b309e50f4e2970cda55aada02d68c4fa705
This moves setting of RABBIT_USERID from stack.sh to lib/rpc_backend
so it may be used in grenade runs, which don't have the defaulted
value from stack.sh. The RABBIT_USERID is needed in order to call
get_transport_url in lib/rpc_backend.
Change-Id: I6f211e9102f79418f9f94a15784f91c4150ab8a7
Future oslo.messaging is going to deprecate usage of driver-specific
options for hosts/port/user/password options.
This change uses transport_url that exists since a while now and
works with all drivers (even devstack handles only the rabbit one).
Change-Id: I3006b96ff93a3468249177c31c359c2f9ddc5db6
Change 4d8c03a3 added logic to enable rabbit on Fedora/CentOS systems
whether or not rabbit is enabled. This corrects that to only enable
rabbit when it is configured as such.
Change-Id: I270e79ff989176770d65df1ac0ac4e2c4382bb9a
Signed-off-by: Kyle Mestery <mestery@mestery.com>
Solve the devstack ./rejoin-stack.sh when is reboot-safe in RHEL 7.
Enable mysql, postgresql, rabbitmq-server, openvswitch service when on boot.
Change-Id: I3ce9fc58ccc76092ad08314de1c3c9339ebfb3b5
Related-Bug: #1486833
I noticed this when debugging some grenade issues failures.
An include of grenade/functions stores the current value of XTRACE
(on) and disables xtrace for the rest of the import.
We then include devstack's "functions" library, which now overwrites
the stored value of XTRACE the current state; i.e. disabled.
When it finishes it restores the prior state (disabled), and then
grenade restores the same value of XTRACE (disabled).
The result is that xtrace is incorrectly disabled until the next time
it just happens to be turned on.
The solution is to name-space the store of the current-value of xtrace
so when we finish sourcing a file, we always restore the tracing value
to what it was when we entered.
Some files had already discovered this. In general there is
inconsistency around the setting of the variable, and a lot of obvious
copy-paste. This brings consistency across all files by using
_XTRACE_* prefixes for the sotre/restore of tracing values.
Change-Id: Iba7739eada5711d9c269cb4127fa712e9f961695
Part of what was decided at summit is devstack needs to return to a
more opinionated stance, the following removes support for non
RabbitMQ messaging. RabbitMQ is used by over 95% of our community
(statistically all of it), so it's a pretty clear line to draw that
this shouldn't be in tree.
iniset_rpc_backend will be our stable hook for other projects that
want to implement this out of tree. The burden on creating those out
of tree plugins will be on those that wish to support those
alternative stacks.
Change-Id: I8073a895c03ec927a2598eff6c2f01e5c82606fc
Some distros have converted to systemd for starting RabbitMQ. This has
resulted in:
---
[Call Trace]
./stack.sh:904:restart_rpc_backend
/home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201:die
[ERROR] /home/stack/projects/openstack/openstack-dev/devstack/lib/rpc_backend:201 Failed to set rabbitmq password
Error on exit
World dumping... see /opt/stack/logs/worlddump-2015-05-29-031618.txt for details
---
Because 'restart_service rabbitmq-server' returns before the server is ready to
accept connections.
Alter the retry loop to only restart the rabbitmq-server every second time
through the loop. Allowing time for the slow rabbit to start.
Closes-Bug: 1449056
Change-Id: Ibb291c1ecfd109f9ed10b5f194933364985cc1ce
For testing we can need to disable or change the rate of the heartbeat
Currently we have to set the value manually in each componments or
to write multiple [[post-config|$<APP>_CONF]] section in local.conf.
This change will allow to configure all componments at once with only two
lines.
Also, we don't set default values to continue to use oslo.messaging
defaults.
Change-Id: Ieaca60ca1cd6d7455b66ce490a9b023df431e9c3
This creates a new pip_install_gr that installs from global
requirements allowed versions. Now that stable branches are getting
capped all of devstack needs to be fixed to do things like this.
Change-Id: I8fd0ef2bfc544ca2576fab09d3018f760b8848fe
I changed it so that rabbitmq always restart.
Current devstack don't restart rabbitmq in case of ubuntu. Because
rabbitmq is running at default.
But this approach have the following bug.
If rabbitmq is already installed and not running , stack.sh will
fail.
So I change it so that rabbitmq always restart.
Closes-bug: #1030798
Change-Id: Ie45446d3817b2f15631f03b2af84749fe936c67b
Stop services from throwing the following warning:
Option "rabbit_password" from group "DEFAULT" is deprecated. Use option
"rabbit_password" from group "oslo_messaging_rabbit".
Same for rabbit_hosts and rabbit_userid
Change-Id: I7da503ef50b3653b888cb243caa74b4253a495e2
iniset_rpc_backend should know what section it needs to set the
config options in better than the callers. The config options
have actually been moved to different sections and the options
in the DEFAULT section are deprecated.
Change-Id: I0e07fe03c7812ef8df49e126bf71c57588635639
swift middleware contained in ceilometer is now deprecated. the
middleware is available in ceilometermiddleware.
Change-Id: I6e41986245f4d95a9385dc7829479ed1199f10ac
MatchMakerRedis is the only tested routing method
for ZeroMQ driver. For others, like MatchMakerLocalhost
and MatchMakerRing, it still takes some time to work
on and completely test.
MatchMakerRedis is enough to run under real-world
deployment.
Change-Id: I3b2e8e68ceebd377479d75bbb8b862ae60cfc826
Partially-Implements: blueprint zeromq
On fedora 21, qpidd cannot authenticate user, because
cyrus-sasl-plain is no more automatically installed.
This change fixes that.
Change-Id: I74452f40723881291b8c7577e5509da1c0e4e6e5
This adds the test infrastructure for testing that unstack.sh and
clean.sh do the right thing, and actually stop what's expected. This
is designed to be used in upstream testing to make unstack and clean a
bit more certain.
It includes numerous fixes to make these pass in an errexit
environment with the gate config. The scripts still don't run under
errexit because we don't assume we've handled all possible cleanup safely.
Change-Id: I774dfb2cc934367eef2bb7ea5123197f6da7565b
This makes a bunch of variable cleanups that will let -o nounset
function, for the time being we hide nounset behind another setting
variable so that it's not on by default.
Because this is bash, and things are only executed on demand, this
probably only works in the config it was run in. Expect cleaning up
all the paths to be something that takes quite a while.
This also includes a new set of unit tests around the trueorfalse
function, because my change in how it worked, didn't. Tests are good
m'kay.
Change-Id: I71a896623ea9e1f042a73dc0678ce85acf0dc87d
Add an option 'ZEROMQ_MATCHMAKER' to indicate which
matchmaker driver to use.
When it indicates 'redis', the dependencies will be
installed.
Change-Id: I910b48347bad0685ea10083a3b0b243524f32095
Partially-Implements: blueprint zeromq
Currently, as almost all the rpc driver of projects
have been switched to oslo.messaging, the object should
be imported via oslo.messaging rather than its own rpc lib.
Change-Id: I9633446e78cb5af21f61a26f6fb365a8ed57a85f
Partially-Implements: blueprint zeromq
Closes-Bug: #1395721
I think this retry check has been broken since we introduced "set -e".
Unfortunately it seems the issue of rabbitmq not starting first-time
persists on centos 7 hosts occasionally, e.g. [1]:
---
+ rabbit_setuser stackrabbit secretrabbit
+ local user=stackrabbit pass=secretrabbit found= out=
++ sudo rabbitmqctl list_users
Error: unable to connect to node 'rabbit@devstack-centos7-rax-iad-100675': nodedown
DIAGNOSTICS
===========
nodes in question: ['rabbit@devstack-centos7-rax-iad-100675']
hosts, their running nodes and ports:
- devstack-centos7-rax-iad-100675: [{rabbitmqctl29293,39511}]
current node details:
- node name: 'rabbitmqctl29293@devstack-centos7-rax-iad-100675'
- home dir: /var/lib/rabbitmq
- cookie hash: KieJnx1pnllKbHVihGcDqA==
---
Fix up this retry while we investigate [2]
[1] http://logs.openstack.org/64/141864/1/check//check-tempest-dsvm-centos7/4308f0c/logs/devstacklog.txt.gz
[2] https://bugzilla.redhat.com/show_bug.cgi?id=1144100
Change-Id: I11fb3728e08adc1e0f7acca63e5a308d24dce78e
Newer versions of rabbitmq (3.3 and later) do not allow the 'guest'
user to access on non-local interfaces.
- Added a new config RABBIT_USERID which defaults to stackrabbit
- Invoked config scripts using that variable
Adopted from:
https://review.openstack.org/#/c/107779/
Change-Id: I43a231c9611b4cc2e390b603aa3bfb49c915bdc5
Closes-Bug: #1343354
Co-Authored-By: Scott Moser <smoser@ubuntu.com>
With gerrit 2.8, and the new change screen, this will trigger syntax
highlighting in gerrit. Thus making reviewing code a lot nicer.
Change-Id: Id238748417ffab53e02d59413dba66f61e724383
If QPID_USERNAME is set, add the user to the QPID broker's
authentication database. Use the value of QPID_PASSWORD as the
password for the user, prompting for a password if QPID_PASSWORD is
not set. This requires that all clients provide this username and
password when connecting to the QPID broker, or the connection will be
rejected.
If QPID_USERNAME is not set (the default), disable QPID broker
authentication. This allows any client to connect to the QPID broker
without needing authentication.
Change-Id: Ibd79873379740930ce5f598018c1ca1fffda7c31
Closes-Bug: 1272399
The python-qpid package is available for Ubuntu trusty, precise, and
all the supported RHEL based platforms. This package is necessary if
qpidd is configured as the RPC backend. It is the client API used to
talk to the broker, and must be installed on each system that
communicates with the broker.
Change-Id: I635d3e857aa4b769a80cb7cde405cfd6cae44d32
Part 3 of a series
Re-order the setup and check bits in the top portion of stack.sh to
have a logical flow with similar things done together.
No behaviour changes are intended aside from the order of execution.
Any such changes are bugs.
* Move logging and error configuration earlier to cover initial project setup.
Change-Id: Ib16bbe20f224b1cf5e86c7a2fda0d9472c108873
This change adds the RPC_MESSAGING_PROTOCOL configuration option that
selects the messaging protocol that is used by the RPC backend and
client.
Some brokers can support different kinds of 'on the wire' messaging
protocols. Qpid, for example, supports both AMQP 0-10 (the default),
and AMQP 1.0. Use the RPC_MESSAGING_PROTOCOL configuration variable
to override the default protocol for those brokers that support
multiple protocol options.
This new option is necessary in order to enable the new AMQP 1.0
oslo.messaging transport as described in the blueprint.
Note well: currently this AMQP 1.0 functionality is only available on
fedora 19+ platforms. Support is WIP on ubuntu/debian and rhel/centos
7. Enabling the RPC_MESSAGING_PROTOCOL option on an unsupported
platform will cause devstack to exit with an approriate error
message.
Change-Id: Ib8dea59922844e87d6c947b5dca557f5b5fc1160
Implements: blueprint amqp10-driver-implementation
The original workaround was for an init script that grabbed stdout.
This was some time ago and I think it's safe to remove.
It can be a problem on Fedora; out-of-the-box some old F20 images can
have a broken dependency when rabbitmq installs. With this in place,
all helpful output goes into the temp log file, but it is never shown
because the install_packages function exits the script. Thus things
just stop and you have no idea why.
Change-Id: I2f5b934492a8c9d7b93e89fdcfa776bf15f25cb8
This allows for easy client configuration against clustered RabbitMQ
setups. Does not break existing configs.
Change-Id: I2b180f8860a727e35d7b465253689e5e8c44eb98
Closes-Bug: 1286411
Check that function calls look like ^function foo {$ in bash8, and fix
all existing failures of that check. Add a note to HACKING.rst
Change-Id: Ic19eecb39e0b20273d1bcd551a42fe400d54e938
Removes the dependence with aptitude by replacing
the call of:
aptitude purge -y ~npackage by apt_get purge -y package*
Change-Id: I08875ffad9dc6293047827666f02453a355b16ea
Closes-Bug: 1281410