In the "DevStack Component Timing" section, display the unaccounted
time.
Also add the units (seconds) to the output to make it clear to viewers.
Change-Id: Iaca82cc54a355f7077e20e548b771e53387f6628
https://review.openstack.org/504171 prevented Python version detection
from failing when python3 is not installed. However, "which python3"
returns a message in stderr when python3 is not there, and this output
can make diskimage-builder get confused when parsing
source-repository-images.
Change-Id: Idb649dc341ede73c39954b0432ef3cacf379ed37
On platforms without python3 installed devstack fails the python
version check. This does it more gracefully.
Change-Id: I4d79a41eb2d66852ab1a1aa8bd383f8e3e89cdc0
lib/neutron service prefixes are neutron-* not q-*. We should install
those packages either way.
The patch moves files/*/neutron into files/*/neutron-common so that we
can correctly match */neutron against service specific dependency files
(f.e. */neutron-agent) and load the common packages if any neutron-*
service is present.
Change-Id: I57b36f2ed3f33737223a35d9ed734bb414f31e0b
1] Process using uwsgi:
uwsgi services doesn't support for graceful shutting down [1].
It requires some changes in unit files [2] including adding below
graceful shutdown hook and changing KillSignal:
--hook-master-start "unix_signal:15 gracefully_kill_them_all
All the steps and changes required are specified in etherpad [1].
2] Non uwsgi services needs below changes:
In [service] section:
a. Add KillMode = process
b. Add TimeoutStopSec = infinity
NOTE:
Creating unit file for services other than uwsgi is handled by the
'write_user_unit_file' function [3]. This function is common for all
the services so this patch adds the above mentioned parameters for
services using ServiceLauncher also though they don't require.
Added a new stackrc variable WORKER_TIMEOUT which is required to add
graceful shutdown support to uwsgi services. It will be set as a value
to 'worker-reload-mercy' [4] in uwsgi file of service. The default
value set to this variable is 90.
[1] https://etherpad.openstack.org/p/uwsgi-issues
[2] https://www.freedesktop.org/software/systemd/man/systemd.kill.html
[3] 2967ca3dfd/functions-common (L1439-L1461)
[4] http://uwsgi-docs.readthedocs.io/en/latest/Options.html#worker-reload-mercy
Co-Authored-By: Dinesh Bhor <dinesh.bhor@nttdata.com>
Change-Id: Ia95291325ce4858b47102dd49504250183f339ab
This tears out the alternative path of using screen, so that we only
use systemd enabled paths. This simplifies the number of ways that
devstack can be run, and provides a much more reliable process
launcher than the screen based approach.
Change-Id: I8c27182f60b0f5310b3a8bf5feb02beb7ffbb26a
Query the python2/python3 interpreter for it's version to fill in
PYTHON3_VERSION and PYTHON2_VERSION defaults. This means on a
python3.6 platform such as Fedora 26, we don't need to override the
default.
Change-Id: Id826f275b99b9f397b95e817941019fc503daa1d
Add a function which tests if a plugin has been enabled with
enable_plugin. This is helpful if two co-ordinating projects want to run
specific setup in devstack in one only if the other is enabled.
Change-Id: Ibf113755595b19d028374cdc1c86e19b5170be4f
This reverts commit 15b0a5f1eb.
The change had an assumption that a service
is properly configured even when it isn't enabled.
The assumption is not true.
Change-Id: Ib5a8ffe63eaec15bc29bfdd133db7169507bab82
Closes-Bug: #1698129
The OSC number remain high, and it's useful to understand how much
time we spend making OSC calls, especially to surface it relative to
other items. The way we embed this in our code makes it hard to
instrument.
This patch creates a wrapper function for OSC which collects the timings
then aliases `openstack` to that function. This means any invocations of
the openstack utility goes through our function while devstack is
running. Because this is an alias it only affects the stack.sh shell and
any subshells.
This also moves the time tracking infrastructure to count in ms,
instead of s, because some of these operations are close enough to a
second that rounding early is losing way to many significant
digits. We divide by 1000 before reporting to the user.
Change-Id: Ic5f1844ce732d447ee980b3c9fdc417f72482609
In the helper functions to check if roles are set and if not add the
role and return the id we weren't actually checking if the role was set.
The reason for this was we grepped for name values while outputing only
uuid values with OSC. Fix for this is straightforward, we just add the
--role argument to OSC which will filter for us then we don't have to
use a grep on the wrong value type.
Change-Id: I2691b347d2a6273100deb4a1750ab353a8e49673
Commit 5edae54 introduced the usage of systemd in Devstack. This allowed
the transition away from 'screen'. Systemd needs "user unit files" to
describe the services. Currently, those unit files get only created when
an openstack service (n-cpu, c-sch, g-api, ...) is in the list of enabled
services (`ENABLED_SERVICES`). This means, when Devstack is fully stacked,
there is no way to start the systemd unit of an openstack service which
is *not* in that list.
This commit changes that behavior, and creates the systemd unit files
independently of the list ENABLED_SERVICES. This means, when Devstack
is fully stacked, I can start a systemd unit of an openstack service which
wasn't in the ENABLED_SERVICES list. This allows more flexible lifecycle
management of openstack services in the gate, which is useful for tests
which test components which are not in the "default configuration" (e.g.
the "nova-serialproxy" service).
The `clean.sh` script purges all traces of systemd user unit files created
by devstack.
Change-Id: I0f7e1ee8723f4de47cbc56b727182f90a2b32bfb
Now that we aren't using native journal messages by default, the
syslog identifier of all the uwsgi processes is the same. We should be
more explicit with those.
Change-Id: Id5406d02407b022d4016517c2e18890973876d88
There are a couple of issues that have ended up being hit by devstack
plugin authors which we can detect and error in a much nicer way
instead of them having a cryptic systemd failure.
Change-Id: I45e4ac363aeefb4503015f9e1b57c58f79b58f40
When transitioning between different wsgi modes, or service modes, we
should really safely stop and cleanup things that are started in any
service mode, which makes it easier to ensure that we don't leave
things around from past runs.
Change-Id: I33acbee39e1a2da2bfd79a5dd54b84a12a778be1
In order to start making the transition in the gate make
USE_SCREEN=False also mean USE_SYSTEMD=True. We'll never actually
declare USE_SYSTEMD=True in the gate (as that doesn't exist for stable
branches), but this will let us roll over the existing transition.
We also have to install systemd-python 234 because we are recording
exception info in the journal, and all versions before that had a bug
in processing that.
Remove the somewhat pointless screen following journalctl commands. We
really don't want or need those, and they tend to build up over time.
Depends-On: I24513f5cbac2c34cf0130bf812ff2df6ad76657c
Change-Id: I6af6d1857effaf662a9d72bd394864934eacbe70
KEYSTONE_SERVICE_API is the keystone endpoint and it is what we should
use. The admin url should DIAF - but until it does, it CERTAINLY should
not be the thing we put into clouds.yaml.
Change-Id: If8196a04f852f633e0b7548793f68c92376aa6da
It turns out that we ended up with duplicate versions of this function
merging on top of each other within 3 days, and gerrit didn't catch
it. Boo gerrit. Boo bash.
Change-Id: Ic6aa2f9bafdec906de2bc51d5929beeec48a6a40
Things like SERVICE_PROTOCOL and KEYSTONE_AUTH_PROTOCOL shouldn't
really be exported in openrc as they encourage using them directly to
build up keystone urls instead of actually using the OS_AUTH_URL.
Remove them.
Change-Id: I4b7cc680f7f14dae29b706a227be540c9e212cad
uwsgi is a different service type under systemd and shouldn't be run as
a standard oneshot type. The uwsgi docs outline a good pattern for
writing systemd unit files:
http://uwsgi-docs.readthedocs.io/en/latest/Systemd.html
This commit takes those suggestions and creates a separate path for
writing uwsgi unit files.
Change-Id: I9b541b86781afdded311dba058cedd783e1a0dfa
During the PTG there was a discussion that the screen developer
workflow wasn't nearly as useful as it once was. There were now too
many services to see them all on one screen, and one of the most
common service restart scenarios was not restarting one service, but a
bunch to get code to take effect.
This implements a 3rd way of running services instead of direct
forking via bash, or running under screen, which is running as systemd
units.
Logging is adjusted because it's redundant to log datetime in oslo.log
when journald has that.
Swift needed to have services launched by absolute path to work.
This is disabled by default, but with instructions on using it. The
long term intent is to make this the way to run devstack, which would
be the same between both the gate and local use.
Some changes were also needed to run_process to pass the run User
in. A hack around the keystone uwsgi launcher was done at the same
time to remove a run_process feature that only keystone uwsgi uses.
Change-Id: I836bf27c4cfdc449628aa7641fb96a5489d5d4e7
The package libvirt-bin is a transitional package in Debian and should
not be used anymore.
Ubuntu Xenial is an exception here.
Because of that this change also adds the possibility to use "not:" to
exclude distros in files/debs/* just as "dist:" limits distros.
Depends-On: Icc59ea79f54d4ff8751f2e353ee3530fff3d961e
Closes-Bug: #1673840
Change-Id: I3998a7178d14ec40eae5cb199d66da9546cd6ccf
When variables use the 'declare' directive, it is by default a local
variable. While other variables have global scope.
For example:
declare -A AN_ARRAY # local in scope
foo=1 # global in scope
This causes errors to occur as some of the variables will be local only
and others will be global.
Update the code, as appropriate, so that variables using the 'declare'
directive also include the '-g' flag to have them also be global. Not
every instance of a declared variable has been updated.
Closes-Bug: #1669509
Co-Authored-By: John L. Villalovos <john.l.villalovos@intel.com>
Change-Id: I2180b68fe861ad19c6d4ec0df0f9f8a528347862
Nova is going to increase the minimum required libvirt
in Pike to 1.2.9 in change:
I9a972e3fde2e4e552f6fc98350820c07873c3de3
Based on the libvirt distro support matrix wiki [1] that
drops support for Ubuntu Trusty and Debian 7.0/Wheezy.
Trusty has libvirt 1.2.2 and Wheezy has 0.9.12 (the Wheezy
support should have been removed long ago apparently). The
7.0 removed here is for Wheezy also based on commit
b2ef890db3.
This does not undo the check for "trusty" with the
EBTABLES_RACE_FIX in lib/nova_plugins/function-libvirt
since you can still force devstack to run on Trusty if
you specify the FORCE=yes variable.
Note that RHEL 7.1 has libvirt 1.2.8 so it won't technically
work with devstack and nova + pike + libvirt, but with the
way os_RELEASE is calculated the minor version is dropped
for RHEL distros so we just get "rhel7".
Also note that this doesn't attempt to continue supporting
Trusty or Wheezy if nova is not configured to use libvirt,
simply in order to start moving forward on devstack distro
support in general and to keep some sanity and closeness
to what we test with in the CI system.
While we're in here, we also drop Fedora 23 and add
Ubuntu Zesty.
[1] https://wiki.openstack.org/wiki/LibvirtDistroSupportMatrix
Depends-On: I9a972e3fde2e4e552f6fc98350820c07873c3de3
Depends-On: If69f99bd789e646b0261e27a8a061efde32436f7
Change-Id: I6617283afd798af37e64913b7865cea3c8a62aba
The proliferation of internal/admin endpoints is mostly legacy and
based on some specific deployment patterns. These are not used by
everyone, and for the devstack case aren't really that useful. We
should simplify our service catalog down to the minimum we need for
development.
Change-Id: Ided7a65c81b3a0b56f0184847fc82e17c29a771e
When command failed and component failed to start, original exit code
was overwritten due to original command being executed in background.
This commit adds information about command's exit code to echoed message
about component's start up failure.
Change-Id: I8a3dd485b1b1f2d70d42c5610baac7c0c713f53a
Signed-off-by: Joanna Taryma <joanna.taryma@intel.com>
Services that run inside Apache use tail -f on the corresponding log
file to display output in the screen session. However, they also use sed
to replace some control characters, and this means that the output is
buffered. This results in debugging experiences that range from
"impossible" (the log you want isn't shown) to "Kafkaesque nightmare"
(the log you want isn't shown, except that sometimes it is, and
sometimes it isn't even though you double-checked and you're completely
sure that you must have output a log, but when you check back later you
realise it actually is and you wonder if history is actually not mutable
after all and begin to question what is real and what is not).
This adds the --unbuffered option to ensure streaming output.
Change-Id: I665ff5f047156401d8152f478d834ac40ff31658
