56 Commits

Author SHA1 Message Date
Davanum Srinivas
aa33c878d8 Fix Cleanups for keystone/placement/nova
* Check KEYSTONE_DEPLOY flag and cleanup appropriately
* When we stop process, we should not wipe uwsgi config we should
  remove files only on cleanup
* We should not call cleanup *BEFORE* configure, we are just wiping
  out the uwsgi ini files
* cleanup_placement should be called from clean.sh

Change-Id: I066f5f87ff22d7da2e3814f8c2de75f2af625d2b
2017-08-17 10:40:38 -04:00
Markus Zoeller
15b0a5f1eb systemd: Always create the systemd unit files
Commit 5edae54 introduced the usage of systemd in Devstack. This allowed
the transition away from 'screen'. Systemd needs "user unit files" to
describe the services. Currently, those unit files get only created when
an openstack service (n-cpu, c-sch, g-api, ...) is in the list of enabled
services (`ENABLED_SERVICES`). This means, when Devstack is fully stacked,
there is no way to start the systemd unit of an openstack service which
is *not* in that list.

This commit changes that behavior, and creates the systemd unit files
independently of the list ENABLED_SERVICES. This means, when Devstack
is fully stacked, I can start a systemd unit of an openstack service which
wasn't in the ENABLED_SERVICES list. This allows more flexible lifecycle
management of openstack services in the gate, which is useful for tests
which test components which are not in the "default configuration" (e.g.
the "nova-serialproxy" service).

The `clean.sh` script purges all traces of systemd user unit files created
by devstack.

Change-Id: I0f7e1ee8723f4de47cbc56b727182f90a2b32bfb
2017-06-14 14:29:39 +10:00
Sean Dague
eaadffe07b Always have ./clean.sh run ./unstack.sh
There was detection code in clean.sh to only run it if a screen
session was found, but in systemd world, that's obviously not
true. This was causing me (and others) substantial confusion.

Change-Id: I204e94cd86b8c67012aabfca74796e593151c3a4
2017-05-04 16:05:19 -04:00
Victor Morales
b612b6281a Use delete action for clean up *.pyc files
Findutils added in release 4.2.3 a new --delete action for deleting
matching files. This action performs better than -exec rm {} \;
because it doesn't have to spawn an external process.  This change
uses a new action whenever is possible.

Change-Id: Iff16a86b18e924cfe78ac7c6107910940ce51e03
2017-01-13 09:16:42 -06:00
Jenkins
77a7296248 Merge "Remove heat code from devstack tree" 2016-12-06 13:36:52 +00:00
Jenkins
25dd43b881 Merge "Remove all *.pyc files in $DEST when executing clean.sh" 2016-11-28 23:45:16 +00:00
xiaolihope
8f985b6ff1 Remove heat code from devstack tree
This removes all of the heat code from the devstack tree, in favor of the
devstack plugin in Heat's tree.

Depends-On: I4bed1e5cef5afa7b049b07640086a86a3f881e13
Depends-On: Ic392bcc24bc374ee8511a94f1d8f6ac23131c7e3

Change-Id: I5b60422bf1f5fa78aa8f3383f7a222e0356d9e42
2016-11-15 15:00:43 +05:30
Chris Dent
4d6017566a Add support for placement API to devstack
Uses lib/placement, but relies on some functionality from
lib/nova. This leads to some weirdness since the nova has
special status in stack.sh. If/when placement is extracted
it may be good to follow the devstack plugin structure
instead.

Because the placement code is currently a part of nova, there
are dependencies in lib/placement on a some $NOVA_* variable
and, if virtenv is being used, the virtualenv used by nova.

Because placement currently runs using nova's configuration
settings, not a lot actually happens in lib/placement: apache
is configured and keystone accounts and endpoints are created.

If PLACEMENT_DB_ENABLED is true then a separate placement db
will be configured.

When complete the initial version of the placement service will
provide support for managing resource providers, inventories and
allocations.

The placement api only runs under mod-wsgi.

Change-Id: I53dd3e6b41de17387a0e179fc9ac64c143b6a9eb
2016-08-30 12:49:47 +00:00
Rob Crittenden
5631ca5e12 Clean up Horizon Apache configuration files in clean.sh
The horizon cleanup function wasn't being called at all during
cleanup which left the Apache configuration.

Change-Id: Iff5336d0c5e79cfc82f1c648afaabb869d86020e
2016-08-02 15:28:19 -04:00
WenyanZhang
8f0e97c146 Remove all *.pyc files in $DEST when executing clean.sh
Currentlly, the *.pyc files could not be removed in any scripts or
functions. But the redundant files would lead stack.sh not to find the
correct script for some versions after branch switched from master to
stable/mitaka in migration_helpers.sync_database_to_version.

So this commit adds the process of cleaning all the *.pyc files in
clean.sh.

It is needed to execute clean.sh before re-stack.sh to prevent the
exception.

Change-Id: I9ba0674d6b20b13c0a26b22cd5d1939daa121a94
Closes-Bug: #1599124
2016-07-29 09:12:55 +08:00
Sean M. Collins
2a242519f7 Begin new lib/neutron
Background for this work can be read on the mailing list:

http://lists.openstack.org/pipermail/openstack-dev/2016-May/094063.html

Usage of the new Neutron is by setting the following in
ENABLED_SERVICES:

* neutron-api
* neutron-l3
* neutron-agent
* neutron-dhcp
* neutron-metadata-agent

For now, the new neutron library supports just the ML2 plugin, with the
Open vSwitch and Linux Bridge agents supported. All other Neutron
plugins should be creating their own DevStack plugin if they wish for
DevStack to support them. Many of them already do.

Other notable changes compared to neutron-legacy:

* Rely on the Neutron defaults, and force Neutron to make
  sane defaults instead of all kinds of knobs in DevStack.

* Default to rootwrap daemon support

* Use the security group driver by default

* interface_driver can now use NEUTRON_AGENT (linuxbridge, openvswitch), since
  they are entrypoints in neutron's setup.cfg

* Use NEUTRON_AGENT variable to determine which agent to run
  Works with NEUTRON_AGENT set to either "linuxbridge" or "openvswitch"
  Default is openvswitch for the time being.

* Set ML2 configuration for VXLAN support

* Remove Xen hypervisor stuff - it should be a plugin

* Move L3 crud into separate service file:

  There's a lot of L3 configuration that was in the main neutron file, but
  a lot of it is self contained and can be moved into its own file.

  The new l3 service file will contain all the previous L3 plumbing and
  configuration that the OpenStack Gate expects, while also eventually
  moving the whole l3 network creation step into a single hook that can be
  overridden by plugins.

* Introduce a check for a function "neutron_plugin_create_initial_networks" which
  will become the mechanism through which different topologies, and
  networking plugins can create and wire the initial networks that are
  created during a stack.sh run.

The new lib/neutron is considered experimental, and followup patches
will build upon this one. Existing users of lib/neutron-legacy should
remain unharmed.

Co-Authored-By: Hirofumi Ichihara <ichihara.hirofumi@lab.ntt.co.jp>
Co-Authored-By: Dean Troyer <dtroyer@gmail.com>
Change-Id: I31b6362c6d9992f425f2dedbbeff2568390a93da
2016-05-09 14:26:08 -04:00
ZhiQiang Fan
0b4a009e56 remove clouds.yaml in clean.sh
devstack failed to install because glance:
Could not determine a suitable URL for the plugin

patch I618ea8e27b49af360c905df85af06d9b1eef8407 tries
to fix this problem, but with a wrong way because path is not
correct, the clouds.yaml is not under /path/to/devstack/~/.config/openstack/
but ~/.config/openstack.

patch I8af6bd465f74099c560dddba6b5221dd79cbc965 tries to
fix this problem, but with a worng way to specify the path,
~$STACK_USER/.config/openstack/clouds.yaml will not expand with
a variable, only const string can.

$ whoami
zqfan
$ touch ~/.config/openstack/clouds.yaml
$ export STACK_USER=zqfan
$ rm -rf ~$STACK_USER/.config/openstack/clouds.yaml
$ ls ~/.config/openstack/
clouds.yaml

Change-Id: I549817d2f4638be615991c1726b39d270ba71357
ref: I618ea8e27b49af360c905df85af06d9b1eef8407
2016-04-15 02:27:35 +08:00
Jenkins
5073896cc9 Merge "Only use lsb_release for distro tags" 2016-02-14 14:42:43 +00:00
Sean Dague
e73f88e70b tweak ps4 for readability
After staring at a bunch of logs, try to crisp up the ps4 output for
maximum readability.

This also adds PS4 to all calling scripts by having a common PS4 in
stackrc. It should make understanding when clean fails a bit more
straight forward.

Change-Id: Ia1f8dff5c1102c32c146a020a9f033c65d2c50de
2016-02-04 07:47:45 -05:00
Steve Martinelli
8e3df96cec use sudo to remove /etc/openstack directory
the /etc/openstack directory needs to be removed with sudo privs,
the loop at the end will not suffice.

Change-Id: Icc0ac45f9216d538ca214176d90241f973a4687b
2016-01-18 14:00:41 -05:00
Steve Martinelli
16246acb4c remove clouds.yaml files when running clean.sh
There have been a few reported errors of folks running into
issues with old clouds.yaml files that linger around. Remove
these if the user runs clean.sh.

1. http://lists.openstack.org/pipermail/openstack-dev/2016-January/084283.html
2. https://ask.openstack.org/en/question/84825/glance-upload_image-fails-in-latest-devstack/
3. https://ask.openstack.org/en/question/86071/devstack-installation-failure/

Change-Id: I618ea8e27b49af360c905df85af06d9b1eef8407
2016-01-18 01:20:53 -05:00
Ian Wienand
7710e7fc27 Only use lsb_release for distro tags
The existing GetOSVersion has a lot of unused code which is wrong in
several ways

 - the only path tested in upstream CI is with lsb_release, because
   it's pre-installed on all nodes

 - the /etc/redhat-release checking probably still works, but is
   unnecessary

 - If using lsb_release, os_UPDATE has never actually been set.

 - the /etc/SuSE-release branch checking is broken if the lsb package
   is actually installed.  lsb checking does not set os_UPDATE but yet
   the SuSE DISTRO setting relies on this to set a patch level (and so
   does some of the rpm tags).  SuSE 11 is up to update 3, but the rpm
   matching is stuck hard-coded to update 2.  I'm guessing
   installation is actually broken there.

 - the debian checking branch is broken.  The VERSION tags have been
   removed and were not supposed to be relied on anyway (see notes in
   [1])

This simplifies things:

 - remove OSX checking (moved here after discussions in
   I31d0fdd30928ecc8d959a95838b1d3affd28ac6f)

 - only use the output of lsb_release.

 - A small best-effort check to pre-install lsb packages if not
   detected (that avoids chicken-egg-problem of package-install
   wrappers relying on os_* flags).

 - The unset os_UPDATE is removed.  It's only previous use was for
   setting separate suse versions in the DISTRO element for matching
   during package installs (since removed)

 - DISTRO setting is modified to use the parts of os_RELEASE it wants.
   Per-above, this is the correct place to parse out specifics.

 - Call out the is_* functions, which are a better way to detect
   platforms

 - Export the variables as read-only, since they shouldn't be reset

[1] http://sources.debian.net/src/base-files/7.5/debian/changelog/

Change-Id: I46a2c36d95327087085df07cb797eb91249a893c
2016-01-12 09:23:35 +11:00
Jim Rollenhagen
47367071cd Remove ironic code from tree
This removes all of the ironic code from the devstack tree, in favor of
the devstack plugin in Ironic's tree.

Depends-On: I659e3de5c64df798441798ff48ba5c9c0506585a
Depends-On: I2c52bc014f1b0dbc6b0ae22a4deb4132b4c28621
Change-Id: I5125fce295c79600781469c2f48bea80e7600081
2015-12-18 17:26:02 +02:00
Ian Wienand
975f4209d0 Save interactive passwords to separate file
The interactive password prompt currently saves to .localrc.auto

However, this is removed when you re-run stack; that is required as it
is how we source the localrc bits of local.conf, and we want the
users' changes to be picked up.

The passwords, however, should remain constant, because everything has
already been setup with them.  So write them to a separate file.  Note
we source before localrc so it can still overwrite them.

Some minor flow-changes too

Change-Id: I9871c8b8c7569626faf552628de69b811ba4dac0
Closes-Bug: #1505872
2015-10-15 10:51:05 +11:00
Chris Dent
e9a4750fe1 Remove ceilometer in favor of plugin
The ceilometer project is moving to using a devstack plugin rather
than having ceilometer in the base devstack. This is to allow
greater control and flexibility.

Change-Id: I413ab159474b7d7231ad66d3a482201f74efe8a8
2015-09-09 16:44:29 +00:00
Mahito
67168e807a Add "source $TOP_DIR/lib/lvm" to clean.sh
When clean.sh is executed, it shows "command not found" messages.
Commands are defined in lib/lvm, however lib/lvm doesn't include clean.sh.
This pache add lib/lvm to clean.sh.

Change-Id: I56672e949d25f7cdcda879badd992f849d06c749
Closes-Bug: 1486392
2015-08-18 23:59:29 -07:00
Jenkins
eb38bd7602 Merge "Write out a clouds.yaml file" 2015-05-21 16:01:08 +00:00
Monty Taylor
61045ca58a Write out a clouds.yaml file
os-client-config consumes clouds.yaml files, which is now supported in
python-openstackclient and shade. It also makes for a non-envvar way of
getting config info into functional tests.

Change-Id: I1150b943f52f10d19f8434b27e8dde73a14d7843
2015-05-14 13:58:19 -04:00
James E. Blair
9e220b9b2b Move trove into in-tree plugin
Once the trove code is copied into the trove repo and it is used
as a devstack-plugin, we can remove trove-specific code from
devstack.

Change-Id: I8f9f1a015edb7ec1033e2eaf0b29ab15d89384ce
Depends-On: I3506dec0e6097f9c2e9267110fdfb768faa23c85
2015-05-14 07:57:23 -07:00
Jenkins
cb88e2b7cb Merge "clean.sh needs to load plugin settings." 2015-04-16 19:19:05 +00:00
ajmiller
e6843e5ea1 clean.sh needs to load plugin settings.
Change-Id: Id957f585d2aa93075b138d462d6076d2d70d450e
2015-04-13 08:16:30 -07:00
Davanum Srinivas
93d09c24e3 Setting LOGFILE to a file in root directory wipes everything
clean.sh picks the parent of LOGFILE and wipes it clean! So if you
set it to a log file in the users root directory, you lose everything

We should delete just the LOGFILE and cleanup LOGDIR and SCREEN_LOGDIR if
they are explicitly set.

Change-Id: I45745427dcaed3dcf0b78cc9ed680833d9d555e8
2015-04-11 18:45:17 -04:00
Dean Troyer
5a9739a4ca Rename lib/neutron to lib/neutron-legacy
Preparing to refactor lib/neutron to support Neutron as the default
network config.  lib/neutron will be renamed internally and refined
to support a couple of specific configurations.

Change-Id: I0d3773d14c4c636a4b915734784e7241f4d15474
2015-03-25 22:22:11 -04:00
Al Miller
a1701fabcf clean.sh needs to call "run_phase clean" for external plugins
Change-Id: I67b970992479e50dc054f8c4a77a20e724e3e305
2015-02-20 08:22:25 -08:00
Dean Troyer
8c2ce6ea72 Virtual environment groundwork
Introduce the tooling to build virtual environments.

* tools/build_venv.sh: build a venv
* introduce lib/stack to house functionality extracted from stack.sh that
  is needed in other places, such as Grenade; start with stack_install_service
  to wrap the venv install mechanics
* declare PROJECT_VENV array to track where project venvs should be installed
* create a venv for each project defined in PROJECT_VENV in stack_install_service()

Change-Id: I508588c0e2541b976dd94569d44b61dd2c35c01c
2015-02-18 20:19:38 -06:00
Dean Troyer
b1d8e8e274 Build wheel cache for venvs
Building a bunch of virtual envs later is going to be tedious if we do not
pre-cache certain annoying-to-build packages.

* tools/build_wheels.sh: pre-build some wheels for annoying package installs
* list distro package dependencies in files/*/venv
* list packages to pre-build as wheels in files/venv-requirements.txt
* install database Python modules when setting up the database

Change-Id: Idff1ea69a5ca12ba56098e664dbf6924fe6a2e47
2015-02-18 15:13:00 -06:00
Sean Dague
9a413abcd4 add gating up/down script for devstack
This adds the test infrastructure for testing that unstack.sh and
clean.sh do the right thing, and actually stop what's expected. This
is designed to be used in upstream testing to make unstack and clean a
bit more certain.

It includes numerous fixes to make these pass in an errexit
environment with the gate config. The scripts still don't run under
errexit because we don't assume we've handled all possible cleanup safely.

Change-Id: I774dfb2cc934367eef2bb7ea5123197f6da7565b
2015-02-11 06:10:38 -05:00
Sean Dague
537532931d Make changes such that -o nounset runs
This makes a bunch of variable cleanups that will let -o nounset
function, for the time being we hide nounset behind another setting
variable so that it's not on by default.

Because this is bash, and things are only executed on demand, this
probably only works in the config it was run in. Expect cleaning up
all the paths to be something that takes quite a while.

This also includes a new set of unit tests around the trueorfalse
function, because my change in how it worked, didn't. Tests are good
m'kay.

Change-Id: I71a896623ea9e1f042a73dc0678ce85acf0dc87d
2015-01-15 13:06:14 -05:00
Sean Dague
2f8e08b572 remove nova baremetal driver support
This has been deleted from the nova tree, we should purge it from
devstack, as it will not work any more.

Change-Id: I20501fec140998b91c9ddfd84b7b10168624430a
2014-12-05 08:31:16 -05:00
Ihar Hrachyshka
a89a0dc624 Revert "cleanup potentially installed older oslo.config"
Icehouse is for long behind our back, so let's remove that hack.

Conflicts:
	lib/oslo

This reverts commit db5fadb5cb768820df54fc3d1c7428a57b511582.

Change-Id: I06d3b0a8779ba51e05c439832ef3b7dbdc97ded1
2014-07-29 13:13:12 +02:00
ZhiQiang Fan
ecd05637a3 Do not clean docs directory
The docs/source has been introduced by change-id:
Ide5f7d980a294c7a9f8a3decaed0939f1c239934, but clean.sh try to clean
whole docs direcotry, which is not right.

Meanwhile, there are directories in FILES_TO_CLEAN variable, which means
rm -f will not work. We need to use rm -rf.

Change-Id: I4e10d70d5638ce7e7a9215f5bd848422aa90aad3
Closes-Bug: #1335399
2014-06-30 09:27:31 +08:00
Gary Kotton
b00e4181a8 Ensure neutron configuration files are cleaned by clean.sh
Add the /etc/neutron directory to the list of directories
that are deleted.

Change-Id: Ia560179a3cf121f8d09a0c10c8b68375fdd13be7
2014-05-01 05:16:59 -07:00
lokesh
559145253d Closes-Bug:1313755 adding code to remove heat conf files
Change-Id: Iefe1a34b09958d21d2516fdce0937e00e32afcb5
2014-04-28 20:48:09 +05:30
Sean Dague
59d9cecb16 bring back oslo_clean
oslo_clean is still needed at this point, removing it was
premature, especially for upgrade testing.

Change-Id: Ic845d835f587923423f83ac698bd825f3fa5dd1f
2014-04-02 18:54:51 -04:00
Sean Dague
099e5e3f81 don't use pip -e install for libraries
libraries in openstack shouldn't be installed editable, as it
causes all manner of issues (especially complicated by the use
of namespace packages). Install these globally as part of the
devstack installation process.

Change-Id: I11acb169e74069be0618e57496ff342f9e788493
2014-04-01 08:48:00 -04:00
Shashank Hegde
cb415697f3 clean.sh removes all the files
clean.sh was incorrectly looping over the list of files to remove. Because of
this the files were not being removed.

Change-Id: Ie0559e1d396a4d35df6a12dfbceefa7eb261bac5
Closes-Bug:1285924
2014-02-27 16:50:05 -08:00
Jenkins
abcc36e121 Merge "reset prereqs status on clean.sh" 2014-02-26 01:09:14 +00:00
Sean Dague
2e2b28b531 reset prereqs status on clean.sh
when running a clean, we should really reset the prereq status as
well, as this should start us back from zeroish.

Change-Id: I5fae151ab13bcf7fb82feb1e91eed19e0215dc59
2014-02-24 06:45:17 -05:00
Masayuki Igawa
1958c1eb5e Remove unnecessary comment out lines
This commit removes some comment-outed codes. If we want to use them, we
can get them from the git repository.

Change-Id: Ie438c43d332d0631750f0ad458653fc40e23faad
2014-02-20 14:35:44 +09:00
Dean Troyer
e4fa721322 Begin is_service_enabled() cleanup
This converts the special cases in the is_service_enabled() function to call
individual functions declared by the projects.  This allows projects that
are not in the DevStack repo and called via the extras.d plugin to handle
an equivalent service alias.

* Ceilometer
* Cinder
* Glance
* Neutron
* Nova
* Swift

TODO: remove the tests from is_service_enabled() after a transition period

Patch Set 2: Rebased

Change-Id: Ic78be433f93a9dd5f46be548bdbd4c984e0da6e7
2014-02-07 10:06:21 -06:00
Dean Troyer
53ffc713b1 clean.sh updates
* Clean out data, log and state dirs
* Include lib/apache to clear is_apache_enabled_service not found error
* Clean errors removing tgt config files

* Clean errors removing VG backing file in lib/cinder

Change-Id: I33dfde17eb8daaaed7f7e76337fe6a8085a266bf
2014-01-25 13:26:58 -06:00
Dean Troyer
b9e25135c5 freshen the LDAP support
* Build the base DN from a given domain name
* Remove all hard-coded names to allow configuration of base DN
* Fix manager DN (cn=Manager,dc=...)
* Add ldap init_ldap()
* Add support for clean.sh

Change-Id: Ieb69be9740653645b8e000574ad3fe59a0f97540
2013-11-26 11:28:54 -06:00
Dean Troyer
cdf3d76647 Add stack phases to extras.d handling
Add hooks to stack.sh, unstack.sh and clean.sh to call the extras.d
scripts at multiple points in stack.sh.  This allows these scripts to
perform installation and startup tasks at similar times as they would
if integrated into stack.sh.

extras.d/70-tempest.sh is present as an example of the structure
of these scripts.

See extras.d/README.md for more information.

Change-Id: Ic1fe522559b94d204d6c0319a2e3d23684c8d028
2013-10-15 16:13:47 -05:00
Sean Dague
d02287e5e6 import lib/oslo so we can clean it up
we were calling cleanup_oslo, however we weren't importing
lib/oslo, so that was just throwing an error message and moving on.

Let's stop doing that and actually clean up oslo.

Change-Id: I48340a8b3d5b50477fb5a7e2ce0bed27deb3ec01
2013-09-11 14:08:59 -04:00
Dean Troyer
2aa2a89cdb Add support for Docker as Nova hypervisor
* Add basic support for hypervisor plugins in lib/nova_plugins
* Add lib/nova_plugins/hypervisor-docker to use Docker as a Nova
  hypervisor.
* Add tools/install_docker.sh to install the Docker daemon and
  registry container, download base image and import
* Configure Nova to use docker plugin
* Add docker exercise and skip unsupported ones

Nova blueprint: new-hypervisor-docker

Change-Id: I9e7065b562dce2ce853def583ab1165886612227
2013-08-28 22:36:48 -05:00