295 Commits

Author SHA1 Message Date
Lakshmi Prasanna Goutham Pratapa
297eb5e88f Add resource Constraints to Nova containers.
This commit will constrain the dimensions of service `Nova`
and sub-containers deployed along with it.

A user can give the dimension values in `/etc/kolla/globals.yml`
the data-types just like stated in this commit.

Reference-Docs:
https://docs.docker.com/config/containers/resource_constraints/

Added Test-cases for the same.

Partially-Implements: blueprint resource-constraints
Change-Id: I6458d8fb7b26a6e7c3a9fd0d674d9cf129b0bf5d
2018-07-12 23:24:05 +05:30
Zuul
fd3f54faed Merge "Add a job for testing deployment of bifrost" 2018-07-04 15:51:45 +00:00
Tone Zhang
3591d0fa9f Allow Kolla Ceph to deploy bluestore OSDs in Kolla-ansible
Support Kolla Ceph to deploy blustore OSDs with Kolla-ansible.

Please refer to [1] for bluestore OSD configuration

The patch includes:
1. Set Ceph OSD store type group_vars/all.yml. The default value
is "bluestore" in Rocky.

2. Make Kolla Ceph to deploy bluestore OSDs with Kolla-ansible

3. Update gate test configuration for Ceph bluestore OSD test

[1]: specs/kolla-ceph-bluestore.rst

Partially-Implements: blueprint kolla-ceph-bluestore
Depends-On: I00eaa600a5e9ad4c1ebca2eeb523bca3d7a25128
Change-Id: I14f20a00654dff32c36d078ebb9005d91a3e60b2
Signed-off-by: Tone Zhang <tone.zhang@arm.com>
2018-06-19 11:13:38 +00:00
Mark Goddard
45a4f9c075 Add a job for testing deployment of bifrost
Deploys a bifrost container using kolla-ansible bifrost-deploy.

IPA and disk images are downloaded rather than built to improve
reliability.

Currently only minimal testing of the deployment is performed, creating
and deleting an ironic node. Ideally we would perform a bare metal node
deployment.

The job is based on CentOS, as Ubuntu bifrost deployment is currently
failing with a python-MySQLdb error.

Change-Id: Ic45094594c21116b5b0d6a606f568fc7954175e3
2018-06-18 13:40:43 +01:00
Zuul
ffa5e57f20 Merge "Add support for enabling ipxe boot with ironic" 2018-05-31 11:37:52 +00:00
Mark Giles
41254b6c46 Add cAdvisor for Prometheus monitoring
cAdvisor (Container Advisor) provides metrics on resource usage and
performance characteristics of running containers.  This change
deploys a cadvisor container and configures prometheus to scrape
data from it.

Change-Id: I55dd4fee954f9be68efda397746861ddaaa0a565
Partially-Implements: blueprint prometheus
2018-05-29 08:55:58 -04:00
Will Szumski
0a1ccc2612 Add support for enabling ipxe boot with ironic
When enable_ironic_ipxe is set in /etc/kolla/globals.yml,
the following happens:

- a new docker container, ironic_ipxe, is created. This contains
  an apache webserver used to serve up the boot images
- ironic is configured to use ipxe

Change-Id: I08fca1864a00afb768494406c49e968920c83ae7
Implements: blueprint ironic-ipxe
2018-05-25 08:20:47 +00:00
Duong Ha-Quang
365e3d3a3f Apply Glance database migration
Apply Glance database upgrade procedure:

https://docs.openstack.org/glance/latest/admin/zero-downtime-db-upgrade.html

Rolling upgrade in Glance requires that glance-registry is not deployed and run

Depends-On: https://review.openstack.org/532128
Depends-On: https://review.openstack.org/566804

Change-Id: Ia5d0cee2385facefda9d423051b74ab51d075f13
Implements: blueprint apply-service-upgrade-procedure
2018-05-09 16:29:27 +00:00
ZhijunWei
585efc27ab Update the Ceph pgs value for tests
refer to https://review.openstack.org/#/c/564169/

Change-Id: I093b0b8fd3b34c9b35149c9e4ae945a8e0e67f53
2018-04-30 15:46:19 +00:00
Zuul
de1777134e Merge "Configure Infoblox as an IPAM driver for neutron" 2018-04-27 10:07:46 +00:00
Mathias Ewald
4d1f37359d Add role to deploy prometheus
This patch adds the ansible role to deploy the prometheus service which
can be used to collect performance metrics accross the environment

Partially-Implements: blueprint prometheus
Change-Id: I908b9c9dad63ab5c9b80be1e3a80a4fc8191cb9e
2018-04-19 10:58:15 -04:00
Gerry Buteau
2f69b3cbc6 Configure Infoblox as an IPAM driver for neutron
This patch allows configuration of the Infoblox
pluggable IPAM driver in neutron [0].

When 'infoblox' is chosen as the driver, an Infoblox
IPAM agent can be started as well.  The agent
allows for enhanced DNS capabilities by listening
for neutron and nova notifications.

[0] https://github.com/openstack/networking-infoblox/blob/master/README.rst

Change-Id: I4f863750a7806a7b6eaf13900d44e5f063afe3de
Depends-On: Ia44f0e0d7a0d60cebf0857ad51700e02eba5099b
Partially-Implements: blueprint neutron-ipam-driver-infoblox
2018-04-16 08:45:40 -04:00
Zuul
d2f5570823 Merge "Fix ipc_mode comparison in the latest docker" 2018-04-09 12:17:53 +00:00
wangwei
193ce726c2 Fix ipc_mode comparison in the latest docker
In old docker, if you do not specify ipc_mode, the default value is empty,
but in the latest docker, such as 17.09.0, if not specified, the default
is "IpcMode": "shareable", which will cause all containers to be deleted
and re-create when to redeploy or upgrade. This commit solves the
problem.

Change-Id: Ia8269b9c8066880e4aee23d6fdea8d9c04c41e44
Closes-Bug: #1747586
2018-04-06 13:35:43 +00:00
Zuul
21acfde7ba Merge "Introduce variable to allow different DVR modes" 2018-04-05 10:03:00 +00:00
Vladislav Belogrudov
1c1d6e20c1 Introduce variable to allow different DVR modes
This feature replaces splitting of compute hosts into inner
and external with a new variable 'neutron_compute_dvr_mode'
that controls whether computes will have full-blown DVR or
internal only (tenant) networking.

Change-Id: I6720ccfcfcec89f9996d4cb5ae60f31eb3113667
Implements: blueprint dvr-mode-property
2018-04-03 19:14:14 +03:00
Jeffrey Zhang
def7d35404 Implement ARA sqlite databases over http feature
More info please check[0][1].

[0] http://lists.openstack.org/pipermail/openstack-dev/2018-March/128902.html
[1] https://ara.readthedocs.io/en/latest/advanced.html

Change-Id: I5cf68c0bb54878e328387143712dec9582be04a1
2018-04-02 14:13:54 +08:00
Zuul
2f73a01828 Merge "Support parsing ini files with no sections" 2018-03-27 08:27:53 +00:00
Eduardo Gonzalez
ea1a1dee0d Verify YAML syntax in gates
This patchset implements yamllint test to all *.yml
files.

Also fixes syntax errors to make jobs to pass.

Change-Id: I3186adf9835b4d0cada272d156b17d1bc9c2b799
2018-03-26 17:56:22 +02:00
Doug Szumski
d32c708290 Support parsing ini files with no sections
Services such as Zookeeper and Kafka use ini files which do not
explicitly specify sections. This change supports merging ini
files with no sections, so that the configuration for these
services follows the example configuration provided for them
as closely as possible.

Closes-Bug: #1756101
Partially-Implements: blueprint monasca-roles
Change-Id: I1061729875e5545c7af7d80779f9c2124b6c7134
2018-03-16 15:26:41 +00:00
Zuul
52615a5355 Merge "Use zuul firewall rules in gate" 2018-03-11 03:44:31 +00:00
Paul Bourke
404d4d0a50 Use zuul firewall rules in gate
Till now we've been flusing iptables in the gates to allow cross node
communication in the multi node ceph jobs. This raised security
concerns, in particular it exposed memcached to the external net.

This patch uses the infra provided role 'multi-node-firewall' in order
to correctly configure iptables. Thanks to Jeremy Stanley and Jeffrey
for help with this.

Closes-Bug: #1749326
Change-Id: Iafaf1cf1d9b0227b0f869969d0bd52fbde3791a0
2018-03-08 16:37:13 +00:00
Jeffrey Zhang
06e7334263 Revert "Revert "Use fernet in gates""
This reverts commit 2a4ce0b8fc7f811d96b7353f5ca46da59cdf2905.

Change-Id: I52efd29791a230df1ce7e3acde4b7184aec34150
2018-03-08 07:50:16 +00:00
Jeffrey Zhang
2a4ce0b8fc Revert "Use fernet in gates"
We have pin keyston to queens release which supports UUID token through
https://review.openstack.org/546475, let us use UUID in queens and
migrate to fernet in rocky cycle.

This reverts commit df0bf1903febd124343ffb7fae398f44b9986422.

Change-Id: Ifb0112315b5047461ce0bf02c754cc0beac52d9a
2018-02-23 10:45:30 +08:00
Eduardo Gonzalez
df0bf1903f Use fernet in gates
we have gates currently broken due using rocky keystone code.
we have no way to test our release until we branch out queens.
this change doesnt change globals default, only our gates tests.

Change-Id: I189aa8cafc73a365668d79e4b186aac21da63a6f
2018-02-19 08:12:11 +01:00
Eduardo Gonzalez
448a10df6c Add container state check in kolla_docker
Missing container status check in recreate_or_restart_container,
this causes if the container is not running (kolla-ansible stop),
to not be started with deploy/reconfigure/upgrade if any other param
changes.

Change-Id: I5cff5f367e963ba8b1807ec46469da817e40e468
Closes-Bug: #1714015
2018-01-26 07:30:11 +00:00
Zuul
39d9327d29 Merge "Fixes ceph deploy gates" 2018-01-25 14:46:34 +00:00
Eduardo Gonzalez
e0d37de9d0 Fixes ceph deploy gates
- Ceph images are not being built when using depends-on a kolla build
  job.
- Sync inventory files with current ones which causes ceph to fail due
  missing groups.
- Small corrections in ceph config.yml syntax.
- fix preparse the disk only once
- enable ceph nfs only when enable_ceph_nfs is true

Co-Authored-By: Jeffrey Zhang <zhang.lei.fly@gmail.com>
Change-Id: Id0c7963bf59e2af4944834dcd16589a638e78ba5
2018-01-24 15:10:01 +08:00
Zuul
c086b2c4ec Merge "Use a random keepalived router id in CI environment" 2018-01-23 14:43:53 +00:00
Jeffrey Zhang
bf4a3c9f6c Use a random keepalived router id in CI environment
Use the same router will result in conflict in the CI environment.

Change-Id: Id1ba69ebbf67b4cead855adbe440343c05ea3d1e
Closes-Bug: #1744913
2018-01-23 12:58:26 +00:00
Zuul
5d4270d282 Merge "Remove the duplicated task for tests/post.yml" 2018-01-17 07:51:29 +00:00
caoyuan
c38ab37766 Remove the unnecessary gather facts
By default, the ansible task will gather facts, it can be configured by
set "father_facts: true", it's unncessary to run setup in tasks, it will
waste much time when hosts is large.

Change-Id: I9462a538c76b6271a34531d8752b72cc695f693e
Closes-Bug: #1743485
2018-01-16 10:25:30 +08:00
caoyuan
18bb9d1a4d Remove the duplicated task for tests/post.yml
Change-Id: Ifdf2ede2cb5f057d1d1f02bfa00ee05872e94b9f
2018-01-15 23:54:50 +08:00
Michal (inc0) Jastrzebski
8bd6dd9be5 Fix kolla gates
multinode gate change introduced bug that breaks kolla deploy gates.
This is fix.

Change-Id: Ibc5fe3adb3555e177d3a6a64250cbc9bf50430d1
2018-01-03 15:06:34 -08:00
Michal (inc0) Jastrzebski
478e2cf9c9 Enable multinode scenarios in zuulv3
Another step to migrate to new CI

Change-Id: Id9bf9b4b1114f5be66f6c028a0db16d869efeceb
Depends-On: Iaacdca31b27b7033ec08499bcea7061b9d3ec279
2017-12-27 09:21:35 -08:00
Jeffrey Zhang
11f9afba1b Pull image from dockerhub rather than tarballs.o.o site
Change-Id: Id71c1ecda9f8647a0eb290fe02a12afb3df09988
2017-11-16 15:44:24 +08:00
Jeffrey Zhang
cdd125117f Optimize zuul v3 jobs
- move check container failure from post.yml to run.yml
- add binary related jobs
- use static kolla-ansible src dir, which is helpful for kolla project
  to use.
- generate correct /etc/hosts by using private ip address and hostname
- fix the wrong api interface in global.yml file

Change-Id: Idfdee6dfe18f0fa2d4f984df59b57553122ce298
2017-10-26 09:58:29 +08:00
Jeffrey Zhang
baa9319a75 Move to zuul v3 in project jobs
Partial-Bug: #1720601
Change-Id: Ibc20a6ae8c645ff82f3c14a6286073dffd4cfae2
2017-10-18 12:31:52 -07:00
Eduardo Gonzalez
fee1538c38 Retrieve fluentd logs in gates
Fluentd send logs to stdout,
this changes creates a file with fluentd logs
output to easy discover missing patterns during fluentd
changes.

Change-Id: I131f95089eac60ccb4c48cf5071c3b44c5ea42ca
2017-09-12 23:03:25 +02:00
Eduardo Gonzalez
c27338bf2f Retrieve ceph logs in gate
Check status of ceph cluster

Change-Id: I4919a32794cc75bd3e8f411a219f778238a334ee
2017-08-29 15:54:26 +00:00
Michal (inc0) Jastrzebski
6119585ae4 Fixing ceph gates
Ceph gates notoriously run out of memory

Change-Id: I5cf5cc5adf5379095ed9e3094494e65b76ff62d8
2017-08-16 12:16:08 -07:00
Jenkins
7e6fb5e308 Merge "Remove unnecessary setUp function in testcase" 2017-08-02 14:59:59 +00:00
Jeffrey Zhang
7f2ade5b5a Stop and start container rather than use restart directly
Docker has an issue[0] when restart container. But stop then start
works.

[0] https://github.com/moby/moby/issues/29704

Change-Id: If0a9c0c257cd72209be8e138a1f0b8871500e089
Closes-Bug: #1707097
2017-07-28 17:26:27 +08:00
Jenkins
71133abce7 Merge "Fix logging collection in gates" 2017-07-27 03:23:02 +00:00
Eduardo Gonzalez
52f73f4061 Fix logging collection in gates
Log retrieval was out of sync since repo split
and from multinode gates.

Many useful information retrieved before like
ps, df, docker info, etc is not in kolla-ansible
gates.

Also, his change fix logs visualization to have
colored view, allowing to easily identify errors.

Change-Id: I948233e26ceb6efc58b962bcb4b710b3f006232b
2017-07-24 14:32:09 +00:00
Jenkins
cd20a68635 Merge "Remove warning during kolla_docker execution" 2017-07-24 10:42:00 +00:00
Bertrand Lallau
2d424ffb14 Remove warning during kolla_docker execution
Ansible check if modules parameters are named like
%password% and allow to hidden log param in this case.
This requires adding "no_log" parameters.
This patch just add "no_log" param in order to avoid
this warning.

Change-Id: I9c1df1093e0fd101090292d6e8bf3527f99aeb17
Closes-Bug: #1702244
2017-07-14 18:43:45 +02:00
Hongbin Lu
3daeea37ea Upgrade from docker-py to docker
The pypi package 'docker-py' [1] has been renamed to 'docker' [2].
It is better to move to the new 'docker' package because the old
package will be deprecated and all the new features will go into
the new package only.

Package 'docker' has been added to requirements [3]. The old
package 'docker-py' is still allowed to be in the global requirements
during the transition period but it should be removed after all or
most of the projects finsih the migration.

[1] https://pypi.python.org/pypi/docker-py
[2] https://pypi.python.org/pypi/docker
[3] https://review.openstack.org/#/c/423715/

Change-Id: Ibcd5a57a1fbf55dcc5a690e41f20917f95b63da0
2017-07-10 14:19:28 +00:00
Michal (inc0) Jastrzebski
339b27c7fe Enable multi-scenario gates
First non-trivial scenario we'll be testing in gates will be cinder +
ceph. This patchset is supposed to create framework to easily add new
testing scenarios.

Change-Id: I6790eaf5cf3297af04a64e4169a39b6e2cd1890f
2017-06-19 10:14:33 -07:00
Jeffrey Zhang
50803ee803 Revert "Enable port_security in gate to fix the neutron broken"
This reverts commit 94a882babe7210cdec0029f4796f2e1b83df116b.

Upstream has fix this issue by Ice89ad9dd486ad5fcac534ef5f7d8aae3b6b0f97

Change-Id: Ib17f93e4030c6f7da4e32ee84fcc5f7139e57895
Closes-Bug: #1694420
2017-06-15 09:25:48 +08:00