854 Commits

Author SHA1 Message Date
Mark Goddard
5fb37ae584 Support editable virtualenv installation for development
An editable installation allows changes to be made to the source code
directly, and have those changes applied immediately without having to
reinstall.

    pip install -e /path/to/kolla-ansible

Change-Id: I023d96d25edd9d2fafd4415743e298af72a861a1
2020-07-02 18:15:40 +01:00
Mark Goddard
4d8d678240 Fix unexpected message without globals.d
Recently a feature was merged to support pulling in multiple
configuration files from a globals.d directory. However, if this
directory does not exist, we get the following error when executing
kolla-ansible:

    find: '/etc/kolla/globals.d': No such file or directory

This change addresses this by redirecting find command stderr to
/dev/null.

TrivialFix

Change-Id: Ie5aa511a5ebf3355817a7c3bb65b09ac5dcf2b67
2020-06-29 14:11:05 +01:00
Zuul
64f05d7a05 Merge "CI: Move NFV reqs installation to where it belongs" 2020-06-19 13:48:44 +00:00
Konstantinos Mouzakitis
f6d8c0d481 Adding support for multiple globals files
Added a spec file for this blueprint.
Changed the kolla-ansible script to accept more than one
globals.yml file. That will still be the main one but operators
will be able to create more, under the /etc/kolla/globals.d
directory.
Also added some paragraphs in the quickstart documentation
about this.
Finally, Adding a release note

Change-Id: I34eb91d0e2ed80694594b8fc6801cf8ad77da754
Implements: blueprint multiple-globals-files
2020-06-18 17:33:51 +00:00
Radosław Piliszek
7316815055 CI: Move NFV reqs installation to where it belongs
This ought to fix #1864238 for the stable branches.

Change-Id: I218905247a4a4003ecfc2c9ab3e47767bb5ab33e
Related-bug: #1864238
2020-06-12 21:33:22 +02:00
Luke Short
e8b2053f51 Do not ask for a SSH key password
This is to avoid waiting for the user to input data to continue
creating sample resources.

Change-Id: I46ff7a4779dae4c9cbe157e1712afa4e53be269e
Signed-off-by: Luke Short <ekultails@gmail.com>
2020-05-28 20:57:53 -04:00
Zuul
05e3597cfa Merge "Check that used Ansible can see Kolla Ansible" 2020-05-24 13:39:58 +00:00
xiaojueguan
593b7b4ce4 fix can not generate ovs-dpdk.conf
Change-Id: I0ecc10f8035f421d7dc1132c2b29db0d9cc84cdf
Closes-Bug: #1879125
2020-05-17 16:28:59 +08:00
Radosław Piliszek
32fc2599a6 Check that used Ansible can see Kolla Ansible
Fix-feature following up on the original check [1] to make it
test the correct interpreter.

Additionally, this change removes last, unneeded call to
random python - getting script directory is perfectly
doable in bash.

All checks are done from Python, not Ansible, due to its
performance. Python version feels snappy (0.2 s to check),
compared to sluggish Ansible (2.0 s to check).
What is more, relying on Ansible would require hacky solutions
to e.g. prevent custom config from interfering with it.
We might be willing to steer Ansible from Python in the future
anyhow.

[1] Icf0399d21b3fde8d530d73e6e7ee4a57665da276

Change-Id: Ib8f2e6b6672e7c06aa94bc226c4d72640d25d8c2
Closes-Bug: #1856346
2020-04-27 17:18:31 +02:00
Radosław Piliszek
3036c7cfda CI: Make bashate happy
Bashate warned on init-swift.sh because of E043 -- arithmetic
compound has inconsistent return semantics: (( next_port++ ))
New Zuul reported that on every change as a warning.
This is fixed here.

This change makes Bashate always produce errors so that we do not
introduce such warnings again.

Change-Id: I40166b377ec2580e17901375b636183bca492d3a
2020-04-24 15:57:55 +02:00
Zuul
5273828fae Merge "Remove support for Python 2 on the host" 2020-04-21 17:01:25 +00:00
Zuul
9f54c6fd1b Merge "Remove support for CentOS 7" 2020-04-20 16:57:53 +00:00
Mark Goddard
284f492861 Remove support for Python 2 on the host
Drops support for creating Python 2 virtualenvs in bootstrap-servers,
and looking for a python2 interpreter in the kolla-ansible script.

Also forces the use of Python 3 as the remote interpreter in CI on
Debian and Ubuntu hosts, since they typically symlink the unversioned
interpreter to python2.7.

Change-Id: Id0e977de381e7faafed738674a140ba36184727e
Partially-Implements: blueprint drop-py2-support
2020-04-20 16:18:31 +00:00
Michal Nasiadka
8a0740df97 OVN Support
Implement OVN Ansible role.

Implements: blueprint ovn-controller-neutron-ansible

Depends-On: https://review.opendev.org/713422
Change-Id: Icd425dea85d58db49c838839d8f0b864b4a89a78
2020-04-09 07:40:12 +02:00
Mark Goddard
f4e20a1f16 Remove support for CentOS 7
CentOS 8 support is now fairly complete - time to drop CentOS 7.

Partially-Implements: blueprint centos-rhel-8

Change-Id: I940b1d3eceb98e16fa366c243672f588b1412d70
2020-04-08 17:05:10 +01:00
Zuul
fa49143f10 Merge "Update hacking for Python3" 2020-04-06 06:43:46 +00:00
Andreas Jaeger
454489762c Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found by updated hacking version.

Remove hacking and friends from lower-constraints, they are not needed
during installation.

Change-Id: I7ef5ac8a89e94f5da97780198619b6facc86ecfe
2020-04-04 17:00:17 +02:00
Radosław Piliszek
ed1b74a35b CI: Avoid tox (and clean up gate setup)
We don't need tox nor tell our users to use it when doing kolla
builds.
Tox is going away from base infra images.
It's already gone from aarch64 (arm64) ones.

Change-Id: I2eb5203ad93c011b8806f4b6fb56db081c14a2cb
2020-04-03 18:04:52 +01:00
Doug Szumski
b758883418 Support setting Kafka storage volume
Not everyone wants Kafka data stored on a Docker volume. This
change allows a user to flexibly control where the data is stored.

Change-Id: I2ba8c7a85c7bf2564f954a43c6e6dbb3257fe902
2020-03-30 11:00:03 +02:00
Marcin Juszkiewicz
9bde29a30b cirros: upgrade to 0.5.1
We released CirrOS 0.5.1. Time to move then.

Change-Id: Ibca24836f19b3cbf6166fa39a3702883938feda8
2020-03-21 15:40:10 +00:00
Radosław Piliszek
e03f06c081 Do not enforce vxlan in init-runonce
The affected command was meant to create a tenant network, so let
us really test it this way.

Not marking CI, because someone may be using this script.

Change-Id: I5abe46948992121a11a36f941d4f8fac1caa92b1
2020-03-20 14:51:02 +01:00
Zuul
0718a1a90c Merge "CI: Enable fluentd in MariaDB jobs" 2020-03-19 17:19:06 +00:00
Zuul
270fb4cf0b Merge "CentOS 8: Use master image tag by default" 2020-03-19 12:55:34 +00:00
Michal Nasiadka
866a6ba16a CI: Enable fluentd in MariaDB jobs
Since fluentd is disabled in MariaDB jobs - haproxy logs are not getting
populated.

Change-Id: I56b3fc1be6940d97905cdb2c4452b846f106c071
Depends-on: https://review.opendev.org/713704
2020-03-19 09:01:23 +00:00
Mark Goddard
bab770a88b CentOS 8: Use master image tag by default
While supporting both CentOS 7 and 8, we used the tag 'master-centos8'
for CentOS 8 images. We are now ready to drop CentOS 7 support, and
Kolla is switching to publish CentOS 8 images using the master tag on
the master branch, so we should use this.

Depends-On: https://review.opendev.org/713265

Partially-Implements: blueprint centos-rhel-8

Change-Id: I07d2c285e3214a6dc827a8e8eacf263048ee099b
2020-03-16 15:58:55 +00:00
Zuul
5a11f14b1b Merge "CI: Debug init-runonce" 2020-03-15 20:18:18 +00:00
Radosław Piliszek
525a7f326f CI: Use network mode = host for builds
This avoids issues with IPv6 not enabled in docker.

Change-Id: I0faa1df8a04acd1746fa8a5f0d8455f819c254b5
2020-03-03 13:15:30 +01:00
Radosław Piliszek
29e4ef1374 CI: Debug init-runonce
Also adds gawk for timestamping.

This helps to correlate init events with failures elsewhere.

Change-Id: I22fdb683ecf9870b2d66fedd6b40b7004317130a
2020-03-01 09:57:35 +01:00
Zuul
dfe2f9d0c7 Merge "CI: Use upper constraints when installing clients" 2020-02-25 15:07:44 +00:00
Mark Goddard
d8733b290e CI: Use upper constraints when installing clients
Clients are starting to release versions that don't support Python 2.
The ironic scenario is currently failing on stable branches for this
reason.

Use upper constraints to avoid installing these new versions on stable
branches.

Change-Id: I4f91b53cbf2297d70da4b54d6c402c1427aacdd9
2020-02-25 11:21:06 +00:00
Radosław Piliszek
effbb20997 Allow to override external network params in init-runonce
I'm not marking it CI-only as ppl seem to be using it.
OTOH, not adding a release note as we are not promoting its usage.

This is to allow us to customize for CI.

Change-Id: I8100a6cb63b1e54078629bd6ca8475dc5896784a
2020-02-23 16:35:50 +01:00
Zuul
0c5432dc8e Merge "Introduce influxdb_datadir_volume" 2020-02-20 20:09:03 +00:00
Will Szumski
4cf7ff9fa2 Introduce influxdb_datadir_volume
This allows you to tune the performance of InfluxDB by locating the
volume on a drive that is separate to the default docker storage.

Change-Id: Iea555a2702b225b30f5d7035b8a703d4f3376ee7
2020-02-19 16:08:33 +00:00
Zuul
990400c76e Merge "Follow up on Ansible version checking" 2020-02-19 10:50:05 +00:00
Dincer Celik
7b0022b957 Follow up on Ansible version checking
The bash part of version checking fails when ANSIBLE_DEBUG is set
while running kolla-ansible, so need to skip it.

Change-Id: I4adc986365c4b0c11eb69e128eb7d037a24a2baf
2020-02-13 21:05:47 +03:00
Zuul
cd3c51197e Merge "Remove kolla-ceph" 2020-02-13 10:09:24 +00:00
Mark Goddard
9b484c25e4 Docs: Document EXTRA_OPTS
Also better organise the 'tips and tricks' section.

Change-Id: If20a89ae93b64b5a1b5895bb9dc73c4e70adaba7
Related-Bug: #1862649
2020-02-12 10:42:42 +00:00
Michal Nasiadka
4e6fe7a6da Remove kolla-ceph
Kolla-Ansible Ceph deployment mechanism has been deprecated in Train [1].

This change removes the Ansible code and associated CI jobs.

[1]: https://review.opendev.org/669214

Change-Id: Ie2167f02ad2f525d3b0f553e2c047516acf55bc2
2020-02-11 11:42:06 +01:00
Radosław Piliszek
287adab05e CentOS 8: Add deploy jobs in CI
Adds new CI job definitions for CentOS 8:

- kolla-ansible-centos8-source
- kolla-ansible-centos8-binary
- kolla-ansible-centos8-source-ceph-ansible
- kolla-ansible-centos8-source-cinder-lvm
- kolla-ansible-centos8-source-mariadb
- kolla-ansible-centos8-source-bifrost
- kolla-ansible-centos8-source-zun
- kolla-ansible-centos8-source-swift
- kolla-ansible-centos8-source-scenario-nfv
- kolla-ansible-centos8-source-ironic
- kolla-ansible-centos8-binary-ironic
- kolla-ansible-centos8-source-masakari
- kolla-ansible-centos8-source-cells

The following jobs are added to the check pipeline:

- kolla-ansible-centos8-source
- kolla-ansible-centos8-binary
- kolla-ansible-centos8-source-cinder-lvm
- kolla-ansible-centos8-source-mariadb
- kolla-ansible-centos8-source-zun
- kolla-ansible-centos8-source-swift
- kolla-ansible-centos8-source-scenario-nfv
- kolla-ansible-centos8-source-ironic
- kolla-ansible-centos8-binary-ironic
- kolla-ansible-centos8-source-cells

The following jobs are not yet passing so are not added to the check
pipeline:

- kolla-ansible-centos8-source-ceph-ansible
- kolla-ansible-centos8-source-bifrost
- kolla-ansible-centos8-source-masakari

The kolla-ansible-centos8-source job is added to the gate.

Upgrade jobs will be added when CentOS 8 support exists in Train.

Depends-On: https://review.opendev.org/704337
Depends-On: https://review.opendev.org/704848
Depends-On: https://review.opendev.org/704965

Co-Authored-By: Mark Goddard <mark@stackhpc.com>

Change-Id: Ibd806feee71721b122b77d7eff33228ca1cc2853
Partially-Implements: blueprint centos-rhel-8
2020-02-06 11:43:50 +00:00
Radosław Piliszek
5b0894299e CI: Replace cinder-lvm scenario with zun scenario
As discussed during the meeting. [1]

[1] http://eavesdrop.openstack.org/meetings/kolla/2020/kolla.2020-02-05-15.00.html

Change-Id: I339fc7f01f640a32ff35d19ded54b6f8e2fdec15
2020-02-05 18:25:18 +01:00
Mark Goddard
f303202850 CI: build swift images for swift scenario
Change-Id: I957fcea4ab180dd82d1faaa9a012f83d3647f043
2020-01-30 10:59:38 +00:00
Mark Goddard
8b86c8194b CI: Reduce unnecessary gate image builds
Due to regex matching of images, we are currently building quite a few
unnecessary images in CI. Images that can unexpectedly match:

* barbican-keystone-listener
* ironic-neutron-agent
* prometheus-haproxy-exporter
* prometheus-memcached-exporter

This change improves the situation by anchoring matches to the start of
the name. A full solution would be to specify every image required for
each job. This would be quite verbose and require maintenance.

Change-Id: I7387ccd0cb394c54c79bfba8b1c09fce7ee2973e
2020-01-28 16:44:27 +00:00
Zuul
0580d00342 Merge "Adds Docker image pruning command" 2020-01-27 16:11:37 +00:00
Dincer Celik
b7728ea890 Adds Docker image pruning command
This change introduces prune-images command.

Uses docker_prune module of Ansible that comes with version 2.8.

Depends-On: https://review.opendev.org/#/c/699333/

Implements: blueprint docker-image-pruning

Change-Id: Icbf374dd50e1cc1f1604bb4fa779b34279efd50c
2020-01-27 13:44:01 +03:00
Zuul
b103989642 Merge "CI: Add Ceph-Ansible jobs" 2020-01-27 09:09:13 +00:00
Michal Nasiadka
d8c15ad4e8 CI: Add Ceph-Ansible jobs
* Adding zuul centos-source/ubuntu-source ceph-ansible jobs
* Jobs will deploy all Ceph integrated OpenStack components, i.e.
  cinder, glance, nova
* Will utilize core openstack testing script

Depends-On: https://review.opendev.org/685032
Depends-On: https://review.opendev.org/698301

Implements: blueprint ceph-ansible
Change-Id: I233082b46785f74014177f579aeac887a25b2ae2
2020-01-24 22:37:03 +01:00
Zuul
39c09d0b60 Merge "Adds maximum supported version check for Ansible" 2020-01-22 13:36:34 +00:00
Dincer Celik
63ab53195c Adds maximum supported version check for Ansible
Maximum supported version is set to 2.9

Updated the minimum supported version to 2.8

Implements: blueprint ansible-max-version

Change-Id: I97cc95e37f49886e6d74f2d5a789b923b14b5a2d
2020-01-09 09:54:22 +03:00
Mark Goddard
350bb171ef Configure Cinder to use lioadm on CentOS/RHEL 8
In CentOS/RHEL 8 there is no scsi-target-utils package, nor is it
available in EPEL. It is removed from kolla in [1]. In RHEL 7 and beyond
the LIO kernel subsystem can be used instead of the tgtd daemon.

This change removes support for the SCSI target daemon on CentOS/RHEL 8.
The 'tgtd' image is no longer available for CentOS/RHEL 8.

[1] https://review.openstack.org/#/c/613815/5

Change-Id: I718fc16cde2dd177b2a1c2f79b932426034897fe
Related: blueprint centos-rhel-8
2020-01-08 10:39:54 +00:00
Zuul
34edd9517e Merge "CI: Test Zun with Cinder LVM backend (iSCSI)" 2019-12-16 11:27:15 +00:00