490 Commits

Author SHA1 Message Date
gugug
f13847a5a2 Remove the congress roles since it has been retired
more info: https://review.opendev.org/#/c/721733/

Depends-On: I561ead226f714d98c8e06e6027715a64c3a8e47e
Depends-On: I21c9ab9820f78cf76adf11c5f0591c60f76372a8
Change-Id: Ic740d090211ee331b374a6dac69dfde466df7200
Co-Authored-By: jacky06 <zhang.min@99cloud.net>
2020-06-20 01:51:03 +00:00
Zuul
e744b9d510 Merge "Remove mongodb integration" 2020-06-19 13:50:04 +00:00
Zuul
64f05d7a05 Merge "CI: Move NFV reqs installation to where it belongs" 2020-06-19 13:48:44 +00:00
Zuul
23cd9fb2ba Merge "CI: use venv only (and not virtualenv)" 2020-06-19 13:48:40 +00:00
gugug
66ea6e099f Remove mongodb integration
more info: a6c97d7284

Change-Id: I778d472cc7f6ca19852482a3e309d793973d75a6
Co-Authored-By: jacky06 <zhang.min@99cloud.net>
2020-06-19 09:07:23 +08:00
Radosław Piliszek
e7247dbd9a CI: use venv only (and not virtualenv)
It seems we used virtualenv for ceph-ansible only.

Change-Id: I7f9002283462dbe4bae3c1d7ff1dedcc4e7d01f2
2020-06-17 18:24:07 +00:00
Michal Nasiadka
3f94ac9ce9 CI: Make ARA rsync quiet
Change-Id: I4d6e4f43bcf3dc31cad0675f94724dbc4fcb466f
2020-06-17 11:37:57 +02:00
Zuul
1e35ef5a26 Merge "Replace internal and external VIP CA with root CA" 2020-06-16 16:01:25 +00:00
Zuul
e7f39d31e9 Merge "Generate Root CA for Self-Signed Certificates" 2020-06-16 11:12:26 +00:00
James Kirsch
e3cd02eda4 Replace internal and external VIP CA with root CA
Replaced "kolla_external_fqdn_cacert" and "kolla_internal_fqdn_cacert" with
"kolla_admin_openrc_cacert". OS_CACERT is now set to the value of
"kolla_admin_openrc_cacert" in the generated admin-openrc.sh file.

Change-Id: If195d5402579cee9a14b91f63f5fde84eb84cccf
Partially-Implements: blueprint add-ssl-internal-network
Depends-On: https://review.opendev.org/#/c/731344/
2020-06-16 11:46:34 +01:00
James Kirsch
a982d3acbb Generate Root CA for Self-Signed Certificates
Update the certificate generation task to create a root CA for the
self-signed certificates. The internal and external facing certificates
are then generated using the root CA.

Updated openstack_cacert to use system CA trust store in CI tests
certificate by default.

Change-Id: I6c2adff7d0128146cf086103ff6060b0dcefa37b
Partially-Implements: blueprint add-ssl-internal-network
2020-06-15 10:29:51 -07:00
Radosław Piliszek
7316815055 CI: Move NFV reqs installation to where it belongs
This ought to fix #1864238 for the stable branches.

Change-Id: I218905247a4a4003ecfc2c9ab3e47767bb5ab33e
Related-bug: #1864238
2020-06-12 21:33:22 +02:00
Zuul
d896179eac Merge "CI: drop pre-ussuri upgrade support" 2020-06-09 17:53:14 +00:00
Marcin Juszkiewicz
09ab5fd0ba CI: drop pre-ussuri upgrade support
Ussuri is Py3 everywhere. We are free to drop any py2 leftovers.

Depends-On: https://review.opendev.org/731805
Change-Id: I1a9c9a14af351cd3e8b01a40f323a82ffa673d35
2020-05-30 15:23:47 +00:00
Doug Szumski
b39a0f805a Switch to Monasca API for logs
The Monasca Log API has been removed and in this change we switch
to using the unified API. If dedicated log APIs are required then
this can be supported through configuration. Out of the box the
Monasca API is used for both logs and metrics which is envisaged to
work for most use cases.

In order to use the unified API for logs, we need to disable the
legacy Kafka client. We also rename the Monasca API config file
to remove a warning about using the old style name.

Depends-On: https://review.opendev.org/#/c/728638
Change-Id: I9b6bf5b6690f4b4b3445e7d15a40e45dd42d2e84
2020-05-23 17:49:32 +01:00
Zuul
b480ecff0c Merge "CI: Discern between Ironic client and grep failure" 2020-05-17 21:02:22 +00:00
Zuul
e2b25023d9 Merge "Add extend_lists option to merge_yaml" 2020-05-13 14:31:47 +00:00
Zuul
c3a4f78d14 Merge "Stop mocking ansible modules globally" 2020-05-12 21:31:13 +00:00
Mark Goddard
100f92563f CI: Blacklist Ansible 2.9.8
Ansible 2.9.8 includes a regression on the fileglob plugin [1] that
causes the HAProxy role to fail.

This change blacklists Ansible 2.9.8 to work around the issue.

[1] https://github.com/ansible/ansible/issues/69450

Change-Id: I12ca3b154fc7fed6a221880596e0acb5f6278bb7
Related-Bug: #1878192
2020-05-12 11:55:44 +00:00
Will Szumski
69a6acf7a8 Add extend_lists option to merge_yaml
This allows you to extend lists in yaml config. This is useful, for
example, in prometheus.yml, where it would be nice to be able to
extend the scrape_configs to include exporters that aren't packaged
with kolla-ansible. This would provide a mechanism to do so.

Change-Id: I7a10e363f42e8ffaae3c0d2c2a758853e2cab7e1
Related: blueprint custom-prometheus-targets
2020-05-11 13:47:12 +01:00
Will Szumski
4fcbdd7740 Stop mocking ansible modules globally
This causes non-local side effects that are hard to track down. E.g:

--- import errors ---
Failed to import test module: tests.test_merge_yaml
Traceback (most recent call last):
  File "/home/will/.pyenv/versions/3.7.7/lib/python3.7/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/will/.pyenv/versions/3.7.7/lib/python3.7/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/will/code/kolla-ansible/tests/test_merge_yaml.py", line 19, in <module>
    from ansible.errors import AnsibleModuleError
ModuleNotFoundError: No module named 'ansible.errors'; 'ansible' is not a package

This `'ansible' is not a package` message occurs because ansible is a Mock.

Depends-On: https://review.opendev.org/#/c/726768/
Change-Id: Iddbdd3d855daadbf12536cc990559e6b8e123051
2020-05-11 13:47:12 +01:00
Radosław Piliszek
7d73246fe7 OVN IPv6
Fixes:
- SB/NB DB address format (single host) for SB/NB DB daemon
- SB/NB DB address format (all hosts) for Neutron / northd /
  ovn-ovs bootstrap
- OVN tests

Change-Id: I539773c48f89b731d068280c228ce11782bf5788
Closes-Bug: #1875222
2020-05-01 18:03:14 +02:00
Hongbin Lu
91678f67af Zun: Add zun-cni-daemon to compute node
Zun has a new component "zun-cni-daemon" which should be
deployed in every compute nodes. It is basically an implementation
of CNI (Container Network Interface) that performs the neutron
port binding.

If users is using the capsule (pod) API, the recommended deployment
option is using "cri" as capsule driver. This is basically to use
a CRI runtime (i.e. CRI plugin for containerd) for supporting
capsules (pods). A CRI runtime needs a CNI plugin which is what
the "zun-cni-daemon" provides.

The configuration is based on the Zun installation guide [1].
It consits of the following steps:
* Configure the containerd daemon in the host. The "zun-compute"
  container will use grpc to communicate with this service.
* Install the "zun-cni" binary at host. The containerd process
  will invoke this binary to call the CNI plugin.
* Run a "zun-cni-daemon" container. The "zun-cni" binary will
  communicate with this container via HTTP.

Relevant patches:
Blueprint: https://blueprints.launchpad.net/zun/+spec/add-support-cri-runtime
Install guide: https://review.opendev.org/#/c/707948/
Devstack plugin: https://review.opendev.org/#/c/705338/
Kolla image: https://review.opendev.org/#/c/708273/

[1] https://docs.openstack.org/zun/latest/install/index.html

Depends-On: https://review.opendev.org/#/c/721044/
Change-Id: I9c361a99b355af27907cf80f5c88d97191193495
2020-04-30 02:22:20 +00:00
Marcin Juszkiewicz
30b8eed1cd CI: Fix Debian aarch64 jobs
Debian defaults to Python2 which is not complete in aarch64 images.
This patch changes CI to always use Python3.

We need to install several Python modules to have working ussuri jobs.

"Failed to import the required Python library (setuptools) on primary's Python /usr/bin/python3."

And then several Python2 ones for train->ussuri upgrade jobs:

"Unable to find any of pip2 to use. pip needs to be installed."

Change-Id: Ia0d3ff15d97d1cabbb0b8e7f32e8712ca3f94732
2020-04-28 19:15:06 +02:00
Radosław Piliszek
1243337a1c CI: Discern between Ironic client and grep failure
Pipe hid an issue that Ironic client was not available in CI,
making it look as if fake-hardware driver was not usable.

This commit applies separation of concerns to easily discern
between the two failures and provide better debugging experience.

Change-Id: Iac3d84b7d707b84a8426c8d787b29a368ff7ae85
Related-Bug: #1872481
2020-04-26 16:05:03 +02:00
Radosław Piliszek
234272eb64 Cleanup py27 support
Removes and/or replaces all mentions of py27.

Cleans up obsolete requirements and their lower-constraints.

Separates test-requirements.

Makes lower-constraints pass outside of CI (MarkupSafe).

Adds FIXMEs about some hacky Mocks that may misbehave.

Change-Id: Ifc090bf3c1db17d8542ee591c91e8225a597bfe2
2020-04-26 12:16:44 +02:00
Radosław Piliszek
3036c7cfda CI: Make bashate happy
Bashate warned on init-swift.sh because of E043 -- arithmetic
compound has inconsistent return semantics: (( next_port++ ))
New Zuul reported that on every change as a warning.
This is fixed here.

This change makes Bashate always produce errors so that we do not
introduce such warnings again.

Change-Id: I40166b377ec2580e17901375b636183bca492d3a
2020-04-24 15:57:55 +02:00
Zuul
ca9e6f1a4e Merge "Improve CI ansible plays to pass ansible-lint" 2020-04-21 21:15:20 +00:00
Marcin Juszkiewicz
f40886f6d2 Improve CI ansible plays to pass ansible-lint
Change-Id: I150c8952aad7b6fdb1260b343c26a96fb2360bce
2020-04-21 18:05:42 +00:00
Zuul
5273828fae Merge "Remove support for Python 2 on the host" 2020-04-21 17:01:25 +00:00
Zuul
9f54c6fd1b Merge "Remove support for CentOS 7" 2020-04-20 16:57:53 +00:00
Mark Goddard
284f492861 Remove support for Python 2 on the host
Drops support for creating Python 2 virtualenvs in bootstrap-servers,
and looking for a python2 interpreter in the kolla-ansible script.

Also forces the use of Python 3 as the remote interpreter in CI on
Debian and Ubuntu hosts, since they typically symlink the unversioned
interpreter to python2.7.

Change-Id: Id0e977de381e7faafed738674a140ba36184727e
Partially-Implements: blueprint drop-py2-support
2020-04-20 16:18:31 +00:00
Zuul
e17ac13c6b Merge "OVN: Followup changes" 2020-04-17 10:47:06 +00:00
Michal Nasiadka
ca380e6d30 OVN: Followup changes
Followup of last reviews from [1].

[1]: https://review.opendev.org/#/c/696841

Change-Id: I7085093b20e8848e09dc521ae9fbf120e909470d
2020-04-16 18:13:35 +02:00
Zuul
2ba903de0d Merge "CI: do not build images on aarch64" 2020-04-16 15:59:13 +00:00
Marcin Juszkiewicz
3d653038dc CI: do not build images on aarch64
We publish those images for a while.

Change-Id: Ifc157b43e87e4f77a1c70d98343bd0ef9cc0de79
2020-04-16 09:29:07 +00:00
Zuul
975db2b2b4 Merge "OVN Support" 2020-04-15 20:51:57 +00:00
Zuul
573390303c Merge "CI: do 12 attempts to connect to test instance" 2020-04-10 20:17:06 +00:00
Marcin Juszkiewicz
bfd661a600 CI: tell Nova which CPU to emulate on AArch64
AArch64 does not have a way to get cpu features from libvirt.

Change-Id: Ieed404e17e8a9829f38c03f7ee7fdb3caa3919e8
2020-04-10 16:50:22 +00:00
Marcin Juszkiewicz
6f24582eb9 CI: do 12 attempts to connect to test instance
AArch64 needs a bit more time to boot testing instance. So give it more
time by doubling amount of connect attempts.

Change-Id: I87ca65691dfbac84349e8af24d2f36f1db1c8be1
2020-04-10 16:50:04 +00:00
Zuul
2d8edc374d Merge "Add support for encrypting backend Keystone HAProxy traffic" 2020-04-09 16:10:50 +00:00
James Kirsch
b475643c11 Add support for encrypting backend Keystone HAProxy traffic
This patch introduces an optional backend encryption for Keystone
service. When used in conjunction with enabling TLS for service API
endpoints, network communcation will be encrypted end to end, from
client through HAProxy to the Keystone service.

Change-Id: I6351147ddaff8b2ae629179a9bc3bae2ebac9519
Partially-Implements: blueprint add-ssl-internal-network
2020-04-09 09:22:55 +00:00
Michal Nasiadka
8a0740df97 OVN Support
Implement OVN Ansible role.

Implements: blueprint ovn-controller-neutron-ansible

Depends-On: https://review.opendev.org/713422
Change-Id: Icd425dea85d58db49c838839d8f0b864b4a89a78
2020-04-09 07:40:12 +02:00
Mark Goddard
f4e20a1f16 Remove support for CentOS 7
CentOS 8 support is now fairly complete - time to drop CentOS 7.

Partially-Implements: blueprint centos-rhel-8

Change-Id: I940b1d3eceb98e16fa366c243672f588b1412d70
2020-04-08 17:05:10 +01:00
Zuul
fa49143f10 Merge "Update hacking for Python3" 2020-04-06 06:43:46 +00:00
Andreas Jaeger
454489762c Update hacking for Python3
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.

Fix problems found by updated hacking version.

Remove hacking and friends from lower-constraints, they are not needed
during installation.

Change-Id: I7ef5ac8a89e94f5da97780198619b6facc86ecfe
2020-04-04 17:00:17 +02:00
Radosław Piliszek
ed1b74a35b CI: Avoid tox (and clean up gate setup)
We don't need tox nor tell our users to use it when doing kolla
builds.
Tox is going away from base infra images.
It's already gone from aarch64 (arm64) ones.

Change-Id: I2eb5203ad93c011b8806f4b6fb56db081c14a2cb
2020-04-03 18:04:52 +01:00
Zuul
0c1b6c8a4b Merge "Use unittest.mock instead of PyPI mock" 2020-04-03 13:29:25 +00:00
Radosław Piliszek
5648ad6e9d Use unittest.mock instead of PyPI mock
Now that py2 is gone, oslotest dropped dependency on mock and will
soon affect Ussuri CI [1], let's use unittest.mock built in py3.

This also fixes py38 jobs and proactively prevents py36 and py37
failing due to [1]. This is because we never included mock in
test-requirements (but in lower-constraints where it does not
really belong at all) and instead relied on oslotest to bring
it in.

[1] https://review.opendev.org/716322

Change-Id: I30e82e2d87418272a71c7ee089a8acdaf8872158
2020-04-02 18:21:01 +02:00
Radosław Piliszek
641b22b2ef CI: Always use upper-constraints
In stable branches we are getting hit by more py2-incompats.
Let's pin u-c in all CI pip invocations.

Change-Id: Ie2bcc7c115cd2aaf4639d90803216011b346daf3
2020-03-30 20:42:08 +02:00