278 lines
12 KiB
YAML
Raw Normal View History

---
nova_services:
nova-api:
container_name: "nova_api"
group: "nova-api"
image: "{{ nova_api_image_full }}"
enabled: True
privileged: True
volumes: "{{ nova_api_default_volumes + nova_api_extra_volumes }}"
dimensions: "{{ nova_api_dimensions }}"
healthcheck: "{{ nova_api_healthcheck }}"
haproxy:
nova_api:
enabled: "{{ enable_nova }}"
mode: "http"
external: false
port: "{{ nova_api_port }}"
listen_port: "{{ nova_api_listen_port }}"
tls_backend: "{{ nova_enable_tls_backend }}"
nova_api_external:
enabled: "{{ enable_nova }}"
mode: "http"
external: true
external_fqdn: "{{ nova_external_fqdn }}"
port: "{{ nova_api_public_port }}"
listen_port: "{{ nova_api_listen_port }}"
tls_backend: "{{ nova_enable_tls_backend }}"
nova_metadata:
enabled: "{{ enable_nova }}"
mode: "http"
external: false
port: "{{ nova_metadata_port }}"
listen_port: "{{ nova_metadata_listen_port }}"
tls_backend: "{{ nova_enable_tls_backend }}"
nova_metadata_external:
enabled: "{{ nova_enable_external_metadata }}"
mode: "http"
external: true
external_fqdn: "{{ nova_metadata_external_fqdn }}"
port: "{{ nova_metadata_port }}"
listen_port: "{{ nova_metadata_listen_port }}"
tls_backend: "{{ nova_enable_tls_backend }}"
nova-scheduler:
container_name: "nova_scheduler"
group: "nova-scheduler"
image: "{{ nova_scheduler_image_full }}"
enabled: True
volumes: "{{ nova_scheduler_default_volumes + nova_scheduler_extra_volumes }}"
dimensions: "{{ nova_scheduler_dimensions }}"
healthcheck: "{{ nova_scheduler_healthcheck }}"
nova-super-conductor:
container_name: "nova_super_conductor"
group: "nova-super-conductor"
enabled: "{{ enable_cells }}"
image: "{{ nova_super_conductor_image_full }}"
volumes: "{{ nova_super_conductor_default_volumes + nova_super_conductor_extra_volumes }}"
dimensions: "{{ nova_super_conductor_dimensions }}"
healthcheck: "{{ nova_super_conductor_healthcheck }}"
####################
# Config Validate
####################
nova_config_validation:
- generator: "/nova/etc/nova/nova-config-generator.conf"
config: "/etc/nova/nova.conf"
####################
# Database
####################
# These are kept for backwards compatibility, as cell0 references them.
nova_database_name: "nova"
nova_database_user: "{% if use_preconfigured_databases | bool and use_common_mariadb_user | bool %}{{ database_user }}{% else %}nova{% endif %}"
Implement IPv6 support in the control plane Introduce kolla_address filter. Introduce put_address_in_context filter. Add AF config to vars. Address contexts: - raw (default): <ADDR> - memcache: inet6:[<ADDR>] - url: [<ADDR>] Other changes: globals.yml - mention just IP in comment prechecks/port_checks (api_intf) - kolla_address handles validation 3x interface conditional (swift configs: replication/storage) 2x interface variable definition with hostname (haproxy listens; api intf) 1x interface variable definition with hostname with bifrost exclusion (baremetal pre-install /etc/hosts; api intf) neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network basic multinode source CI job for IPv6 prechecks for rabbitmq and qdrouterd use proper NSS database now MariaDB Galera Cluster WSREP SST mariabackup workaround (socat and IPv6) Ceph naming workaround in CI TODO: probably needs documenting RabbitMQ IPv6-only proto_dist Ceph ms switch to IPv6 mode Remove neutron-server ml2_type_vxlan/vxlan_group setting as it is not used (let's avoid any confusion) and could break setups without proper multicast routing if it started working (also IPv4-only) haproxy upgrade checks for slaves based on ipv6 addresses TODO: ovs-dpdk grabs ipv4 network address (w/ prefix len / submask) not supported, invalid by default because neutron_external has no address No idea whether ovs-dpdk works at all atm. ml2 for xenapi Xen is not supported too well. This would require working with XenAPI facts. rp_filter setting This would require meddling with ip6tables (there is no sysctl param). By default nothing is dropped. Unlikely we really need it. ironic dnsmasq is configured IPv4-only dnsmasq needs DHCPv6 options and testing in vivo. KNOWN ISSUES (beyond us): One cannot use IPv6 address to reference the image for docker like we currently do, see: https://github.com/moby/moby/issues/39033 (docker_registry; docker API 400 - invalid reference format) workaround: use hostname/FQDN RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4. This is due to old RabbitMQ versions available in images. IPv4 is preferred by default and may fail in the IPv6-only scenario. This should be no problem in real life as IPv6-only is indeed IPv6-only. Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will no longer be relevant as we supply all the necessary config. See: https://github.com/rabbitmq/rabbitmq-server/pull/1982 For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed to work well). Older Ansible versions are known to miss IPv6 addresses in interface facts. This may affect redeploys, reconfigures and upgrades which run after VIP address is assigned. See: https://github.com/ansible/ansible/issues/63227 Bifrost Train does not support IPv6 deployments. See: https://storyboard.openstack.org/#!/story/2006689 Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c Implements: blueprint ipv6-control-plane Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-11 20:47:00 +02:00
nova_database_address: "{{ database_address | put_address_in_context('url') }}:{{ database_port }}"
nova_cell0_database_name: "{{ nova_database_name }}_cell0"
nova_cell0_database_user: "{{ nova_database_user }}"
nova_cell0_database_address: "{{ nova_database_address }}"
nova_cell0_database_password: "{{ nova_database_password }}"
nova_api_database_name: "nova_api"
nova_api_database_user: "{% if use_preconfigured_databases | bool and use_common_mariadb_user | bool %}{{ database_user }}{% else %}nova_api{% endif %}"
Implement IPv6 support in the control plane Introduce kolla_address filter. Introduce put_address_in_context filter. Add AF config to vars. Address contexts: - raw (default): <ADDR> - memcache: inet6:[<ADDR>] - url: [<ADDR>] Other changes: globals.yml - mention just IP in comment prechecks/port_checks (api_intf) - kolla_address handles validation 3x interface conditional (swift configs: replication/storage) 2x interface variable definition with hostname (haproxy listens; api intf) 1x interface variable definition with hostname with bifrost exclusion (baremetal pre-install /etc/hosts; api intf) neutron's ml2 'overlay_ip_version' set to 6 for IPv6 on tunnel network basic multinode source CI job for IPv6 prechecks for rabbitmq and qdrouterd use proper NSS database now MariaDB Galera Cluster WSREP SST mariabackup workaround (socat and IPv6) Ceph naming workaround in CI TODO: probably needs documenting RabbitMQ IPv6-only proto_dist Ceph ms switch to IPv6 mode Remove neutron-server ml2_type_vxlan/vxlan_group setting as it is not used (let's avoid any confusion) and could break setups without proper multicast routing if it started working (also IPv4-only) haproxy upgrade checks for slaves based on ipv6 addresses TODO: ovs-dpdk grabs ipv4 network address (w/ prefix len / submask) not supported, invalid by default because neutron_external has no address No idea whether ovs-dpdk works at all atm. ml2 for xenapi Xen is not supported too well. This would require working with XenAPI facts. rp_filter setting This would require meddling with ip6tables (there is no sysctl param). By default nothing is dropped. Unlikely we really need it. ironic dnsmasq is configured IPv4-only dnsmasq needs DHCPv6 options and testing in vivo. KNOWN ISSUES (beyond us): One cannot use IPv6 address to reference the image for docker like we currently do, see: https://github.com/moby/moby/issues/39033 (docker_registry; docker API 400 - invalid reference format) workaround: use hostname/FQDN RabbitMQ may fail to bind to IPv6 if hostname resolves also to IPv4. This is due to old RabbitMQ versions available in images. IPv4 is preferred by default and may fail in the IPv6-only scenario. This should be no problem in real life as IPv6-only is indeed IPv6-only. Also, when new RabbitMQ (3.7.16/3.8+) makes it into images, this will no longer be relevant as we supply all the necessary config. See: https://github.com/rabbitmq/rabbitmq-server/pull/1982 For reliable runs, at least Ansible 2.8 is required (2.8.5 confirmed to work well). Older Ansible versions are known to miss IPv6 addresses in interface facts. This may affect redeploys, reconfigures and upgrades which run after VIP address is assigned. See: https://github.com/ansible/ansible/issues/63227 Bifrost Train does not support IPv6 deployments. See: https://storyboard.openstack.org/#!/story/2006689 Change-Id: Ia34e6916ea4f99e9522cd2ddde03a0a4776f7e2c Implements: blueprint ipv6-control-plane Signed-off-by: Radosław Piliszek <radoslaw.piliszek@gmail.com>
2019-09-11 20:47:00 +02:00
nova_api_database_address: "{{ database_address | put_address_in_context('url') }}:{{ database_port }}"
####################
# Database sharding
####################
nova_database_shard_root_user: "{% if enable_proxysql | bool %}root_shard_{{ nova_database_shard_id }}{% else %}{{ database_user }}{% endif %}"
nova_database_shard:
users:
- user: "{{ nova_database_user }}"
password: "{{ nova_database_password }}"
- user: "{{ nova_api_database_user }}"
password: "{{ nova_api_database_password }}"
rules:
- schema: "{{ nova_database_name }}"
shard_id: "{{ nova_database_shard_id }}"
- schema: "{{ nova_cell0_database_name }}"
shard_id: "{{ nova_database_shard_id }}"
- schema: "{{ nova_api_database_name }}"
shard_id: "{{ nova_database_shard_id }}"
####################
# Docker
####################
nova_tag: "{{ openstack_tag }}"
nova_super_conductor_image: "{{ docker_registry ~ '/' if docker_registry else '' }}{{ docker_namespace }}/{{ docker_image_name_prefix }}nova-conductor"
nova_super_conductor_tag: "{{ nova_tag }}"
nova_super_conductor_image_full: "{{ nova_super_conductor_image }}:{{ nova_super_conductor_tag }}"
nova_scheduler_image: "{{ docker_registry ~ '/' if docker_registry else '' }}{{ docker_namespace }}/{{ docker_image_name_prefix }}nova-scheduler"
nova_scheduler_tag: "{{ nova_tag }}"
nova_scheduler_image_full: "{{ nova_scheduler_image }}:{{ nova_scheduler_tag }}"
nova_api_image: "{{ docker_registry ~ '/' if docker_registry else '' }}{{ docker_namespace }}/{{ docker_image_name_prefix }}nova-api"
nova_api_tag: "{{ nova_tag }}"
nova_api_image_full: "{{ nova_api_image }}:{{ nova_api_tag }}"
nova_api_dimensions: "{{ default_container_dimensions }}"
nova_scheduler_dimensions: "{{ default_container_dimensions }}"
nova_super_conductor_dimensions: "{{ default_container_dimensions }}"
nova_api_enable_healthchecks: "{{ enable_container_healthchecks }}"
nova_api_healthcheck_interval: "{{ default_container_healthcheck_interval }}"
nova_api_healthcheck_retries: "{{ default_container_healthcheck_retries }}"
nova_api_healthcheck_start_period: "{{ default_container_healthcheck_start_period }}"
nova_api_healthcheck_test: ["CMD-SHELL", "healthcheck_curl {{ 'https' if nova_enable_tls_backend | bool else 'http' }}://{{ api_interface_address | put_address_in_context('url') }}:{{ nova_api_listen_port }} "]
nova_api_healthcheck_timeout: "{{ default_container_healthcheck_timeout }}"
nova_api_healthcheck:
interval: "{{ nova_api_healthcheck_interval }}"
retries: "{{ nova_api_healthcheck_retries }}"
start_period: "{{ nova_api_healthcheck_start_period }}"
test: "{% if nova_api_enable_healthchecks | bool %}{{ nova_api_healthcheck_test }}{% else %}NONE{% endif %}"
timeout: "{{ nova_api_healthcheck_timeout }}"
nova_scheduler_enable_healthchecks: "{{ enable_container_healthchecks }}"
nova_scheduler_healthcheck_interval: "{{ default_container_healthcheck_interval }}"
nova_scheduler_healthcheck_retries: "{{ default_container_healthcheck_retries }}"
nova_scheduler_healthcheck_start_period: "{{ default_container_healthcheck_start_period }}"
nova_scheduler_healthcheck_test: ["CMD-SHELL", "healthcheck_port nova-scheduler {{ om_rpc_port }}"]
nova_scheduler_healthcheck_timeout: "{{ default_container_healthcheck_timeout }}"
nova_scheduler_healthcheck:
interval: "{{ nova_scheduler_healthcheck_interval }}"
retries: "{{ nova_scheduler_healthcheck_retries }}"
start_period: "{{ nova_scheduler_healthcheck_start_period }}"
test: "{% if nova_scheduler_enable_healthchecks | bool %}{{ nova_scheduler_healthcheck_test }}{% else %}NONE{% endif %}"
timeout: "{{ nova_scheduler_healthcheck_timeout }}"
nova_super_conductor_enable_healthchecks: "{{ enable_container_healthchecks }}"
nova_super_conductor_healthcheck_interval: "{{ default_container_healthcheck_interval }}"
nova_super_conductor_healthcheck_retries: "{{ default_container_healthcheck_retries }}"
nova_super_conductor_healthcheck_start_period: "{{ default_container_healthcheck_start_period }}"
nova_super_conductor_healthcheck_test: ["CMD-SHELL", "healthcheck_port nova-conductor {{ om_rpc_port }}"]
nova_super_conductor_healthcheck_timeout: "{{ default_container_healthcheck_timeout }}"
nova_super_conductor_healthcheck:
interval: "{{ nova_super_conductor_healthcheck_interval }}"
retries: "{{ nova_super_conductor_healthcheck_retries }}"
start_period: "{{ nova_super_conductor_healthcheck_start_period }}"
test: "{% if nova_super_conductor_enable_healthchecks | bool %}{{ nova_super_conductor_healthcheck_test }}{% else %}NONE{% endif %}"
timeout: "{{ nova_super_conductor_healthcheck_timeout }}"
nova_api_default_volumes:
- "{{ node_config_directory }}/nova-api/:{{ container_config_directory }}/:ro"
- "/etc/localtime:/etc/localtime:ro"
- "{{ '/etc/timezone:/etc/timezone:ro' if ansible_facts.os_family == 'Debian' else '' }}"
- "/lib/modules:/lib/modules:ro"
- "kolla_logs:/var/log/kolla/"
- "{{ kolla_dev_repos_directory ~ '/nova/nova:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/nova' if nova_dev_mode | bool else '' }}"
nova_scheduler_default_volumes:
- "{{ node_config_directory }}/nova-scheduler/:{{ container_config_directory }}/:ro"
- "/etc/localtime:/etc/localtime:ro"
- "{{ '/etc/timezone:/etc/timezone:ro' if ansible_facts.os_family == 'Debian' else '' }}"
- "kolla_logs:/var/log/kolla/"
- "{{ kolla_dev_repos_directory ~ '/nova/nova:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/nova' if nova_dev_mode | bool else '' }}"
nova_super_conductor_default_volumes:
- "{{ node_config_directory }}/nova-super-conductor/:{{ container_config_directory }}/:ro"
- "/etc/localtime:/etc/localtime:ro"
- "{{ '/etc/timezone:/etc/timezone:ro' if ansible_facts.os_family == 'Debian' else '' }}"
- "kolla_logs:/var/log/kolla/"
- "{{ kolla_dev_repos_directory ~ '/nova/nova:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/nova' if nova_dev_mode | bool else '' }}"
# Used by bootstrapping containers.
nova_api_bootstrap_default_volumes:
- "{{ node_config_directory }}/nova-api-bootstrap/:{{ container_config_directory }}/:ro"
- "/etc/localtime:/etc/localtime:ro"
- "{{ '/etc/timezone:/etc/timezone:ro' if ansible_facts.os_family == 'Debian' else '' }}"
- "kolla_logs:/var/log/kolla/"
- "{{ kolla_dev_repos_directory ~ '/nova/nova:/var/lib/kolla/venv/lib/python' ~ distro_python_version ~ '/site-packages/nova' if nova_dev_mode | bool else '' }}"
nova_extra_volumes: "{{ default_extra_volumes }}"
nova_api_extra_volumes: "{{ nova_extra_volumes }}"
nova_scheduler_extra_volumes: "{{ nova_extra_volumes }}"
nova_super_conductor_extra_volumes: "{{ nova_extra_volumes }}"
# Used by bootstrapping containers.
nova_api_bootstrap_extra_volumes: "{{ nova_extra_volumes }}"
####################
# OpenStack
####################
nova_internal_base_endpoint: "{{ nova_internal_fqdn | kolla_url(internal_protocol, nova_api_port) }}"
nova_public_base_endpoint: "{{ nova_external_fqdn | kolla_url(public_protocol, nova_api_public_port) }}"
Standardize use and construction of endpoint URLs The goal for this push request is to normalize the construction and use of internal, external, and admin URLs. While extending Kolla-ansible to enable a more flexible method to manage external URLs, we noticed that the same URL was constructed multiple times in different parts of the code. This can make it difficult for people that want to work with these URLs and create inconsistencies in a large code base with time. Therefore, we are proposing here the use of "single Kolla-ansible variable" per endpoint URL, which facilitates for people that are interested in overriding/extending these URLs. As an example, we extended Kolla-ansible to facilitate the "override" of public (external) URLs with the following standard "<component/serviceName>.<companyBaseUrl>". Therefore, the "NAT/redirect" in the SSL termination system (HAproxy, HTTPD or some other) is done via the service name, and not by the port. This allows operators to easily and automatically create more friendly URL names. To develop this feature, we first applied this patch that we are sending now to the community. We did that to reduce the surface of changes in Kolla-ansible. Another example is the integration of Kolla-ansible and Consul, which we also implemented internally, and also requires URLs changes. Therefore, this PR is essential to reduce code duplicity, and to facility users/developers to work/customize the services URLs. Change-Id: I73d483e01476e779a5155b2e18dd5ea25f514e93 Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2020-06-16 16:03:31 -03:00
nova_legacy_internal_endpoint: "{{ nova_internal_base_endpoint }}/v2/%(tenant_id)s"
nova_legacy_public_endpoint: "{{ nova_public_base_endpoint }}/v2/%(tenant_id)s"
Standardize use and construction of endpoint URLs The goal for this push request is to normalize the construction and use of internal, external, and admin URLs. While extending Kolla-ansible to enable a more flexible method to manage external URLs, we noticed that the same URL was constructed multiple times in different parts of the code. This can make it difficult for people that want to work with these URLs and create inconsistencies in a large code base with time. Therefore, we are proposing here the use of "single Kolla-ansible variable" per endpoint URL, which facilitates for people that are interested in overriding/extending these URLs. As an example, we extended Kolla-ansible to facilitate the "override" of public (external) URLs with the following standard "<component/serviceName>.<companyBaseUrl>". Therefore, the "NAT/redirect" in the SSL termination system (HAproxy, HTTPD or some other) is done via the service name, and not by the port. This allows operators to easily and automatically create more friendly URL names. To develop this feature, we first applied this patch that we are sending now to the community. We did that to reduce the surface of changes in Kolla-ansible. Another example is the integration of Kolla-ansible and Consul, which we also implemented internally, and also requires URLs changes. Therefore, this PR is essential to reduce code duplicity, and to facility users/developers to work/customize the services URLs. Change-Id: I73d483e01476e779a5155b2e18dd5ea25f514e93 Signed-off-by: Rafael Weingärtner <rafael@apache.org>
2020-06-16 16:03:31 -03:00
nova_internal_endpoint: "{{ nova_internal_base_endpoint }}/v2.1"
nova_public_endpoint: "{{ nova_public_base_endpoint }}/v2.1"
nova_logging_debug: "{{ openstack_logging_debug }}"
openstack_nova_auth: "{{ openstack_auth }}"
nova_services_require_policy_json:
- nova-api
Restart all nova services after upgrade During an upgrade, nova pins the version of RPC calls to the minimum seen across all services. This ensures that old services do not receive data they cannot handle. After the upgrade is complete, all nova services are supposed to be reloaded via SIGHUP to cause them to check again the RPC versions of services and use the new latest version which should now be supported by all running services. Due to a bug [1] in oslo.service, sending services SIGHUP is currently broken. We replaced the HUP with a restart for the nova_compute container for bug 1821362, but not other nova services. It seems we need to restart all nova services to allow the RPC version pin to be removed. Testing in a Queens to Rocky upgrade, we find the following in the logs: Automatically selected compute RPC version 5.0 from minimum service version 30 However, the service version in Rocky is 35. There is a second issue in that it takes some time for the upgraded services to update the nova services database table with their new version. We need to wait until all nova-compute services have done this before the restart is performed, otherwise the RPC version cap will remain in place. There is currently no interface in nova available for checking these versions [2], so as a workaround we use a configurable delay with a default duration of 30 seconds. Testing showed it takes about 10 seconds for the version to be updated, so this gives us some headroom. This change restarts all nova services after an upgrade, after a 30 second delay. [1] https://bugs.launchpad.net/oslo.service/+bug/1715374 [2] https://bugs.launchpad.net/nova/+bug/1833542 Change-Id: Ia6fc9011ee6f5461f40a1307b72709d769814a79 Closes-Bug: #1833069 Related-Bug: #1833542
2019-06-17 13:48:13 +01:00
nova_enable_external_metadata: "no"
nova_api_workers: "{{ openstack_service_workers }}"
nova_superconductor_workers: "{{ openstack_service_workers }}"
nova_metadata_api_workers: "{{ openstack_service_workers }}"
nova_scheduler_workers: "{{ openstack_service_workers }}"
nova_enable_nova_legacy_service: false
####################
# Keystone
####################
nova_ks_services:
- name: "nova_legacy"
type: "compute_legacy"
description: "OpenStack Compute Service (Legacy 2.0)"
endpoints:
- {'interface': 'internal', 'url': '{{ nova_legacy_internal_endpoint }}'}
- {'interface': 'public', 'url': '{{ nova_legacy_public_endpoint }}'}
enabled: "{{ nova_enable_nova_legacy_service | bool }}"
- name: "nova"
type: "compute"
description: "OpenStack Compute Service"
endpoints:
- {'interface': 'internal', 'url': '{{ nova_internal_endpoint }}'}
- {'interface': 'public', 'url': '{{ nova_public_endpoint }}'}
nova_ks_users:
- project: "service"
user: "{{ nova_keystone_user }}"
password: "{{ nova_keystone_password }}"
role: "admin"
nova_ks_user_roles:
- project: "service"
user: "{{ nova_keystone_user }}"
role: "service"
####################
# Notification
####################
nova_notification_topics:
- name: notifications
enabled: "{{ enable_ceilometer | bool or enable_neutron_infoblox_ipam_agent | bool }}"
- name: "{{ designate_notifications_topic_name }}"
enabled: "{{ designate_enable_notifications_sink | bool }}"
nova_enabled_notification_topics: "{{ nova_notification_topics | selectattr('enabled', 'equalto', true) | list }}"
####################
# Kolla
####################
nova_git_repository: "{{ kolla_dev_repos_git }}/{{ project_name }}"
nova_dev_repos_pull: "{{ kolla_dev_repos_pull }}"
nova_dev_mode: "{{ kolla_dev_mode }}"
nova_source_version: "{{ kolla_source_version }}"
####################
# TLS
####################
nova_enable_tls_backend: "{{ kolla_enable_tls_backend }}"