52 Commits

Author SHA1 Message Date
Giulio Fidente
0b1afb48e5 Allows for configuration of the Ceph cluster name
To be able to support multiple Ceph cluster, an initial step is
to allow for configuration of each cluster name.

Depends-On: I8d5293eaaf104b6374dfa13992a67ddc37397f10
Implements: blueprint custom-ceph-cluster-name
Change-Id: I1b4d51ca6a2d08fa7a68eea680eb104eff732057
2018-02-20 11:35:01 +01:00
Lars Kellogg-Stedman
b20bce1bf0 logging: use service_config_settings for fluentd
The initial fluentd client implementation predates the introduction of
service_config_settings, and necessitated some invasive changes to
what is now common/serivces.yaml. This commit modifies existing
services to use the service_config_settings based configuration
mechanism supported by more recent versions of the fluentd support in
puppet-tripleo.

Partial-bug: #1715187
Depends-On: I3149902401d68d6fd236073a73a20f982d4b952a
Depends-On: I2b057190ec0e4e75ee4ee47ebe0164c2644e5ab7
Depends-On: Ie7df4b8b94cb0ae38096ab95800f211ef1cd8455
Change-Id: I28028ffa00df2da8e0478a551d3de89c3ee46e1f
2018-02-07 16:37:00 +01:00
marios
dec003def8 Convert tags to when statements for Q major upgrade workflow
This converts "tags: stepN" to "when: step|int == N" for the direct
execution as an ansible playbook, with a loop variable 'step'.
The tasks all include the explicit cast |int.

This also adds a set_fact task for handling of the package removal
with the UpgradeRemovePackages parameter (no change to the interface)

The yaml-validate also now checks for duplicate 'when:' statements

Q upgrade spec @ Ibde21e6efae3a7d311bee526d63c5692c4e27b28
Related Blueprint: major-upgrade-workflow
[0]: 394a92f761/tripleo_common/utils/config.py (L141)
Change-Id: I6adc5619a28099f4e241351b63377f1e96933810
2018-01-08 13:57:47 +02:00
Lee Yarwood
80e9bb33a9 nova: Add VerifyGlanceSignatures compute param
This controls image signature verification during instance creation on
the compute host.

Change-Id: I0d80cbd38eb4e3d110443f6b9a8a7c7643c43453
2017-12-20 10:07:06 +00:00
Oliver Walsh
4e6eeb1625 Improve comment re nova live_migration_tunnelled setting
Change-Id: Ie565cb16bb69f0eb98bcfabfb95f67c71f492bd1
2017-12-18 10:27:18 +00:00
Saravanan KR
d0702e82b5 Add a tag to all the role specific parameters
With parameter tags, it is possible to categorize the parameters.
In this patch, all role-specific parameters of the services are
categorized as role_specific, which will help in adding validation
during the deployment (to ensure the provided role-specific
parameter is actually implemented as role-specific). This patch
adds only the tags, and the validation will done via workflows.

Change-Id: Ic053111298e7872a3a3cd11e6249dbd85707cc29
2017-12-07 12:20:11 +05:30
Moshe Levi
05dbb4e001 Fix applying pci whitelist in nova compute
This commit Iccf86f42448d2e7e682728cb82808f91b6d8124e
remove the nova::compute::pci_passthrough which is
deprecate. Therefore we need to update the nova-compute.yaml
according to this change.

Change-Id: I015936b199e0a7b3c0bf5b7b091c50730140b916
2017-11-26 09:24:16 +00:00
Carlos Camacho
927495fe3d Change template names to queens
The new master branch should point now to queens instead of pike.

So, HOT templates should specify that they might contain features
for queens release [1]

[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#queens

Change-Id: I7654d1c59db0c4508a9d7045f452612d22493004
2017-11-23 10:15:32 +01:00
Emilien Macchi
24f859c018 Fix nova-cpu/collectd hieradata
Probably a typo, never caught or even tested.

Change-Id: Iaf75edb421a19cb69bf3ead59c83bf812c653f0b
Closes-Bug: #1729479
2017-11-01 18:22:16 -07:00
Oliver Walsh
17fd16b9f2 Support for Ocata-Pike live-migration over ssh
In Ocata all live-migration over ssh is performed on the default ssh port (22).
In Pike the containerized live-migration over ssh is on port 2022 as the
docker host's sshd is using port 22.

To allow live migration during upgrade we need to temporarily pin the Pike
computes to port 22 and in the final converge we can switch over to port 2022.

This also changes the default port to 2022 for baremetal computes in Pike to
enable live-migration between baremetal and containerized computes.

Change-Id: Icb9bfdd9a99dc1dce28eb95c50a9a36bffa621b1
Depends-On: I0b80b81711f683be539939e7d084365ff63546d3
Closes-Bug: 1714171
2017-09-07 12:20:34 +01:00
Jan Provaznik
ad8589212c Let mds create manila key and fs
ceph-ansible will take care of setting up client keys both
in ceph and on client side. It will also create filesystem
for manila. To assure that manila manifest can work in future
both with puppet and with ceph-ansible, creation of filesystem
is moved to ceph-mds manifest and creation of manila key on ceph
side is moved to ceph-base (so manila key is always created),
manila key is added to ceph-external for external ceph deployments.
Key creation is removed from manila.pp in patch
I2b5567a39ac8737e80758b705818cc1807dc8bf1

Change-Id: I6308a317ffe0af244396aba5197c85e273e69f68
Related-To: Ia3ef9e9a2b159dacea01e38762145ff2bcc7ba27
Depends-On: I3f18bbe476c4f43fa4e162cc66c5df443122cd0c
2017-08-18 16:22:10 +02:00
Giulio Fidente
c20033524d Set virsh secret with an init step when using Ceph
Run virsh secret-define and secret-set-value in an init step
instead of relying on the puppet-nova exec.

Co-Authored-By: Jiri Stransky <jistr@redhat.com>
Change-Id: Ic950e290af1c66d34b40791defbdf4f8afaa11da
Closes-Bug: #1709583
2017-08-09 16:19:39 +02:00
Ben Nemec
7f84409a6a Make UpgradeLevelNovaCompute parameters consistent
There is logic in nova-base.yaml that depends on the default for
this parameter being '', and the nova-compute service only needs it
set to auto during upgrade.  That will be done by [1] anyway, so it
doesn't matter what the default is.  It's also not clear to me that
the nova-compute task is even needed now that we're post-Ocata, but
that's not a change I feel comfortable making.

1: https://github.com/openstack/tripleo-heat-templates/blob/master/environments/major-upgrade-composable-steps.yaml

Change-Id: Iccfcb5b68e406db1b942375803cfedbb929b4307
Partial-Bug: 1700664
2017-08-02 16:20:12 -05:00
Oliver Walsh
4a7f3398f1 Add support for nova live/cold-migration with containers
Updates hieradata for changes in https://review.openstack.org/471950.
Creates a new service - NovaMigrationTarget. On baremetal this just configures
live/cold-migration. On docker is includes a container running a second sshd
services on an alternative port.
Configures /var/lib/nova/.ssh/config and mounts in nova-compute and libvirtd
containers.

Change-Id: Ic4b810ff71085b73ccd08c66a3739f94e6c0c427
Implements: blueprint tripleo-cold-migration
Depends-On: I6c04cebd1cf066c79c5b4335011733d32ac208dc
Depends-On: I063a84a8e6da64ae3b09125cfa42e48df69adc12
2017-07-23 02:26:55 +01:00
Giulio Fidente
baf6eee501 Adds network/cidr mapping into a new service property
Makes it possible to resolve network subnets within a service
template; the data is transported into a new property ServiceData
wired into every service which hopefully is generic enough to
be extended in the future and transport more data.

Data can be consumed in service templates to set config values
which need to know what is the subnet where a deamon operates (for
example the Ceph Public vs Cluster network).

Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
2017-07-14 13:44:04 +02:00
Keith Schincke
6d36dd0602 Update NovaCompute to consume CephClientKey
It is not necessary to get the Ceph key issueing a get-key to the Ceph
cluster; this change provides the libvirt key via parameter instead.

Change-Id: Iff3dbcb0f1b4d2373570e184e636a71553cea708
2017-07-03 15:47:06 +00:00
Saravanan KR
7a368550ab Role Specific parameter for nova-compute service
The parameters NovaVcpuPinSet, NovaReservedHostMemory and
NovaPCIPassthrough are modified to support role-specific
parameter inputs.
Change-Id: I7c11e8fc2c933f424318e457cb1e96acb8df2ec7
2017-06-08 11:34:13 +05:30
Sven Anderson
cbf74d6e61 Increase default for NovaReservedHostMemory to 4096
Idle compute nodes are found to already consume ~1.5GB of memory, so
2GB is a bit tight.  Increasing to 4GB to be on the safe side.  Also
see https://bugzilla.redhat.com/show_bug.cgi?id=1341178

Change-Id: Ic95984b62a748593992446271b197439fa12b376
2017-06-01 13:38:42 +02:00
Carlos Camacho
0a0e2ee629 Update the template_version alias for all the templates to pike.
Master is now the development branch for pike
changing the release alias name.

Change-Id: I938e4a983e361aefcaa0bd9a4226c296c5823127
2017-05-19 09:58:07 +02:00
Saravanan KR
a096ddab34 Add role specific information to the service template
When a service is enabled on multiple roles, the parameters for the
service will be global. This change enables an option to provide
role specific parameter to services and other templates.

Two new parameters - RoleName and RoleParameters, are added to the
service template. RoleName provides the role name of on which the
current instance of the service is being applied on. RoleParameters
provides the list of parameters which are configured specific to the
role in the environment file, like below:

  parameters_default:
      # Default value for applied to all roles
      NovaReservedHostMemory: 2048
      ComputeDpdkParameters:
          # Applied only to ComputeDpdk role
          NovaReservedHostMemory: 4096

In above sample, the cluster contains 2 roles - Compute, ComputeDpdk.
The values of ComputeDpdkParameters will be passed on to the templates
as RoleParameters while creating the stack for ComputeDpdk role. The
parameter which supports role specific configuration, should find the
parameter first in in the RoleParameters list, if not found, then the
default (for all roles) should be used.
Implements: blueprint tripleo-derive-parameters

Change-Id: I72376a803ec6b2ed93903cc0c95a6ffce718b6dc
2017-05-15 10:06:46 +05:30
Jenkins
8af47c20f3 Merge "[N->O] Add openstack-nova-migration to compute nodes." 2017-05-04 23:05:16 +00:00
Sofer Athlan-Guyot
29a8a46d98 [N->O] Add openstack-nova-migration to compute nodes.
This add openstack-nova-migration on the compute during the upgrade.

Closes-Bug: #1687081

Depends-on: Iab022bdfb655e3c52fecebf416e75c9e981072ab
Depends-on: I02dc8934521340f42ac44a7d16889f6d79620c33

Change-Id: I3db2a3188e538eeaef61769d38f0166545444cfe
2017-05-03 20:21:44 +00:00
Oliver Walsh
3d8af2fcf8 Restrict nova migration ssh tunnel
Specify the allowed networks for migration ssh tunneling.

bp tripleo-cold-migration

Change-Id: Iab022bdfb655e3c52fecebf416e75c9e981072ab
Depends-on: Idb56acd1e1ecb5a5fd4d942969be428cc9cbe293
2017-05-03 20:21:37 +00:00
Jenkins
cac8a68460 Merge "Add migration SSH tunneling support" 2017-04-19 01:39:27 +00:00
Oliver Walsh
0271a63e52 Add migration SSH tunneling support
This enables nova cold migration.

This also switches to SSH as the default transport for live-migration.
The tripleo-common mistral action that generates passwords supplies the
MigrationSshKey parameter that enables this.
The TCP transport is no longer used for live-migration and the firewall
port has been closed.

Change-Id: I4e55a987c93673796525988a2e4cc264a6b5c24f
Depends-On: I367757cbe8757d11943af7e41af620f9ce919a06
Depends-On: I9e7a1862911312ad942233ac8fc828f4e1be1dcf
Depends-On: Iac1763761c652bed637cb7cf85bc12347b5fe7ec
2017-04-15 16:53:40 +01:00
Saravanan KR
57c06ddefd Modify pci_passthrough hiera value as string
Hiera value of nova::compute::pci_passthrough should be a string.
It has been modified to JSON with the heira hook changes. Modifying
it again back to string.
Closes-Bug: #1675036

Change-Id: I441907ff313ecc5b7b4da562c6be195687fc6c76
2017-03-28 14:03:20 +05:30
Sofer Athlan-Guyot
fb78213782 Put service stop at step1 and quiesce at step2.
In the previous release[1], the services were stopped before the
pacemaker services, so that they get a chance to send last message to
the database/rabbitmq queue:

Let's do the upgrade in the same order.

[1] https://github.com/openstack/tripleo-heat-templates/blob/stable/newton/extraconfig/tasks/major_upgrade_controller_pacemaker_2.sh#L13-L71

Change-Id: I1c4045e8b9167396c9dfa4da99973102f1af1218
2017-02-28 19:20:13 +01:00
Steven Hardy
5353f1c7c9 Add nova service support for composable upgrades
Co-Authored-By: Mathieu Bultel <mbultel@redhat.com>
Co-Authored-By: Oliver Walsh <owalsh@redhat.com>

Change-Id: Iafad800a6819d7e75fdaab60d328999d3d3c037f
Partially-Implements: blueprint overcloud-upgrades-per-service
Related-Bug: #1662344
2017-02-14 23:23:33 +00:00
Lars Kellogg-Stedman
490c19bb38 implement a collectd composable service
The collectd composable service permits an operator to configure
collectd metrics collection as part of the overcloud install.

Depends-on: I03cfbd96778a76125d18e2ca2f48d96e292608de
Change-Id: I143565329f5128f15cc39c9b62a6b242666383ab
2017-02-07 11:54:14 +00:00
Steven Hardy
3c6ec654b4 Bump template version for all templates to "ocata"
Heat now supports release name aliases, so we can replace
the inconsistent mix of date related versions with one consistent
version that aligns with the supported version of heat for this
t-h-t branch.

This should also help new users who sometimes copy/paste old templates
and discover intrinsic functions in the t-h-t docs don't work because
their template version is too old.

Change-Id: Ib415e7290fea27447460baa280291492df197e54
2016-12-23 11:43:39 +00:00
Dan Prince
7876851011 Hiera optimization: use a new hiera hook
This patch optimizes how we deploy hiera by using a new
heat hook specifically designed to help compose hiera
within heat templates. As part of this change:

 - we update all the 'hiera' software configurations to set the group to hiera
   instead of os-apply-config.

 - The new format uses JSON instead of YAML. The hook actually writes
   out the hiera JSON directly so no conversion takes place. Arrays,
   Strings, Booleans all stay in their native formats. As such we can avoid
   having to do many of the awkward string and list conversions in t-h-t to
   support the previous YAML formatting.

 - The new hook prefers JSON over YAML so upgrading users will have the
   new files prefered. (we will post a cleanup routine for the old files
   soon but this isn't a new behavior, JSON is now simply prefered.)

 - A lot of services required edits to account for default settings that
   worked in YAML that no longer work correctly in the native JSON
   format. In almost all these cases I think the resulting codes looks
   cleaner and is more explicit with regards to what is getting
   configured in hiera on the actual nodes.

Depends-On: I6a383b1ad4ec29458569763bd3f56fd3f2bd726b
Closes-bug: #1596373

Change-Id: Ibe7e2044e200e2c947223286fdf4fd5bcf98c2e1
2016-11-30 22:16:13 -05:00
Jenkins
f6ffad6515 Merge "Set VNC URL parameters for nova-compute" 2016-09-19 15:57:19 +00:00
Juan Antonio Osorio Robles
67ae241b7e Set VNC URL parameters for nova-compute
These are needed so the computes can advertize the VNC URL correctly.

Change-Id: Ic3eba9fe929ce396b584249eb84415de09ab1b62
Closes-Bug: #1623607
2016-09-17 16:45:28 +00:00
Lars Kellogg-Stedman
0d9298bb8f Add fluentd client service
This implements support for installing fluentd agents as a composable
service on the overcloud.

Depends-On: I2e1abe4d8c8359e56ff626255ee50c9cacca1940

Implements: tripleo-opstools-centralized-logging
Change-Id: I23b0e23881b742158fcfb6b8c145a3211d45086e
2016-09-17 01:31:12 +00:00
Ben Nemec
57898b4162 De-bracket vncproxy_host in compute profile
This is done in the vncproxy profile, but for some reason is not in
the compute one.  It causes hiera to explode when the brackets are
left, so we need to do the bracket stripping here too.

Also switches both places to just use the host_nobrackets version
of the endpoint instead of stripping them with str_replace.

Change-Id: I7ccd84b575fd652f6412fdb1869c31c79a7bf53b
Closes-Bug: 1618623
2016-09-12 16:35:22 +02:00
Martin Mágr
25ad7b8e1e Availability monitoring agents support
- adds possibility to install sensu-client on all nodes
- each composable service has it's own subscription

Co-Authored-By: Emilien Macchi <emilien@redhat.com>
Co-Authored-By: Michele Baldessari <michele@redhat.com>
Implements: blueprint tripleo-opstools-availability-monitoring
Change-Id: I6a215763fd0f0015285b3573305d18d0f56c7770
2016-08-31 09:22:59 -04:00
karthik s
09a5ff6c79 Customize the reservation of memory for host processes
Allows the operator to alter the memory reservation for host processes

Implements: blueprint tripleo-ovs-dpdk

Change-Id: If7a099cde2e8854a14a75c1304620cc444eefcf2
Signed-off-by: karthik s <ksundara@redhat.com>
2016-08-30 08:42:54 +05:30
karthik s
67d3a774e5 Configure the pci_passthrough_whitelist via THT
It allows the operator to configure pci_passthrough_whitelist
in nova.conf for each of the compute nodes.

implements: blueprint tripleo-sriov

Depends-On: I5ed53cfffe80dbbbb9dcee7c2ea6037afbed2382
Change-Id: Ic5e099fe788046363536f913272b2814abe165fa
Signed-off-by: karthik s <ksundara@redhat.com>
2016-08-27 08:25:54 +05:30
karthik s
1087a8b496 Customize vcpu_pin_set in nova.conf
Allow the operator to customize the vcpu_pin_set configuration
in nova.conf

Implements: blueprint tripleo-ovs-dpdk

Change-Id: I7c76a303f4736a4439fa15c5641e576140f1c7b6
Signed-off-by: karthik s <ksundara@redhat.com>
2016-08-26 17:35:37 +05:30
Dan Prince
b31d80f74c Move network bind IPs out of compute.yaml
This patch moves the local bind host hiera data out
of compute.yaml and into composable services.

Change-Id: Iae4ca707c429cc8f5ec4d1d514ae7da0bf557dfd
2016-08-24 12:41:05 -04:00
Dan Prince
3b62761d2f Add DefaultPasswords to composable services
This patch adds a new DefaultPasswords parameter to
composable services. This is needed to help provide
access to top level password resources that overcloud.yaml
currently manages (passwords for Rabbit, Mysql, etc.).

Moving the RandomString resources into composable services
would cause them to regenerate within the stack. With this
approach we can leave them where they are while we deprecate
the top level mechanism and move the code that uses the
passwords into the composable services.

Change-Id: I4f21603c58a169a093962594e860933306879e3f
2016-08-18 12:45:30 -04:00
Giulio Fidente
885b37c80e Pass ServiceNetMap to services
This will be needed to pick the network where the service has
to bind to from within the service template.

Change-Id: I52652e1ad8c7b360efd2c7af199e35932aaaea8c
2016-08-18 12:36:18 -04:00
Dan Prince
4e05d138c6 Move Nova settings out of puppet/compute.yaml
This finishes moving most of the config settings out of
compute.yaml for Nova and into the proper nova-* services.

Only the bind port/VIP related Nova settings remain now and those
will be dealt with in a follow up patch.

Change-Id: I1c40e7d54c11dfff2aaa6438c7701e98da17ebe6
Related-Bug: #1604412
2016-08-15 13:07:58 -04:00
Jenkins
9aec3de5b8 Merge "Convert service_name to underscore syntax" 2016-07-29 08:52:05 +00:00
Steven Hardy
7df649f59e Convert service_name to underscore syntax
Currently we use hyphens, e.g cinder-api, but in overcloud.yaml
we have a lot of references to services (e.g for AllNodesConfig)
by underscore, e.g cinder_api.  To enable dynamic generation of
this data, we need the service name in underscore format.

Change-Id: Ief13dfe5d8d7691dfe2534ad5c39d7eacbcb6f70
2016-07-28 16:31:36 +01:00
Emilien Macchi
315fa31963 Migrate Puppet Hieradata to composable services
Migrate puppet/hieradata/*.yaml parameters to puppet/services/*.yaml
except for some services that are not composable yet.

Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
Change-Id: I7e5f8b18ee9aa63a1dffc6facaf88315b07d5fd7
2016-07-27 12:23:38 -04:00
Giulio Fidente
2720b5d4c8 Cleanup templates from the shared CephCluster config
Removes from the templates the old CephCluster configuration and
deployment which before roles was distributing the shared settings
for the Ceph cluster configuration.

Change-Id: Ia704f5d7add85e52dd477f4bc758aa0a02e4b39b
2016-07-25 12:55:21 +02:00
Jenkins
f00ed98048 Merge "Move nova::db data within service template" 2016-07-25 08:12:49 +00:00
Giulio Fidente
55e84b6100 Move nova::db data within service template
Change-Id: I86752248e59a2e98f8ff9b2c5998839f9ade4779
2016-07-22 15:21:37 +02:00
Dan Prince
6b30ff11d4 Add 'service_name' to composable services
This patch adds a new service_name section to each composable
service. We now have an explicit unit test check to ensure that
service_name exists in tools/yaml-validate.py.

This patch also wires service_names into hieradata on each
of the roles so that tools can access the deployed services locally
during deployment and upgrades.

Change-Id: I60861c5aa760534db3e314bba16a13b90ea72f0c
2016-07-22 07:29:39 -04:00