23 Commits

Author SHA1 Message Date
Carlos Camacho
0a0e2ee629 Update the template_version alias for all the templates to pike.
Master is now the development branch for pike
changing the release alias name.

Change-Id: I938e4a983e361aefcaa0bd9a4226c296c5823127
2017-05-19 09:58:07 +02:00
Saravanan KR
a096ddab34 Add role specific information to the service template
When a service is enabled on multiple roles, the parameters for the
service will be global. This change enables an option to provide
role specific parameter to services and other templates.

Two new parameters - RoleName and RoleParameters, are added to the
service template. RoleName provides the role name of on which the
current instance of the service is being applied on. RoleParameters
provides the list of parameters which are configured specific to the
role in the environment file, like below:

  parameters_default:
      # Default value for applied to all roles
      NovaReservedHostMemory: 2048
      ComputeDpdkParameters:
          # Applied only to ComputeDpdk role
          NovaReservedHostMemory: 4096

In above sample, the cluster contains 2 roles - Compute, ComputeDpdk.
The values of ComputeDpdkParameters will be passed on to the templates
as RoleParameters while creating the stack for ComputeDpdk role. The
parameter which supports role specific configuration, should find the
parameter first in in the RoleParameters list, if not found, then the
default (for all roles) should be used.
Implements: blueprint tripleo-derive-parameters

Change-Id: I72376a803ec6b2ed93903cc0c95a6ffce718b6dc
2017-05-15 10:06:46 +05:30
Thomas Herve
687c53a05a Remove yaql call when building logging_groups
yaql calls are fairly expensive. Let's try to not nest them when we can
avoid it.

Change-Id: I5e7dbc42be625bbfe7989867794a67ebae08687d
2017-04-10 10:15:52 +00:00
Dan Prince
ad2ea290be docker: new hybrid deployment architecture and configuration
This patch implements a new docker deployment architecture that
should us to install docker services in a stepwise manner alongside
of baremetal puppet services. This works by using Yaql to select
docker specific services (docker/services/*.yaml) vs the puppet
specific ones and then applying the selected Json to relevant Heat
software deployments for docker and baremetal puppet in a stepwise
fashion.

Additionally the new architecture
leverages new composable services interfaces from Newton to
allow configuration of per-service container configuration
sets (directories that are bind mounted into kolla containers) by
using the Kolla containers themselves. It does this by spinning up
a throw away "configuration only" version of the container being
configured itself, then running the puppet apply in that container and
copying the generated config files into /var/lib/config-data. This
avoids having to install all of the OpenStack dependency packages
in the heat-agent-container itself (our previous approach) and should
allow us to configure a much wider variety of container config files
that would otherwise be impossible with the previous shared approach.

The new approach (combined) should allow us to configure containers in
both the undercloud and overcloud and incrementally add CI coverage to
services as we containerize them.

Co-Authored-By: Martin André <m.andre@redhat.com>
Co-Authored-By: Ian Main <imain@redhat.com>
Co-Authored-By: Flavio Percoco <flavio@redhat.com>

Change-Id: Ibcff99f03e6751fbf3197adefd5d344178b71fc2
2017-02-15 12:56:44 -05:00
Steven Hardy
1cdc514871 Add support for batched upgrades to composable upgrades
Some services (e.g ceph mon) require upgrading in batches (the old
upgrade architecture did the ceph mon upgrade one controller at a
time).  This interface enables doing the same, and over time we
can probably move more services into this interface (e.g when
services support rolling upgrades) to reduce downtime.

Change-Id: If581f301a5493ef33ac1386bdc22f9fca4f2544e
Partially-Implements: blueprint overcloud-upgrades-per-service
2017-01-25 21:03:01 +00:00
Steven Hardy
3c6ec654b4 Bump template version for all templates to "ocata"
Heat now supports release name aliases, so we can replace
the inconsistent mix of date related versions with one consistent
version that aligns with the supported version of heat for this
t-h-t branch.

This should also help new users who sometimes copy/paste old templates
and discover intrinsic functions in the t-h-t docs don't work because
their template version is too old.

Change-Id: Ib415e7290fea27447460baa280291492df197e54
2016-12-23 11:43:39 +00:00
Juan Antonio Osorio Robles
d2da59065d Add hook to generate metadata from service profiles
This enables the deployer to dynamically add nova metadata to the
servers based on the output of service profiles that implement the
metadata_settings key in the role_data output for the profiles.

One can set an implementation via the OS::TripleO::ServerMetadataHook
resource, which currently is set as OS::Heat::None. So, because of
the default implementation, if left untouched it actually does
nothing.

Currently, besides the list, which is metadata_settings, this hook also
takes the name of the node that it's setting the metadata for.

This is useful for nova vendordata plugins that can parse said metadata.

Change-Id: I8a937f711f0b90156fbb6c4632760435ef846474
2016-12-22 10:06:17 +00:00
Steven Hardy
dbece39f54 Initial support for composable upgrades with Heat+Ansible
This shows how we could wire in the upgrade steps using Ansible
as was previously proposed e.g in https://review.openstack.org/#/c/321416/
but it's more closely integrated with the new composable services
architecture.

It's also very similar to the approach taken by SpinalStack where
ansible snippets per-service were combined then run in a series of
steps using Ansible tags.

This patch just enables upgrade of keystone - we'll add support for
other patches in subsequent patches.

Partially-Implements: blueprint overcloud-upgrades-per-service
Change-Id: I39f5426cb9da0b40bec4a7a3a4a353f69319bdf9
2016-12-01 13:40:50 +00:00
Dan Prince
93b4d836ee Handle null role_data in services
This patch updates the Yaql expressions that work on role_data
so that they evaluate properly when the get_attr for role_data
is null.

I hit issues using this for the heat undercloud installer and this
seems to resolve them.

Change-Id: I0493d0525cd3ad280339f26ef9d3aa311af9962e
2016-11-10 07:42:13 -05:00
Lars Kellogg-Stedman
dd162f42c7 restore missing fluentd client functionality
in the great rebase following the JINJA ALL THE THINGS changes we lost
critical functionality in the fluentd client service.  This review
restores the missing features.

Change-Id: I7c23f16f81e75f3da6a24587b2eb8385b3e920a4
Closes-bug: 1630692
2016-10-05 13:39:06 -04:00
Dan Prince
9d67d7b3b1 Move keystone::auth into service_config_settings
This patch moves the keystone::auth settings for all
services into the new service_config_settings section. This
is important because we execute the keystone commands via
puppet only on the role containing the keystone service
and without these settings it will fail.

Note that yaql merging/filtering is used here to ensure that
service_config_settings is optional in service templates,
and also that we'll only deploy hieradata for a given
service on a node running the service (the key in
the service_config_settings map must match the service_name
in the service template for this to work).

e.g the following will result in only deploying keystone: 123
in hiera on the role running the "keystone" service,
regardless of which service template defines it.

  service_config_settings:
    keystone:
      keystone: 123

Co-Authored-By: Steven Hardy <shardy@redhat.com>
Change-Id: I0c2fce037a1a38772f998d582a816b4b703f8265
Closes-bug: 1620829
2016-09-23 07:43:21 -04:00
Steven Hardy
a99c06aa96 Tolerate missing keys from role_data in service templates
Currently we have a few keys which may be considered optional,
such as monitoring_subscription, logging* and global_config_settings.

Currently we dereference these directly via get_attr, but this will
break when heat output validation is fixed, ref bug #1599114 is fixed
(patches are up for this, so it may be soon).

Change-Id: If4eed1ca39c10ace9b1cb5ce2dc4b9c70a3dd2f4
Partial-Bug: #1620829
2016-09-22 14:21:24 +01:00
Lars Kellogg-Stedman
0d9298bb8f Add fluentd client service
This implements support for installing fluentd agents as a composable
service on the overcloud.

Depends-On: I2e1abe4d8c8359e56ff626255ee50c9cacca1940

Implements: tripleo-opstools-centralized-logging
Change-Id: I23b0e23881b742158fcfb6b8c145a3211d45086e
2016-09-17 01:31:12 +00:00
Jenkins
56486057c7 Merge "Add global_config_settings to services' output" 2016-09-09 09:20:21 +00:00
Martin Mágr
25ad7b8e1e Availability monitoring agents support
- adds possibility to install sensu-client on all nodes
- each composable service has it's own subscription

Co-Authored-By: Emilien Macchi <emilien@redhat.com>
Co-Authored-By: Michele Baldessari <michele@redhat.com>
Implements: blueprint tripleo-opstools-availability-monitoring
Change-Id: I6a215763fd0f0015285b3573305d18d0f56c7770
2016-08-31 09:22:59 -04:00
Giulio Fidente
7aeb416528 Add global_config_settings to services' output
The new global_config_settings output is meant to contain
additional hieradata distributed by the services to all role types.

Change-Id: I1beff8f7b12a4178e6081de6134cd8e67794d710
2016-08-26 11:11:47 +02:00
Dan Prince
3b62761d2f Add DefaultPasswords to composable services
This patch adds a new DefaultPasswords parameter to
composable services. This is needed to help provide
access to top level password resources that overcloud.yaml
currently manages (passwords for Rabbit, Mysql, etc.).

Moving the RandomString resources into composable services
would cause them to regenerate within the stack. With this
approach we can leave them where they are while we deprecate
the top level mechanism and move the code that uses the
passwords into the composable services.

Change-Id: I4f21603c58a169a093962594e860933306879e3f
2016-08-18 12:45:30 -04:00
Giulio Fidente
885b37c80e Pass ServiceNetMap to services
This will be needed to pick the network where the service has
to bind to from within the service template.

Change-Id: I52652e1ad8c7b360efd2c7af199e35932aaaea8c
2016-08-18 12:36:18 -04:00
Steven Hardy
23e13cc1e0 Filter null/None service names
To enable use of the service_names lists in overcloud.yaml we need
to strip any null/None values or list processing becomes cumbersome.

These happen because we're currently disabling some services via
OS::Heat::None in the resource_registry, it may be possible to remove
when we've got a fully composable approach to generating the
*Services lists.

Change-Id: I8ef53b4279d93850c207c73aab0d75095a2497a2
Partially-Implements: blueprint custom-roles
2016-07-27 17:04:59 +01:00
Dan Prince
6b30ff11d4 Add 'service_name' to composable services
This patch adds a new service_name section to each composable
service. We now have an explicit unit test check to ensure that
service_name exists in tools/yaml-validate.py.

This patch also wires service_names into hieradata on each
of the roles so that tools can access the deployed services locally
during deployment and upgrades.

Change-Id: I60861c5aa760534db3e314bba16a13b90ea72f0c
2016-07-22 07:29:39 -04:00
Dan Prince
68562d07cc Output generic role_data in services.yaml
This patch updates puppet/services/services.yaml (currently the only
interface for 'services' in t-h-t) so that we return a more generic
'role_data' Heat output.

This is a move towards making the services themselves a bit more generic
so we can accommodate other deployment types (containers, etc.)

Change-Id: I8bc32c59a48e6d5f0caa2f26fab394d5d992a4a5
2016-07-01 10:05:04 +01:00
Giulio Fidente
a6438a2082 Pass MysqlVirtualIP via EndpointMap
By passing the MysqlVirtualIP via the EndpointMap we won't need it
to be provided as a parameter to the services.

This follows what is already happening for the glance registry
service with I9186e56cd4746a60e65dc5ac12e6595ac56505f0.

Change-Id: Iad2ab389bf64d0fc8b06eb0e7d29b5370ff27dff
Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
2016-05-30 10:22:59 +03:00
Dan Prince
0bcf693a73 Configure ControllerServices via resource chains
This patch wires in a new for Mitaka Heat feature
that allows us to dynamically include a set of nested
stacks representing individual services via a Heat resource chain.
Follow on patches will use this interface to decompose the controller
role into isolated services.

Co-Authored-By: Steve Hardy <shardy@redhat.com>
Depends-On: If510abe260ea7852dfe2d1f7f92b529979483068
Change-Id: I84c97a76159704c2d6c963bc4b26e365764b1366
2016-03-31 16:09:17 -04:00