55 Commits

Author SHA1 Message Date
Zuul
d01d2ea23e Merge "logging: use service_config_settings for fluentd" 2018-02-12 19:40:36 +00:00
Zuul
4e3cd0df30 Merge "Always evaluate step first in conditional" 2018-02-12 19:39:06 +00:00
Lukas Bezdicka
0cb5c847f3 Always evaluate step first in conditional
If we use variables defined in later step in conditional before
checking which step are we on we will fail.

Resolves: rhbz#1535457
Closes-Bug: #1743764
Change-Id: Ic21f6eb5c4101f230fa894cd0829a11e2f0ef39b
2018-02-09 17:12:29 +01:00
Pradeep Kilambi
b2b37b6bda Add S3 storage driver as an option and its config
S3 storage driver is a supported option for gnocchi upstream.

The puppet change has been added in this commit
I5a49fa34d344c4464ea883bf96350f00b0a17eac

Change-Id: Ib44278ef87c8c4d1c39a50d24c1f02fe1fe4042e
2018-02-08 10:50:12 -05:00
Lars Kellogg-Stedman
b20bce1bf0 logging: use service_config_settings for fluentd
The initial fluentd client implementation predates the introduction of
service_config_settings, and necessitated some invasive changes to
what is now common/serivces.yaml. This commit modifies existing
services to use the service_config_settings based configuration
mechanism supported by more recent versions of the fluentd support in
puppet-tripleo.

Partial-bug: #1715187
Depends-On: I3149902401d68d6fd236073a73a20f982d4b952a
Depends-On: I2b057190ec0e4e75ee4ee47ebe0164c2644e5ab7
Depends-On: Ie7df4b8b94cb0ae38096ab95800f211ef1cd8455
Change-Id: I28028ffa00df2da8e0478a551d3de89c3ee46e1f
2018-02-07 16:37:00 +01:00
Pradeep Kilambi
25b0b97c38 Add incoming storage driver param
Change-Id: I5b71099d2e1c25b86ceed430f15ee28ef0f37f5c
2018-02-01 19:25:11 +00:00
marios
dec003def8 Convert tags to when statements for Q major upgrade workflow
This converts "tags: stepN" to "when: step|int == N" for the direct
execution as an ansible playbook, with a loop variable 'step'.
The tasks all include the explicit cast |int.

This also adds a set_fact task for handling of the package removal
with the UpgradeRemovePackages parameter (no change to the interface)

The yaml-validate also now checks for duplicate 'when:' statements

Q upgrade spec @ Ibde21e6efae3a7d311bee526d63c5692c4e27b28
Related Blueprint: major-upgrade-workflow
[0]: 394a92f761/tripleo_common/utils/config.py (L141)
Change-Id: I6adc5619a28099f4e241351b63377f1e96933810
2018-01-08 13:57:47 +02:00
Zuul
0d77ce6615 Merge "Revert "Add upgrade task to run gnocchi upgrade"" 2017-12-14 03:22:21 +00:00
Mehdi Abaakouk (sileht)
11e2cf07c5 Revert "Add upgrade task to run gnocchi upgrade"
This reverts commit 60925faefc58d76adf3914f96c636ca2a5b8c783.

The issue still occurs with this.

Also gnocchi-upgrade should have already run in step4: a327583643/manifests/profile/base/gnocchi/api.pp (L92)

Change-Id: I8ce02ab1425e891b6608363250910bf1f57914fc
2017-12-13 14:53:39 +00:00
Carlos Camacho
927495fe3d Change template names to queens
The new master branch should point now to queens instead of pike.

So, HOT templates should specify that they might contain features
for queens release [1]

[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#queens

Change-Id: I7654d1c59db0c4508a9d7045f452612d22493004
2017-11-23 10:15:32 +01:00
Pradeep Kilambi
60925faefc Add upgrade task to run gnocchi upgrade
Closes-bug: #1724328

Change-Id: Id7fed3746733c0ea0804532beda627c69e4ce078
2017-11-20 15:13:56 -05:00
Thomas Herve
8008089de2 Use list_concat in place of yaql
Where applicable, use list_concat instead of yaql to build new lists: it
should be more resilient to errors, easier to debug, and less expensive.

Change-Id: I6d3dbc7ee8eac50f46023a35af4ec7f2d378fd87
Related-Bug: #1714005
2017-08-30 15:43:16 +02:00
Giulio Fidente
baf6eee501 Adds network/cidr mapping into a new service property
Makes it possible to resolve network subnets within a service
template; the data is transported into a new property ServiceData
wired into every service which hopefully is generic enough to
be extended in the future and transport more data.

Data can be consumed in service templates to set config values
which need to know what is the subnet where a deamon operates (for
example the Ceph Public vs Cluster network).

Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
2017-07-14 13:44:04 +02:00
Carlos Camacho
0a0e2ee629 Update the template_version alias for all the templates to pike.
Master is now the development branch for pike
changing the release alias name.

Change-Id: I938e4a983e361aefcaa0bd9a4226c296c5823127
2017-05-19 09:58:07 +02:00
Saravanan KR
a096ddab34 Add role specific information to the service template
When a service is enabled on multiple roles, the parameters for the
service will be global. This change enables an option to provide
role specific parameter to services and other templates.

Two new parameters - RoleName and RoleParameters, are added to the
service template. RoleName provides the role name of on which the
current instance of the service is being applied on. RoleParameters
provides the list of parameters which are configured specific to the
role in the environment file, like below:

  parameters_default:
      # Default value for applied to all roles
      NovaReservedHostMemory: 2048
      ComputeDpdkParameters:
          # Applied only to ComputeDpdk role
          NovaReservedHostMemory: 4096

In above sample, the cluster contains 2 roles - Compute, ComputeDpdk.
The values of ComputeDpdkParameters will be passed on to the templates
as RoleParameters while creating the stack for ComputeDpdk role. The
parameter which supports role specific configuration, should find the
parameter first in in the RoleParameters list, if not found, then the
default (for all roles) should be used.
Implements: blueprint tripleo-derive-parameters

Change-Id: I72376a803ec6b2ed93903cc0c95a6ffce718b6dc
2017-05-15 10:06:46 +05:30
Emilien Macchi
a6041608ca upgrades: deploy mod_ssl when upgrading apache
1) When Apache is upgraded, install mod_ssl rpm.
   See https://bugs.launchpad.net/tripleo/+bug/1682448
   to understand why we need mod_ssl.

2) All services that run Apache for API will use the snippet from
   Apache service to deploy mod_ssl, so we don't duplicate the code
   in all services. It's using the same mechanism as ovs upgrade to
   compile upgrade_tasks between both services.

Change-Id: Ia2f6fea45c2c09790c49baab19b1efcab25e9a84
Closes-Bug: #1686503
2017-04-26 20:08:26 +00:00
Emilien Macchi
91053af09d Allow to configure policy.json for OpenStack projects
For both containers and classic deployments, allow to configure
policy.json for all OpenStack APIs with new parameters (hash,
empty by default).

Example of new parameter: NovaApiPolicies.
See environments/nova-api-policy.yaml for how the feature can be used.

Note: use it with extreme caution.

Partial-implement: blueprint modify-policy-json
Change-Id: I1144f339da3836c3e8c8ae4e5567afc4d1a83e95
2017-03-28 22:21:28 +00:00
Emilien Macchi
0f48e7b35e gnocchi: deploy services with Keystone v3 endpoints
* Move swift_authurl to gnocchi-base hieradata, where other swift auth
  credentials live and switch it to versionless keystone endpoint.
* Force swift_auth_version to 3 for Keystone v3.
* Switch auth_uri to use versionless Keystone endpoint.
* Switch auth_url to use Keystone admin endpoint (instead of internal).
* Remove old parameters from gnocchi::api, not used anymore.

Partial-blueprint: keystone-v3

Change-Id: I2feed8b1219069128faa1a1e8dcd2ddfbae7e40a
2017-03-13 08:17:26 -04:00
Sofer Athlan-Guyot
fb78213782 Put service stop at step1 and quiesce at step2.
In the previous release[1], the services were stopped before the
pacemaker services, so that they get a chance to send last message to
the database/rabbitmq queue:

Let's do the upgrade in the same order.

[1] https://github.com/openstack/tripleo-heat-templates/blob/stable/newton/extraconfig/tasks/major_upgrade_controller_pacemaker_2.sh#L13-L71

Change-Id: I1c4045e8b9167396c9dfa4da99973102f1af1218
2017-02-28 19:20:13 +01:00
Juan Antonio Osorio Robles
d1eb0bc0dc Use Keystone internal endpoint instead of admin for services
The admin endpoint is listening on the ctlplane network by default;
services should ideally be using the internal api network for this kind
of traffic, as the ctlplane network is mostly for provisioning. On the
other hand, the admin endpoint shouldn't be as relevant with services
switching to keystone v3.

Change-Id: I1213a83ef8693c1cca1d20de974f7949a801d9f1
2017-02-14 02:41:13 +00:00
Mathieu Bultel
8a77f29e58 Remove gnocchi db sync from the ansible upgrade step
Since gnocchi db sync for upgrade is made by puppet
we need to remove it from ansible tasks

Change-Id: I4a26513c3e3d0dcf77c007ea865364e38da3a94a
2017-02-10 17:22:56 +01:00
Jenkins
6ada7c69ef Merge "Remove precheck on services which run on httpd for upgrade" 2017-02-06 22:01:44 +00:00
Mathieu Bultel
7a14359fbc Remove precheck on services which run on httpd for upgrade
Those services is not handle with systemctl

Change-Id: Ia57dffd42a11070696fda14f1e91de2993e63479
2017-02-06 16:39:54 +01:00
Pradeep Kilambi
1cb731bae8 Remove old host param
Change-Id: Ib9e1a4ccdf447455a330687184eae471b9f3f4d4
Depends-On: I2b48d23006e38f56f04456b4556374bf0fcdb14a
2017-02-02 15:53:29 -05:00
marios
d14c56e1b6 Adds a pre-upgrade check that service is running (step0)
Adds a step0 for most services to check that the state is running
before continuing with any of the other upgrades steps (these are
tagged step0).

You can skip this service check by overriding the
SkipUpgradeConfigTags parameter as follows:

parameter_defaults:
  SkipUpgradeConfigTags: validation

Co-Authored-By: Steven Hardy <shardy@redhat.com>
Change-Id: Ie276f153015f671b720b6ed5beaac1b921661909
2017-01-27 11:20:15 +02:00
Steven Hardy
5750df8829 Add telemetry service support for composable upgrades
Change-Id: I62735676b45a881a7dac24171b26d88d6eb60d4a
Partially-Implements: blueprint overcloud-upgrades-per-service
2017-01-25 21:04:18 +00:00
Juan Antonio Osorio Robles
80086fd342 Add metadata settings for needed kerberos principals
These are only used for TLS-everywhere, and fills up the kerberos
principals that will need to be created for the certs used by the
overcloud. With this, the metadata hook will format these principals
correctly and will further pass them on to the nova metadata service.
Where they can be used if there's a plugin enabled.

bp tls-via-certmonger
bp novajoin

Change-Id: I873094bb69200052febda629fda698a7a782c031
2017-01-25 00:33:11 +02:00
Jenkins
7057ec6f7c Merge "Set gnocchi wsgi display name" 2017-01-03 10:32:05 +00:00
Steven Hardy
3c6ec654b4 Bump template version for all templates to "ocata"
Heat now supports release name aliases, so we can replace
the inconsistent mix of date related versions with one consistent
version that aligns with the supported version of heat for this
t-h-t branch.

This should also help new users who sometimes copy/paste old templates
and discover intrinsic functions in the t-h-t docs don't work because
their template version is too old.

Change-Id: Ib415e7290fea27447460baa280291492df197e54
2016-12-23 11:43:39 +00:00
Pradeep Kilambi
efd28dbfa5 Set gnocchi wsgi display name
Depends-On: Ice921f0fdd4bec6de50e62c39c447ee40dc0e8f5

Change-Id: I4109ac83c32ee2365695611009579a8b117134ff
2016-12-20 11:12:54 -05:00
Juan Antonio Osorio Robles
b4cd2ed1ee Use network-based fqdn entry from hiera instead of the custom fact
This changes how we get the network-based FQDNs for the specific
services, from using the custom fact, to the new hiera entries.

Change-Id: Iae668a5d89fb7bee091db4a761aa6c91d369b276
2016-12-01 11:18:23 +02:00
Dan Prince
7876851011 Hiera optimization: use a new hiera hook
This patch optimizes how we deploy hiera by using a new
heat hook specifically designed to help compose hiera
within heat templates. As part of this change:

 - we update all the 'hiera' software configurations to set the group to hiera
   instead of os-apply-config.

 - The new format uses JSON instead of YAML. The hook actually writes
   out the hiera JSON directly so no conversion takes place. Arrays,
   Strings, Booleans all stay in their native formats. As such we can avoid
   having to do many of the awkward string and list conversions in t-h-t to
   support the previous YAML formatting.

 - The new hook prefers JSON over YAML so upgrading users will have the
   new files prefered. (we will post a cleanup routine for the old files
   soon but this isn't a new behavior, JSON is now simply prefered.)

 - A lot of services required edits to account for default settings that
   worked in YAML that no longer work correctly in the native JSON
   format. In almost all these cases I think the resulting codes looks
   cleaner and is more explicit with regards to what is getting
   configured in hiera on the actual nodes.

Depends-On: I6a383b1ad4ec29458569763bd3f56fd3f2bd726b
Closes-bug: #1596373

Change-Id: Ibe7e2044e200e2c947223286fdf4fd5bcf98c2e1
2016-11-30 22:16:13 -05:00
Juan Antonio Osorio Robles
bf97a0fb63 Enable internal TLS for gnocchi
This adds the necessary hieradata for enabling TLS in the internal
network for gnocchi.

bp tls-via-certmonger

Depends-On: Ie983933e062ac6a7f0af4d88b32634e6ce17838b
Change-Id: Iad8d4949ada8b8fd52e0d0bd345b6fb1ca65827b
2016-10-20 12:22:51 +03:00
Juan Antonio Osorio Robles
17c821b071 Enable proxy headers parsing for Gnocchi
http_proxy_to_wsgi middleware was recently added to Gnocchi [1] and
in order to take it into use, we need to enable it via hiera.

[1] Ic5526cf37e70335fa2cc70946a271253f227f129
Related-Bug: #1590608

Change-Id: I145dcfa3455ca1541cbf6b5fc4b601f0813619c0
2016-10-13 08:57:37 +03:00
Dan Prince
7ba5525207 Move db::mysql into service_config_settings
This patch movs the various db::mysql hiera settings into a
'mysql' specific service_config_settings section for each
service so that these will only get applied on the MySQL service
node. This follows a similar puppet-tripleo change where we
create the actual databases for all services locally on
the MySQL service node to avoid permission issues.

Change-Id: Ic0692b1f7aa8409699630ef3924c4be98ca6ffb2
Closes-bug: #1620595
Depends-On: I05cc0afa9373429a3197c194c3e8f784ae96de5f
Depends-On: I5e1ef2dc6de6f67d7c509e299855baec371f614d
2016-09-28 07:01:49 -04:00
Dan Prince
9d67d7b3b1 Move keystone::auth into service_config_settings
This patch moves the keystone::auth settings for all
services into the new service_config_settings section. This
is important because we execute the keystone commands via
puppet only on the role containing the keystone service
and without these settings it will fail.

Note that yaql merging/filtering is used here to ensure that
service_config_settings is optional in service templates,
and also that we'll only deploy hieradata for a given
service on a node running the service (the key in
the service_config_settings map must match the service_name
in the service template for this to work).

e.g the following will result in only deploying keystone: 123
in hiera on the role running the "keystone" service,
regardless of which service template defines it.

  service_config_settings:
    keystone:
      keystone: 123

Co-Authored-By: Steven Hardy <shardy@redhat.com>
Change-Id: I0c2fce037a1a38772f998d582a816b4b703f8265
Closes-bug: 1620829
2016-09-23 07:43:21 -04:00
Juan Antonio Osorio Robles
2fc2770529 Make apache-based services use network-dependent servername
Currently the servername is incorrectly set for the services running
over apache. It currently takes the default value which is just the
regular FQDN, when the services actually might be running on
different IPs that require alternative FQDNs.

This fixes that by filling that value from a fact in hiera that's
dependant on the service's network.

Closes-Bug: #1625677
Change-Id: Ib7ea5fd2d18a376eaa2f5a3fa5687cb9b719a8e2
2016-09-21 09:31:33 +00:00
Lars Kellogg-Stedman
0d9298bb8f Add fluentd client service
This implements support for installing fluentd agents as a composable
service on the overcloud.

Depends-On: I2e1abe4d8c8359e56ff626255ee50c9cacca1940

Implements: tripleo-opstools-centralized-logging
Change-Id: I23b0e23881b742158fcfb6b8c145a3211d45086e
2016-09-17 01:31:12 +00:00
Martin Mágr
25ad7b8e1e Availability monitoring agents support
- adds possibility to install sensu-client on all nodes
- each composable service has it's own subscription

Co-Authored-By: Emilien Macchi <emilien@redhat.com>
Co-Authored-By: Michele Baldessari <michele@redhat.com>
Implements: blueprint tripleo-opstools-availability-monitoring
Change-Id: I6a215763fd0f0015285b3573305d18d0f56c7770
2016-08-31 09:22:59 -04:00
Dan Prince
e3cb92a5db Mv Nova, Neutron, Horizon out of controller.yaml
This patch moves the settings for Nova, Neutron, and Horizon
out of controller.yaml.

Also fixes the NovaPassword settings in nova-base.yaml
so they don't use get_input.

Also, creates a new apache.yaml base service to contain shared
apache settings for several services which use Apache for WSGI.

Co-Authored-By: Giulio Fidente <gfidente@redhat.com>

Change-Id: I35d909bd5abc23976b5732a2b9af31cf1448838e
Related-bug: #1604414
2016-08-30 08:59:07 -04:00
Jenkins
4c6a101511 Merge "Set the default gnocchi driver to swift" 2016-08-29 11:04:32 +00:00
Dan Prince
dd9c8d6e9c Mv ceilo, aodh, and gnocchi out of controller.yaml
This patch moves the remaining hiera settings for
Ceilometer, Aodh, and Gnocchi out of controller.yaml
and into the respective composable services.

Change-Id: I01377aa5c121ecbb4a96b4f0525924c7bbf12198
Related-bug: #1604414
2016-08-25 20:27:11 -04:00
Jenkins
b61f1a33d5 Merge "Enable gnocchi systemd services" 2016-08-25 13:26:19 +00:00
Pradeep Kilambi
780750e881 Set the default gnocchi driver to swift
Until mitaka we have been using file driver. Swift is much more
scalable so lets use that instead as the default.

Change-Id: Idb0d55f99dbdd9d89881ce981d489756eb508fc0
2016-08-24 18:13:15 +00:00
Michele Baldessari
fbbd134f6b Enable gnocchi systemd services
Marian Krcmari noticed that on a freshly deployed HA Newton system
gnocchi-metricd and gnocchi-statsd are down. This likely came about when
gnocchi moved to composable roles
(I40a3815923099d00a0f3fc1d88a942784e7c6fb9) and those settings made
sense when it was managed by pacemaker.

Also make sure that all services are disabled when managed via
pacemaker and add manage_service: false for gnocchi-api for pacemaker
which was missing.

Change-Id: If7d681e0fa3e84379ac83c8a55b481369fe2849e
2016-08-18 23:18:30 +02:00
Dan Prince
3b62761d2f Add DefaultPasswords to composable services
This patch adds a new DefaultPasswords parameter to
composable services. This is needed to help provide
access to top level password resources that overcloud.yaml
currently manages (passwords for Rabbit, Mysql, etc.).

Moving the RandomString resources into composable services
would cause them to regenerate within the stack. With this
approach we can leave them where they are while we deprecate
the top level mechanism and move the code that uses the
passwords into the composable services.

Change-Id: I4f21603c58a169a093962594e860933306879e3f
2016-08-18 12:45:30 -04:00
Giulio Fidente
885b37c80e Pass ServiceNetMap to services
This will be needed to pick the network where the service has
to bind to from within the service template.

Change-Id: I52652e1ad8c7b360efd2c7af199e35932aaaea8c
2016-08-18 12:36:18 -04:00
Emilien Macchi
a19da8ef49 gnocchi: use new authtoken parameters
Configure Gnocchi with authtoken new class in the Puppet module, and
also remove the useless parameters that didn't exist in the module.

Change-Id: I414990c4fd5c5c1cd43d50c7a3947a4a29f4587a
2016-08-16 21:54:51 -04:00
Juan Antonio Osorio Robles
ba1fb563d1 Move gnocchi's kestone::auth parameters to API profile
In the move to composable services, these parameters are not
necessary in the controller, but in the profile itself. They are not
yet in use but will be used to populate the keystone endpoint.

Change-Id: Iab3ab05e16872d94d3b3ab4827e2f87f4970aee3
2016-08-09 16:29:07 +03:00
Jenkins
9aec3de5b8 Merge "Convert service_name to underscore syntax" 2016-07-29 08:52:05 +00:00