41 Commits

Author SHA1 Message Date
Emilien Macchi
6c6c784865 Remove glance-base service
glance-base is not useful anymore since we only run Glance API service
and there is no plan yet to add new services for Glance. Let's cleanup
this useless service and consolidate glance-api service.

Change-Id: I73cd0def2ae73e0bd52104c6710998df4a0d2e58
2017-03-07 17:03:18 -05:00
Emilien Macchi
7c84a9b390 upgrades/validation: only run validation when services exist
During upgrades, validation test if a service is running before the
upgrade process starts.
In some cases, servies doesn't exist yet so we don't want to run the
validation.

This patch makes sure we check if the service is actually present on the
system before validating it's running correctly.

Also it makes sure that services are enabled before trying to stop them.
It allows use-cases where we want to add new services during an upgrade.
Also install new packages of services added in Ocata, so we can validate
upgrades on scenarios jobs.

Change-Id: Ib48fb6b1557be43956557cbde4cbe26b53a50bd8
2017-03-01 19:49:00 +00:00
Sofer Athlan-Guyot
fb78213782 Put service stop at step1 and quiesce at step2.
In the previous release[1], the services were stopped before the
pacemaker services, so that they get a chance to send last message to
the database/rabbitmq queue:

Let's do the upgrade in the same order.

[1] https://github.com/openstack/tripleo-heat-templates/blob/stable/newton/extraconfig/tasks/major_upgrade_controller_pacemaker_2.sh#L13-L71

Change-Id: I1c4045e8b9167396c9dfa4da99973102f1af1218
2017-02-28 19:20:13 +01:00
Michele Baldessari
90431683b5 Make the DB URIs host-independent for all services
When fixing LP#1643487 we added ?bind_address to all DB URIs.
Since this clashes with Cellsv2 due to the URIs becoming host
dependent, we need a new approach to pass bind_address to pymysql
that leaves the DB URIs host-independent.

In change Iff8bd2d9ee85f7bb1445aa2e1b3cfbff1f397b18 we first create a
/etc/my.cnf.d/tripleo.cnf file with a [tripleo] section with the correct
bind-address option.

In this change we make sure that the DB URIs will point to the added
file and to the specific section containing the necessary bind-address
option. We do introduce a new MySQLClient profile which will hold all
this more client-specific configuration so that this change can fit
better in the composable roles work. Also, in the future it might
contain the necessary configuration for SSL for example.

Note that in case the /etc/my.cnf.d/tripleo.cnf file does not exist
(because it is created via the mysqlclient profile), things keep on
working as usual and the bind-address option simply won't be set, which
has no impact on hosts where there are no VIPs.

Co-Authored-By: Damien Ciabrini <dciabrin@redhat.com>

Change-Id: Ieac33efe38f32e949fd89545eb1cd8e0fe114a12
Related-Bug: #1643487
Closes-Bug: #1663181
Closes-Bug: #1664524
Depends-On: Iff8bd2d9ee85f7bb1445aa2e1b3cfbff1f397b18
2017-02-17 17:22:42 +01:00
Juan Antonio Osorio Robles
d1eb0bc0dc Use Keystone internal endpoint instead of admin for services
The admin endpoint is listening on the ctlplane network by default;
services should ideally be using the internal api network for this kind
of traffic, as the ctlplane network is mostly for provisioning. On the
other hand, the admin endpoint shouldn't be as relevant with services
switching to keystone v3.

Change-Id: I1213a83ef8693c1cca1d20de974f7949a801d9f1
2017-02-14 02:41:13 +00:00
marios
ec5ba081c4 Remove [heat,glance,ironic,cinder,keystone] db sync from ansible
These are handle by puppet as usual (puppet run comes after the
ansible steps) so remove them from these remaining upgrade_tasks

Change-Id: Ic341f31251622ccb11a5f7818b2edf7a82391560
2017-02-13 13:54:21 +02:00
marios
d14c56e1b6 Adds a pre-upgrade check that service is running (step0)
Adds a step0 for most services to check that the state is running
before continuing with any of the other upgrades steps (these are
tagged step0).

You can skip this service check by overriding the
SkipUpgradeConfigTags parameter as follows:

parameter_defaults:
  SkipUpgradeConfigTags: validation

Co-Authored-By: Steven Hardy <shardy@redhat.com>
Change-Id: Ie276f153015f671b720b6ed5beaac1b921661909
2017-01-27 11:20:15 +02:00
Juan Antonio Osorio Robles
a88261aa05 Pass parameters for TLS proxy in front of Glance-API
If TLS in the internal network is enabled, we run glance-api beind a
TLS proxy (which is actually httpd's mod_proxy). This passes the
necessary hieradata.

bp tls-via-certmonger
Change-Id: I693213a1f35021b540202240e512d121cc1cd0eb
Depends-On: Id35a846d43ecae8903a0d58306d9803d5ea00bee
2017-01-24 17:52:22 +00:00
Steven Hardy
df1e016ad7 Don't start all services during upgrade steps
Currently we start all OpenStack services in step6, but puppet
already does this, and sometimes services require configuration
to account for the new version after the yum update before they
will start.

So instead of reimplementing that configuration management in
ansible, just defer starting the services until puppet has run
which will happen right after the ansible upgrade steps complete.

Note there are some DB sync operations etc that we may also be able
to remove and let puppet do those steps, but I've left those in
for now, as we know there are some actions during that phase
e.g nova cells setup, which aren't yet handled by puppet.

Change-Id: Idc8e253167a4bc74b086830cfabf28d4aab97d28
2017-01-19 13:27:58 +00:00
Steven Hardy
d5d8701c45 Disable glance registry during upgrade
Change-Id: I447ce74cca93fcae87ca608ecc8eeb2721fecefb
2017-01-19 13:27:58 +00:00
Emilien Macchi
4ccb27ab81 Remove Glance Registry service
Glance registry is not required for the v2 of the API and there are
plans to deprecate it in the glance community.

Let's remove v1 support since it has been deprecated for a while in
Glance.

Depends-On: I77db1e1789fba0fb8ac014d6d1f8f5a8ae98ae84
Co-Authored: Flavio Percoco <flaper87@gmail.com>
Change-Id: I0cd722e8c5a43fd19336e23a7fada71c257a8e2d
2017-01-16 17:04:19 -05:00
Steven Hardy
9245880ae4 Add glance service support for composable upgrades
Change-Id: I730abee756598c0a23209a53e52cc83e0b815a50
Partially-Implements: blueprint overcloud-upgrades-per-service
2017-01-12 09:51:47 +00:00
Jenkins
0bfe7c9279 Merge "DB connection: prevent src address from binding to a VIP" 2017-01-04 16:43:34 +00:00
Damien Ciabrini
56ebc7e58d DB connection: prevent src address from binding to a VIP
When a service connects to the database VIP from the node hosting this
VIP, the resulting TCP socket has a src address which is by default
bound to the VIP as well. If the VIP is failed over to another node
while the socket's Send-Q is not empty, TCP keepalive won't engage and
the service will become unavailable for a very long time (by default
more than 10m).

To prevent failover issues, DB connections should have the src address
of their TCP socket bound to the IP of the network interface used for
MySQL traffic. This is achieved by passing a new option to the
database connection URIs. This option is available starting from
PyMySQL 0.7.9-2.

We use a new intermediate variable in hiera to hold the IP to be used
as a source address for all DB connections. All services adapt their
database URI accordingly.

Moreover, a new YAML validation check is added to guarantee that new
services will construct their database URI appropriately.

Change-Id: Ic69de63acbfb992314ea30a3a9b17c0b5341c035
Closes-Bug: #1643487
2017-01-03 10:56:02 +01:00
Steven Hardy
3c6ec654b4 Bump template version for all templates to "ocata"
Heat now supports release name aliases, so we can replace
the inconsistent mix of date related versions with one consistent
version that aligns with the supported version of heat for this
t-h-t branch.

This should also help new users who sometimes copy/paste old templates
and discover intrinsic functions in the t-h-t docs don't work because
their template version is too old.

Change-Id: Ib415e7290fea27447460baa280291492df197e54
2016-12-23 11:43:39 +00:00
Dan Prince
04486223fd Allow Glance API and Registry to be split
The glance-api and glance-registry services are currently coupled
in that some of the hiera settings in the API are required for
the registry to run correctly (the backend settings).

This patch moves some of the common settings into glance-base and
then updates the glance-api and glance-registry services to
supply that service.

Change-Id: Ie3d7e24c7fd475e3f6ad542c1654eb7dbd9d9b35
Closes-bug: #1628582
2016-10-12 08:10:02 -04:00
Pradeep Kilambi
6d9f97f359 Set the notification driver for glance
Need to set the right default notification driver for glance so
telemetry receives them accordingly. Without this tempest tests
fail.

Closes-bug: #1631939

Change-Id: I1cee5467d077eea6142076925646f7d0cdae96c7
2016-10-10 08:46:46 -04:00
Dan Prince
7ba5525207 Move db::mysql into service_config_settings
This patch movs the various db::mysql hiera settings into a
'mysql' specific service_config_settings section for each
service so that these will only get applied on the MySQL service
node. This follows a similar puppet-tripleo change where we
create the actual databases for all services locally on
the MySQL service node to avoid permission issues.

Change-Id: Ic0692b1f7aa8409699630ef3924c4be98ca6ffb2
Closes-bug: #1620595
Depends-On: I05cc0afa9373429a3197c194c3e8f784ae96de5f
Depends-On: I5e1ef2dc6de6f67d7c509e299855baec371f614d
2016-09-28 07:01:49 -04:00
Dan Prince
9d67d7b3b1 Move keystone::auth into service_config_settings
This patch moves the keystone::auth settings for all
services into the new service_config_settings section. This
is important because we execute the keystone commands via
puppet only on the role containing the keystone service
and without these settings it will fail.

Note that yaql merging/filtering is used here to ensure that
service_config_settings is optional in service templates,
and also that we'll only deploy hieradata for a given
service on a node running the service (the key in
the service_config_settings map must match the service_name
in the service template for this to work).

e.g the following will result in only deploying keystone: 123
in hiera on the role running the "keystone" service,
regardless of which service template defines it.

  service_config_settings:
    keystone:
      keystone: 123

Co-Authored-By: Steven Hardy <shardy@redhat.com>
Change-Id: I0c2fce037a1a38772f998d582a816b4b703f8265
Closes-bug: 1620829
2016-09-23 07:43:21 -04:00
Joe Talerico
e6ecdb8b18 Glance worker count fix
This patch changes the default value and type of the Glance worker
configuration to allow it to be unset and allow a system dependent
default to be used (e.g. processorcount or some derivative value). The
previous default of 0 would result in a single self contained process,
which while suitable for debugging and testing is not appropriate for
production deployments.

Partial-Bug: #1626126
Change-Id: I58a6a72a581e7083e1dc4e5ca568fdd3fdd6cdf1
2016-09-21 12:22:36 -02:30
Lars Kellogg-Stedman
0d9298bb8f Add fluentd client service
This implements support for installing fluentd agents as a composable
service on the overcloud.

Depends-On: I2e1abe4d8c8359e56ff626255ee50c9cacca1940

Implements: tripleo-opstools-centralized-logging
Change-Id: I23b0e23881b742158fcfb6b8c145a3211d45086e
2016-09-17 01:31:12 +00:00
Juan Antonio Osorio Robles
af5f892692 Set client protocol for glance registry client
To communicate to glance registry, glance API has several parameters
that it uses to form the URI. Right now we are defaulting to http,
when we enable TLS everywhere, this will break. So setting the value
from the endpoint map should fix it.

Closes-Bug: #1623477
Change-Id: Id86787cbaa6f87fdcf9c26111c228fd59fbba012
2016-09-14 15:38:42 +03:00
Jenkins
d6837ea4a6 Merge "Availability monitoring agents support" 2016-09-02 10:00:14 +00:00
Emilien Macchi
98c6bdaa99 Last round of modern authtoken update
It updates Glance, Neutron and Swift to deploy authtoken with modern
pattern.

Change-Id: Icfaf011ea4a23bc47d2fb45e8768f8238532dab3
2016-08-31 18:42:44 +00:00
Martin Mágr
25ad7b8e1e Availability monitoring agents support
- adds possibility to install sensu-client on all nodes
- each composable service has it's own subscription

Co-Authored-By: Emilien Macchi <emilien@redhat.com>
Co-Authored-By: Michele Baldessari <michele@redhat.com>
Implements: blueprint tripleo-opstools-availability-monitoring
Change-Id: I6a215763fd0f0015285b3573305d18d0f56c7770
2016-08-31 09:22:59 -04:00
Dan Prince
43476e235c Move glance/heat hiera settings to services
This patch removes the remaining bind IP, and password
settings for Heat and Glance into the composable services.

Change-Id: I17abcb2a08a1972cbcf8163f6608ac22ddfc15f7
Related-bug: #1604414
2016-08-25 08:21:56 -04:00
Dan Prince
3b62761d2f Add DefaultPasswords to composable services
This patch adds a new DefaultPasswords parameter to
composable services. This is needed to help provide
access to top level password resources that overcloud.yaml
currently manages (passwords for Rabbit, Mysql, etc.).

Moving the RandomString resources into composable services
would cause them to regenerate within the stack. With this
approach we can leave them where they are while we deprecate
the top level mechanism and move the code that uses the
passwords into the composable services.

Change-Id: I4f21603c58a169a093962594e860933306879e3f
2016-08-18 12:45:30 -04:00
Giulio Fidente
885b37c80e Pass ServiceNetMap to services
This will be needed to pick the network where the service has
to bind to from within the service template.

Change-Id: I52652e1ad8c7b360efd2c7af199e35932aaaea8c
2016-08-18 12:36:18 -04:00
Juan Antonio Osorio Robles
219acaa5c2 Move setting of Glance's keystone region to API profile
This is not necessary in the controller.yaml and is more appropriate
in the profile.

Change-Id: Ie2badbd87eabb8404acff77e9aa5d091fbdd1499
2016-08-08 16:58:06 +03:00
Michele Baldessari
81de065665 Next generation HA architecture work
This is the THT part that brings us the next generation architecture
as described in the following spec:
https://review.openstack.org/#/c/299628/

Blueprint:
https://blueprints.launchpad.net/tripleo/+spec/ha-lightweight-architecture

So far we tested deployment + tripleo.sh --overcloud-pingtest and
failover + tripleo.sh --overcloud-pingtest

Note that many of the Pacemaker template files become redundant with
this change, but to simplify the process of getting this change landed,
those templates will not be removed until a future commit.

Depends-On: I5e7585c08675d8a4bd071523b94210d325d79b59

Change-Id: I00bccb2563c006f80baed623b64f1e17af20dd4e
Implements: blueprint ha-lightweight-architecture
Co-Author: cmsj@tenshu.net
2016-08-04 15:07:39 +02:00
Juan Antonio Osorio Robles
c2c8f0f39a Enable glance to use the SSL middleware
The http_proxy_to_wsgi middleware was recently added to glance as
default in the pipeline [1].

We already enable this middleware for nova, cinder and heat.

[1] I481d88020b6e8420ce4b9072dd30ec82fe3fb4f7

Change-Id: I4a8f7fc079ca93c50aa0ef7b0548dc64f6c5cfa0
Depends-On: I51fbc6050dfbdc72f7ee56a2d17dd5223a208a17
2016-07-29 20:37:20 +00:00
Jenkins
9aec3de5b8 Merge "Convert service_name to underscore syntax" 2016-07-29 08:52:05 +00:00
Steven Hardy
7df649f59e Convert service_name to underscore syntax
Currently we use hyphens, e.g cinder-api, but in overcloud.yaml
we have a lot of references to services (e.g for AllNodesConfig)
by underscore, e.g cinder_api.  To enable dynamic generation of
this data, we need the service name in underscore format.

Change-Id: Ief13dfe5d8d7691dfe2534ad5c39d7eacbcb6f70
2016-07-28 16:31:36 +01:00
Emilien Macchi
315fa31963 Migrate Puppet Hieradata to composable services
Migrate puppet/hieradata/*.yaml parameters to puppet/services/*.yaml
except for some services that are not composable yet.

Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
Change-Id: I7e5f8b18ee9aa63a1dffc6facaf88315b07d5fd7
2016-07-27 12:23:38 -04:00
Dan Prince
5195d7f891 Composable firewall rules
Split out the firewall rules in puppet/hieradata/controller.yaml
into the composable services

Depends-On: Id370362ab57347b75b1ab25afda877885b047263
Change-Id: Icaecab100d3f278035fbbb3facb9bf6c62c76c03
2016-07-25 15:24:16 +02:00
Dan Prince
6b30ff11d4 Add 'service_name' to composable services
This patch adds a new service_name section to each composable
service. We now have an explicit unit test check to ensure that
service_name exists in tools/yaml-validate.py.

This patch also wires service_names into hieradata on each
of the roles so that tools can access the deployed services locally
during deployment and upgrades.

Change-Id: I60861c5aa760534db3e314bba16a13b90ea72f0c
2016-07-22 07:29:39 -04:00
Giulio Fidente
794fece5cc Switch Ceph Monitor/OSD/Client/External to composable roles
Change-Id: I1921115cb6218c7554348636c404245c79937673
Depends-On: I7ac096feb9f5655003becd79d2eea355a047c90b
Depends-On: I871ef420700e6d0ee5c1e444e019d58b3a9a45a6
2016-07-04 16:38:40 +02:00
Giulio Fidente
a6438a2082 Pass MysqlVirtualIP via EndpointMap
By passing the MysqlVirtualIP via the EndpointMap we won't need it
to be provided as a parameter to the services.

This follows what is already happening for the glance registry
service with I9186e56cd4746a60e65dc5ac12e6595ac56505f0.

Change-Id: Iad2ab389bf64d0fc8b06eb0e7d29b5370ff27dff
Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
2016-05-30 10:22:59 +03:00
Juan Antonio Osorio Robles
995ad9c32b Pass parameters to manage endpoints via puppet
This commit passes the necessary hieradata in order to create
the endpoints, users and roles of the services in keystone via
puppet.

Change-Id: I2470dfa4661be7ba8218f6035fffa05f547214f0
2016-05-04 17:23:52 +03:00
Giulio Fidente
7e08362835 Wire missing RabbitClientPort into Glance API role
Change-Id: I0ebb5a1e504dd3ffef8ec15c721cf9a9bce6f05b
2016-04-29 10:24:29 -05:00
Dan Prince
7588f74919 composable glance services
Adds new puppet and puppet pacemaker specific services for
Glance API and Glance Registry.

The Pacemaker templates extend the default glance services and
swap in the pacemaker specific puppet-tripleo profile instead.
In the case of pacemaker glance-registry there is no separate
puppet manifest so only the configuration parameters are maintained
there. (Due to the way the pacemaker glance constraints are written
the pacemaker varients of this service can't be split out...)

Depends-On: Ifc388f7058ccfff2818f531bcbc00c7179874bbc
Change-Id: I00a8c916129af43cda225754eb10370289bb4b41
2016-04-21 01:03:51 +00:00