76 Commits

Author SHA1 Message Date
Steven Hardy
47f2579fa2 Don't assume default network names in net_ip*map
This needs to handle a ServiceNetMap containing non-default
network names when they are overridden via the *NetName parameters.

Closes-Bug: #1651541
Change-Id: I95d808444642a37612a495e822e50449a7e7da63
2017-02-17 13:38:31 +00:00
Steven Hardy
3c6ec654b4 Bump template version for all templates to "ocata"
Heat now supports release name aliases, so we can replace
the inconsistent mix of date related versions with one consistent
version that aligns with the supported version of heat for this
t-h-t branch.

This should also help new users who sometimes copy/paste old templates
and discover intrinsic functions in the t-h-t docs don't work because
their template version is too old.

Change-Id: Ib415e7290fea27447460baa280291492df197e54
2016-12-23 11:43:39 +00:00
Michele Baldessari
8d796ea0e4 Add a per service bootstrap node variable
In order to call commands that need to be run on a single node, we
create a new per-service variable that will contain the first node of
each role containing the service.

Change-Id: I03e8685f939e8ae1fcd8b16883b559615042505d
Partial-Bug: #1615983
2016-12-21 11:52:57 +01:00
Dan Prince
7876851011 Hiera optimization: use a new hiera hook
This patch optimizes how we deploy hiera by using a new
heat hook specifically designed to help compose hiera
within heat templates. As part of this change:

 - we update all the 'hiera' software configurations to set the group to hiera
   instead of os-apply-config.

 - The new format uses JSON instead of YAML. The hook actually writes
   out the hiera JSON directly so no conversion takes place. Arrays,
   Strings, Booleans all stay in their native formats. As such we can avoid
   having to do many of the awkward string and list conversions in t-h-t to
   support the previous YAML formatting.

 - The new hook prefers JSON over YAML so upgrading users will have the
   new files prefered. (we will post a cleanup routine for the old files
   soon but this isn't a new behavior, JSON is now simply prefered.)

 - A lot of services required edits to account for default settings that
   worked in YAML that no longer work correctly in the native JSON
   format. In almost all these cases I think the resulting codes looks
   cleaner and is more explicit with regards to what is getting
   configured in hiera on the actual nodes.

Depends-On: I6a383b1ad4ec29458569763bd3f56fd3f2bd726b
Closes-bug: #1596373

Change-Id: Ibe7e2044e200e2c947223286fdf4fd5bcf98c2e1
2016-11-30 22:16:13 -05:00
Dan Prince
9f8944111c Split out hosts config deployment
This patch moves the hosts configuration into its own deployment.
It will continue to use os-apply-config as something that is
required early on in the bootstrapping (it needs to be
configured before puppet runs for example).

The motivation here is so we can refactor all-nodes-config.yaml to use a
new hiera hook that that avoids os-apply-config entirely.

Change-Id: Ib3e4380f205358b27d22a1102b663cf300b1ed86
Partial-bug: #1596373
2016-10-13 14:15:54 -04:00
Jenkins
8b9e9ff4cc Merge "Add flag for internal TLS" 2016-10-13 11:42:23 +00:00
Jenkins
f9c3481c3e Merge "restore missing fluentd client functionality" 2016-10-06 12:34:31 +00:00
Lars Kellogg-Stedman
dd162f42c7 restore missing fluentd client functionality
in the great rebase following the JINJA ALL THE THINGS changes we lost
critical functionality in the fluentd client service.  This review
restores the missing features.

Change-Id: I7c23f16f81e75f3da6a24587b2eb8385b3e920a4
Closes-bug: 1630692
2016-10-05 13:39:06 -04:00
Steven Hardy
6a408ff27e Select per-network hostnames for service_node_names
Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
Depends-On: Ic6fec1057439ed9122d44ef294be890d3ff8a8ee
Change-Id: I754c4a41d8a294a4c7c18bd282ae014efd4b9b16
Closes-Bug: #1628521
2016-10-05 16:24:54 +03:00
Jenkins
81b9392886 Merge "Make keystone api network hiera composable" 2016-10-04 05:18:43 +00:00
Steven Hardy
c947008d9e Make keystone api network hiera composable
These hard-coded references to the Controller role mean that
things won't work if the keystone service is moved to any other
role, so we need to generate the lists dynamically based on the
enabled services for each role.

Change-Id: I5f1250a8a1a38cb3909feeb7d4c1000fd0fabd14
Closes-Bug: #1629096
2016-09-30 16:41:27 +01:00
Juan Antonio Osorio Robles
d2af1b887a Add flag for internal TLS
This sets up a flag that tells the profiles to use TLS (this will happen
in the internal network).

bp tls-via-certmonger

Change-Id: If47febb5b38b1c65f60f9de87a34cb31936a7c0d
2016-09-30 04:32:08 +00:00
Juan Antonio Osorio Robles
57f14d99c1 Add option to specify Certmonger CA
This will be used for internal (or even public) TLS, for when
certmonger is generating the certificates. This same setting is used
for the undercloud with the generate_service_certificate option.

Change-Id: Ic54fe512b9ed5c71417a66491b7954e653f660b6
2016-09-29 17:14:56 +00:00
Jenkins
3cc34744f1 Merge "Generate VIP info for ctlplane VIP, not management" 2016-09-09 18:53:07 +00:00
Jenkins
1be640af93 Merge "Add keystone networks for the different endpoints" 2016-09-09 11:28:14 +00:00
Jenkins
27c38e3784 Merge "Add service networks to hieradata" 2016-09-09 07:46:08 +00:00
Juan Antonio Osorio Robles
975fbb4dca Generate VIP info for ctlplane VIP, not management
The management network does not have a VIP, so it's been wrong to
generate a cloud name and hieradata for this. Instead, the network
that actually needs a name and a hosts entry is the ctlplane network,
which actually has a VIP and there are services that use it.

bp tls-via-certmonger

Closes-Bug: #1621742
Change-Id: I163b2c7b5684da6dc290636f54eefe3f2b0c3e3f
2016-09-09 08:43:24 +03:00
Juan Antonio Osorio Robles
50b5f6ae43 Add keystone networks for the different endpoints
Keystone doesn't provide different flags to indicate that both of its
endpoints are enabled. So currently we have to manually add its
network to all-nodes-config.

bp tls-via-certmogner

Change-Id: Ibecd78706e84853107f698ba411a0c05e6f5be52
2016-09-08 11:12:08 +03:00
Juan Antonio Osorio Robles
2932988126 Add VIP names to allNodesConfig
The nodes need to be aware of the fqdn's for the specific endpoints
in the cloud. This could be either to set the entries in /etc/hosts
or to select an appropriate hostname for a certificate to be
generated.

bp tls-via-certmonger

Change-Id: I9b4645b937a344f46ec18a9a68c5afa2bc5206d0
2016-09-07 03:06:30 +00:00
Jiri Stransky
9d07e18cad Use Redis VIP when deploying with keepalived
Previously we weren't creating Redis VIP in keepalived, causing Redis to
be unusable in non-HA deployments. This is now fixed.

Depends-On: I0bb37f6fb3eed022288b2dcfc7a88e8ff88a7ace
Change-Id: I0ecfda1e6ad5567f6f58d60bf418bc91761833ab
Closes-Bug: #1618510
2016-09-06 15:34:13 +02:00
Jiri Stransky
280a70bfaf Set Redis VIP on all nodes
Move Redis VIP from controller-only to all nodes so that we don't assume
where Redis is deployed.

Change-Id: I55f8d48e3e077951fbcc88158dd6f21a2fe5f457
Related-Bug: #1618510
Partially-Implements: blueprint custom-roles
2016-09-06 15:34:13 +02:00
Juan Antonio Osorio Robles
393c15550a Add service networks to hieradata
This adds a mapping of which service is on which network. This
information can be used to fetch a certificate depending on the
network (since they use different hostnames).

Change-Id: I176245da591bea28aeabf3d2b552f24456c98c43
2016-09-06 12:08:19 +03:00
Jenkins
4488b0fdf8 Merge "Move VIP-related data to AllNodesConfig" 2016-09-06 08:44:17 +00:00
Jenkins
578c4f9e46 Merge "Add default for DeployIdentifier in nested templates" 2016-09-05 12:50:48 +00:00
Juan Antonio Osorio Robles
ddfbf35374 Move VIP-related data to AllNodesConfig
This makes it easier to access the VIP data for other node types and
de-ties this from the controller role.

Change-Id: I71125576ec93889fed134b92fb59f7e7dc9920c4
2016-09-05 13:48:06 +03:00
Steven Hardy
24c596468b Remove hard-coded per-service rabbitmq hieradata
To avoid the hard-coded references which won't work with
composable roles, we instead default to the rabbitmq_node_ips
list in the per-service puppet-tripleo profiles.

Change-Id: I76b7e06781fdd5d969503b6d73423bb3f5f7a41f
Depends-On: Ie53c93456529420588eb1927703ea91b54095d87
Partially-Implements: blueprint custom-roles
2016-09-03 19:45:11 +00:00
Steven Hardy
bf3bc0d551 Generate composable service node_names lists
Some puppet interfaces require a comma separated list of hostnames
where a service is running, so generate it in a similar way to th
service ips.

Change-Id: Icdf5d993d089dc94035194bdbd52299fcbc793be
Partially-Implements: blueprint custom-roles
2016-09-02 13:00:49 +00:00
Steven Hardy
9579bbff10 Add default for DeployIdentifier in nested templates
Until we fix the bug where at validation time heat doesn't know
if a parent passes a value into the nested template, this may
be a workaround for validation failing where no default is found.

Change-Id: I02b0764ac29700cd29584e356ac0cfebcda09a36
Closes-Bug: #1619352
2016-09-01 16:37:23 +01:00
Steven Hardy
10a24b2faa Move ceph cluster config out of allNodesConfig
Pass the list of ceph nodes to the ceph_mon profile via
the service template - this requires some fixup to the
profile to handle the ipv6 case.

Note this also aligns the ServiceNetMap keys so that the
composable node_ips logic will generate the lists when
the ceph_mon service is enabled.

Change-Id: If8a5c65f17e677fe62243b3aa746fd642f72d2b0
Depends-On: I481dd2cd2cde7f1491080e6d9c7dcb7047c22de1
Partially-Implements: blueprint custom-roles
2016-08-30 12:51:50 +02:00
Steven Hardy
2291cdda04 Create composable mapping between enabled services and role ips
Currently we have a hard-coded list of ips for various services that
run on the controller, instead we can dynamically generate that list
of per-service ips, initially only for the controller but this approach
can be extended so it works for any role.

Change-Id: I3c8a946e439539d239ad7281a1395414df0893eb
Partially-Implements: blueprint custom-roles
2016-08-28 10:26:42 +01:00
Steven Hardy
753131d6b5 Create hiera service_enabled keys for enabled services
This adds a list of all enabled service_names in the
enabled_services key, and also generates some boolean
values e.g service_name_enabled, which is more convenient
for some usage (such as haproxy where we need an easy way to
set a flag saying if a given service is enabled)

Partially-Implements: blueprint custom-roles
Change-Id: I62273f403838893602816204d9bc50d516c0057f
2016-08-27 11:33:23 +03:00
Tim Rozet
783975c72c Adds OpenDaylight composable service
Introduces environment files for deploying OpenDaylight in two ways:
 - ODL only managing L2 as an ML2 plugin
 - ODL managing L2 and L3 DVR, by replacing NeutronL3Agent

Two services are added.  One to install ODL and configure OVS on the
Controllers, and another service to only configure OVS on compute nodes.

Paritally-Implements: blueprint opendaylight-integration

Depends-On: I666dc0874f1d11a72a62d796f4f6d41f7aa87a3f

Change-Id: Ide69e20cbf2ec6151953cb23e51478b770aca17f
Signed-off-by: Tim Rozet <trozet@redhat.com>
2016-08-26 09:35:25 -04:00
Steven Hardy
65619843d2 Remove deprecated node_ips hiera keys
This aligns with the new naming conventions in puppet-tripleo, so
the keys can be more easily generated from the service_names.

Change-Id: Idb4a740e70257e3c69d8ec7d0c88594cc091b6a7
Partially-Implements: blueprint custom-roles
Depends-On: I423b544df174254ac511b906b0c570e701678022
2016-08-12 21:28:03 +01:00
Steven Hardy
e8683a863d Align node_ips hiera keys with the service name.
To enable composable generation of this switch the key names
to align with the service_name of each service.

Note that this should depend on I423b544df174254ac511b906b0c570e701678022
and previously passed CI with that defined, but because we now run
gate validation jobs on puppet-tripleo it's impossible to land, so
this now contains both old and new hiera keys temporarily, which will
be removed when the puppet-tripleo patch lands.

Change-Id: I7febf28bf409e25e8e5961ab551b6d56bb11e0c6
Partially-Implements: blueprint custom-roles
2016-08-11 11:53:37 +01:00
Jenkins
ff43f1d38a Merge "Convert AllNodesConfig hosts config to a map" 2016-08-08 12:55:28 +00:00
Ryan Hefner
b99733d08a Enable Manila integration - as a composable controller service
Allows the installation and configuration of Manila.
Supports the generic driver only. This has a dependency on the
puppet-tripleo classes for manila where the puppet specific
config now lives.

The review at https://review.openstack.org/#/c/315658/ has been
merge into this one, as of v68, so manila lands as a composable
service. This was brought up on the mailing list at [1]

[1] http://lists.openstack.org/pipermail/openstack-dev/2016-May/096126.html

Co-Authored-By: Marios Andreou <marios@redhat.com>
Implements: blueprint composable-services-within-roles
Depends-On: I444916d60a67bf730bf4089323dba1c1429e2e71
Depends-On: I9eda4b3364e5c59342761a1ec71b0eb567c69cf1
Depends-On: I571b65a5402c1028418476a573ebeb9450ed00c9
Change-Id: I7acebac4354fca1f8d7ff6c343c1346bf29b81c6
2016-08-02 17:18:07 +03:00
Steven Hardy
7879f9d06b Convert AllNodesConfig hosts config to a map
Currently we have hard-coded parameters for each role, but to enable
custom roles, we need to pass a generic hosts list that can be joined
for all enabled roles.

Change-Id: I0606f462ff61c3a541342b63fee7d46ebfd1f4e0
Partially-Implements: blueprint custom-roles
2016-07-29 15:13:13 +01:00
Emilien Macchi
315fa31963 Migrate Puppet Hieradata to composable services
Migrate puppet/hieradata/*.yaml parameters to puppet/services/*.yaml
except for some services that are not composable yet.

Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
Change-Id: I7e5f8b18ee9aa63a1dffc6facaf88315b07d5fd7
2016-07-27 12:23:38 -04:00
Steven Hardy
b345dbea16 Combine BootstrapNodeDeployment with AllNodesDeployment
Currently we have a special controller-only deployment which writes
the name/ip of the "bootstrap node", e.g the cluster master, which
defaults to the first node in the Controller ResourceGroup.

Now we're moving to fully composable services/roles, it's possible
folks will want to deploy services that expect to detect the bootstrap
node (e.g so only one node does a DB sync) for non-controller roles.

So, take this opportunity to combine the bootstrap node deployment with
the "all nodes" data, such that we deploy the same data for all roles.

Because the boostrap node data is per role cluster, rather than truly
global, we pass it via input_values into each per-role Deployment.

At some future point we might consider renaming this, e.g to
something which describes per-cluster config vs "all nodes",
but as a first step let's just rationalize the resources.

Change-Id: I4011526a13c51b3d0f95c17fe8ed38115b4fdce4
2016-07-05 10:58:31 +01:00
Giulio Fidente
794fece5cc Switch Ceph Monitor/OSD/Client/External to composable roles
Change-Id: I1921115cb6218c7554348636c404245c79937673
Depends-On: I7ac096feb9f5655003becd79d2eea355a047c90b
Depends-On: I871ef420700e6d0ee5c1e444e019d58b3a9a45a6
2016-07-04 16:38:40 +02:00
Imre Farkas
dfbc9380aa Basic support for deploying Ironic in overcloud
Note that this change is not enough yet to deploy bare metal instances,
it only deploys Ironic services themselves and makes sure they work.

Also it does not support HA for now.

Co-Authored-By: Dmitry Tantsur <dtansur@redhat.com>
Partially-implements: blueprint ironic-integration
Change-Id: I541be905022264e2d4828e7c46338f2e300df540
2016-06-29 15:59:08 +02:00
Jenkins
1bbf7a27e3 Merge "Fix distinguishing between stack-create and stack-update" 2016-05-04 10:01:28 +00:00
Pradeep Kilambi
0970068cbb Deploy Gnocchi as a Ceilometer metrics storage backend
* Deploy Gnocchi API.
* Storage backends: swift, rbd and file.
* Indexer backend default to mysql
* Configure Ceilometer to send metrics datas to Gnocchi
* Pacemaker config

Depends-On: Ic8778a3104e0ed0460423e4bf857682220dc5802
Depends-On: I7d2eb9405e0171fc54fa0b616122f69db5f51ce2

Co-Authored-By: Pradeep Kilambi <pkilambi@redhat.com>

Change-Id: Ifde17b1ab8fa2b30544633e455e1c7eb475705aa
2016-04-11 12:27:38 -04:00
Jiri Stransky
aa0bd9eb1b Fix distinguishing between stack-create and stack-update
Previously we tried to use UpdateIdentifier for two different things:
tell whether to perform package update, and also to tell whether the
top-level stack is being created or updated (which was incorrect and
resulted in bug 1567384, and an attempt to work around that bug resulted
in bug 1567385).

We cannot use Heat's "action" conditionals in some cases, because they
refer to the direct parent stack, which can yield undesirable results
when introducing new nested stacks or temporarily no-opping something
and then adding it back (in both these cases, "action" would be
considered "CREATE", even though the top-level stack is in "UPDATE").

So tripleoclient passes a new parameter StackAction to tell whether the
top-level stack is being created or updated, and we make use of
that. (It seems there's no better way of getting this info from within
the nested Heat stacks.)

Change-Id: Ie14ddbff15e7ed21aaa3fcdacf36e0040f912382
Depends-On: I9dc3b4cd8a6a71df34d8babf0e4c6505041f5311
Closes-Bug: #1567384
Related-Bug: #1567385
2016-04-11 14:31:42 +02:00
Pradeep Kilambi
2018c38ed4 Deploy Aodh services, replacing Ceilometer Alarm
Ceilometer Alarm is deprecated in Liberty by Aodh.

This patch:
* manage Aodh Keystone resources
* deploy Aodh API under WSGI, Notifier, Listener and Evaluator
* manage new parameters to customize Aodh deployment
* uses ceilometer DB for the upgrade path
* pacemaker config
* Add migration logic to remove pcs resources

Depends-On: I5333faa72e52d2aa2a622ac2d4b60825aadc52b5
Depends-On: Ib6c9c4c35da3fb55e0ca8e2d5a58ebaf4204d792

Co-Authored-By: Emilien Macchi <emilien@redhat.com>

Change-Id: Ib47a22884afb032ebc1655e1a4a06bfe70249134
2016-03-20 10:27:21 -04:00
Jenkins
bb05fa304a Merge "Fixup the memcached servers string in nova.conf for v6" 2016-03-08 09:11:05 +00:00
Jenkins
89b1177f58 Merge "Fix rabbit_hosts list for glance-api for IPv6" 2016-03-07 16:24:01 +00:00
marios
c0f8a2aad6 Fixup the memcached servers string in nova.conf for v6
As discussed at https://bugzilla.redhat.com/show_bug.cgi?id=1299265
when providing a list of IPv6 addresses as the memcache_node_ips
the resulting nova.conf entry can't be parsed properly.

This adds a memcache_node_ips_v6 which has the required format like
inet6:[ADDR1],inet6:[ADDR2],inet6:[ADDR3]

Closes-Bug: 1536103
Change-Id: I7f95fa063cbba279c4c2e270841f0a279d2be2f6
2016-03-07 09:46:09 +02:00
James Slagle
8da8b84560 Revert "Deploy Aodh services, replacing Ceilometer Alarm"
This just a revert to see if reverting this gets back to a normal CI run time.

This reverts commit f72aed85594f223b6f888e6d0af3c880ea581a66.

Change-Id: I04a0893f6cf69f547a4db26261005e580e1fc90b
2016-03-04 23:05:35 -05:00
Giulio Fidente
164d537378 Fix rabbit_hosts list for glance-api for IPv6
Previously we were always appending the :port suffix to the list
of rabbitmq nodes but the syntax was invalid for IPv6.

This change wires rabbit_hosts from the templates as it happens
already for the other services. Port can be customized using
rabbit_port.

Change-Id: Iecc7a97d46d7de17e85398c57996c104c9125b0e
2016-03-05 02:36:49 +01:00