128 Commits

Author SHA1 Message Date
Harald Jensås
2f2d8183e6 L3 routed networks - subnet fixed_ips (3/3)
When using neutron routed networks we need to specify
either the subnet or a ip address in the fixed-ips-request
when creating neutron ports.

a) For the Vip's:

Adds VipSubnetMap and VipSubnetMapDefaults parameters in
service_net_map.yaml. The two maps are merged, so that the
operator can override the subnet where VIP port should be
hosted. For example:

parameter_defaults:
  VipSubnetMap:
    ctlplane: ctlplane-leaf1
    InternalApi: internal_api_leaf1
    Storage: storage_leaf1
    redis: internal_api_leaf1

b) For overcloud node ports:

Enrich 'networks' in roles defenition to include both
network and subnet data. Changes the list to a map
instead of a list of strings. New schema:

- name: <role_name>
  networks:
    <network_name>
      subnet: <subnet_name>

For backward compatibility a conditional is used to check
if the data is a map or not. In either case the internal
list of role networks is created as '_role_networks' in
the jinja2 templates.

When the data is a map, and the map contains the 'subnet'
key the subnet specified in roles_data.yaml is used as
the subnet in the fixed-ips-reqest when ports are created.
If subnet is not set (or role.networks is not a map) the
default will be {{network.name_lower}}_subnet.

Also, since the fixed_ips request passed to Vip ports are no
longer [] by default, the conditinal has been updated to
test for 'ip_address' entries in the request.

Partial: blueprint tripleo-routed-networks-templates
Depends-On: I773a38fd903fe287132151a4d178326a46890969
Change-Id: I77edc82723d00bfece6752b5dd2c79137db93443
2019-01-03 19:07:20 +01:00
Harald Jensås
e644e3dda9 Add MTU to neutron networks and nic-config templates
Neutron has support[1] to set the guaranteed MTU for
networks and network segments so that this is exposed
to plug-ins. In interest of supporting the use of
plug-ins to configure network devices in the future
this change adds MTU property on neutron networks.

The new (optional) property 'mtu' in the network
defenitions in 'network_data.yaml' is used to control
the MTU settings. By default the mtu is '1500'.

We already configure the MTU on the ctlplane neutron
networks, this adds the MTU to composable networks.

Also update the nic-config sample templates to include
mtu settings. A heat value resource is added to
nic-config templates to get the required minimum
viable MTU value for bridges, bonds and member
interfaces to ensure the MTU is large enough to allow
the largest configured MTU to traverse the path.

Closes-Bug: #1790537
Change-Id: I11e38f82eb9040f77412fe8ad200fcc48031e2f8
2018-12-22 17:03:09 +01:00
Thomas Herve
862f52cce0 Put user data in the main stack
We create user data per instance, but two are global for all, and the
last one per role, so we can move it up the stack.

Change-Id: I1330e54744adef9be159edd8f01aefa3db85a480
2018-12-07 15:45:10 +01:00
Zuul
769f18f0f5 Merge "Check for available networks for a role" 2018-12-05 19:01:14 +00:00
Zuul
a80bb71642 Merge "ctlplane pre-alloc IPs - deprecated/non-deprecated role name" 2018-12-04 14:03:07 +00:00
Harald Jensås
7a1cd822f6 ctlplane pre-alloc IPs - deprecated/non-deprecated role name
By default, Compute role template set's the deprecated_param_ips
parameter in roles data. This forces the use of the deprecated
names in paramer_defaults when using predictable IPs for the
ctlplane network.

To allow the user to either use the deprecated role name, or the
non deprecated role name in parameters defaults extend the
ctlplane_fixed_ip_set contition to use or logic to test for data
in either the deprecated name parameter or the new parameter.

In the server resource use yaql to pick the first element that
is not empty. The non-deprecated parameter name is prioritiezed.

Change-Id: Iedc65064c5efaa618c3d54df10bf09296829efd2
Closes-Bug: #1805482
2018-11-27 20:20:50 +01:00
Harald Jensås
055e252872 Add ctlplane interface routes
For the isolated networks we use the subnets host_routes
to set and get the routes for overcloud node interfaces.

This change add's this to the ctlplane interface.

Partial: blueprint tripleo-routed-networks-templates
Change-Id: Id4cf0cc17bc331ae27f8d0ef8f285050330b7be0
2018-11-26 17:49:31 +01:00
Rabi Mishra
5d275fb922 Check for available networks for a role
For network isolation, we specifcy available networks for role.
Therefore, there is no point in creating noop network resources for
networks that are not available/connected. This results in redundant
host entries for not available networks on overcloud nodes.

If a network is not available for a role we don't need to create
those extra noop resources.

For Undercloud/Standalone role we keep all networks in roles data
as the default ServiceNetMap specifies non ctlplane networks though
they map to ctlplane.

Change-Id: I07822ec0cba7eed352c0010eb893b5e5a522e95c
Closes-Bug: #1800811
2018-11-19 10:14:34 +05:30
Juan Antonio Osorio Robles
cb3c72f37d Remove references to logging_source
This has been unused for a while, and even deprecation was scheduled
(although the patch never merged [1]). So, in order to stop folks
getting confused with this, it's being removed.

[1] https://review.openstack.org/#/c/543871/

Change-Id: Iada64874432146ef311682f26af5990469790ed2
2018-10-08 13:43:47 +03:00
Steve Baker
6bbc3b51df docker-puppet.py: used dedicated hiera entry, not uuid
Currently it is not possible to do per-node customization inside
docker-puppet.py because it overrides the fact 'uuid'.

This change adds a dedicated docker_puppet entry in hiera.yaml so that
docker-puppet.py needs to do nothing special for
/etc/puppet/hieradata/docker_puppet.json to be included in the hiera
merge.

Change-Id: Icf37dcd63e0152ee15e9f0079b45e31a4f8d9fbb
Depends-On: https://review.openstack.org/#/c/605478/
Closes-Bug: #1761624
2018-10-01 12:21:46 -04:00
Zuul
85da81624f Merge "Remove references to logging_group" 2018-09-19 08:01:19 +00:00
Zuul
8f513c4f2f Merge "Simplify ssh known_hosts entries for non-default port" 2018-09-08 09:14:43 +00:00
Zuul
0165b1c046 Merge "Always lowercase role name" 2018-08-31 08:43:45 +00:00
Oliver Walsh
c70d197d36 Simplify ssh known_hosts entries for non-default port
'[host]*' matches both default port and non-default port.

Change-Id: Id83bed36f3ab7f8d0fbdbd03f3960307af62fc84
Related-bug: #1789452
2018-08-30 12:23:45 +01:00
Sergii Golovatiuk
0cf4bff9e6 Always lowercase role name
Due to [1] ansible always access servers lowcase. Also, in respect to [2], this
patch lowercase name which is use in fqdn, hostname, ssh_known_hosts and other
places.

[1] aa4278e5f3
[2] https://tools.ietf.org/html/rfc4343

Change-Id: Ib25832496d6504def436414b9c2903cbfe5854d4
Resolves: rhbz#1619556
2018-08-30 00:21:05 +02:00
Juan Antonio Osorio Robles
90234f4f2a Remove references to logging_group
This has been unused for a while, and even deprecation was scheduled
(although the patch never merged [1]). So, in order to stop folks
getting confused with this, it's being removed.

[1] https://review.openstack.org/#/c/543871/

Change-Id: Icc6b51044ccc826f5b629eb1abd3342813ed84c0
2018-08-29 13:43:30 +03:00
Oliver Walsh
876683f317 Include ssh known_hosts entries for non-default port
The ssh client no longer appears to accept the regular known hosts entry when
the target is running on a non-default port.
Adding '[host]:*' should fix this, regardless of the port.
However this does not work for the default port so we must include both.

Change-Id: I519ff6053676870dff1bdff60fb1f6b2aa5ee8c9
Closes-bug: #1789452
2018-08-28 16:38:14 +01:00
Zuul
69c0bd6a8b Merge "host_routes using get_attr (Composable Networks)" 2018-08-14 19:40:25 +00:00
Jill Rouleau
6a1ac35275 DeployedServer support for cidr via get_attr
Change I92ee0f9a2107cdf1ca5903d3756a235a79c36c73 implemented a
conditional for ControlPlaneSubnetCidr to be resolved using get_attr
and a str_split. DeployedServer documentation specifies that this
value should be provided to DeployedServerPortMap as a numeric value
(ie; 24) rather than full cidr (ie; 192.168.24.0/24), which causes the
str_split to fail.

This change provides support for both via a yaql expression, and adds
a deprecation note to switch to the full cidr format.

Change-Id: I6d4422b4f1aa52aee2954eb447008d3d24fe8736
2018-08-07 09:53:06 -07:00
Harald Jensås
d27ec26e56 host_routes using get_attr (Composable Networks)
Uses get_attr on the port resource to resolve attribute
value from neutron subnet 'host_routes' attribute and
pass it to the parameter '{{network.name}}InterfaceRoutes'
in network configuration templates.

A conditional is used in puppet/role.role.j2.yaml. The
user provided parameter value is used whenever it is
not the default: []. This allow advanced user's to
override the routes in neutron.

Co-Authored-By: Dan Sneddon <dsneddon@redhat.com>
Partial: blueprint tripleo-routed-networks-templates
Change-Id: Ie44b211c4aeab9ca79d7994f31961e34aa3517e6
2018-07-30 10:03:34 +02:00
Zuul
3e875145cd Merge "DnsServers using get_attr" 2018-07-25 00:53:28 +00:00
Zuul
7ce066eba3 Merge "EC2MetadatIp using get_attr" 2018-07-25 00:36:06 +00:00
Zuul
ad5d4d1db1 Merge "ControlPlaneDefaultRoute using get_attr" 2018-07-24 01:34:22 +00:00
Zuul
dfc09b6ff1 Merge "ControlPlaneSubnetCidr using get_attr" 2018-07-24 01:34:20 +00:00
Zuul
a9fda38818 Merge "Fix HostnameMap lookup - replace str_replace with yaql" 2018-07-20 04:31:45 +00:00
Harald Jensås
aeb783fdcf Fix HostnameMap lookup - replace str_replace with yaql
Using str_replace when looking up hostnames in
HostnameMap may return unpredictable results.
For example if the map contain:

HostnameMap:
  overcloud-compute-12: cmp-12-london
  overcloud-compute-123: cmp-123-berlin

When we do string replace for a hostname that is
not in the map (example: 'overcloud-compute-124')
it will match on the substring. The result would
be 'cmp-12-london4'.

This change replaces the str_replace with a yaql
expression that will return the value of they key
matching the hostname provided. If the key is not
in the map, the hostname is returned.

Closes-Bug: #1781560
Change-Id: Ia3a6a3539cd566131cd81b4d755190aa5b61a573
2018-07-19 07:24:29 +00:00
Harald Jensås
52ec1a018b DnsServers using get_attr
Nameservers are configured on the ctlplane subnets by the
undercloud installer, the nameservers are used early during
the deployment, prior to running os-net-config.

Remove the default DnsServer's in THT, replacing it with
an empty list and use get_attr to get the values for
DnsServers for the overcloud from the ctlplane subnet(s).

A conditinal is used in  puppet/role.role.j2.yaml so that
the parameter value is used whenever it is not [] (default)
to provide backwards compatibilityi and in case the user
want to use different DnsServers for the overcloud and
undercloud.

Partial: blueprint tripleo-routed-networks-templates
Change-Id: I5f33e06ca3f4b13cc355e02156edd9d8a1f773cd
2018-07-14 09:19:12 +02:00
Harald Jensås
19381ecffb EC2MetadatIp using get_attr
The route to metadata service is set up in host_routes
of ctlplane subnets by extraconf post deploy::
  extraconfig/post_deploy/undercloud_ctlplane_network.py

Use get_attr on the server resource to resolve attribute
value from the subnet(s) and pass it to the parameter
'EC2MetadatIp' used in the THT/network/config/* templates.

Changes the default for 'EC2MetadatIp' to ''.
Removes the comment that the value should be overriden in
parameters_defaults. It also removes the parameter from
network-environment templates.

A conditinal is used in  puppet/role.role.j2.yaml so that
the parameter value is used whenever it is not '' (the
default) to provide backwards compatibility in case the
user set a different value for this parameter in
network-environment.yaml.

When deploying a routed control plane the network config
templates would previously need to be updated to carry
'EC2MetadatIpLeafX' parameters for each leaf.  By getting
the value to pass from the server resource this change
reduces the required nic-config template customisation.
(Reduces the risk of user error.)

Partial: blueprint tripleo-routed-networks-templates
Change-Id: I9c019ec840a44ca8c5f98be55daea365bc6554ec
2018-07-14 09:18:58 +02:00
Harald Jensås
c649cf0545 ControlPlaneDefaultRoute using get_attr
Use get_attr on the server resource to resolve attribute
value from the subnet(s) and pass it to the parameter
'ControlPlaneDefaultRoute' used in the THT/network/config/*
templates.

Changes the default for 'ControlPlaneDefaultRoute' to ''
as well as the comment that the value should be overriden
in parameters_defaults. It also removes the parameter from
network-environment templates.

A conditinal is used in  puppet/role.role.j2.yaml so that
the parameter value is used whenever it is not '' (the
default) to provide backwards compatibility in case the
user set a different value (different from the one used in
undercloud.conf) for this parameter in
network-environment.yaml.

When deploying a routed control plane the network config
templates would previously need to be updated to carry
'ControlPlaneXDefaultRoute' parameters for each leaf. With
8 Leafs in addition to the network local to the undercloud
that is 8 parameters less to place in the configuration.
By getting the value to pass from the server resource this
change reduces the required nic-config template
customisation (reduces the risk of user error).

Partial: blueprint tripleo-routed-networks-templates
Change-Id: I5139249d55e9ac01761c270b8c0f31ef35595940
2018-07-14 09:11:39 +02:00
Harald Jensås
6ab86a3ebe ControlPlaneSubnetCidr using get_attr
Use get_attr on the server resource to resolve attribute
value from the subnet(s) and pass it to the parameter
'ControlPlaneSubnetCidr' used in the THT/network/config/*
templates.

As the value is now resolved from resource attributes,
this changes the default for 'ControlPlaneSubnetCidr' to ''
as well as the comment that these value should be overriden
in parameters_defaults. It also removes the parameter from
network-environment templates.

A conditinal is used in  puppet/role.role.j2.yaml so that
the parameter value is used whenever it is not '' (the
default) to provide backwards compatibility in case the user
set a different value (different from the one used in
undercloud.conf) for this parameter in
network-environment.yaml.

When deploying a routed control plane the network config
templates would previously need to be updated to carry
'ControlPlaneXSubnetCidr' parameter (in case the subnet
mask is not the same for all the routed network leafs).
With 8 Leafs in addition to the network local to the
undercloud that is 8 parameters less to place in the
configuration. By getting the value to pass from the
server resource this change reduces the required nic-config
template customisation (reduces the risk of user error).

Partial: blueprint tripleo-routed-networks-templates
Change-Id: I92ee0f9a2107cdf1ca5903d3756a235a79c36c73
2018-07-14 09:11:28 +02:00
Harald Jensås
80d154d709 Give neutron ports names: $HOSTNAME_$NETWORKNAME
Set the name property of neutron ports created for
composable networks to:
 $HOSTNAME_$NETWORKNAME

This is helpful when troubleshooting, a simple port
list will tell us what port and ip is assigned to the
different overcloud nodes.

Change-Id: I9d067654a81826d79574d17bd073747b2f59d114
2018-07-13 10:13:34 +02:00
Zuul
beeda14b65 Merge "Add ServerDeletionPolicy parameter" 2018-07-12 14:59:03 +00:00
Harald Jensås
7817fccc5b Add support for {{role.name}}IPs in Composable Role
Use the {{role.name}}IPs for network port resources in
Composable Role to enable setting predictable ip address
on network ports.

We currently support predictable ip addressing using the
from_pool templates. The from_pool creates a 'fake' neutron
port. Because of this we loose access to properties of
the ports subnet, properties that is useful to simplyfy
advanced configurations such as routed spine-and-leaf.

Creating a neutron port also open the possibility to use ML2
plugins in the future (Neutron Ansible ML2) to do port
binding that configures network devices.

The parameter (interface) to use this is the same as with
from_pool. Example:

parameter_defaults:
  {role.name}IPs:
    {network.name_lower}:
    - 10.0.0.1
    {network.name_lower}:
    - 172.16.1.1

We just don't need to override the resource registry. I.e
the following is not used/required.
resource_registry:
  ``OS::TripleO::{{role.name}}::Ports::{{network.name}}Port``

NOTE: This does not remove the from_pool templates. We may
      want to deprecate them, but in that case we need to
      ensure it is possible to upgrade an environment that
      used them by changeing the resource registry to use
      the default port templates in this change.

Related: blueprint tripleo-routed-networks-templates
Change-Id: I3f9b55ffbf6b6626b4d0dfbec3867811cb74de14
2018-07-06 21:54:43 +02:00
Steven Hardy
372e1b580d Add ServerDeletionPolicy parameter
With this we could potentially delete or update the heat stack without
deleting any servers, e.g if we wanted to switch from heat/nova managed
server resources to some which are externally managed via Ironic directly.

Partially-Implements: blueprint nova-less-deploy
Change-Id: I2b871c1f096013956904441cddc36cb2404e477a
2018-07-02 09:39:14 +01:00
Jill Rouleau
c16167f3d9 Enable Ansible error handling per role
Enable any_errors_fatal and max_fail_percentage Ansible options
to be set per TripleO role.  This change also provides a
structure by which future per-role Ansible options can readily
be added to group_vars.

Closes-Bug: 1760989
Change-Id: I47954717f42f14bae8d9fd2bd17cd8ea1fd787b3
2018-06-21 09:40:29 -07:00
baiwenteng
66b8d58ee8 Fix typo
replace "configuation" with "configuration"
replace "assigment" with "assignment"

Change-Id: Icd4735ffbdea14997557f8db0566cb190027101b
2018-06-12 11:51:24 +08:00
Zuul
096cef093c Merge "Manage public certificate with ansible" 2018-06-01 17:37:48 +00:00
Zuul
1785d431b2 Merge "Add ability to pre-assign IPs by role on ctlplane" 2018-05-31 14:26:10 +00:00
Cédric Jeanneret
59b762658d Manage public certificate with ansible
This is basically a rewrite of the bash script pushed by
puppet/extraconfig/tls/tls-cert-inject.yaml

UpgradeImpact: NodeTLSData is not used anymore

Change-Id: Iaf7386207e5bd8b336759f51e4405fe15114123a
2018-05-31 14:50:00 +02:00
Dan Sneddon
393476fda3 Add ability to pre-assign IPs by role on ctlplane
This change adds the ability to pre-assign IP addresses
that will be used on the ctlplane network for each node.
The functionality is similar to the existing ips-from-pool
templates, but the IP will be passed to the Nova server
resource rather than a dedicated Neutron port (as happens
with the isolated networks templates). This allows for
compatibility with legacy installations for upgrades.
In testing, it also appears that the fixed IP can be
changed in a stack update, but more testing will have to
be done. Note that if an IP address is defined for
some nodes but not others, nodes without a fixed IP
will get an IP assigned automatically by Neutron.

Change-Id: I67513f54a60f5a50a2bc435099fbb2a643adc277
2018-05-18 15:42:41 -07:00
Carlos Camacho
44ef2a3ec1 Change template names to rocky
The new master branch should point now to rocky.

So, HOT templates should specify that they might contain features
for rocky release [1]

Also, this submission updates the yaml validation to use only latest
heat_version alias. There are cases in which we will need to set
the version for specific templates i.e. mixed versions, so there
is added a variable to assign specific templates to specific heat_version
aliases, avoiding the introductions of error by bulk replacing the
the old version in new releases.

[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#rocky
Change-Id: Ib17526d9cc453516d99d4659ee5fa51a5aa7fb4b
2018-05-09 08:28:42 +02:00
Zuul
1e2cdd60aa Merge "Support SshKnownHostsDeployment with config-download" 2018-03-29 21:45:09 +00:00
James Slagle
fa4b3e2a3c Honor {{role.name}}ExtraConfig with deprecated params
{{role.name}}ExtraConfig was previously ignored if the role used
deprecated params in roles_data.yaml. This was due to the usage of
server_resource_name in the ExtraConfig resource, where
service_resource_name also defaulted to
deprecated_service_resource_name. So, the new {{role.name}}ExtraConfig
was never actually used.

Change-Id: I83e57317e2c56260957be90c66290a41a926835a
Closes-Bug: #1758343
2018-03-24 13:17:18 +01:00
Zuul
3eb0c62e47 Merge "Remove unused minor update code" 2018-03-19 12:34:21 +00:00
James Slagle
088d5c12f0 Support SshKnownHostsDeployment with config-download
Add support for the SshKnownHostsDeployment resources to
config-download. Since the deployment resources relied on Heat outputs,
they were not supported with the default handling from tripleo-common
that relies on the group_vars mechanism.

Instead, this patch refactors the templates to add the known hosts
entries as global_vars to deploy_steps_playbook.yaml, and then includes
the new tripleo-ssh-known-hosts role from tripleo-common to apply the
same configuration that the Heat deployment did.

Since these deployments no longer need to be triggered when including
config-download-environment.yaml, a mapping is added that can be
overridden to OS::Heat::None to disable the deployment resources when
using config-download.

The default behavior when not using config-download remains unchanged.

Closes-Bug: #1746336
Change-Id: Ia334fe6adc9a8ab228f75cb1d0c441c1344e2bd9
2018-03-19 07:50:06 -04:00
Jiri Stransky
a782462a1a Remove unused minor update code
Since Pike, minor updates are done via the composable services
framework. The old shell script approach hasn't been used/tested for 2
releases now, and should be dropped.

Also drop the UpdateWorkflow interface. Before we started doing
upgrades via Ansible, we used this pluggable resource interface to
perform oneshot operations like migrations to WSGI or AODH
services. Nowadays this interface is not referenced from anywhere and
we'd probably rather do similar operations via Ansible tasks.

Change-Id: I6c5eafe76eb53bc38d100a9ba132dd8fe6dd2d5f
2018-03-15 18:27:14 +01:00
Steven Hardy
3a7baa8fa6 Convert ServiceNetMap evals to hiera interpolation
Since https://review.openstack.org/#/c/514707/ added the net_ip_map
to hieradata, we can look up the per-network bind IPs via hiera
interpolation instead of heat map_replace.

In some cases the ServiceNetMap lookup is used for other things,
but anywhere we make use of the "magic" translation via NetIpMap
is changed the same way.

This will enable more of the configuration data to be exposed per
role vs per node in a future patch (to simplify our ansible
workflow).

Co-authored-by: Bogdan Dobrelya <bdobreli@redhat.com>
Change-Id: Ie3da9fedbfce87e85f74d8780e7ad1ceadda79c8
2018-03-10 08:18:30 +00:00
Jiri Stransky
293dc73265 Fix invalid Ceph and BlockStorage role template
The roles would get generated with deprecated parameter group, but no
parameter in that group. Heat would then refuse that template.

Change-Id: I526c8177d1a759ae9e48cdb8b94fc2aa7fe3c6fb
Closes-Bug: #1750828
2018-02-22 11:08:36 +01:00
Zuul
e8c37e90a4 Merge "Add subnet property to ctlplane network for server resources" 2018-02-08 01:27:21 +00:00
James Slagle
e727fd4169 Remove redundant action key on NetworkDeployment
action was specified twice on NetworkDeployment. The first occurrence is
just ignored by Heat, so remove it for clarity.

Change-Id: Iff24a65c09f37f2777787e7436f7902f5e6a122f
Closes-Bug: #1747072
2018-02-02 15:08:07 -05:00