47 Commits

Author SHA1 Message Date
Zuul
9f978d7425 Merge "Add new encryption middleware to swift proxy" 2018-02-21 16:09:54 +00:00
Thiago da Silva
ab1a421cc6 Add new encryption middleware to swift proxy
Enabling data-at-rest encryption and integration
with barbican to swift proxy

Related-Change-Id: I78c6003f5f599a422193dc47422ee607ce05c715
Related-Change-Id: I1ceda973733acb081967ab04a5fd57eb1609c9a7
Change-Id: I26cf063fe410689530ee507cc2f79e93b5e71732
Signed-off-by: Thiago da Silva <thiago@redhat.com>
2018-02-07 16:05:37 -05:00
Zuul
bfc61c8030 Merge "Configure auth_uri for swift proxy explicitly without suffix" 2018-01-16 00:18:31 +00:00
Juan Antonio Osorio Robles
b9870d4c6c Configure auth_uri for swift proxy explicitly without suffix
This is meant for backporting to pike, since in that version, swift
proxy is the only service left that's configured with keystone v2.0.
This fixes that.

Change-Id: I403868e36000abd10be756dcbdb4ce32700f3fec
Closes-Bug: #1742654
2018-01-11 12:12:16 +02:00
marios
dec003def8 Convert tags to when statements for Q major upgrade workflow
This converts "tags: stepN" to "when: step|int == N" for the direct
execution as an ansible playbook, with a loop variable 'step'.
The tasks all include the explicit cast |int.

This also adds a set_fact task for handling of the package removal
with the UpgradeRemovePackages parameter (no change to the interface)

The yaml-validate also now checks for duplicate 'when:' statements

Q upgrade spec @ Ibde21e6efae3a7d311bee526d63c5692c4e27b28
Related Blueprint: major-upgrade-workflow
[0]: 394a92f761/tripleo_common/utils/config.py (L141)
Change-Id: I6adc5619a28099f4e241351b63377f1e96933810
2018-01-08 13:57:47 +02:00
Carlos Camacho
927495fe3d Change template names to queens
The new master branch should point now to queens instead of pike.

So, HOT templates should specify that they might contain features
for queens release [1]

[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#queens

Change-Id: I7654d1c59db0c4508a9d7045f452612d22493004
2017-11-23 10:15:32 +01:00
Ade Lee
c9b7091536 Ensure Debug is a boolean
Oslo does not like it when Debug is not a proper python boolean
Closes-Bug: 1719929

Change-Id: Ib6c3969d4dd75d5fb2cc274266c060acff8d5571
2017-09-27 13:22:07 -04:00
Ben Nemec
c54e9b681b Make various password descriptions consistent
Since these are obviously global parameters they shouldn't specify
what will be using them because they are used in multiple places.

Change-Id: I5054c2d67dffe802e37f8391dd7bad4721e29831
Partial-Bug: 1700664
2017-07-21 18:39:28 +00:00
Giulio Fidente
baf6eee501 Adds network/cidr mapping into a new service property
Makes it possible to resolve network subnets within a service
template; the data is transported into a new property ServiceData
wired into every service which hopefully is generic enough to
be extended in the future and transport more data.

Data can be consumed in service templates to set config values
which need to know what is the subnet where a deamon operates (for
example the Ceph Public vs Cluster network).

Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
2017-07-14 13:44:04 +02:00
Pradeep Kilambi
142b5a2889 Disable swift middleware ceilometer pipeline by default
This generates tons of unnecessary events when gnocchi uses swift backend.
We end up filtering most of these anyway. So lets disable this so it
doesn't put useless load. Also changing the default project to service as
thats what gnocchi uses to authenticate with swift.

Closes-bug: #1693339

Change-Id: I40f47d46fdb06f31a739b590bf653bca71e33f61
2017-06-28 07:55:05 -04:00
Pradeep Kilambi
37447494de Add ignore_projects to filter gnocchi events
Without this, ceilometer db gets hammered with gnocchi swift events.
Keystone creds are required so middleware can query for id.

Related change:  I5c0f4f1a2c7fe7eb39ea6441970e9ac0946a4ec1

Change-Id: I9a7a80252703e470a69dc10352e7ece45ab23150
2017-05-25 16:44:16 +00:00
Carlos Camacho
0a0e2ee629 Update the template_version alias for all the templates to pike.
Master is now the development branch for pike
changing the release alias name.

Change-Id: I938e4a983e361aefcaa0bd9a4226c296c5823127
2017-05-19 09:58:07 +02:00
Saravanan KR
a096ddab34 Add role specific information to the service template
When a service is enabled on multiple roles, the parameters for the
service will be global. This change enables an option to provide
role specific parameter to services and other templates.

Two new parameters - RoleName and RoleParameters, are added to the
service template. RoleName provides the role name of on which the
current instance of the service is being applied on. RoleParameters
provides the list of parameters which are configured specific to the
role in the environment file, like below:

  parameters_default:
      # Default value for applied to all roles
      NovaReservedHostMemory: 2048
      ComputeDpdkParameters:
          # Applied only to ComputeDpdk role
          NovaReservedHostMemory: 4096

In above sample, the cluster contains 2 roles - Compute, ComputeDpdk.
The values of ComputeDpdkParameters will be passed on to the templates
as RoleParameters while creating the stack for ComputeDpdk role. The
parameter which supports role specific configuration, should find the
parameter first in in the RoleParameters list, if not found, then the
default (for all roles) should be used.
Implements: blueprint tripleo-derive-parameters

Change-Id: I72376a803ec6b2ed93903cc0c95a6ffce718b6dc
2017-05-15 10:06:46 +05:30
Juan Antonio Osorio Robles
dba8795b26 Add parameters for internal TLS for swift proxy
This adds the necessary parameter for swift proxy to be terminiated
internally by a TLS proxy.

bp tls-via-certmonger

Change-Id: I3cb9d53d75f982068f1025729c1793efaee87380
Depends-On: I6e7193cc5b4bb7e56cc89e0a293c91b0d391c68e
2017-04-05 06:24:15 +00:00
Jenkins
09af40f5db Merge "Set number of Swift proxy server workers to auto" 2017-03-09 11:28:55 +00:00
Sofer Athlan-Guyot
fb78213782 Put service stop at step1 and quiesce at step2.
In the previous release[1], the services were stopped before the
pacemaker services, so that they get a chance to send last message to
the database/rabbitmq queue:

Let's do the upgrade in the same order.

[1] https://github.com/openstack/tripleo-heat-templates/blob/stable/newton/extraconfig/tasks/major_upgrade_controller_pacemaker_2.sh#L13-L71

Change-Id: I1c4045e8b9167396c9dfa4da99973102f1af1218
2017-02-28 19:20:13 +01:00
Juan Antonio Osorio Robles
e8df83570e Enable TLS for swift-proxy's ceilometer notifications
If the message broker is using TLS, we enable it for these
notifications.

Change-Id: I4f37e77ae12e9582fab7d326ebd4c70127c5445f
Depends-On: If23d1f0d20264faaddc2e5ad54863483fa43ed41
2017-02-23 16:02:16 +02:00
Juan Antonio Osorio Robles
d1eb0bc0dc Use Keystone internal endpoint instead of admin for services
The admin endpoint is listening on the ctlplane network by default;
services should ideally be using the internal api network for this kind
of traffic, as the ctlplane network is mostly for provisioning. On the
other hand, the admin endpoint shouldn't be as relevant with services
switching to keystone v3.

Change-Id: I1213a83ef8693c1cca1d20de974f7949a801d9f1
2017-02-14 02:41:13 +00:00
Juan Antonio Osorio Robles
ad4cc3e9ad Add ability to toggle swift's ceilometer transport_url SSL
So, if RabbitClientUseSSL is set, this will enable TLS for the
swift's ceilometer message broker connection.

Change-Id: Ide70a509aefc9e7eb9d7cc5b3a60520fa42b4010
Depends-On: I8b7457b6233c4f88af2d7bc1b9304fcccb6edf61
2017-01-31 22:40:11 +00:00
Steven Hardy
04084ba43f Add swift service support for composable upgrades
Co-Authored-By: Sofer Athlan-Guyot <sathlang@redhat.com>
Partially-Implements: blueprint overcloud-upgrades-per-service
Closes-Bug: #1655651
Change-Id: I83134f51d152f3b97f9a570bbd9a67c753982810
2017-01-19 15:23:09 +00:00
Steven Hardy
2dee58a85a Swift proxy align *-quotas with puppet-swift syntax
puppet-swift has hard-coded sections which expect these to be
*_quotas, without matching the pipeline to the sections swift
proxy fails to start.

Change-Id: I3ee94a9bc4b046051e5d814e82a69f759bea1296
Closes-Bug: #1657167
2017-01-19 15:10:12 +00:00
Christian Schwede
892827c639 Set number of Swift proxy server workers to auto
Setting the default Swift proxy workers to 0 actually results in a
single Swift proxy worker, no matter how many CPU cores are available.
This is not the default Swift setting and is most likely not sufficient.

Setting this to auto uses the default in Swift, which equals to the
number of CPU cores.

Closes-Bug: 1655070
Change-Id: Ic321b6111f8697ba3cc1554611fee44c2e540759
2017-01-09 19:05:51 +00:00
Steven Hardy
3c6ec654b4 Bump template version for all templates to "ocata"
Heat now supports release name aliases, so we can replace
the inconsistent mix of date related versions with one consistent
version that aligns with the supported version of heat for this
t-h-t branch.

This should also help new users who sometimes copy/paste old templates
and discover intrinsic functions in the t-h-t docs don't work because
their template version is too old.

Change-Id: Ib415e7290fea27447460baa280291492df197e54
2016-12-23 11:43:39 +00:00
Dan Prince
1a9c2022cd Decouple swift-proxy from ceilometer
This patch updates the swift-proxy base profile so that
we now explicitly set the rabbit_port. This allows us
to remove the use of puppet-ceilometer default settings
in the puppet-tripleo modules change ID here:
I8d9f69f5e9160543b372bd9886800f16f625fdc6

It also adds a new boolean parameter that allows the
end user to disable the swift ceilometer pipeline
by setting SwiftCeilometerPipelineEnabled to false.

This two settings allow Swift to once again be installed
on a machine without configuring Ceilometer.

Depends-On: Id1584df5e5bb90f8087ae25eecc4834179b6fc21

Change-Id: Ief5399d7ea4d26e96ce54903a69d660fa4fe3ce9
Related-bug: #1648736
2016-12-11 20:30:09 -05:00
Jenkins
52d9139135 Merge "adding swift middleware that is typically enabled by default" 2016-11-28 09:48:31 +00:00
Christian Schwede
ab8b13d09b Make Ceilometer notifications non-blocking
Ceilometer notifications can be sent in a background thread, unblocking
the Swift proxy in case the RabbitMQ is not processing notifications
quick enough or even unavailable.

There is a default queue size of 1000 notifications. If more messages
are added to the queue these will be discarded, and a warning log entry
will be emitted.

Change-Id: I98022dcbf661a5bb7425f49ba8525225d61212dc
2016-11-18 15:00:23 +01:00
Thiago da Silva
5e52fb0ecc adding swift middleware that is typically enabled by default
Adding these features are typically enabled by default
in any swift cluster.

See upstream sample:
https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample

Change-Id: I29915d1b86da5c47ec34acfb89ab8234e153bf31
Signed-off-by: Thiago da Silva <thiago@redhat.com>
Depends-On: Ie323f68255a73d46e774cbf49d9353c3bf90c35e
2016-11-09 18:40:26 -05:00
Jenkins
3ddf0dd3ef Merge "set url_base option in static web middleware" 2016-11-09 16:30:18 +00:00
Thiago da Silva
14829560b6 set url_base option in static web middleware
Depends-On: Icf45cf2aece398b836c87ddffde5d3056e96dc4d

Change-Id: I3577dc38a0b52092ee5e98a381eb52c3d2768c10
Signed-off-by: Thiago da Silva <thiago@redhat.com>
2016-11-08 16:37:51 -05:00
Emilien Macchi
a560e98874 swift/proxy: remove swift::proxy::ceilometer::rabbit_host
The param is now managed in puppet-tripleo like other services.

Change-Id: I306aa6ac6e2cfc0d4602e15e11564a6be096a121
Depends-On: Ibc0ed642931dd3ada7ee594bb8c70a1c3462206d
2016-11-04 15:50:46 +00:00
Christian Schwede
38fe61be95 Fix Swift proxy pipeline ordering
The Ceilometer middleware is in the wrong place; actually any middleware
should be deployed after catch_errors to catch any errors that would
otherwise crash the proxy service. Additionally the ceilometer
middleware should be deployed after any authentication middleware.

Closes-Bug: 1637471
Co-Authored-By: Thiago da Silva <thiago@redhat.com>
Change-Id: I710ff2f51271a78582fa502e7eecfa687800c664
2016-10-28 13:33:31 +02:00
Pradeep Kilambi
f1b509c8c8 Include ceilometer in swift proxy pipeline
new ceilometermiddleware is available and integrated into
puppet-swift. Lets leverage it and include it in the
swift proxy pipeline. The correcponding puppet triple
change for this is Ie49f4a750368ff174b23b8d6baa743d0956d727e

Closes-Bug: #1631108

Change-Id: I82da0240d60d1eed54f1c0927e6157bb63025a19
2016-10-20 19:57:33 +00:00
Christian Schwede
28a2a6d56b Enable object versioning in Swift proxy
Tempest expects object versioning to be enabled by default in Swift;
if not it has to be disabled explicitly in the Tempest config.

This is a commonly used middleware, therefore it should be enabled
in the overcloud proxy nodes as well.

Closes-Bug: 1632215
Depends-On: I07a206473ff7939749e3eba1dfe3ea8c4526eb5c
Change-Id: I4eae08ff3f9a3a2f829c3497c1c2aaee8e7f8554
2016-10-11 07:07:04 +00:00
Dan Prince
9d67d7b3b1 Move keystone::auth into service_config_settings
This patch moves the keystone::auth settings for all
services into the new service_config_settings section. This
is important because we execute the keystone commands via
puppet only on the role containing the keystone service
and without these settings it will fail.

Note that yaql merging/filtering is used here to ensure that
service_config_settings is optional in service templates,
and also that we'll only deploy hieradata for a given
service on a node running the service (the key in
the service_config_settings map must match the service_name
in the service template for this to work).

e.g the following will result in only deploying keystone: 123
in hiera on the role running the "keystone" service,
regardless of which service template defines it.

  service_config_settings:
    keystone:
      keystone: 123

Co-Authored-By: Steven Hardy <shardy@redhat.com>
Change-Id: I0c2fce037a1a38772f998d582a816b4b703f8265
Closes-bug: 1620829
2016-09-23 07:43:21 -04:00
Jenkins
d6837ea4a6 Merge "Availability monitoring agents support" 2016-09-02 10:00:14 +00:00
Emilien Macchi
98c6bdaa99 Last round of modern authtoken update
It updates Glance, Neutron and Swift to deploy authtoken with modern
pattern.

Change-Id: Icfaf011ea4a23bc47d2fb45e8768f8238532dab3
2016-08-31 18:42:44 +00:00
Martin Mágr
25ad7b8e1e Availability monitoring agents support
- adds possibility to install sensu-client on all nodes
- each composable service has it's own subscription

Co-Authored-By: Emilien Macchi <emilien@redhat.com>
Co-Authored-By: Michele Baldessari <michele@redhat.com>
Implements: blueprint tripleo-opstools-availability-monitoring
Change-Id: I6a215763fd0f0015285b3573305d18d0f56c7770
2016-08-31 09:22:59 -04:00
Dan Prince
a356bb65c9 Move Swift hiera settings into composable services
This moves the swift local bind and hash prefix settings
into the relevant swift-* composable services.

Change-Id: I807ff14c4cc9afa39efee13849e0f8c22718f1c0
2016-08-25 20:27:11 -04:00
Dan Prince
3b62761d2f Add DefaultPasswords to composable services
This patch adds a new DefaultPasswords parameter to
composable services. This is needed to help provide
access to top level password resources that overcloud.yaml
currently manages (passwords for Rabbit, Mysql, etc.).

Moving the RandomString resources into composable services
would cause them to regenerate within the stack. With this
approach we can leave them where they are while we deprecate
the top level mechanism and move the code that uses the
passwords into the composable services.

Change-Id: I4f21603c58a169a093962594e860933306879e3f
2016-08-18 12:45:30 -04:00
Giulio Fidente
885b37c80e Pass ServiceNetMap to services
This will be needed to pick the network where the service has
to bind to from within the service template.

Change-Id: I52652e1ad8c7b360efd2c7af199e35932aaaea8c
2016-08-18 12:36:18 -04:00
Jenkins
9aec3de5b8 Merge "Convert service_name to underscore syntax" 2016-07-29 08:52:05 +00:00
Steven Hardy
7df649f59e Convert service_name to underscore syntax
Currently we use hyphens, e.g cinder-api, but in overcloud.yaml
we have a lot of references to services (e.g for AllNodesConfig)
by underscore, e.g cinder_api.  To enable dynamic generation of
this data, we need the service name in underscore format.

Change-Id: Ief13dfe5d8d7691dfe2534ad5c39d7eacbcb6f70
2016-07-28 16:31:36 +01:00
Emilien Macchi
315fa31963 Migrate Puppet Hieradata to composable services
Migrate puppet/hieradata/*.yaml parameters to puppet/services/*.yaml
except for some services that are not composable yet.

Co-Authored-By: Juan Antonio Osorio Robles <jaosorior@redhat.com>
Change-Id: I7e5f8b18ee9aa63a1dffc6facaf88315b07d5fd7
2016-07-27 12:23:38 -04:00
Dan Prince
5195d7f891 Composable firewall rules
Split out the firewall rules in puppet/hieradata/controller.yaml
into the composable services

Depends-On: Id370362ab57347b75b1ab25afda877885b047263
Change-Id: Icaecab100d3f278035fbbb3facb9bf6c62c76c03
2016-07-25 15:24:16 +02:00
Dan Prince
6b30ff11d4 Add 'service_name' to composable services
This patch adds a new service_name section to each composable
service. We now have an explicit unit test check to ensure that
service_name exists in tools/yaml-validate.py.

This patch also wires service_names into hieradata on each
of the roles so that tools can access the deployed services locally
during deployment and upgrades.

Change-Id: I60861c5aa760534db3e314bba16a13b90ea72f0c
2016-07-22 07:29:39 -04:00
Jiri Stransky
26e639f4a4 Parametrize and increase default of swift-proxy node_timeout
We've had problems uploading images on slow virtualized environments,
where the requests to swift would time out. The issue has been seen both
on undercloud and overcloud. The default timeout is 10 seconds, the
issue has been reportedly reproduced on undercloud with node_timeout as
high as 30 seconds, but not with 60 seconds yet. Set the default timeout
to 60 seconds on overcloud too.

Change-Id: I7d486cf4dc9768ddbf71ab71e92db8d2ef29978e
Closes-Bug: #1594725
2016-06-21 11:37:28 +02:00
Steven Hardy
d372a3b76f Convert Swift proxy to composable services format
Switch the swift proxy service to use the new composable
services format.

Change-Id: Idc9ac64818882e73836ac99bbad56eec184c9a5d
Partially-Implements: blueprint composable-services-within-roles
Depends-On: I6bd72284911f3f449157a6fc00b76682dd53bd8c
2016-05-31 10:55:48 +02:00