48 Commits

Author SHA1 Message Date
Christian Berendt
c073b2d897 Remove EC2 API related configurations and tasks
The Nova EC2 API is disabled by default, the default value
of the enabled_apis parameter in nova.conf is "osapi_compute, metadata"
The EC2 API is marked as deprecated and will be removed from Nova in
the future.

Change-Id: I6b9d66017e066cde5749be45b367194d2192ead3
Closes-bug: #1586605
2016-06-16 12:16:07 +02:00
Michal Rostecki
a5c1aa0b81 Run HAProxy in a chroot jail
It's impossible to drop root for the HAProxy container.
But HAProxy provides a possibility to use a chroot jail.

When attaching to the HAProxy container, we see that
the root directory is changed:

    $ sudo docker exec -ti haproxy bash
    (haproxy)[root@operator /]# ls -di /
    259 /

Co-Authored-By: Vikram Hosakote <vhosakot@cisco.com>

Closes-Bug: #1552289

Change-Id: I9d55e9b741b8560cac53dc8b837a24a3029a4dc0
2016-05-06 10:23:49 +02:00
Dave McCowan
4c8227ec8b Add TLS to Kibana Web Interface
Use HAProxy to terminate a TLS connection on port 5601 for the
Kibana dashboard when TLS is enabled for Kolla. x-forwarded-for
and x-forwarded-proto headers are set to give Kibana the info it
needs to write returned URLs.

Change-Id: I03a2dd3a8e2513d38281b30bf4bae6449fec0316
Closes-bug: #1566117
2016-04-15 14:06:36 -04:00
Jeffrey Zhang
5680bf0f24 Enable the kibana access from the external network
Closes-Bug: #1564440
Change-Id: I8b9ff303d131fab18e51e9f99c64f4e4004f41b4
2016-03-31 22:34:14 +08:00
akwasniewska
ba62740a93 Add kibana authentication using HAproxy
Change-Id: Ib501571dd34cb68924775ce738499d63df5718dd
Closes-Bug: 1556487
2016-03-24 11:46:06 +01:00
SamYaple
9500bd239a haproxy formatting was incorrect
haproxy 1.6+ does not allow the formatting that was used for stats
listener. We need to adjust it to the correct syntax

TrivialFix

Change-Id: I5f0111c756d40a0cf7385e6963ebbb57adb36b35
2016-03-14 00:24:38 +00:00
Éric Lemoine
95e8ac65d1 Do not make Kibana available at the external VIP
Kibana is a tool for operators. It should not be accessible though
the external VIP.

Closes-Bug: #1554977
Change-Id: I1dc101de18e4e01ebde9d317ab7e3193e307a14e
2016-03-09 10:12:59 +01:00
Dave McCowan
3b378b11ed Glance registry should only be accessible from internal network
When configured with a separate external VIP, glance registry
should listen on only the internal VIP.

TrivialFix

Change-Id: Ie186f2ea391b53b9ea0cb230c573c9e09efd44b2
2016-03-07 22:17:34 -05:00
Jenkins
08101fd465 Merge "Make Heka send logs to Elasticsearch" 2016-03-05 15:36:44 +00:00
Éric Lemoine
491aff0b88 Make Heka send logs to Elasticsearch
This patch includes changes relative to integrating Heka with
Elasticsearch and Kibana.

The main change is the addition of an Heka ElasticSearchOutput plugin
to make Heka send the logs it collects to Elasticsearch.

Since Logstash is not used the enable_elk deploy variable is renamed
to enable_central_logging.

If enable_central_logging is false then Elasticsearch and Kibana are
not started, and Heka won't attempt to send logs to Elasticsearch.

By default enable_central_logging is set to false. If
enable_central_logging is set to true after deployment then the Heka
container needs to be recreated (for Heka to get the new
configuration).

The Kibana configuration used property names that are deprecated in
Kibana 4.2. This is changed to use non-deprecated property names.

Previously logs read from files and from Syslog had a different Type
in Heka. This is changed to always use "log" for the Type. In this
way just one index instead of two is used in Elasticsearch, making
things easier to the user on the visualization side.

The HAProxy configuration is changed to add entries for Kibana.
Kibana server is now accessible via the internal VIP, and also via
the external VIP if there's one configured.

The HAProxy configuration is changed to add an entry for
Elasticsearch. So Elasticsearch is now accessible via the internal
VIP. Heka uses that channel for communicating with Elasticsearch.

Note that currently the Heka logs include "Plugin
elasticsearch_output" errors when Heka starts. This occurs when Heka
starts processing logs while Elasticsearch is not yet started. These
are transient errors that go away when Elasticsearch is ready. And
with buffering enabled on the ElasticSearchOuput plugin logs will be
buffered and then retransmitted when Elasticsearch is ready.

Change-Id: I6ff7a4f0ad04c4c666e174693a35ff49914280bb
Implements: blueprint central-logging-service
2016-03-05 11:51:49 +01:00
Dave McCowan
3daded6242 Add TLS protection on external API endpoints
TLS can be used to encrypt and authenticate the connection with
OpenStack endpoints.  This patch provides the necessary
parameters and changes the resulting service configurations to
enable TLS for the Kolla deployed OpenStack cloud.

The new input parameters are:

kolla_enable_tls_external: "yes" or "no" (default is "no")
kolla_external_fqdn_cert: "/etc/kolla/certificates/haproxy.pem"
kolla_external_fqdn_cacert: "/etc/kolla/certificates/haproxy-ca.crt"

Implements: blueprint kolla-ssl

Change-Id: I48ef8a781c3035d58817f9bf6f36d59a488bab41
2016-03-03 14:44:37 -05:00
SamYaple
d3cfb2052a Change kolla_internal_address variable
Due to poor planning on our variable names we have a situation where
we have "internal_address" which must be a VIP, but "external_address"
which should be a DNS name. Now with two vips "external_vip_address"
is a new variable.

This corrects that issue by deprecating kolla_internal_address and
replacing it with 4 nicely named variables.

kolla_internal_vip_address
kolla_internal_fqdn
kolla_external_vip_address
kolla_external_fqdn

The default behaviour will remain the same, and the way the variable
inheritance is setup the kolla_internal_address variable can still be
set in globals.yml and propogate out to these 4 new variables like it
normally would, but all reference to kolla_internal_address has been
completely removed.

Change-Id: I4556dcdbf4d91a8d2751981ef9c64bad44a719e5
Partially-Implements: blueprint ssl-kolla
2016-02-26 20:00:09 +00:00
Jenkins
bb48523f9f Merge "Use passed client IP address in various audit logs" 2016-02-24 17:00:35 +00:00
Dave McCowan
b770339534 Use passed client IP address in various audit logs
HAProxy: change to use option forwardfor to pass origin IP address
to backend via X-Forwarded-For header

Keystone: Apache does the audit logs for keystone.  Change the
LogFormat to display the passed address instead of the connection
address which is that of the load balancer.

Nova, Cinder, Glance: these services can make use of the address
passed in X-Forwarded-For.  With this setting the API logs for
these services include the client IP address.

Change-Id: Ia861ecc11a7c7d463d0366586926d1a842853f69
Closes-Bug: #1548935
2016-02-24 09:51:13 -05:00
Dave McCowan
34c2cb8e64 Implement Option for Two VIPs to separate API traffic
To improve security, operators have asked for two VIPs for
their cloud.

VIP 1 is the internal VIP that can reach internal and admin endpoints.
In addition, the internal VIP can also reach other internal services,
such as the database and message services.
VIP 2 is the external VIP that can only reach public endpoints.

With one VIP only, all services are reached at the same address.

To add a second VIP, this patch adds two new configuration parameters.

kolla_external_vip_address: is an IPv4 address to use for created VIP
kolla_external_vip_interface: is the network interface to use for VIP
In this scenario, the first VIP (the internal VIP), is defined by
the original parameters (kolla_internal address and network_interface).

When using two VIPs, the existing kolla_external_address parameter
should be/point to/resolve to the kolla_external_vip_address.

Closes-bug: 1535333

Change-Id: I5bfcefaf7899298455cdade8209c34324aebfecb
2016-02-24 09:18:06 -05:00
Éric Lemoine
3fca4ccbaf Make Heka collect HAProxy and Keepalived logs
Partially implements: blueprint heka
Change-Id: I7e2bf4e520fa14fd40e3b329f3b2998ae6ea47f4
2016-02-19 20:32:10 +00:00
SamYaple
7ddad3f3d5 Move haproxy socket to volume
Partial-Bug: #1544545
Change-Id: I292bcaeacb080ff4c5ab6b42b7d899039d6b19c4
2016-02-17 14:45:59 +00:00
Hui Kang
3ae0063944 Add an ansible role for mongodb
Change-Id: Ia6ac371845d1f2b545406d096b35a6d5f68be5f6
Implements: blueprint ansible-mongodb
2016-01-27 10:48:34 -05:00
Andrey Shestakov
1a368c9134 Add RadosGW container
Implements blueprint radosgw-container

Change-Id: Idc88a67a0979be626d3eaa9b2b9a527010aa2006
2015-12-29 12:49:03 +00:00
OTSUKA, Yuanying
903c7d4c3f Fix retrieving api_interface
In heterogeneous environment, api_interfaces are different each other.
So we should specify it from hostvars.

Implements: bp configure-network-interface
Change-Id: Id15d70bfb9ebb62a64a3847a6b77407efb171dbe
2015-12-16 15:55:11 +09:00
Jenkins
84b6f7edc4 Merge "Add Ansible support for Magnum" 2015-11-13 17:01:04 +00:00
Michal Rostecki
2cd5d9ed74 Add Ansible support for Magnum
Change-Id: Idb25ac4d3148c9b9400cf675ac2e47d35cce6224
Implements: blueprint ansible-magnum
2015-11-13 11:31:34 +01:00
Jenkins
44b5a8867e Merge "Finish implementation of spice" 2015-11-06 21:57:08 +00:00
Sam Yaple
99932d650d Finish implementation of spice
Due bad rebases there is a huge section of the spice patch missing
from the implementation unfortunately. This patch finishes the rest
of this patch out properly.

Change-Id: I693c6745e9594fd91eb6453f6de9dfcbd410e89c
Paritally-Implements: blueprint nova-proxies
2015-11-06 03:48:07 +00:00
Jenkins
037bdca42d Merge "bootstrap takes place on the wrong node" 2015-11-06 03:32:36 +00:00
Sam Yaple
e2bc9dcf79 bootstrap takes place on the wrong node
The bootstrap must occur on the nova-api node due to binding in the
nova-api directory (same goes for all other services)

Closes-Bug: #1513439
Backport: Liberty
Change-Id: Iab88b49712828085e4d7e7f85e6d8f0b7999a9bf
2015-11-05 16:30:44 +00:00
Sam Yaple
347730cec8 Remove vip for rabbitmq
Adjust all the configs to list all the rabbitmq hosts rather than
running rabbitmq through the VIP. This is made possible by clusterer
which has already merged.

Change-Id: I5db48f5f10ec68f4c8863a29bc13984f6845a4f9
Partially-Implements: blueprint rabbitmq-clusterer
2015-11-05 15:43:20 +00:00
Vladislav Belogrudov
f4eb89f8b0 HAProxy does not have murano-api configuration
This commit fixes configuration for murano-api HA

Change-Id: I6b6f2c266cbb4c8131a949e31094572dad91fc40
Closes-Bug: #1510921
Backport: Liberty
2015-11-01 20:27:02 +00:00
James McCarthy
c879b1f271 Updated haproxy config to include swift api
Closes-bug: #1503197
backport: liberty
Change-Id: I5c233298312dabeb43efa4821855c77fd81aae67
2015-10-06 10:30:04 +01:00
Jeff Peeler
f5a50a1d7d Add Ansible support for Ironic
Configuration based off upstream documentation here:
http://docs.openstack.org/developer/ironic/deploy/install-guide.html

A few notes:
-ironic-api is not configured to use mod_wsgi
-several places it's noted that discoverd is going away and needs to be
replaced with ironic-inspector - (sqlite connection should be changed
too)
-currently enabling ironic reconfigures nova compute (driver and
scheduler) as well as changes neutron network settings
-a nice enhancement would be to configure the web console

Required post-deployment configuration:

Create the flat network to launch the instances:

neutron net-create --tenant-id $TENANT_ID sharednet1 --shared \
--provider:network_type flat --provider:physical_network physnet1

neutron subnet-create sharednet1 $NETWORK_CIDR --name $SUBNET_NAME \
--ip-version=4 --gateway=$GATEWAY_IP --allocation-pool \
start=$START_IP,end=$END_IP --enable-dhcp

And then the above ID is used to set cleaning_network_uuid in the neutron
section of ironic.conf.

Change-Id: I572e7ff1f23c4e57a2c50817cafe9269fd9950dd
Implements: blueprint ironic-container
2015-09-28 16:10:43 -04:00
Sam Yaple
ae30857144 bind haproxy to single interface
Change-Id: I0c9d85ec7da98d32b94d1e61538fe20333536d39
Closes-Bug: #1500159
2015-09-27 09:17:51 +00:00
Sam Yaple
9f99d63ea2 Stop haproxy from listening on ports it shouldnt
Haproxy is currently setup to listen on all services, even ones that
aren't being installed (e.g. cinder or swift). This patch places
conditionals around those groups.

Change-Id: Ia1ff873ce075768dfebf442aabf13604076ce637
Closes-Bug: #1500157
2015-09-27 08:46:04 +00:00
Hui Kang
83157d1a19 Add nova_novncproxy port to haproxy configuration
Make novncproxy_port configurable

Change-Id: I54f7918b1927562baf2e125b7f5c8edaec87e993
Closes-Bug: #1497673
2015-09-22 15:48:09 +00:00
Qiu Yu
c7ceca711a Expose HAProxy stats for management purpose
Change-Id: Ia7263a9993e29c8f3db71e86264b786398224223
Closes-Bug: #1496658
2015-09-17 09:55:27 +08:00
Steven Dake
5e521f0550 Add Ansible support for Heat
This changes bootstrapping of the Heat container to bootstrap
the Heat container with a heat domain user.  This requires some
work from bootstrap.yml to pass in several environment variables
needed by the heat domain setup script.

Co-Authored-By: Sam Yaple <sam@yaple.net>
Change-Id: Iab05983754fa514835cb5ff54d775faa18773110
Partially-implements: blueprint ansible-heat
2015-08-29 23:58:54 -07:00
Sam Yaple
2868390503 Update Haproxy for Horizon
Change-Id: I725efca9f26b73abd855e6fc90fc1694c4a9d22a
Parially-Implements: blueprint ansible-horizon
2015-08-25 07:41:13 +00:00
Sam Yaple
cf3fd4043a Update mariadb confs and ports
Cleanup all options in galera.cnf. Bind to all interfaces and ports
appropriately.

Change-Id: I516613d09673ba61aadda2c7bbb4abbbe4ea47ac
Partially-Implements: blueprint update-configs
Closes-Bug: #1478330
2015-08-13 02:27:38 +00:00
Jenkins
15f058d243 Merge "Update nova.conf and ports" 2015-08-12 20:22:37 +00:00
Jenkins
15041bd4cc Merge "Update rabbitmq confs and ports" 2015-08-12 20:22:30 +00:00
Swapnil Kulkarni
18f0e536f6 Add ansible support for Cinder
Change-Id: I0694e5ad95aa7f924ebd3a29dff1f85e6307b7ae
Partially-implements: blueprint ansible-cinder
Partially-Implements: blueprint remove-abstraction-ansible
Co-Authored-By: Ryan Hallisey <rhallise@redhat.com>
2015-08-12 03:41:48 +00:00
Sam Yaple
d91346377e Update nova.conf and ports
Cleanup all options in the minimal nova.conf. Remove options where
the default value was specified explicitly. Updated ports and bindings
to be configurable.

Partially-Implements: blueprint update-configs
Change-Id: I0bca7a8f9c4c6fa40145d66a95de7e98edc0edce
2015-08-10 23:54:52 +00:00
Sam Yaple
3a78bbdef0 Update rabbitmq confs and ports
Cleanup all options in the rabbitmq confs. Allow all ports to be
configurable.

Change-Id: I9b3b485a4f3a25d20c0f19d13638f717daa169dc
Partially-Implements: blueprint update-configs
2015-08-10 05:02:25 +00:00
Sam Yaple
1ab8bc184c Fixes group name for keepalived.conf
Also included is removing the executable bit on haproxy.cfg.j2 as it
should not have those permissions in the repo. It has no affect on the
templating process.

Change-Id: I9c76e528896bdf1799b8eeb62ae77bc4ad0b4449
Closes-Bug: #1482832
2015-08-08 05:28:07 +00:00
Sam Yaple
8ede380f1d Add check_script for haproxy in keepalived
This patch checks that haproxy is alive and running. It does this by
using socat to talk to the haproxy socket. That socket will only
respond successfully when haproxy is active and functional.

Change-Id: I528588d5742071103c28109a69842a6f935232c2
Closes-Bug: #1478570
2015-08-05 11:10:53 +00:00
Jenkins
e9db94135d Merge "Rename role name to service name" 2015-07-31 09:03:50 +00:00
Sam Yaple
0382909dbb Rename role name to service name
The original purpose for having an abstract like 'database' rather than
the service name of 'mariadb' has been change. Our direction is different
and this patch reflects consistent naming throughout

Change-Id: I704896191cc5243f9dab2a4cca9120e9dc2ceb2c
Closes-Bug: #1478328
2015-07-30 19:35:17 +00:00
Sam Yaple
3bc9adf9b2 Forces metadata to bind to single interface
Also adds haproxy config for metadata.

Change-Id: I65c1ae5f491d66bf4dccaa38a9a2ad4026c1f72e
Closes-Bug: #1478333
2015-07-30 19:10:11 +00:00
Michal Jastrzebski (inc0)
2d920745a4 Ansible install of HAProxy
This commit consists HAProxy ansible bits including config generation,
container deployment and hot reloads.

Closes-Bug: #1477915
Co-Authored-By: Sam Yaple <sam@yaple.net>
Change-Id: Ie93fa68fdb6b2885889c992ff1267d38b68e0cbc
Partially-implements: blueprint ansible-service
2015-07-28 15:46:42 +00:00