Added a heat template to build Faafo

The added template builds Faafo as an all-in-one install.

Change-Id: Ie5e2df904c7f1fb0cee6c1238c5523495d1b93f2
This commit is contained in:
Martin Paulo 2015-11-13 10:29:01 +11:00 committed by Diane Fleming
parent a6b7010488
commit 6bc079176f
6 changed files with 223 additions and 118 deletions

View File

@ -0,0 +1,90 @@
heat_template_version: 2014-10-16
description: |
A template to bring up the faafo application as an all in one install
parameters:
key_name:
type: string
description: Name of an existing KeyPair to enable SSH access to the instances
default: id_rsa
constraints:
- custom_constraint: nova.keypair
description: Must already exist on your cloud
flavor:
type: string
description: The flavor the application is to use
constraints:
- custom_constraint: nova.flavor
description: Must be a valid flavor provided by your cloud provider.
image_id:
type: string
description: ID of the image to use to create the instance
constraints:
- custom_constraint: glance.image
description: Must be a valid image on your cloud
faafo_source:
type: string
description: The http location of the faafo application install script
default: https://git.openstack.org/cgit/openstack/faafo/plain/contrib/install.sh
resources:
security_group:
type: OS::Neutron::SecurityGroup
properties:
description: "SSH and HTTP for the all in one server"
rules: [
{remote_ip_prefix: 0.0.0.0/0,
protocol: tcp,
port_range_min: 22,
port_range_max: 22},
{remote_ip_prefix: 0.0.0.0/0,
protocol: tcp,
port_range_min: 80,
port_range_max: 80},]
server:
type: OS::Nova::Server
properties:
image: { get_param: image_id }
flavor: { get_param: flavor }
key_name: { get_param: key_name }
security_groups:
- {get_resource: security_group}
user_data_format: RAW
user_data:
str_replace:
template: |
#!/usr/bin/env bash
curl -L -s faafo_installer | bash -s -- \
-i faafo -i messaging -r api -r worker -r demo
wc_notify --data-binary '{"status": "SUCCESS"}'
params:
wc_notify: { get_attr: ['wait_handle', 'curl_cli'] }
faafo_installer: { get_param: faafo_source }
wait_handle:
type: OS::Heat::WaitConditionHandle
wait_condition:
type: OS::Heat::WaitCondition
depends_on: server
properties:
handle: { get_resource: wait_handle }
count: 1
# we'll give it 10 minutes
timeout: 600
outputs:
faafo_ip:
description: The faafo url
value:
list_join: ['', ['Faafo can be found at: http://', get_attr: [server, first_address]]]

View File

@ -56,7 +56,7 @@ Fractal worker can use for configuration.
Using Object Storage instead of a database
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We haven't quite figured out how to do this yet, but the general steps involve
We have not quite figured out how to do this yet, but the general steps involve
changing the fractal upload code to store metadata with the object in swift,
then changing the API code such as "list fractals" to query swift to get the
metadata. If you do this, you should be able to stop using a database.
@ -66,7 +66,7 @@ metadata. If you do this, you should be able to stop using a database.
Next steps
~~~~~~~~~~
Wow! If you've made it through this section, you know more than the authors of
Wow! If you have made it through this section, you know more than the authors of
this guide know about working with OpenStack clouds.
Perhaps you can `contribute

View File

@ -191,10 +191,10 @@ How the Fractals application interacts with OpenStack
The magic revisited
~~~~~~~~~~~~~~~~~~~
So what exactly was that request doing at the end of the previous
section? Let us look at it again. (Note that in this subsection, we are
just explaining what you have already done in the previous section; you
do not need to execute these commands again.)
So what exactly was that request doing at the end of the previous section?
Let us look at it again. In this subsection, we are just explaining what you
have already done in the previous section; you do not need to run these
commands again.
.. only:: shade
@ -277,7 +277,7 @@ your cloud provider to confirm the user name.
:start-after: step-2
:end-before: step-3
.. note:: User data in openstacksdk must be encoded to base64.
.. note:: User data in openstacksdk must be encoded to Base64.
After the instance is created, cloud-init downloads and runs a script called
:code:`install.sh`. This script installs the Fractals application. Cloud-init

View File

@ -5,17 +5,17 @@ Networking
.. todo:: Latter part of the chapter (LBaaS) needs to use Fractals app
entities for the examples.
Prior to this chapter, all of the nodes that comprise the fractal application
were attached to the same network.
In previous chapters, all nodes that comprise the fractal application were
attached to the same network.
In this section of the tutorial, we introduce the Networking API.
This will enable us to build networking topologies that separate
public traffic accessing the application from traffic between the API
and the worker components. We also introduce load balancing for
resilience, and create a secure back-end network for communication between
the database, webserver, file storage, and worker components.
This chapter introduces the Networking API. This will enable us to build
networking topologies that separate public traffic accessing the application
from traffic between the API and the worker components. We also introduce
load balancing for resilience, and create a secure back-end network for
communication between the database, webserver, file storage, and worker
components.
.. warning:: This section assumes your cloud provider has implemented the
.. warning:: This section assumes that your cloud provider has implemented the
OpenStack Networking API (neutron). Users of clouds which have
implemented legacy networking (nova-network) will have access to
networking via the Compute API. Log in to the Horizon dashboard
@ -60,20 +60,19 @@ the database, webserver, file storage, and worker components.
.. warning:: PHP-OpenCloud supports the OpenStack Networking API,
but this section has not been completed.
Working with the CLI
~~~~~~~~~~~~~~~~~~~~
Work with the CLI
~~~~~~~~~~~~~~~~~
As SDKs don't currently fully support the OpenStack Networking API, this section
uses the command-line clients.
Because the SDKs do not fully support the OpenStack Networking API, this
section uses the command-line clients.
Install the 'neutron' command-line client by following this guide:
Use this guide to install the 'neutron' command-line client:
http://docs.openstack.org/cli-reference/content/install_clients.html
Then set up the necessary variables for your cloud in an 'openrc' file
using this guide:
Use this guide to set up the necessary variables for your cloud in an 'openrc' file:
http://docs.openstack.org/cli-reference/content/cli_openrc.html
Ensure you have an openrc.sh file, source it and then check your
Ensure you have an openrc.sh file, source it, and then check that your
neutron client works: ::
$ cat openrc.sh
@ -91,8 +90,8 @@ neutron client works: ::
Networking segmentation
~~~~~~~~~~~~~~~~~~~~~~~
In traditional datacenters, network segments are
dedicated to specific types of network traffic.
In traditional data centers, network segments are dedicated to specific types
of network traffic.
The fractal application we are building contains three types of
network traffic:
@ -130,7 +129,7 @@ would be similar to the following diagram:
}
}
In this network layout, we are assuming that the OpenStack cloud in which
In this network layout, we assume that the OpenStack cloud in which
you have been building your application has a public network and tenant router
that was previously created by your cloud provider or by yourself, following
the instructions in the appendix.
@ -194,8 +193,8 @@ worker back end is as follows:
* Allocate floating ips and assign them to the web server nodes.
Creating networks
~~~~~~~~~~~~~~~~~
Create networks
~~~~~~~~~~~~~~~
Most cloud providers will make a public network accessible to you.
We will attach a router to this public network to grant Internet access
@ -206,7 +205,7 @@ be accessed from the Internet.
Let's just confirm that we have a public network by listing the networks our
tenant has access to. The public network doesn't have to be named public -
it could be 'external', 'net04_ext' or something else - the important thing
is it exists and can be used to reach the internet.
is it exists and can be used to reach the Internet.
::
@ -256,7 +255,7 @@ Next, create a network and subnet for the workers.
| tenant_id | f77bf3369741408e89d8f6fe090d29d2 |
+-------------------+--------------------------------------------+
Now, create a network and subnet for the webservers.
Now, create a network and subnet for the web servers.
::
@ -336,7 +335,7 @@ Next, create a network and subnet for the API servers.
| tenant_id | 0cb06b70ef67424b8add447415449722 |
+-------------------+--------------------------------------------+
Now that you've got the networks created, go ahead and create two
Now that you have got the networks created, go ahead and create two
Floating IPs, for web servers. Ensure that you replace 'public' with
the name of the public/external network offered by your cloud provider.
@ -439,9 +438,9 @@ Now, attach our router to the worker, api, and webserver subnets.
Booting a worker
----------------
Now that you've prepared the networking infrastructure, you can go
Now that you have prepared the networking infrastructure, you can go
ahead and boot an instance on it. Ensure you use appropriate flavor
and image values for your cloud - see :doc:`getting_started` if you've not
and image values for your cloud - see :doc:`getting_started` if you have not
already.
.. todo:: Show how to create an instance in libcloud using the network

View File

@ -2,20 +2,14 @@
Orchestration
=============
.. todo:: Needs to be restructured so that the fractals app is used as the example for the explanatory material.
.. note:: Sorry! We're not quite happy with this chapter. It will give you an introduction to heat,
but it's a little dry at the moment. We'd like to write a template for the Fractals app instead
of using the "hello world" style ones, so stay tuned!
Throughout this guide, we've talked about the importance of durability and scalability
for your cloud-based applications. In most cases, really achieving these qualities means
automating tasks such as scaling and other operational tasks.
This guide describes the importance of durability and scalability for your
cloud-based applications. In most cases, you must automate tasks, such as
scaling and other operational tasks, to achieve these goals.
The Orchestration module provides a template-based way to describe a cloud
application, then coordinates running the needed OpenStack API calls to run
cloud applications. The templates allow you to create most OpenStack resource
types, such as instances, networking information, volumes, security groups
cloud applications. The templates enable you to create most OpenStack resource
types, such as instances, networking information, volumes, security groups,
and even users. It also provides more advanced functionality, such as
instance high availability, instance auto-scaling, and nested stacks.
@ -25,45 +19,52 @@ The OpenStack Orchestration API contains the following constructs:
* Resources
* Templates
Stacks are created from Templates, which contain Resources. Resources
are an abstraction in the HOT (Heat Orchestration Template) template language, which enables you to define different
cloud resources by setting the `type` attribute.
You create stacks from templates, which contain resources. Resources are an
abstraction in the HOT (Heat Orchestration Template) template language, which
enables you to define different cloud resources by setting the :code:`type`
attribute.
For example, you might use the Orchestration API to create two compute
instances by creating a Stack and by passing a Template to the Orchestration API.
That Template would contain two Resources with the `type` attribute set to `OS::Nova::Server`.
instances by creating a stack and by passing a template to the Orchestration
API. That template would contain two resources with the :code:`type` attribute
set to :code:`OS::Nova::Server`.
That's a simplistic example, of course, but the flexibility of the Resource object
enables the creation of Templates that contain all the required cloud
infrastructure to run an application, such as load balancers, block storage volumes,
compute instances, networking topology, and security policies.
That is a simplistic example, of course, but the flexibility of the resource
object enables the creation of templates that contain all the required cloud
infrastructure to run an application, such as load balancers, block storage
volumes, compute instances, networking topology, and security policies.
.. note:: The Orchestration module isn't deployed by default in every cloud. If these commands don't work, it means the Orchestration API isn't available; ask your support team for assistance.
.. note:: The Orchestration module is not deployed by default in every cloud.
If these commands do not work, it means the Orchestration API is not
available; ask your support team for assistance.
This section introduces the `HOT templating language <http://docs.openstack.org/developer/heat/template_guide/hot_guide.html>`_,
and takes you through some of the common calls you will make when working with OpenStack Orchestration.
This section introduces the
`HOT templating language <http://docs.openstack.org/developer/heat/template_guide/hot_guide.html>`_,
and takes you through some common OpenStack Orchestration calls.
Unlike previous sections of this guide, in which you used your SDK to programmatically interact with
OpenStack, in this section you'll be using the Orchestration API directly through Template files,
so we'll work from the command line.
Unlike in previous sections where you used your SDK to programatically
interact with OpenStack, you use the 'heat' command-line client to access the
Orchestration API directly through template files.
Install the 'heat' commandline client by following this guide:
Use this guide to install the 'heat' command-line client:
http://docs.openstack.org/cli-reference/content/install_clients.html
then set up the necessary variables for your cloud in an 'openrc' file using this guide:
Then, use this guide to set up the necessary variables for your cloud in an 'openrc' file:
http://docs.openstack.org/cli-reference/content/cli_openrc.html
.. only:: dotnet
.. warning:: the .NET SDK does not currently support OpenStack Orchestration
.. warning:: the .NET SDK does not currently support OpenStack Orchestration.
.. only:: fog
.. note:: fog `does support OpenStack Orchestration <https://github.com/fog/fog/tree/master/lib/fog/openstack/models/orchestration>`_.
.. note:: fog `does support OpenStack Orchestration
<https://github.com/fog/fog/tree/master/lib/fog/openstack/models/orchestration>`_.
.. only:: jclouds
.. warning:: Jclouds does not currently support OpenStack Orchestration. See this `bug report <https://issues.apache.org/jira/browse/JCLOUDS-693>`_.
.. warning:: Jclouds does not currently support OpenStack Orchestration.
See this `bug report <https://issues.apache.org/jira/browse/JCLOUDS-693>`_.
.. only:: libcloud
@ -71,52 +72,51 @@ http://docs.openstack.org/cli-reference/content/cli_openrc.html
.. only:: pkgcloud
.. note:: Pkgcloud supports OpenStack Orchestration :D:D:D but this section is `not written yet <https://github.com/pkgcloud/pkgcloud/blob/master/docs/providers/openstack/orchestration.md>`_
.. note:: Pkgcloud supports OpenStack Orchestration :D:D:D but this section
is `not written yet <https://github.com/pkgcloud/pkgcloud/blob/master/docs/providers/openstack/orchestration.md>`_
.. only:: openstacksdk
.. warning:: OpenStack SDK does not currently support OpenStack Orchestration.
.. warning:: The OpenStack SDK does not currently support OpenStack Orchestration.
.. only:: phpopencloud
.. note:: PHP-opencloud supports orchestration :D:D:D but this section is not written yet.
.. note:: PHP-opencloud supports OpenStack Orchestration :D:D:D but this section is not written yet.
HOT Templating Language
-----------------------
The best place to learn about the template syntax for OpenStack Orchestration is the
`Heat Orchestration Template (HOT) Guide <http://docs.openstack.org/developer/heat/template_guide/hot_guide.html>`_
You should read the HOT Guide first to learn how to create basic templates, their inputs and outputs.
The best place to learn about the OpenStack Orchestration template syntax is the
`Heat Orchestration Template (HOT) Guide <http://docs.openstack.org/developer/heat/template_guide/hot_guide.html>`_.
Read this guide to learn about basic templates and their inputs and outputs.
Working with Stacks: Basics
---------------------------
Work with stacks: Basics
------------------------
.. todo::
**Stack create**
This section needs to have a HOT template written for deploying the Fractal Application
The following example uses the
`hello_faafo <https://git.openstack.org/cgit/openstack/api-site/plain/firstapp/samples/heat/hello_faafo.yaml>`_ Hot template to
demonstrate how to create a compute instance that builds and runs the Fractal
application as an all-in-one installation. These configuration settings are
passed in as parameters:
.. todo::
Replace the hello_world.yaml templte with the Fractal template
* Stack create
In the following example, we use the `hello_world <https://github.com/openstack/heat-templates/blob/master/hot/hello_world.yaml>`_ Hot template to demonstrate creating
a Nova compute instance, with a few configuration settings passed in, such as an administrative password and the unique identifier (UUID)
of an image:
- The flavor to use
- Your ssh key name
- The unique identifier (UUID) of the image to use
::
$ wget https://raw.githubusercontent.com/openstack/heat-templates/master/hot/hello_world.yaml
$ heat stack-create --template-file hello_world.yaml \
--parameters admin_pass=Test123\;key_name=test\;image=5bbe4073-90c0-4ec9-833c-092459cc4539 hello_world
$ wget https://git.openstack.org/cgit/openstack/api-site/plain/firstapp/samples/heat/hello_faafo.yaml
$ heat stack-create --template-file hello_faafo.yaml \
--parameters flavor=m1.small\;key_name=test\;image_id=5bbe4073-90c0-4ec9-833c-092459cc4539 hello_faafo
+--------------------------------------+-------------+--------------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+-------------+--------------------+----------------------+
| 0db2c026-fb9a-4849-b51d-b1df244096cd | hello_world | CREATE_IN_PROGRESS | 2015-04-01T03:20:25Z |
| 0db2c026-fb9a-4849-b51d-b1df244096cd | hello_faafo | CREATE_IN_PROGRESS | 2015-04-01T03:20:25Z |
+--------------------------------------+-------------+--------------------+----------------------+
The resulting stack creates a Nova instance automatically, which you can see here:
The resulting stack automatically creates a Nova instance, as follows:
::
@ -124,10 +124,10 @@ The resulting stack creates a Nova instance automatically, which you can see her
+--------------------------------------+---------------------------------+--------+------------+-------------+------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+---------------------------------+--------+------------+-------------+------------------+
| 9bdf0e2f-415e-43a0-90ea-63a5faf86cf9 | hello_world-server-dwmwhzfxgoor | ACTIVE | - | Running | private=10.0.0.3 |
| 9bdf0e2f-415e-43a0-90ea-63a5faf86cf9 | hello_faafo-server-dwmwhzfxgoor | ACTIVE | - | Running | private=10.0.0.3 |
+--------------------------------------+---------------------------------+--------+------------+-------------+------------------+
Verify that the stack was successfully created using the following command:
Use the following command to verify that the stack was successfully created:
::
@ -135,21 +135,38 @@ Verify that the stack was successfully created using the following command:
+--------------------------------------+-------------+-----------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+-------------+-----------------+----------------------+
| 0db2c026-fb9a-4849-b51d-b1df244096cd | hello_world | CREATE_COMPLETE | 2015-04-01T03:20:25Z |
| 0db2c026-fb9a-4849-b51d-b1df244096cd | hello_faafo | CREATE_COMPLETE | 2015-04-01T03:20:25Z |
+--------------------------------------+-------------+-----------------+----------------------+
Remove the stack:
The stack reports an initial :code:`CREATE_IN_PROGRESS` status. When all
software has been installed, the status changes to :code:`CREATE_COMPLETE`.
You might have to run the :code:`stack-list` command a few times before
the stack creation is complete.
**Show information about the stack**
Run this command to get more information about the stack:
::
$ heat stack-delete hello_world
$ heat stack-show hello_faafo
The `outputs` property shows the URL through which you can access the Fractal
app. You can SSH into the instance.
**Remove the stack**
::
$ heat stack-delete hello_faafo
+--------------------------------------+-------------+--------------------+----------------------+
| id | stack_name | stack_status | creation_time |
+--------------------------------------+-------------+--------------------+----------------------+
| 0db2c026-fb9a-4849-b51d-b1df244096cd | hello_world | DELETE_IN_PROGRESS | 2015-04-01T03:20:25Z |
| 0db2c026-fb9a-4849-b51d-b1df244096cd | hello_faafo | DELETE_IN_PROGRESS | 2015-04-01T03:20:25Z |
+--------------------------------------+-------------+--------------------+----------------------+
Verify that the removal of the stack has deleted the nova instance:
Verify the nova instance was deleted when the stack was removed:
::
@ -159,40 +176,39 @@ Verify that the removal of the stack has deleted the nova instance:
+----+------+--------+------------+-------------+----------+
+----+------+--------+------------+-------------+----------+
While this stack is not very interesting - it just starts a single instance - it
is possible to make very complicated templates that involve dozens of instances
or adds and removes instances based on demand. Continue to the next section to
learn more.
While this stack starts a single instance that builds and runs the Fractal app
as an all-in-one installation, you can make very complicated templates that
impact dozens of instances or that add and remove instances on demand.
Continue to the next section to learn more.
Working with Stacks: Advanced
-----------------------------
Work with stacks: Advanced
--------------------------
.. todo:: needs more explanatory material
.. todo:: needs a heat template that uses fractal app
With the use of the Orchestration API, the Fractal app can create an autoscaling
group for all parts of the application, in order to dynamically provision more
compute resources during periods of heavy utilization, and also terminate compute
instances to scale down, as demand decreases.
With the Orchestration API, the Fractal app can create an auto-scaling group
for all parts of the application to dynamically provision more compute
resources during periods of heavy utilization, and also terminate compute
instances to scale down as demand decreases.
There are two helpful articles available to learn about autoscaling with the
Orchestration API:
To learn about auto-scaling with the Orchestration API, read these articles:
* http://superuser.openstack.org/articles/simple-auto-scaling-environment-with-heat
* http://superuser.openstack.org/articles/understanding-openstack-heat-auto-scaling
An example template that creates an auto-scaling wordpress instance can be found in
For an example template that creates an auto-scaling Wordpress instance, see
`the heat template repository <https://github.com/openstack/heat-templates/blob/master/hot/autoscaling.yaml>`_
Next Steps
Next steps
----------
You should now be fairly confident working with the Orchestration service.
There are several calls we did not cover. To see these and more,
refer to the volume documentation of your SDK, or try a different step in the tutorial, including:
You should now be fairly confident working with the Orchestration service. To
see the calls that we did not cover, see the volume documentation for your
SDK. Or, try one of these tutorial steps:
* :doc:`/networking` - to learn about more complex networking
* :doc:`/advice` - for advice for developers new to operations
* :doc:`/craziness` - to see all the crazy things we think ordinary folks won't want to do ;)
* :doc:`/networking`: Learn about complex networking.
* :doc:`/advice`: Get advice about operations.
* :doc:`/craziness`. Learn some crazy things that you might not think to do ;)

View File

@ -51,7 +51,7 @@ fractals can be generated depends on the number of workers. With just
one worker, we can only produce one fractal at a time. Before long, it
will be clear that we need more resources.
.. note:: If you don't have a working application, follow the steps in
.. note:: If you do not have a working application, follow the steps in
:doc:`introduction` to create one.
.. todo:: Ensure we have the controller_ip even if this is a new
@ -331,7 +331,7 @@ This was obviously a very manual process - figuring out we needed more
workers and then starting new ones required some effort. Ideally the
system would do this itself. If your application has been built to
detect these situations, you can have it automatically request and
remove resources, but you don't actually need to do this work
remove resources, but you do not actually need to do this work
yourself. Instead, the OpenStack Orchestration service can monitor
load and start instances as appropriate. See :doc:`orchestration` to find
out how to set that up.