ab93e636b8
Brief Summary:
Added Modules and Lab Sections of Aptira's Existing
OpenStack Training Docs. Please do refer Full Summary
for more details.
For those who want to review this and save some time on
building it, I have hosted the content on
http://office.aptira.in
Please talk to Sean Robetrs if you are concerened about
repetition of Doc Content or similar issues like short URLs
etc., this is supposed to be a rough patch and not final.
Full Summary:
Added the following modules.
1. Module001 - Introduction To OpenStack.
- Brief Overview of OpenStack.
- Basic Concepts
- Detailed Description of Core
Projects (Grizzly) under OpenStack.
- All But Networking and Swift.
2. Module002 - OpenStack Networking In detail.
3. Module003 - OpenStack Object Storage In detail.
4. Lab001 - OpenStack Control Node and Compute Node.
5. Lab002 - OpenStack Network Node.
Full Summary added due to the size of the commit. I Apologize
for the size of this commit and will try not to commit huge
content like in this patch. The reason for the size of this
commit is to meet OpenStack Training Sprint day.
bp/training-manuals
Change-Id: Ie3c44527992868b4d9571b66cc1c048e558ec669
441 lines
19 KiB
XML
441 lines
19 KiB
XML
<?xml version="1.0" encoding="utf-8"?>
|
||
<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:id="module001-ch008-queues-messaging">
|
||
<title>OpenStack Messaging and Queues</title>
|
||
<figure>
|
||
<title>Messaging in OpenStack</title>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="figures/image04.png"/>
|
||
</imageobject>
|
||
</mediaobject>
|
||
</figure>
|
||
<para>AMQP is the messaging technology chosen by the OpenStack
|
||
cloud. The AMQP broker, either RabbitMQ or Qpid, sits between any
|
||
two Nova components and allows them to communicate in a loosely
|
||
coupled fashion. More precisely, Nova components (the compute
|
||
fabric of OpenStack) use Remote Procedure Calls (RPC hereinafter)
|
||
to communicate to one another; however such a paradigm is built
|
||
atop the publish/subscribe paradigm so that the following benefits
|
||
can be achieved:</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Decoupling between client and servant (such as the client
|
||
does not need to know where the servant’s reference
|
||
is).</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>Full a-synchronism between client and servant (such as the
|
||
client does not need the servant to run at the same time of
|
||
the remote call).</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>Random balancing of remote calls (such as if more servants
|
||
are up and running, one-way calls are transparently dispatched
|
||
to the first available servant).</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<para>Nova uses direct, fanout, and topic-based exchanges. The
|
||
architecture looks like the one depicted in the figure
|
||
below:</para>
|
||
<figure>
|
||
<title>AMQP</title>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="figures/image24.png"/>
|
||
</imageobject>
|
||
</mediaobject>
|
||
</figure>
|
||
<para>Nova implements RPC (both request+response, and one-way,
|
||
respectively nicknamed ‘rpc.call’ and ‘rpc.cast’) over AMQP by
|
||
providing an adapter class which take cares of marshaling and
|
||
unmarshaling of messages into function calls. Each Nova service
|
||
(for example Compute, Scheduler, etc.) create two queues at the
|
||
initialization time, one which accepts messages with routing keys
|
||
‘NODE-TYPE.NODE-ID’ (for example compute.hostname) and another,
|
||
which accepts messages with routing keys as generic ‘NODE-TYPE’
|
||
(for example compute). The former is used specifically when
|
||
Nova-API needs to redirect commands to a specific node like
|
||
‘euca-terminate instance’. In this case, only the compute node
|
||
whose host’s hypervisor is running the virtual machine can kill
|
||
the instance. The API acts as a consumer when RPC calls are
|
||
request/response, otherwise is acts as publisher only.</para>
|
||
<para><guilabel>Nova RPC Mappings</guilabel></para>
|
||
<para>The figure below shows the internals of a message broker
|
||
node (referred to as a RabbitMQ node in the diagrams) when a
|
||
single instance is deployed and shared in an OpenStack cloud.
|
||
Every Nova component connects to the message broker and,
|
||
depending on its personality (for example a compute node or a
|
||
network node), may use the queue either as an Invoker (such as
|
||
API or Scheduler) or a Worker (such as Compute or Network).
|
||
Invokers and Workers do not actually exist in the Nova object
|
||
model, but we are going to use them as an abstraction for sake
|
||
of clarity. An Invoker is a component that sends messages in the
|
||
queuing system via two operations: 1) rpc.call and ii) rpc.cast;
|
||
a Worker is a component that receives messages from the queuing
|
||
system and reply accordingly to rcp.call operations.</para>
|
||
<para>Figure 2 shows the following internal elements:</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para><emphasis role="bold">Topic Publisher:</emphasis>a Topic
|
||
Publisher comes to life when an rpc.call or an rpc.cast
|
||
operation is executed; this object is instantiated and used to
|
||
push a message to the queuing system. Every publisher connects
|
||
always to the same topic-based exchange; its life-cycle is
|
||
limited to the message delivery.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Direct Consumer:</emphasis>a
|
||
Direct Consumer comes to life if (an only if) a rpc.call
|
||
operation is executed; this object is instantiated and used to
|
||
receive a response message from the queuing system; Every
|
||
consumer connects to a unique direct-based exchange via a
|
||
unique exclusive queue; its life-cycle is limited to the
|
||
message delivery; the exchange and queue identifiers are
|
||
determined by a UUID generator, and are marshaled in the
|
||
message sent by the Topic Publisher (only rpc.call
|
||
operations).</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Topic Consumer:</emphasis>a Topic
|
||
Consumer comes to life as soon as a Worker is instantiated and
|
||
exists throughout its life-cycle; this object is used to
|
||
receive messages from the queue and it invokes the appropriate
|
||
action as defined by the Worker role. A Topic Consumer
|
||
connects to the same topic-based exchange either via a shared
|
||
queue or via a unique exclusive queue. Every Worker has two
|
||
topic consumers, one that is addressed only during rpc.cast
|
||
operations (and it connects to a shared queue whose exchange
|
||
key is ‘topic’) and the other that is addressed only during
|
||
rpc.call operations (and it connects to a unique queue whose
|
||
exchange key is ‘topic.host’).</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Direct Publisher:</emphasis>a
|
||
Direct Publisher comes to life only during rpc.call operations
|
||
and it is instantiated to return the message required by the
|
||
request/response operation. The object connects to a
|
||
direct-based exchange whose identity is dictated by the
|
||
incoming message.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Topic Exchange:</emphasis>The
|
||
Exchange is a routing table that exists in the context of a
|
||
virtual host (the multi-tenancy mechanism provided by Qpid or
|
||
RabbitMQ); its type (such as topic vs. direct) determines the
|
||
routing policy; a message broker node will have only one
|
||
topic-based exchange for every topic in Nova.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Direct Exchange:</emphasis>this is
|
||
a routing table that is created during rpc.call operations;
|
||
there are many instances of this kind of exchange throughout
|
||
the life-cycle of a message broker node, one for each rpc.call
|
||
invoked.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Queue Element:</emphasis>A Queue
|
||
is a message bucket. Messages are kept in the queue until a
|
||
Consumer (either Topic or Direct Consumer) connects to the
|
||
queue and fetch it. Queues can be shared or can be exclusive.
|
||
Queues whose routing key is ‘topic’ are shared amongst Workers
|
||
of the same personality.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<figure>
|
||
<title>RabbitMQ</title>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="figures/image20.png"/>
|
||
</imageobject>
|
||
</mediaobject>
|
||
</figure>
|
||
<para><guilabel>RPC Calls</guilabel></para>
|
||
<para>The diagram below shows the message flow during an rp.call
|
||
operation:</para>
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>a Topic Publisher is instantiated to send the message
|
||
request to the queuing system; immediately before the
|
||
publishing operation, a Direct Consumer is instantiated to
|
||
wait for the response message.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>once the message is dispatched by the exchange, it is
|
||
fetched by the Topic Consumer dictated by the routing key
|
||
(such as ‘topic.host’) and passed to the Worker in charge of
|
||
the task.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>once the task is completed, a Direct Publisher is
|
||
allocated to send the response message to the queuing
|
||
system.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>once the message is dispatched by the exchange, it is
|
||
fetched by the Direct Consumer dictated by the routing key
|
||
(such as ‘msg_id’) and passed to the Invoker.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
<figure>
|
||
<title>RabbitMQ</title>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="figures/image28.png"/>
|
||
</imageobject>
|
||
</mediaobject>
|
||
</figure>
|
||
<para><guilabel>RPC Casts</guilabel></para>
|
||
<para>The diagram below the message flow during an rp.cast
|
||
operation:</para>
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>A Topic Publisher is instantiated to send the message
|
||
request to the queuing system.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>Once the message is dispatched by the exchange, it is
|
||
fetched by the Topic Consumer dictated by the routing key
|
||
(such as ‘topic’) and passed to the Worker in charge of the
|
||
task.</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
<figure>
|
||
<title>RabbitMQ</title>
|
||
<mediaobject>
|
||
<imageobject>
|
||
<imagedata fileref="figures/image20.png"/>
|
||
</imageobject>
|
||
</mediaobject>
|
||
</figure>
|
||
<para><guilabel>AMQP Broker Load</guilabel></para>
|
||
<para>At any given time the load of a message broker node running
|
||
either Qpid or RabbitMQ is function of the following
|
||
parameters:</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Throughput of API calls: the number of API calls (more
|
||
precisely rpc.call ops) being served by the OpenStack cloud
|
||
dictates the number of direct-based exchanges, related
|
||
queues and direct consumers connected to them.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>Number of Workers: there is one queue shared amongst
|
||
workers with the same personality; however there are as many
|
||
exclusive queues as the number of workers; the number of
|
||
workers dictates also the number of routing keys within the
|
||
topic-based exchange, which is shared amongst all
|
||
workers.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<para>The figure below shows the status of a RabbitMQ node after
|
||
Nova components’ bootstrap in a test environment. Exchanges and
|
||
queues being created by Nova components are:</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Exchanges</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>nova (topic exchange)</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para>Queues</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<orderedlist>
|
||
<listitem>
|
||
<para>compute.phantom (phantom is hostname)</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>compute</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>network.phantom (phantom is hostname)</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>network</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>scheduler.phantom (phantom is hostname)</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para>scheduler</para>
|
||
</listitem>
|
||
</orderedlist>
|
||
<para><guilabel>RabbitMQ Gotchas</guilabel></para>
|
||
<para>Nova uses Kombu to connect to the RabbitMQ environment.
|
||
Kombu is a Python library that in turn uses AMQPLib, a library
|
||
that implements the standard AMQP 0.8 at the time of writing.
|
||
When using Kombu, Invokers and Workers need the following
|
||
parameters in order to instantiate a Connection object that
|
||
connects to the RabbitMQ server (please note that most of the
|
||
following material can be also found in the Kombu documentation;
|
||
it has been summarized and revised here for sake of
|
||
clarity):</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para><emphasis role="bold">Hostname:</emphasis> The hostname
|
||
to the AMQP server.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Userid:</emphasis> A valid
|
||
username used to authenticate to the server.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Password:</emphasis> The password
|
||
used to authenticate to the server.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Virtual_host:</emphasis> The name
|
||
of the virtual host to work with. This virtual host must exist
|
||
on the server, and the user must have access to it. Default is
|
||
“/”.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Port:</emphasis> The port of the
|
||
AMQP server. Default is 5672 (amqp).</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<para>The following parameters are default:</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para><emphasis role="bold">Insist:</emphasis> insist on
|
||
connecting to a server. In a configuration with multiple
|
||
load-sharing servers, the Insist option tells the server that
|
||
the client is insisting on a connection to the specified
|
||
server. Default is False.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Connect_timeout:</emphasis> the
|
||
timeout in seconds before the client gives up connecting to
|
||
the server. The default is no timeout.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">SSL:</emphasis> use SSL to connect
|
||
to the server. The default is False.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<para>More precisely Consumers need the following
|
||
parameters:</para>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para><emphasis role="bold">Connection:</emphasis> the above
|
||
mentioned Connection object.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Queue:</emphasis>name of the
|
||
queue.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Exchange:</emphasis>name of the
|
||
exchange the queue binds to.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Routing_key:</emphasis>the
|
||
interpretation of the routing key depends on the value of the
|
||
exchange_type attribute.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para><emphasis role="bold">Direct exchange:</emphasis>if the
|
||
routing key property of the message and the routing_key
|
||
attribute of the queue are identical, then the message is
|
||
forwarded to the queue.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Fanout
|
||
exchange:</emphasis>messages are forwarded to the queues bound
|
||
the exchange, even if the binding does not have a key.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Topic exchange:</emphasis>if the
|
||
routing key property of the message matches the routing key of
|
||
the key according to a primitive pattern matching scheme, then
|
||
the message is forwarded to the queue. The message routing key
|
||
then consists of words separated by dots (”.”, like domain
|
||
names), and two special characters are available; star (“”)
|
||
and hash (“#”). The star matches any word, and the hash
|
||
matches zero or more words. For example ”.stock.#” matches the
|
||
routing keys “usd.stock” and “eur.stock.db” but not
|
||
“stock.nasdaq”.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para><emphasis role="bold">Durable:</emphasis>this flag
|
||
determines the durability of both exchanges and queues;
|
||
durable exchanges and queues remain active when a RabbitMQ
|
||
server restarts. Non-durable exchanges/queues (transient
|
||
exchanges/queues) are purged when a server restarts. It is
|
||
worth noting that AMQP specifies that durable queues cannot
|
||
bind to transient exchanges. Default is True.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Auto_delete:</emphasis>if set, the
|
||
exchange is deleted when all queues have finished using it.
|
||
Default is False.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Exclusive:</emphasis>exclusive
|
||
queues (such as non-shared) may only be consumed from by the
|
||
current connection. When exclusive is on, this also implies
|
||
auto_delete. Default is False.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Exchange_type:</emphasis>AMQP
|
||
defines several default exchange types (routing algorithms)
|
||
that covers most of the common messaging use cases.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold"
|
||
>Auto_ack:</emphasis>acknowledgement is handled automatically
|
||
once messages are received. By default auto_ack is set to
|
||
False, and the receiver is required to manually handle
|
||
acknowledgment.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">No_ack:</emphasis>it disable
|
||
acknowledgement on the server-side. This is different from
|
||
auto_ack in that acknowledgement is turned off altogether.
|
||
This functionality increases performance but at the cost of
|
||
reliability. Messages can get lost if a client dies before it
|
||
can deliver them to the application.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Auto_declare:</emphasis>if this is
|
||
True and the exchange name is set, the exchange will be
|
||
automatically declared at instantiation. Auto declare is on by
|
||
default. Publishers specify most the parameters of Consumers
|
||
(such as they do not specify a queue name), but they can also
|
||
specify the following:</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">Delivery_mode:</emphasis>the
|
||
default delivery mode used for messages. The value is an
|
||
integer. The following delivery modes are supported by
|
||
RabbitMQ:</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<itemizedlist>
|
||
<listitem>
|
||
<para><emphasis role="bold">1 or “transient”:</emphasis>the
|
||
message is transient. Which means it is stored in memory only,
|
||
and is lost if the server dies or restarts.</para>
|
||
</listitem>
|
||
<listitem>
|
||
<para><emphasis role="bold">2 or “persistent”:</emphasis>the
|
||
message is persistent. Which means the message is stored both
|
||
in-memory, and on disk, and therefore preserved if the server
|
||
dies or restarts.</para>
|
||
</listitem>
|
||
</itemizedlist>
|
||
<para>The default value is 2 (persistent). During a send
|
||
operation, Publishers can override the delivery mode of messages
|
||
so that, for example, transient messages can be sent over a
|
||
durable queue.</para>
|
||
</chapter>
|