Octavia Controller Worker specification

Initial draft of a spec for the Controller Worker, formerly known as
deploy-worker.

Change-Id: If17bac321275b6cc1137574f37837dfe220606d8

specs/version0.5/controller-worker.rst

@@ -0,0 +1,270 @@
+..
+ This work is licensed under a Creative Commons Attribution 3.0 Unported
+ License.
+
+ http://creativecommons.org/licenses/by/3.0/legalcode
+
+==================================
+Controller Worker (deploy-worker)
+==================================
+
+Launchpad blueprint:
+
+https://blueprints.launchpad.net/octavia/+spec/controller-worker
+
+Octavia is an operator-grade reference implementation for Load Balancing as a
+Service (LBaaS) for OpenStack.  The component of Octavia that does the load
+balancing is known as Amphora.
+
+The component of Octavia that provides command and control of the Amphora is
+the Octavia controller.
+
+Problem description
+===================
+
+Components of the Octavia controller require a shared library that provides
+the orchestration of create/update/delete actions for Octavia objects such as
+load balancers and listeners.
+
+It is expected that this library will be used by the Queue Consumer to service
+API requests, by the Housekeeping Manager to manage the spare Amphora pool,
+and by the Health Manager to fail over failed objects.
+
+Proposed change
+===============
+
+The Controller Worker will be implemented as a class that provides methods to
+facilitate the create/update/delete actions.  This class will be responsible
+for managing the number of simultaneous operations being executed by
+coordinating through the Octavia database.
+
+The Controller Worker will provide a base class that sets up and initilizes
+the TaskFlow engines required to complete the action.  Users of the library
+will then call the appropriate method for the action.  These methods setup
+and launch the appropriate flow.  Each flow will be contained in a seperate
+class for code reuse and supportability.
+
+The Controller Worker library will provide the following methods:
+
+.. code:: python
+
+    def create_amphora(self):
+        """
+            Creates an Amphora.
+
+            :returns: amphora_id
+        """
+        raise NotImplementedError
+
+    def delete_amphora(self, amphora_id):
+        """
+            Deletes an existing Amphora.
+
+            :returns: None
+            :raises AmphoraNotFound: The referenced Amphora was not found
+        """
+        raise NotImplementedError
+
+    def create_load_balancer(self, load_balancer_id):
+        """
+            Creates a load balancer by allocating Amphorae.
+
+            :returns: None
+            :raises NoSuitableAmphora: Unable to allocate an Amphora.
+        """
+        raise NotImplementedError
+
+    def update_load_balancer(self, load_balancer_updates, load_balancer_id):
+        """
+            Updates a load balancer.
+
+            :returns: None
+            :raises LBNotFound: The referenced load balancer was not found
+        """
+        raise NotImplementedError
+
+    def delete_load_balancer(self, load_balancer_id):
+        """
+            Deletes a load balancer by de-allocating Amphorae.
+
+            :returns: None
+            :raises LBNotFound: The referenced load balancer was not found
+        """
+        raise NotImplementedError
+
+    def create_listener(self, listener_id):
+        """
+            Creates a listener.
+
+            :returns: None
+            :raises NoSuitableLB: Unable to find the load balancer
+        """
+        raise NotImplementedError
+
+    def update_listener(self, listener_updates, listener_id):
+        """
+            Updates a listener.
+
+            :returns: None
+            :raises ListenerNotFound: The referenced listener was not found
+        """
+        raise NotImplementedError
+
+    def delete_listener(self, listener_id):
+        """
+            Deletes a listener.
+
+            :returns: None
+            :raises ListenerNotFound: The referenced listener was not found
+        """
+        raise NotImplementedError
+
+    def create_pool(self, pool_id):
+        """
+            Creates a node pool.
+
+            :returns: None
+            :raises NoSuitableLB: Unable to find the load balancer
+        """
+        raise NotImplementedError
+
+    def update_pool(self, pool_updates, pool_id):
+        """
+            Updates a node pool.
+
+            :returns: None
+            :raises PoolNotFound: The referenced pool was not found
+        """
+        raise NotImplementedError
+
+    def delete_pool(self, pool_id):
+        """
+            Deletes a node pool.
+
+            :returns: None
+            :raises PoolNotFound: The referenced pool was not found
+        """
+        raise NotImplementedError
+
+    def create_health_monitor(self, health_monitor_id):
+        """
+            Creates a health monitor.
+
+            :returns: None
+            :raises NoSuitablePool: Unable to find the node pool
+        """
+        raise NotImplementedError
+
+    def update_health_monitor(self, health_monitor_updates, health_monitor_id):
+        """
+            Updates a health monitor.
+
+            :returns: None
+            :raises HMNotFound: The referenced health monitor was not found
+        """
+        raise NotImplementedError
+
+    def delete_health_monitor(self, health_monitor_id):
+        """
+            Deletes a health monitor.
+
+            :returns: None
+            :raises HMNotFound: The referenced health monitor was not found
+        """
+        raise NotImplementedError
+
+    def create_member(self, member_id):
+        """
+            Creates a pool member.
+
+            :returns: None
+            :raises NoSuitablePool: Unable to find the node pool
+        """
+        raise NotImplementedError
+
+    def update_member(self, member_updates, member_id):
+        """
+            Updates a pool member.
+
+            :returns: None
+            :raises MemberNotFound: The referenced member was not found
+        """
+        raise NotImplementedError
+
+    def delete_member(self, member_id):
+        """
+            Deletes a pool member.
+
+            :returns: None
+            :raises MemberNotFound: The referenced member was not found
+        """
+        raise NotImplementedError
+
+Alternatives
+------------
+This code could be included in the Queue Consumer component of the controller.
+However this would not allow the library to be shared with other components of
+the controller, such as the Health Manager
+
+Data model impact
+-----------------
+
+
+REST API impact
+---------------
+None
+
+Security impact
+---------------
+
+
+Notifications impact
+--------------------
+
+
+Other end user impact
+---------------------
+
+
+Performance Impact
+------------------
+
+
+Other deployer impact
+---------------------
+
+
+Developer impact
+----------------
+
+
+Implementation
+==============
+
+Assignee(s)
+-----------
+Michael Johnson <johnsom>
+
+Work Items
+----------
+
+
+Dependencies
+============
+https://blueprints.launchpad.net/octavia/+spec/amphora-driver-interface
+https://blueprints.launchpad.net/octavia/+spec/neutron-network-driver
+https://blueprints.launchpad.net/octavia/+spec/nova-compute-driver
+
+Testing
+=======
+Unit tests
+
+Documentation Impact
+====================
+None
+
+References
+==========
+https://blueprints.launchpad.net/octavia/+spec/health-manager
+https://blueprints.launchpad.net/octavia/+spec/housekeeping-manager
+https://blueprints.launchpad.net/octavia/+spec/queue-consumer

specs/version0.5/controller.dot

@@ -27,7 +27,7 @@ digraph G {
       queue [label="Queue\nConsumer", fontcolor=white, color=forestgreen, style=filled];
       health [label="Health\nManager", fontcolor=white, color=forestgreen, style=filled];
       house [label="Housekeeping\n(Spares/Cleanup)\nManager", fontcolor=white, color=forestgreen, style=filled];
-      deploy [label="Deploy\nWorker", fontcolor=white, color=forestgreen, style=filled, shape=hexagon];
+      ctrl [label="Controller\nWorker", fontcolor=white, color=forestgreen, style=filled, shape=hexagon];
       proxy [label="Services\nProxy", fontcolor=white, color=forestgreen, style=filled];
 
 
@@ -61,20 +61,20 @@ digraph G {
    amp [label="Amphorae", fontcolor=black, color=coral2, style=filled];
 
 
-   deploy -> queue [dir="both"];
+   ctrl -> queue [dir="both"];
    db -> api -> oslo -> queue [dir="both"];
-   db -> deploy [dir="both"];
+   db -> ctrl [dir="both"];
    db -> queue [dir="both"];
    db -> health [dir="both"];
    db -> house [dir="both"];
    db -> msg [dir="both"];
-   nova -> deploy [dir="both"];
+   nova -> ctrl [dir="both"];
    nova -> house [dir="both"];
-   neutron -> deploy [dir="both"];
+   neutron -> ctrl [dir="both"];
    neutron -> house [dir="both"];
    proxy -> swift [dir="both"];
    proxy -> amp [dir="both"];
-   cert -> deploy [dir="both"];
+   cert -> ctrl [dir="both"];
    cert -> bbq [dir="both"];
    stats -> ceilo [dir="both"];
    msg -> amp [ltail=cluster1];

specs/version0.5/controller.rst

@@ -50,7 +50,7 @@ The Octavia controller will consist of the following components:
 * Queue Consumer
 * Certificate Library
 * Compute Driver
-* Deploy Worker
+* Controller Worker
 * Health Manager
 * Housekeeping Manager
 * Network Driver
@@ -85,7 +85,7 @@ The Queue Consumer is event driven and tasked with servicing requests from the
 API components via an Oslo messaging.  It is also the primary lifecycle
 management component for Amphora.
 
-To service requests the Queue Consumer will spawn a Deploy Worker process.
+To service requests the Queue Consumer will spawn a Controller Worker process.
 Spawning a separate process makes sure that the Queue Consumer can continue to
 service API requests while the longer running deployment process is
 progressing.
@@ -103,31 +103,31 @@ complete the requested action.
 The Compute Driver abstracts the implementation of instantiating the virtual
 machine, container, appliance, or device that the Amphora will run in.
 
-**Deploy Worker**
+**Controller Worker**
 
-The Deploy Worker is spawned from the Queue Consumer or the Health
+The Controller Worker is spawned from the Queue Consumer or the Health
 Manager.  It interfaces with the compute driver (in some deployment scenarios),
 network driver, and Amphora driver to activate Amphora instances,
 load balancers, and listeners.
 
-When a request for a new instance or failover is received the Deploy Worker
+When a request for a new instance or failover is received the Controller Worker
 will have responsibility for connecting the appropriate networking ports to the
 Amphora via the network driver and triggering a configuration push via the
 Amphora driver.  This will include validating that the targeted Amphora
 has the required networks plumbed to the Amphora.
 
-The Amphora configured by the Deploy Worker may be an existing Amphora
+The Amphora configured by the Controller Worker may be an existing Amphora
 instance, a new Amphora from the spares pool, or a newly created Amphora.
 This determination will be made based on the apolocation requirements of
 the load balancer, the load balancer count on the existing Amphora, and
 the availability of ready spare Amphora in the spares pool.
 
-The Deploy Worker will be responsible for passing in the required metadata
+The Controller Worker will be responsible for passing in the required metadata
 via config drive when deploying an Amphora.  This metadata will include:
 a list of controller IP addresses, controller certificate authority
 certificate, and the Amphora certificate and key file.
 
-The main flow of the Deploy Worker is described in the
+The main flow of the Controller Worker is described in the
 amphora-lifecycle-management specification as the Activate Amphora sequence.
 
 **Certificate Library**

specs/version0.5/queue-consumer.rst

@@ -10,16 +10,16 @@ Queue Consumer
 https://blueprints.launchpad.net/octavia/+spec/queue-consumer
 
 This blueprint describes how Oslo messages are consumed, processed and
-delegated from the API-controller queue to the deploy worker component of
+delegated from the API-controller queue to the controller worker component of
 Octavia. The component that is responsible for these activities is called the
 Queue Consumer.
 
 Problem description
 ===================
 Oslo messages need to be consumed by the controller and delegated to the proper
-deploy worker. Something needs to interface with the API-controller queue and
+controller worker. Something needs to interface with the API-controller queue
-spawn the deploy workers. That "something" is what we are calling the Queue
+and spawn the controller workers. That "something" is what we are calling the
-Consumer.
+Queue Consumer.
 
 Proposed change
 ===============
@@ -27,15 +27,15 @@ The major component of the Queue Consumer will be be a class that acts as a
 consumer to Oslo messages. It will be responsible for configuring and starting
 a server that is then able to receive messages. There will be a one-to-one
 mapping between API methods and consumer methods (see code snippet below).
-Corresponding deploy workers will be spawned depending on which consumer
+Corresponding controller workers will be spawned depending on which consumer
 methods are called.
 
 The threading will be handled by Oslo messaging using the 'eventlet' executor.
 Using the 'eventlet' executor will allow for message throttling and removes
-the need for the deploy workers to manage threads. The benefit of using the
+the need for the controller workers to manage threads. The benefit of using the
 'eventlet' executor is that the Queue Consumer will not have to spawn threads
 at all, since every message received will be in its own thread already. This
-means that the Queue Consumer doesn't spawn a deploy worker, rather it just
+means that the Queue Consumer doesn't spawn a controller worker, rather it just
 starts the execution of the deploy code.
 
 An 'oslo_messaging' configuration section will need to be added to octavia.conf
@@ -43,9 +43,10 @@ for Oslo messaging options. For the Queue Consumer, the 'rpc_thread_pool_size'
 config option will need to be added. This option will determine how many
 consumer threads will be able to read from the queue at any given time (per
 consumer instance) and serve as a throttling mechanism for message consumption.
-For example, if 'rpc_thread_pool_size' is set to 1 thread then only one deploy
+For example, if 'rpc_thread_pool_size' is set to 1 thread then only one
-worker will be able to conduct work. When that deploy worker completes its
+controller worker will be able to conduct work. When that controller worker
-task then a new message can be consumed and a new deploy worker flow started.
+completes its task then a new message can be consumed and a new controller
+worker flow started.
 
 Below are the planned interface methods for the queue consumer. The Queue
 Consumer will be listening on the **OCTAVIA_PROV** (short for octavia
@@ -55,12 +56,12 @@ particular interface method. The *context* parameter is a dictionary and is
 reserved for metadata. For example, the Neutron LBaaS agent leverages this
 parameter to send additional request information. Additionally, update methods
 include a *\*_updates* parameter than includes the changes that need to be
-made. Thus, the deploy workers responsible for the update actions will need to
+made. Thus, the controller workers responsible for the update actions will
-query the database to retrieve the old state and combine it with the updates to
+need to query the database to retrieve the old state and combine it with the
-provision appropriately. If a rollback or exception occur, then the deploy
+updates to provision appropriately. If a rollback or exception occur, then the
-worker will only need to update the provisioning status to **ERROR** and will
+controller worker will only need to update the provisioning status to **ERROR**
-not need to worry about making database changes to attributes of the object
+and will not need to worry about making database changes to attributes of the
-being updated.
+object being updated.
 
 .. code:: python
 
@@ -116,16 +117,16 @@ Alternatives
 There are a variety of ways to consume from Oslo messaging. For example,
 instead of having a single consumer on the controller we could have multiple
 consumers (i.e. one for CREATE messages, one for UPDATE messages, etc.).
-However, since we merely need something to pass messages off to deploy workers
+However, since we merely need something to pass messages off to controller
-other options are overkill.
+workers other options are overkill.
 
 Data model impact
 -----------------
 While there is no direct data model impact it is worth noting that the API
 will not be persisting updates to the database. Rather, delta updates will pass
-from the user all the way to the deploy worker. Thus, when the deploy worker
+from the user all the way to the controller worker. Thus, when the controller
-successfully completes the prescribed action, only then will it persist the
+worker successfully completes the prescribed action, only then will it persist
-updates to the database. No API changes are necessary for create and update
+the updates to the database. No API changes are necessary for create and update
 actions.
 
 REST API impact
@@ -173,7 +174,7 @@ Work Items
 
 Dependencies
 ============
-https://blueprints.launchpad.net/octavia/+spec/deploy-worker
+https://blueprints.launchpad.net/octavia/+spec/controller-worker
 
 Testing
 =======