openstack/designate
Code Issues Proposed changes

re-org docs & add details on production architecture

Browse Source 
bug# 1201494

Change-Id: Ie2bd28a1cb7e9f3408e44e2164d72de2c857fc6a
This commit is contained in:
Simon McCartney 2013-07-18 22:33:10 +01:00
parent e2419eae65
commit fce9ffb5dd
15 changed files with 230 additions and 272 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

17
README.md
View File

@ -39,6 +39,18 @@ pip install -r requirements.txt -r test-requirements.txt
python setup.py develop
````
## Building the documentation
To build the documentation from the restructred text source, do the following:
````
cd doc
pip install -r requirements.txt
sphinx-build source/ html/
````
now point your browser at html/index.html
(the official documentation is published to [readthedocs](http://designate.readthedocs.org/en/latest/install.html) by the
maintainers.
## Contributing
Install the git-review package to make life easier
@ -60,4 +72,9 @@ git rebase -i
git-review
````
## Testing
Execute a single test using py27 (test is CentralServiceTest.test_count_domains)
````
tox -e py27 -- designate/tests/test_central/test_service.py:CentralServiceTest.test_count_domains
````

63
doc/source/architecture.rst
View File

@ -13,9 +13,6 @@
License for the specific language governing permissions and limitations
under the License.
.. _architecture:
============
Architecture
============
@ -23,22 +20,54 @@ Architecture
.. index::
double: architecture; brief
Brief overview
++++++++++++++
:term:`central` is the central component that stores data persistantly in
:term:`storage` which is a backend using typically SQLAlchemy, MongoDB or
others.
High Level Topology
-----------------------
:term:`agent` is the component that interacts with DNS servers themselves,
these are usually Bind, PowerDNS or others. Agents communicate to
:term:`central` via the :term:`mq`.
.. image:: images/Designate-Simple.png
:term:`api` standard OpenStack alike REST api service that communicates via
:term:`mq` to do actions like adding records and so on.
.. _designate-api:
:term:`sink` is a service which listens for notications from Nova/Quantum,
and takes action based on the event, and the configured plugins.
Designate API
-----------------------
designate-api provides the standard OpenStack style REST api service, accepting http requests, validating authentication tokens with Keystone & passing them to the :ref:`designate-central` service over AMQP. Multiple versions of the api can be hosted, as well as api-extensions, allowing for pluggable extensions to the core api.
:term:`storage`s are drivers for a particular SQL/NoSQL server.
Although designate-api is capable of handling https traffic, it's typical to terminate https elsewhere, e.g. by placing nginx infront of designate-api or by letting the the external facing load balancers terminate https.
:term:`backend`s are drivers for a particular DNS server.
.. _designate-central:
Designate Central
-----------------------
designate-central is the service that handles RPC requests via the MQ, it coordinates the persistant storage of data & manipulating backend DNS servers via backends or agents. storage is provided via plugins, typically SQLAlchemy, although MongoDB or other storage drivers should be possible.
.. _designate-sink:
Designate Sink
-----------------------
designate-sink is an optional service which listens for event notifications, such as compute.instance.create.end, handlers are available for Nova and Neutron. Notification events can then be used to trigger record creation & deletion.
The current sink implementations generate simple forward lookup A records, using a format specified in :ref:`handler-nova` configuration. Any field in the event notification can be used to generate a record.
.. _dns-backend:
DNS Backend
-----------------------
backends are drivers for a particular DNS server.
Designate supports multiple backend implementations, PowerDNS, BIND, MySQL BIND and dnsmasq, you are also free to implement your own backend to fit your needs, as well as extensions to provide extra functionality to complement existing backends.
.. _designate-agent:
Designate Agent
-----------------------
designate-agents are optional components that interact with DNS servers. Agents are alternatives to backend plugins in :ref:`designate-central`. Agents communicate with :ref:`designate-central` via the :ref:`message-queue`.
.. _message-queue:
Message Queue
-----------------------
Designate uses oslo.rpc for messaging between components, therefore it inherits a requirement for a supported messaging bus (RabbitMQ, Qpid, ZeroMQ etc). Typically this means a RabbitMQ setup is dedicated to Designate, but as only a single virtualhost is required for a normal installation, you’re free to use other RabbitMQ instances as you see fit.
.. _database:
Database/Storage
-----------------------
storage drivers are drivers for a particular SQL/NoSQL server, designate needs a SQLAlchemy supported storage engine for the persistent storage of data, the recommended driver is MySQL.

1
doc/source/configuration.rst
View File

@ -93,6 +93,7 @@ Parameter Default Note
domain_id None The domain ID to use
=========================== ==================================== ==============================================================
.. _handler-nova:
Handler - Nova *
================

BIN
doc/source/diagrams/Designate-MultiZone.shapes Normal file
View File

Binary file not shown.

BIN
doc/source/diagrams/Designate-PowerDNS-Detail.shapes Normal file
View File

Binary file not shown.

BIN
doc/source/diagrams/Designate-Simple.shapes Normal file
View File

Binary file not shown.

9
doc/source/diagrams/README Normal file
View File

@ -0,0 +1,9 @@
*.shapes files are from the OSX App Shapes (http://shapesapp.com/)
It's hoped that this diagrams can be replaced by something from a multi-platform tool such as:
* Dia
* http://sphinx-doc.org/ext/graphviz.html (see http://graphviz.org/content/cluster as a possible example)
* aafig (https://bitbucket.org/birkenfeld/sphinx-contrib)

41
doc/source/getting-involved.rst Normal file
View File

@ -0,0 +1,41 @@
================
Getting Involved
================
#openstack-dns
--------------
There is an active IRC channel at irc://freenode.net/#openstack-dns, where many of the designate contributors can be found, as
well as users from various organisations.
Contributing
------------
We welcome fixes, extensions, documentation, pretty much anything that helps improve Designate, contributing is easy & follows
the standard OpenStack `Gerrit workflow`_, if you're looking for something to do, you could always checkout the blueprint_ & bug_
lists.
Assuming you've already got a working :ref:`Development Environment`, here's a quick summary:
Install the git-review package to make life easier
.. code-block:: shell-session
pip install git-review
Branch, work, & submit:
.. code-block:: shell-session
# cut a new branch, tracking master
git checkout --track -b bug/id origin/master
# work work work
git add stuff
git commit
# rebase/squash to a single commit before submitting
git rebase -i
# submit
git-review
.. _Gerrit workflow: https://wiki.openstack.org/wiki/Gerrit_Workflow
.. _blueprint: https://blueprints.launchpad.net/designate
.. _bug: https://bugs.launchpad.net/designate

25
doc/source/gettingstarted.rst → doc/source/getting-started.rst
View File

@ -13,28 +13,22 @@
License for the specific language governing permissions and limitations
under the License.
.. _gettingstarted:
.. _getting-started:
========================
Getting Started
========================
Designate provides DNS As A Service (DNSaaS) for OpenStack:
Designate is comprised of three designate components :ref:`designate-api`, :ref:`designate-central` and :ref:`designate-sink`, supported by a few standard open source components. For more info see :doc:`architecture`.
- REST API for domain/record management
- Multi-tenant
- Integrated with Keystone for authentication
- Framework in place to integrate with Nova and Quantum notifications (for auto-generated records)
- Support for PowerDNS and Bind9 out of the box
Designate is comprised of three components. For more info see :doc:`architecture`.
This guide will show how to install a simple development environment for Designate, with a PowerDNS backend.
You will be able to see Designate in action, although not to its full potential.
This guide will walk you through setting up a development environment for Designate, using PowerDNS as the DNS
backend, where possible the simplest options have been chosen for you. For a more complete discussion on
installation & configuration options, please see :doc:`architecture` and :doc:`production-architecture`.
For this guide you will need access to an Ubuntu Server (12.04).
.. _Development Environment:
Development Environment
+++++++++++++++++++++++
@ -114,6 +108,7 @@ Configure Designate
.. index::
double: configure; designate
::
$ editor designate.conf
@ -174,9 +169,9 @@ You’ll now be seeing the log from the API service.
Exercising the API
==================
Using a web browser, curl statement, or a Rest client calls can be made to the Designate API using the following format where “command” is any of the commands listed in the Designate Documentation_.
Using a web browser, curl statement, or a REST client calls can be made to the Designate API using the following format where “command” is any of the commands listed in the :doc:`rest`
.. _Documentation: http://designate.readthedocs.org/en/latest/rest.html
.. _Designate REST Documentation:
http://IP.Address:9001/v1/command

17
doc/source/glossary.rst
View File

@ -21,17 +21,34 @@ Glossary
============
.. glossary::
agent
Software service running on nodes that executes actions towards typically
nameservers like BIND or similar.
central
Software service running on a central management node that stores
information persistantly in a backend storage using a configurable driver
like SQLAlchemy or other.
api
HTTP REST API service for Designate
mq
A message queue, typically something like RabbitMQ or ZeroMQ
storage
A backend for storing data/information persistantly. Typically MongoDB or
a SQL based server software.
node
a server, physical or virtual, running some aspect of the designate system.
sink
Optional Software Service that is configured to listen to events from the Nova
or Neutron event queue and use the central service to generate simple A records
for instances as they are created, and delete A records as they are deleted.
backend
A backend is used by the central service to manipulate the data served by a DNS
server. There are backends that support PowerDNS, BIND, MySQL BIND & dnsmasq.

BIN
doc/source/images/Designate-MultiZone.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 36 KiB

BIN
doc/source/images/Designate-Simple.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 42 KiB

28
doc/source/index.rst
View File

@ -3,21 +3,33 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to designate's documentation!
===================================
Designate, a DNSaaS component for OpenStack
====================================================
Contents:
Designate provides DNSaaS services for OpenStack:
* REST API for domain & record management
* Multi-tenant
* Integrated with Keystone for authentication
* Framework in place to integrate with Nova and Neutron notifications (for auto-generated records)
* Support for PowerDNS and Bind9 out of the box
This document describes Designate for users & contributors of the project.
This documentation is generated by the Sphinx toolkit and lives in the `source tree`_. Additional documentation on Designate may
also be found on the `OpenStack wiki`_.
.. toctree::
:maxdepth: 2
:maxdepth: 1
architecture
gettingstarted
install
getting-started
getting-involved
configuration
rest
devstack
related
production-architecture
glossary
@ -28,3 +40,7 @@ Indices and tables
* :ref:`modindex`
* :ref:`search`
.. _OpenStack Wiki: https://wiki.openstack.org/wiki/Designate
.. _source tree: https://github.com/stackforge/designate

234
doc/source/install.rst
View File

@ -1,234 +0,0 @@
..
Copyright 2012 Endre Karlson for Bouvet ASA
Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.
.. _install:
========================
Install
========================
Designate is comprised of three components for more info on these please
see :doc:`architecture`.
.. note::
Designate makes extensive use of the messaging bus, but has not
yet been tested with ZeroMQ. We recommend using RabbitMQ for now.
From Source / GIT
+++++++++++++++++
Common Steps
============
.. index::
double: installing; common_steps
.. note::
The below operations should take place underneath your <project>/etc folder.
1. Install system package dependencies (Ubuntu)::
$ apt-get install python-pip python-virtualenv
$ apt-get install rabbitmq-server
$ apt-get build-dep python-lxml
2. Clone the Designate repo off of Stackforge::
$ git clone https://github.com/stackforge/designate.git
$ cd designate
3. Setup virtualenv::
.. note::
This is to not interfere with system packages etc.
$ virtualenv --no-site-packages .venv
$ . .venv/bin/activate
4. Install Designate and it's dependencies::
$ cd designate
$ pip install -r requirements.txt -r test-requirements.txt
$ python setup.py develop
Copy sample config files::
$ cd etc/designate
$ ls *.sample | while read f; do cp $f $(echo $f | sed "s/.sample$//g"); done
5. Install a DNS server::
1. PowerDNS (recommended)::
Install PowerDNS::
$ DEBIAN_FRONTEND=noninteractive apt-get install pdns-server pdns-backend-sqlite3
Update path to SQLite database to `$CHECKOUT_PATH/powerdns.sqlite`::
$ editor /etc/powerdns/pdns.d/pdns.local.gsqlite3
Restart PowerDNS::
$ service pdns restart
2. BIND9::
Install BIND9::
$ apt-get install bind9
Include the Designate managed zones::
$ editor /etc/bind/named.conf
Add the following line to the file::
include "$CHECKOUT_PATH/bind9/zones.config"
Disable AppArmor for BIND9::
$ touch /etc/apparmor.d/disable/usr.sbin.named
$ service apparmor reload
Restart BIND9::
$ service bind9 restart
7. If you intend to run Designate as a non-root user, then sudo permissions
need to be granted::
$ echo "designate ALL=(ALL) NOPASSWD:ALL" | sudo tee -a /etc/sudoers.d/90-designate
$ sudo chmod 0440 /etc/sudoers.d/90-designate
8. Configure the common settings::
Change the necessary configuration settings in the DEFAULT section, the file
is in the `etc/designate` folder::
$ editor designate.conf
Refer to :doc:`configuration` details on configuring the service.
Note on running processes
=========================
You can start each of the processes mentioned below in for example a screen
session to view output
Installing the Central Service
==============================
.. index::
double: installing; central
1. See `Common Steps`_ before proceeding.
2. Configure the :term:`central` service::
Change the necessary configuration settings in the service:central,
storage:sqlalchemy sections and, optionally the backend:powerdns section::
$ editor designate.conf
Refer to :doc:`configuration` details on configuring the service.
.. note::
Pay particular attention to the "backend_driver" setting, along with the
two sql_connection strings for service:central, and backend:powerdns.
These *must* point at different databases.
3. Initialize and sync the Designate database::
$ designate-manage database-init
$ designate-manage database-sync
4. Initialize and sync the PowerDNS database, if necessay::
$ designate-manage powerdns database-init
$ designate-manage powerdns database-sync
5. Start the central service::
$ designate-central
Installing the Agent Service
============================
.. index::
double: installing; agent
.. note::
The agent service is only required when zone configuration must be "manually"
propagated to multiple servers. For example, remote BIND9 instances needs to,
at the very least, know about all of the zones it is responsible for.
For the purposes of a single BIND9 quickstart.. You can skip this service.
If you use PowerDNS, this service can always be skipped.
1. See `Common Steps`_ before proceeding.
2. Configure the :term:`agent` service::
Change the necessary configuration settings in the service:agent,
and backend:bind9 sections::
$ editor designate.conf
Refer to :doc:`configuration` details on configuring the service.
3. Start the agent service::
$ designate-agent
Installing the API Service
==========================
.. index::
double: installing; api
.. note::
The API Server needs to able to talk to Keystone for AuthN + Z and
communicates via MQ to other services.
1. See `Common Steps`_ before proceeding.
2. Configure the :term:`api` service::
Change the necessary configuration settings in the service:api section::
$ editor designate.conf
.. note::
Pay particular attention to the "auth_strategy" setting, "noauth" disables
all authentication, and keystone requires setup in the api-paste.
If using the keystone auth_strategy, change the necessary configuration
settings in the paste deploy config::
$ editor api-paste.conf
Refer to :doc:`configuration` details on configuring the service.
3. Start the API service::
$ designate-api

67
doc/source/production-architecture.rst Normal file
View File

@ -0,0 +1,67 @@
=============================
Production Architecture
=============================
Multi-Zone Architecture
-----------------------
.. image:: images/Designate-MultiZone.png
Outline
-------
This document outlines what a production environment hosting Designate could look like, it follows an in-cloud model, where Designate would be hosted on instances in an OpenStack cloud. It's supposed to complement the
:doc:`architecture` document, please start there if you are unfamiliar with the designate components.
Designate Dependencies
----------------------
Designate has been designed to integrate with Keystone, or a Keystone-like service, for authentication & authorization, in a production environment it should rely on your Keystone service, and
be registered in your service catalog.
Expectations
------------
This architecture expects your environment to have an external loadbalancer that is the first touch point for customer traffic, this will distribute requests across the available API nodes,
which should span your AZs & regions where possible.
Roles
-----
A Designate deploy breaks down into several key roles:
- `Designate API`_
- `Designate Central`_
- `Message Queue`_
- `Database`_ (MySQL or derivative)
- `DNS Backend`_
Designate API
~~~~~~~~~~~~~~~~~~~
Typically, API nodes would be made available in multiple AZs, providing redundancy should an individual AZ have issues.
In a Multi-AZ deployment, the API nodes should be configured to talk to all members of the MQ Cluster - so that in the event of MQ node failing, requests continue to flow to the MQ.
Designate Central
~~~~~~~~~~~~~~~~~~~~~~~
In a Multi-AZ deployment, the Central nodes should be configured to talk to all members of the MQ Cluster - so that in the event of MQ node failing, requests continue to be processed.
Designate Sink
~~~~~~~~~~~~~~~~~~~~~~~
In a Multi-AZ deployment, the sink node should be configured to talk to all members of the MQ Cluster - so that in the event of MQ node failing, requests continue to flow to the MQ.
Message Queue
~~~~~~~~~~~~~
An AMQP implementation is required for all communication between api & central nodes, in practice this means an RabbitMQ installation, preferably a cluster that spans across the AZs in a given region.
Database
~~~~~~~~~~~~~~~~
Designate needs a SQLAlchemy supported :ref:`database` engine for the persistent storage of data, the recommended driver is MySQL.
In a Multi-AZ environment, a MySQL Galera Cluster, built using Percona's MySQL packages has performed well.
DNS Backend
~~~~~~~~~~~
Designate supports multiple backend implementations, PowerDNS, BIND, MySQL BIND and dnsmasq, you are also free to implement your own backend to fit your needs, as well as extensions to provide extra functionality to complement existing backends.
There are various ways to provide a highly available authorative DNS service, here are some suggestions:
* Multiple PowerDNS instances using the same database being maintained by :ref:`designate-central`, optionally using MySQL Replication to propagate the data to multiple locations.
* DNS AXFR (Zone Transfer) multiple slave DNS server get notified of zone updates from a DNS server being managed by :ref:`designate-central`.