From 7710b1670e8bf2d4459e7fe816a64f8b63a51283 Mon Sep 17 00:00:00 2001 From: vmahe Date: Wed, 25 Nov 2015 16:57:50 +0100 Subject: [PATCH] Provide detailed information on architecture Added new global architecture diagram using Dia tool and a more detailed description of each component. Exported this diagram in SVG format. The source code of the diagram is stored in /doc/images_src/ folder. Moved also the architecture.rst file from the /dev folder to the root folder of the Watcher doc. Change-Id: I74379390178673dcb6a043967b31e38ca9560bb3 --- doc/source/architecture.rst | 170 ++++++++++++++++ doc/source/dev/architecture.rst | 9 - doc/source/image_src/architecture.dia | Bin 0 -> 3085 bytes doc/source/images/architecture.svg | 275 ++++++++++++++++++++++++++ doc/source/index.rst | 2 +- 5 files changed, 446 insertions(+), 10 deletions(-) create mode 100644 doc/source/architecture.rst delete mode 100644 doc/source/dev/architecture.rst create mode 100644 doc/source/image_src/architecture.dia create mode 100644 doc/source/images/architecture.svg diff --git a/doc/source/architecture.rst b/doc/source/architecture.rst new file mode 100644 index 000000000..a7c8cef39 --- /dev/null +++ b/doc/source/architecture.rst @@ -0,0 +1,170 @@ +.. _architecture: + +=================== +System Architecture +=================== + + +This page presents the current technical Architecture of the Watcher system. + +.. _architecture_overview: + +Overview +======== + +Below you will find a diagram, showing the main components of Watcher: + +.. image:: ./images/architecture.svg + :width: 100% + + +.. _components_definition: + +Components +========== + +.. _amqp_bus_definition: + +AMQP Bus +-------- + +The AMQP message bus handles asynchronous communications between the different +Watcher components. + +.. _cluster_history_db_definition: + +Cluster History Database +------------------------ + +This component stores the data related to the +:ref:`Cluster History `. + +It can potentially rely on any appropriate storage system (InfluxDB, OpenTSDB, +MongoDB,...) but will probably be more performant when using +`Time Series Databases `_ +which are optimized for handling time series data, which are arrays of numbers +indexed by time (a datetime or a datetime range). + +.. _watcher_api_definition: + +Watcher API +----------- + +This component implements the REST API provided by the Watcher system to the +external world. + +It enables the :ref:`Administrator ` of a +:ref:`Cluster ` to control and monitor the Watcher system +via any interaction mechanism connected to this API: + +- :ref:`CLI ` +- Horizon plugin +- Python SDK + +You can also read the detailed description of `Watcher API`_. + +.. _watcher_applier_definition: + +Watcher Applier +--------------- + +This component is in charge of executing the :ref:`Action Plan ` +built by the :ref:`Watcher Decision Engine `. + +It connects to the :ref:`message bus ` and launches the +:ref:`Action Plan ` whenever a triggering message is +received on a dedicated AMQP queue. + +The triggering message contains the Action Plan UUID. + +It then gets the detailed information about the +:ref:`Action Plan ` from the +:ref:`Watcher Database ` which contains the list +of :ref:`Actions ` to launch. + +It then loops on each :ref:`Action `, gets the associated +class and calls the execute() method of this class. +Most of the time, this method will first request a token to the Keystone API +and if it is allowed, sends a request to the REST API of the OpenStack service +which handles this kind of :ref:`atomic Action `. + +Note that as soon as :ref:`Watcher Applier ` starts +handling a given :ref:`Action ` from the list, a +notification message is sent on the :ref:`message bus ` +indicating that the state of the action has changed to **ONGOING**. + +If the :ref:`Action ` is successful, the :ref:`Watcher Applier ` +sends a notification message on :ref:`the bus ` informing +the other components of this. + +If the :ref:`Action ` fails, the +:ref:`Watcher Applier ` tries to rollback to the +previous state of the :ref:`Managed resource ` +(i.e. before the command was sent to the underlying OpenStack service). + +.. _watcher_cli_definition: + +Watcher CLI +----------- + +The watcher command-line interface (CLI) can be used to interact with the +Watcher system in order to control it or to know its current status. + +Please, read `the detailed documentation about Watcher CLI `_ + +.. _watcher_database_definition: + +Watcher Database +---------------- + +This database stores all the Watcher domain objects which can be requested +by the :ref:`Watcher API ` or the +:ref:`Watcher CLI `: + +- :ref:`Audit templates ` +- :ref:`Audits ` +- :ref:`Action plans ` +- :ref:`Actions ` +- :ref:`Goals ` + +The Watcher domain being here "*optimization of some resources provided by an +OpenStack system*". + +.. _watcher_decision_engine_definition: + +Watcher Decision Engine +----------------------- + +This component is responsible for computing a set of potential optimization +:ref:`Actions ` in order to fulfill the :ref:`Goal ` +of an :ref:`Audit `. + +It first reads the parameters of the :ref:`Audit ` from the +associated :ref:`Audit Template ` and knows the +:ref:`Goal ` to achieve. + +It then selects the most appropriate :ref:`Strategy ` +depending on how Watcher was configured for this :ref:`Goal `. + +The :ref:`Strategy ` is then dynamically loaded (via +`stevedore `_). The +:ref:`Watcher Decision Engine ` calls the +**execute()** method of the :ref:`Strategy ` class which +generates a set of :ref:`Actions `. + +These :ref:`Actions ` are scheduled in time by the +:ref:`Watcher Planner ` (i.e., it generates an +:ref:`Action Plan `). + +In order to compute the potential :ref:`Solution ` for the +Audit, the :ref:`Strategy ` relies on two sets of data: + +- the current state of the :ref:`Managed resources ` + (e.g., the data stored in the Nova database) +- the data stored in the :ref:`Cluster History Database ` + which provides information about the past of the :ref:`Cluster ` + +So far, only one :ref:`Strategy ` can be associated to a +given :ref:`Goal ` via the main Watcher configuration file. + +.. _Watcher API: webapi/v1.html diff --git a/doc/source/dev/architecture.rst b/doc/source/dev/architecture.rst deleted file mode 100644 index 875deb765..000000000 --- a/doc/source/dev/architecture.rst +++ /dev/null @@ -1,9 +0,0 @@ -.. _architecture: - -=================== -System Architecture -=================== - -Please go to `Wiki Watcher Architecture `_ - -.. _API service: ../webapi/v1.html diff --git a/doc/source/image_src/architecture.dia b/doc/source/image_src/architecture.dia new file mode 100644 index 0000000000000000000000000000000000000000..2153355fe8f11df580f543f99036b0eb2c857862 GIT binary patch literal 3085 zcmV+o4D$0IiwFP!000023+-K7Z`(!|e$THE)K_Ud_e)i$$YqNbXuDl>7Yp>opd?!6 zh9(7)vg3W&-@a!^CDw(sC{e@2&IoV<%H#PsB+vQ2b7s!)haW%Q#)G>s%cC^891?_w zgD{z;^C($d4*&JfpT>Il|h=gf&Fg%ZBP_hG+h9!!;5t{7hC;2(f2Y~gwrexJ~VfY;bY7xT`$9Ix7*ugnn%#2 zcv$W=Y3RrPyU}LdDu+hN;`MiL_;=MQt-pBetL{V_L5kZTTSUn|4qThKx)2m;N-10n zVYI=DEAr`@91l0`7cTA>F6$RA%CDAbR%Ag`>_bk|G!BELLM^gYcsfo#3u1W0o*Gt% zQCvqwksdhz>mbhC450e@v$Y*-xr;2CKW(^MEgE{ckLJbA)kjB{tLpv1QSWzA9!=x0 z?(CzaI9vA*=jnd;?v&p9@y|Ox4OMTA^f1?<2*vUAuvkU&FhB5^ZMSLY>1N&R;?Py! zZP)g~RA23obqNs%4`H_M{_P`857sw$?Qu4)LOT6VI4jn}`zcQEXE#As490`E>Br%t z>;Tab%`b<4$J^IucSO(|xW2s$v`jxgMPrSO#%eSs1ZlB5^nJ(CgfgA(f@Cm`AX&uW z8pt5b)<9+=FjB;@5#dOm1{%iU?bR&Jl6`@2=HZn{r90Ul;wM)k43YwGd}If#n=o43 z6wP2%A_)-0eEO)gJsIP%ne|d|RuWtLGy8SPTyk&`z7CD_tt~t#&)& zzCqeQ!G}%aDou(yAp8B`I=GGE2RNS~$%liyc(9HeJaF}=KZo&MSVXg6lV6TMgFU0K zy(b!Pe|R_0ZEmP@pcF7U2S<%07npQ|+qKKaa1!HgK%m1H$!%DFPs3z&TO*;K5>a(J zph?SIJkt21HYjyKtt$9EydPA@bXOJ(f>{yWbr%Vwt4ILyR9KA03K@qA0Z^nx(Gwz( zA%>3xqAtZj=V7%{28a;OIIy%9*l5j!LuEjiTO~l(Zij-XqKs1h?3Dn6AH1v6eErV*4!zhcOPCGPB*~mThTss$mtw$ZZ$n$s$U^Zjgh=o)0s?d@h;g0@08fO$s}p2| znWuXu&0^d(X>T_lP8=brJ0GlA^@&VpQ9)~aw@|(k_&$cr~ZR6x~IAn7chkQ6qBF>a1QJ{3) zCrLCYmI>8>H5xP2M}Fk=aka{h_+*GrhWKPiPsxxsmHQmLSuW!!%zQGWw`2$dPb$EZ z08MGXq^verN%>XhaINUVVv z@cft{Tvi(Z5i6%<4QCi&8>WZux5I6hOwSM(;<3I_}bDBl}O%uN!zn}H^z*|}Xf5Ed(eUV;2++{j*m#pGNj3jf(ToREA zMzF~kBW@2E=p47a)p>C3SW7siDPt;9tcv%j@*$}YNx$WgbQ!M}QR1W0o}yBQ7%PJe zgR$^S?t3D3(s^4Q26xODW~UAqbx7?Nhp%zegh7Z}q5{V|%|D-Ecs?ZcA!*MX_Ir94 z_;|FRc$C?>Q@a+0Y{wZsB6Uh!ows`71Q7A0?6Nv&)-62?^1bz#4>z*HEi z-B=lWnUG{vwOm2n4lxhM;eq^^+Q!zRqWK}3&qb?9{R_8enuS3RVa+MK|3y7jqL zyj$$&lhUq}((l43PH)2^%m!b!D)k$A83&U#!$?O85hJiqM)#GB?p%;c4Wc$cwsoOS zg`=?#2mu@>vE~y_k==T8#1I@g-{7xdlr@to)SjkosHjh%z1##^_u?*6ty~{-#EskP zE`lfro3T(rQmrTOa0(?zHCKX9U3%8qcp8V=yv++meh1qZ7Imo(QJ38QTh&)qrF$6+ zTvPk#11q@Q{}zCxAj~E=2*QMOYvfFDX`ziKuJhzg18p{OjLga9rQXWFRx3B>v+};S zRHdDl#8ZyI!t7qtSYYl_1aqWHAiE@vPZ-=wQZD$T+cBDh{nySg#CGfAB_^^@+Ror?%8<=Xs? zKuSsKa_yx>!dO@&TSves5HJ@uK`OOztmYd>SwH1M%2mYV^es%ip8twcUVlZ4zkC*A}wdyzy9>T0V z{Cu>&NS`YBAgH^G@@Q(&%gDV=a_+Zak=Uqj@IJ~v)G3e7Yu4?QF)*l0A7Q6|0YXo> zLP}P$_%yiNum}!C|MO^8I4vmdbivxC`?xgpv6h$-hv`1~xw^O83m3{I^?fD#N-!J9 zubk!juA?|^1@X1-ch#oWVPD~HEVOf$0^E7^iz9G>j7j-4busKdk=R-z00 zK5ON6Z%}~Y6^vjLjuEyA6vB{Z&sULkzjSR>nX+pvtG7{V!W84itFo_GmD#lbUX^_# zs_b37%571?&rx2a*@IVSeO4$?W=3|O5v{#A^Wv;8an^cju5*bXU+;CG5aGt@`@wmY z9UBvjKwX%L-HVEq^K)7fEu7BJHIA~mDV!!fu>X(^-sF*OF;b`R3D{~rk32D-=jZxf b)U5tia=to%pG6kjzJB$8#d~z<+Nl5lb$9(q literal 0 HcmV?d00001 diff --git a/doc/source/images/architecture.svg b/doc/source/images/architecture.svg new file mode 100644 index 000000000..48b9b2c03 --- /dev/null +++ b/doc/source/images/architecture.svg @@ -0,0 +1,275 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + AMQP + bus + + + + + + + + + + + + + + + + + + + Watcher + Decision Engine + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Watcher + Actions Applier + + + + + + + + + + + + + + + + + + + Watcher + API + + + + + + + + + + + + + + + + + + + Watcher + CLI + + + + + + + Horizon + + + + + + + + + + + + + + + + + + + Watcher + plugin + + + + + + + Nova + + + + + + + MariaDB Database + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Ceilometer API + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Strategy + + + + + + + + + + + + + + + + + + + + + + + Watcher DB + + + + + + + + Cluster History DB + + + + + + + + diff --git a/doc/source/index.rst b/doc/source/index.rst index 2c4baf728..115d6cbd2 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -22,7 +22,7 @@ Introduction :maxdepth: 1 dev/glossary - dev/architecture + architecture dev/environment dev/contributing dev/plugins