diff --git a/.gitignore b/.gitignore deleted file mode 100644 index a8b42eb..0000000 --- a/.gitignore +++ /dev/null @@ -1 +0,0 @@ -*.retry diff --git a/LICENSE b/LICENSE deleted file mode 100644 index 20d40b6..0000000 --- a/LICENSE +++ /dev/null @@ -1,674 +0,0 @@ - GNU GENERAL PUBLIC LICENSE - Version 3, 29 June 2007 - - Copyright (C) 2007 Free Software Foundation, Inc. - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - Preamble - - The GNU General Public License is a free, copyleft license for -software and other kinds of works. - - The licenses for most software and other practical works are designed -to take away your freedom to share and change the works. By contrast, -the GNU General Public License is intended to guarantee your freedom to -share and change all versions of a program--to make sure it remains free -software for all its users. We, the Free Software Foundation, use the -GNU General Public License for most of our software; it applies also to -any other work released this way by its authors. You can apply it to -your programs, too. - - When we speak of free software, we are referring to freedom, not -price. Our General Public Licenses are designed to make sure that you -have the freedom to distribute copies of free software (and charge for -them if you wish), that you receive source code or can get it if you -want it, that you can change the software or use pieces of it in new -free programs, and that you know you can do these things. - - To protect your rights, we need to prevent others from denying you -these rights or asking you to surrender the rights. Therefore, you have -certain responsibilities if you distribute copies of the software, or if -you modify it: responsibilities to respect the freedom of others. - - For example, if you distribute copies of such a program, whether -gratis or for a fee, you must pass on to the recipients the same -freedoms that you received. You must make sure that they, too, receive -or can get the source code. And you must show them these terms so they -know their rights. - - Developers that use the GNU GPL protect your rights with two steps: -(1) assert copyright on the software, and (2) offer you this License -giving you legal permission to copy, distribute and/or modify it. - - For the developers' and authors' protection, the GPL clearly explains -that there is no warranty for this free software. For both users' and -authors' sake, the GPL requires that modified versions be marked as -changed, so that their problems will not be attributed erroneously to -authors of previous versions. - - Some devices are designed to deny users access to install or run -modified versions of the software inside them, although the manufacturer -can do so. This is fundamentally incompatible with the aim of -protecting users' freedom to change the software. The systematic -pattern of such abuse occurs in the area of products for individuals to -use, which is precisely where it is most unacceptable. Therefore, we -have designed this version of the GPL to prohibit the practice for those -products. If such problems arise substantially in other domains, we -stand ready to extend this provision to those domains in future versions -of the GPL, as needed to protect the freedom of users. - - Finally, every program is threatened constantly by software patents. -States should not allow patents to restrict development and use of -software on general-purpose computers, but in those that do, we wish to -avoid the special danger that patents applied to a free program could -make it effectively proprietary. To prevent this, the GPL assures that -patents cannot be used to render the program non-free. - - The precise terms and conditions for copying, distribution and -modification follow. - - TERMS AND CONDITIONS - - 0. Definitions. - - "This License" refers to version 3 of the GNU General Public License. - - "Copyright" also means copyright-like laws that apply to other kinds of -works, such as semiconductor masks. - - "The Program" refers to any copyrightable work licensed under this -License. Each licensee is addressed as "you". "Licensees" and -"recipients" may be individuals or organizations. - - To "modify" a work means to copy from or adapt all or part of the work -in a fashion requiring copyright permission, other than the making of an -exact copy. The resulting work is called a "modified version" of the -earlier work or a work "based on" the earlier work. - - A "covered work" means either the unmodified Program or a work based -on the Program. - - To "propagate" a work means to do anything with it that, without -permission, would make you directly or secondarily liable for -infringement under applicable copyright law, except executing it on a -computer or modifying a private copy. Propagation includes copying, -distribution (with or without modification), making available to the -public, and in some countries other activities as well. - - To "convey" a work means any kind of propagation that enables other -parties to make or receive copies. Mere interaction with a user through -a computer network, with no transfer of a copy, is not conveying. - - An interactive user interface displays "Appropriate Legal Notices" -to the extent that it includes a convenient and prominently visible -feature that (1) displays an appropriate copyright notice, and (2) -tells the user that there is no warranty for the work (except to the -extent that warranties are provided), that licensees may convey the -work under this License, and how to view a copy of this License. If -the interface presents a list of user commands or options, such as a -menu, a prominent item in the list meets this criterion. - - 1. Source Code. - - The "source code" for a work means the preferred form of the work -for making modifications to it. "Object code" means any non-source -form of a work. - - A "Standard Interface" means an interface that either is an official -standard defined by a recognized standards body, or, in the case of -interfaces specified for a particular programming language, one that -is widely used among developers working in that language. - - The "System Libraries" of an executable work include anything, other -than the work as a whole, that (a) is included in the normal form of -packaging a Major Component, but which is not part of that Major -Component, and (b) serves only to enable use of the work with that -Major Component, or to implement a Standard Interface for which an -implementation is available to the public in source code form. A -"Major Component", in this context, means a major essential component -(kernel, window system, and so on) of the specific operating system -(if any) on which the executable work runs, or a compiler used to -produce the work, or an object code interpreter used to run it. - - The "Corresponding Source" for a work in object code form means all -the source code needed to generate, install, and (for an executable -work) run the object code and to modify the work, including scripts to -control those activities. However, it does not include the work's -System Libraries, or general-purpose tools or generally available free -programs which are used unmodified in performing those activities but -which are not part of the work. For example, Corresponding Source -includes interface definition files associated with source files for -the work, and the source code for shared libraries and dynamically -linked subprograms that the work is specifically designed to require, -such as by intimate data communication or control flow between those -subprograms and other parts of the work. - - The Corresponding Source need not include anything that users -can regenerate automatically from other parts of the Corresponding -Source. - - The Corresponding Source for a work in source code form is that -same work. - - 2. Basic Permissions. - - All rights granted under this License are granted for the term of -copyright on the Program, and are irrevocable provided the stated -conditions are met. This License explicitly affirms your unlimited -permission to run the unmodified Program. The output from running a -covered work is covered by this License only if the output, given its -content, constitutes a covered work. This License acknowledges your -rights of fair use or other equivalent, as provided by copyright law. - - You may make, run and propagate covered works that you do not -convey, without conditions so long as your license otherwise remains -in force. You may convey covered works to others for the sole purpose -of having them make modifications exclusively for you, or provide you -with facilities for running those works, provided that you comply with -the terms of this License in conveying all material for which you do -not control copyright. Those thus making or running the covered works -for you must do so exclusively on your behalf, under your direction -and control, on terms that prohibit them from making any copies of -your copyrighted material outside their relationship with you. - - Conveying under any other circumstances is permitted solely under -the conditions stated below. Sublicensing is not allowed; section 10 -makes it unnecessary. - - 3. Protecting Users' Legal Rights From Anti-Circumvention Law. - - No covered work shall be deemed part of an effective technological -measure under any applicable law fulfilling obligations under article -11 of the WIPO copyright treaty adopted on 20 December 1996, or -similar laws prohibiting or restricting circumvention of such -measures. - - When you convey a covered work, you waive any legal power to forbid -circumvention of technological measures to the extent such circumvention -is effected by exercising rights under this License with respect to -the covered work, and you disclaim any intention to limit operation or -modification of the work as a means of enforcing, against the work's -users, your or third parties' legal rights to forbid circumvention of -technological measures. - - 4. Conveying Verbatim Copies. - - You may convey verbatim copies of the Program's source code as you -receive it, in any medium, provided that you conspicuously and -appropriately publish on each copy an appropriate copyright notice; -keep intact all notices stating that this License and any -non-permissive terms added in accord with section 7 apply to the code; -keep intact all notices of the absence of any warranty; and give all -recipients a copy of this License along with the Program. - - You may charge any price or no price for each copy that you convey, -and you may offer support or warranty protection for a fee. - - 5. Conveying Modified Source Versions. - - You may convey a work based on the Program, or the modifications to -produce it from the Program, in the form of source code under the -terms of section 4, provided that you also meet all of these conditions: - - a) The work must carry prominent notices stating that you modified - it, and giving a relevant date. - - b) The work must carry prominent notices stating that it is - released under this License and any conditions added under section - 7. This requirement modifies the requirement in section 4 to - "keep intact all notices". - - c) You must license the entire work, as a whole, under this - License to anyone who comes into possession of a copy. This - License will therefore apply, along with any applicable section 7 - additional terms, to the whole of the work, and all its parts, - regardless of how they are packaged. This License gives no - permission to license the work in any other way, but it does not - invalidate such permission if you have separately received it. - - d) If the work has interactive user interfaces, each must display - Appropriate Legal Notices; however, if the Program has interactive - interfaces that do not display Appropriate Legal Notices, your - work need not make them do so. - - A compilation of a covered work with other separate and independent -works, which are not by their nature extensions of the covered work, -and which are not combined with it such as to form a larger program, -in or on a volume of a storage or distribution medium, is called an -"aggregate" if the compilation and its resulting copyright are not -used to limit the access or legal rights of the compilation's users -beyond what the individual works permit. Inclusion of a covered work -in an aggregate does not cause this License to apply to the other -parts of the aggregate. - - 6. Conveying Non-Source Forms. - - You may convey a covered work in object code form under the terms -of sections 4 and 5, provided that you also convey the -machine-readable Corresponding Source under the terms of this License, -in one of these ways: - - a) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by the - Corresponding Source fixed on a durable physical medium - customarily used for software interchange. - - b) Convey the object code in, or embodied in, a physical product - (including a physical distribution medium), accompanied by a - written offer, valid for at least three years and valid for as - long as you offer spare parts or customer support for that product - model, to give anyone who possesses the object code either (1) a - copy of the Corresponding Source for all the software in the - product that is covered by this License, on a durable physical - medium customarily used for software interchange, for a price no - more than your reasonable cost of physically performing this - conveying of source, or (2) access to copy the - Corresponding Source from a network server at no charge. - - c) Convey individual copies of the object code with a copy of the - written offer to provide the Corresponding Source. This - alternative is allowed only occasionally and noncommercially, and - only if you received the object code with such an offer, in accord - with subsection 6b. - - d) Convey the object code by offering access from a designated - place (gratis or for a charge), and offer equivalent access to the - Corresponding Source in the same way through the same place at no - further charge. You need not require recipients to copy the - Corresponding Source along with the object code. If the place to - copy the object code is a network server, the Corresponding Source - may be on a different server (operated by you or a third party) - that supports equivalent copying facilities, provided you maintain - clear directions next to the object code saying where to find the - Corresponding Source. Regardless of what server hosts the - Corresponding Source, you remain obligated to ensure that it is - available for as long as needed to satisfy these requirements. - - e) Convey the object code using peer-to-peer transmission, provided - you inform other peers where the object code and Corresponding - Source of the work are being offered to the general public at no - charge under subsection 6d. - - A separable portion of the object code, whose source code is excluded -from the Corresponding Source as a System Library, need not be -included in conveying the object code work. - - A "User Product" is either (1) a "consumer product", which means any -tangible personal property which is normally used for personal, family, -or household purposes, or (2) anything designed or sold for incorporation -into a dwelling. In determining whether a product is a consumer product, -doubtful cases shall be resolved in favor of coverage. For a particular -product received by a particular user, "normally used" refers to a -typical or common use of that class of product, regardless of the status -of the particular user or of the way in which the particular user -actually uses, or expects or is expected to use, the product. A product -is a consumer product regardless of whether the product has substantial -commercial, industrial or non-consumer uses, unless such uses represent -the only significant mode of use of the product. - - "Installation Information" for a User Product means any methods, -procedures, authorization keys, or other information required to install -and execute modified versions of a covered work in that User Product from -a modified version of its Corresponding Source. The information must -suffice to ensure that the continued functioning of the modified object -code is in no case prevented or interfered with solely because -modification has been made. - - If you convey an object code work under this section in, or with, or -specifically for use in, a User Product, and the conveying occurs as -part of a transaction in which the right of possession and use of the -User Product is transferred to the recipient in perpetuity or for a -fixed term (regardless of how the transaction is characterized), the -Corresponding Source conveyed under this section must be accompanied -by the Installation Information. But this requirement does not apply -if neither you nor any third party retains the ability to install -modified object code on the User Product (for example, the work has -been installed in ROM). - - The requirement to provide Installation Information does not include a -requirement to continue to provide support service, warranty, or updates -for a work that has been modified or installed by the recipient, or for -the User Product in which it has been modified or installed. Access to a -network may be denied when the modification itself materially and -adversely affects the operation of the network or violates the rules and -protocols for communication across the network. - - Corresponding Source conveyed, and Installation Information provided, -in accord with this section must be in a format that is publicly -documented (and with an implementation available to the public in -source code form), and must require no special password or key for -unpacking, reading or copying. - - 7. Additional Terms. - - "Additional permissions" are terms that supplement the terms of this -License by making exceptions from one or more of its conditions. -Additional permissions that are applicable to the entire Program shall -be treated as though they were included in this License, to the extent -that they are valid under applicable law. If additional permissions -apply only to part of the Program, that part may be used separately -under those permissions, but the entire Program remains governed by -this License without regard to the additional permissions. - - When you convey a copy of a covered work, you may at your option -remove any additional permissions from that copy, or from any part of -it. (Additional permissions may be written to require their own -removal in certain cases when you modify the work.) You may place -additional permissions on material, added by you to a covered work, -for which you have or can give appropriate copyright permission. - - Notwithstanding any other provision of this License, for material you -add to a covered work, you may (if authorized by the copyright holders of -that material) supplement the terms of this License with terms: - - a) Disclaiming warranty or limiting liability differently from the - terms of sections 15 and 16 of this License; or - - b) Requiring preservation of specified reasonable legal notices or - author attributions in that material or in the Appropriate Legal - Notices displayed by works containing it; or - - c) Prohibiting misrepresentation of the origin of that material, or - requiring that modified versions of such material be marked in - reasonable ways as different from the original version; or - - d) Limiting the use for publicity purposes of names of licensors or - authors of the material; or - - e) Declining to grant rights under trademark law for use of some - trade names, trademarks, or service marks; or - - f) Requiring indemnification of licensors and authors of that - material by anyone who conveys the material (or modified versions of - it) with contractual assumptions of liability to the recipient, for - any liability that these contractual assumptions directly impose on - those licensors and authors. - - All other non-permissive additional terms are considered "further -restrictions" within the meaning of section 10. If the Program as you -received it, or any part of it, contains a notice stating that it is -governed by this License along with a term that is a further -restriction, you may remove that term. If a license document contains -a further restriction but permits relicensing or conveying under this -License, you may add to a covered work material governed by the terms -of that license document, provided that the further restriction does -not survive such relicensing or conveying. - - If you add terms to a covered work in accord with this section, you -must place, in the relevant source files, a statement of the -additional terms that apply to those files, or a notice indicating -where to find the applicable terms. - - Additional terms, permissive or non-permissive, may be stated in the -form of a separately written license, or stated as exceptions; -the above requirements apply either way. - - 8. Termination. - - You may not propagate or modify a covered work except as expressly -provided under this License. Any attempt otherwise to propagate or -modify it is void, and will automatically terminate your rights under -this License (including any patent licenses granted under the third -paragraph of section 11). - - However, if you cease all violation of this License, then your -license from a particular copyright holder is reinstated (a) -provisionally, unless and until the copyright holder explicitly and -finally terminates your license, and (b) permanently, if the copyright -holder fails to notify you of the violation by some reasonable means -prior to 60 days after the cessation. - - Moreover, your license from a particular copyright holder is -reinstated permanently if the copyright holder notifies you of the -violation by some reasonable means, this is the first time you have -received notice of violation of this License (for any work) from that -copyright holder, and you cure the violation prior to 30 days after -your receipt of the notice. - - Termination of your rights under this section does not terminate the -licenses of parties who have received copies or rights from you under -this License. If your rights have been terminated and not permanently -reinstated, you do not qualify to receive new licenses for the same -material under section 10. - - 9. Acceptance Not Required for Having Copies. - - You are not required to accept this License in order to receive or -run a copy of the Program. Ancillary propagation of a covered work -occurring solely as a consequence of using peer-to-peer transmission -to receive a copy likewise does not require acceptance. However, -nothing other than this License grants you permission to propagate or -modify any covered work. These actions infringe copyright if you do -not accept this License. Therefore, by modifying or propagating a -covered work, you indicate your acceptance of this License to do so. - - 10. Automatic Licensing of Downstream Recipients. - - Each time you convey a covered work, the recipient automatically -receives a license from the original licensors, to run, modify and -propagate that work, subject to this License. You are not responsible -for enforcing compliance by third parties with this License. - - An "entity transaction" is a transaction transferring control of an -organization, or substantially all assets of one, or subdividing an -organization, or merging organizations. If propagation of a covered -work results from an entity transaction, each party to that -transaction who receives a copy of the work also receives whatever -licenses to the work the party's predecessor in interest had or could -give under the previous paragraph, plus a right to possession of the -Corresponding Source of the work from the predecessor in interest, if -the predecessor has it or can get it with reasonable efforts. - - You may not impose any further restrictions on the exercise of the -rights granted or affirmed under this License. For example, you may -not impose a license fee, royalty, or other charge for exercise of -rights granted under this License, and you may not initiate litigation -(including a cross-claim or counterclaim in a lawsuit) alleging that -any patent claim is infringed by making, using, selling, offering for -sale, or importing the Program or any portion of it. - - 11. Patents. - - A "contributor" is a copyright holder who authorizes use under this -License of the Program or a work on which the Program is based. The -work thus licensed is called the contributor's "contributor version". - - A contributor's "essential patent claims" are all patent claims -owned or controlled by the contributor, whether already acquired or -hereafter acquired, that would be infringed by some manner, permitted -by this License, of making, using, or selling its contributor version, -but do not include claims that would be infringed only as a -consequence of further modification of the contributor version. For -purposes of this definition, "control" includes the right to grant -patent sublicenses in a manner consistent with the requirements of -this License. - - Each contributor grants you a non-exclusive, worldwide, royalty-free -patent license under the contributor's essential patent claims, to -make, use, sell, offer for sale, import and otherwise run, modify and -propagate the contents of its contributor version. - - In the following three paragraphs, a "patent license" is any express -agreement or commitment, however denominated, not to enforce a patent -(such as an express permission to practice a patent or covenant not to -sue for patent infringement). To "grant" such a patent license to a -party means to make such an agreement or commitment not to enforce a -patent against the party. - - If you convey a covered work, knowingly relying on a patent license, -and the Corresponding Source of the work is not available for anyone -to copy, free of charge and under the terms of this License, through a -publicly available network server or other readily accessible means, -then you must either (1) cause the Corresponding Source to be so -available, or (2) arrange to deprive yourself of the benefit of the -patent license for this particular work, or (3) arrange, in a manner -consistent with the requirements of this License, to extend the patent -license to downstream recipients. "Knowingly relying" means you have -actual knowledge that, but for the patent license, your conveying the -covered work in a country, or your recipient's use of the covered work -in a country, would infringe one or more identifiable patents in that -country that you have reason to believe are valid. - - If, pursuant to or in connection with a single transaction or -arrangement, you convey, or propagate by procuring conveyance of, a -covered work, and grant a patent license to some of the parties -receiving the covered work authorizing them to use, propagate, modify -or convey a specific copy of the covered work, then the patent license -you grant is automatically extended to all recipients of the covered -work and works based on it. - - A patent license is "discriminatory" if it does not include within -the scope of its coverage, prohibits the exercise of, or is -conditioned on the non-exercise of one or more of the rights that are -specifically granted under this License. You may not convey a covered -work if you are a party to an arrangement with a third party that is -in the business of distributing software, under which you make payment -to the third party based on the extent of your activity of conveying -the work, and under which the third party grants, to any of the -parties who would receive the covered work from you, a discriminatory -patent license (a) in connection with copies of the covered work -conveyed by you (or copies made from those copies), or (b) primarily -for and in connection with specific products or compilations that -contain the covered work, unless you entered into that arrangement, -or that patent license was granted, prior to 28 March 2007. - - Nothing in this License shall be construed as excluding or limiting -any implied license or other defenses to infringement that may -otherwise be available to you under applicable patent law. - - 12. No Surrender of Others' Freedom. - - If conditions are imposed on you (whether by court order, agreement or -otherwise) that contradict the conditions of this License, they do not -excuse you from the conditions of this License. If you cannot convey a -covered work so as to satisfy simultaneously your obligations under this -License and any other pertinent obligations, then as a consequence you may -not convey it at all. For example, if you agree to terms that obligate you -to collect a royalty for further conveying from those to whom you convey -the Program, the only way you could satisfy both those terms and this -License would be to refrain entirely from conveying the Program. - - 13. Use with the GNU Affero General Public License. - - Notwithstanding any other provision of this License, you have -permission to link or combine any covered work with a work licensed -under version 3 of the GNU Affero General Public License into a single -combined work, and to convey the resulting work. The terms of this -License will continue to apply to the part which is the covered work, -but the special requirements of the GNU Affero General Public License, -section 13, concerning interaction through a network will apply to the -combination as such. - - 14. Revised Versions of this License. - - The Free Software Foundation may publish revised and/or new versions of -the GNU General Public License from time to time. Such new versions will -be similar in spirit to the present version, but may differ in detail to -address new problems or concerns. - - Each version is given a distinguishing version number. If the -Program specifies that a certain numbered version of the GNU General -Public License "or any later version" applies to it, you have the -option of following the terms and conditions either of that numbered -version or of any later version published by the Free Software -Foundation. If the Program does not specify a version number of the -GNU General Public License, you may choose any version ever published -by the Free Software Foundation. - - If the Program specifies that a proxy can decide which future -versions of the GNU General Public License can be used, that proxy's -public statement of acceptance of a version permanently authorizes you -to choose that version for the Program. - - Later license versions may give you additional or different -permissions. However, no additional obligations are imposed on any -author or copyright holder as a result of your choosing to follow a -later version. - - 15. Disclaimer of Warranty. - - THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY -APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT -HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY -OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, -THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR -PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM -IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF -ALL NECESSARY SERVICING, REPAIR OR CORRECTION. - - 16. Limitation of Liability. - - IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING -WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS -THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY -GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE -USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF -DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD -PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), -EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF -SUCH DAMAGES. - - 17. Interpretation of Sections 15 and 16. - - If the disclaimer of warranty and limitation of liability provided -above cannot be given local legal effect according to their terms, -reviewing courts shall apply local law that most closely approximates -an absolute waiver of all civil liability in connection with the -Program, unless a warranty or assumption of liability accompanies a -copy of the Program in return for a fee. - - END OF TERMS AND CONDITIONS - - How to Apply These Terms to Your New Programs - - If you develop a new program, and you want it to be of the greatest -possible use to the public, the best way to achieve this is to make it -free software which everyone can redistribute and change under these terms. - - To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -state the exclusion of warranty; and each file should have at least -the "copyright" line and a pointer to where the full notice is found. - - - Copyright (C) - - This program is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program. If not, see . - -Also add information on how to contact you by electronic and paper mail. - - If the program does terminal interaction, make it output a short -notice like this when it starts in an interactive mode: - - Copyright (C) - This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - -The hypothetical commands `show w' and `show c' should show the appropriate -parts of the General Public License. Of course, your program's commands -might be different; for a GUI interface, you would use an "about box". - - You should also get your employer (if you work as a programmer) or school, -if any, to sign a "copyright disclaimer" for the program, if necessary. -For more information on this, and how to apply and follow the GNU GPL, see -. - - The GNU General Public License does not permit incorporating your program -into proprietary programs. If your program is a subroutine library, you -may consider it more useful to permit linking proprietary applications with -the library. If this is what you want to do, use the GNU Lesser General -Public License instead of this License. But first, please read -. \ No newline at end of file diff --git a/README.rst b/README.rst index a0fa75e..9fc0f22 100644 --- a/README.rst +++ b/README.rst @@ -1,41 +1,4 @@ -ara-infra -========= +This project has been moved +--------------------------- -``ara-infra`` is a collection of playbooks, roles, tests, scripts, tools and -documentation helpful in the context of managing the CI/CD of ARA itself. - -It is meant to do things like: -- Deploy the infrastructure server, ``infra.recordsansible.org`` -- Deploy the website `ara.recordsansible.org `_ -- Deploy `teamchat `_ for bridging ARA's Slack and IRC communities -- Carry common integration tests between the different ARA projects - -If you're looking to use ARA Records Ansible for reporting on your playbooks, -the ARA documentation is available on `ara.readthedocs.io `_. - -Contributors -============ - -See contributors on GitHub_. - -.. _GitHub: https://github.com/ansible-community/ara-infra/graphs/contributors - -Copyright -========= - -:: - - Copyright (c) 2018 Red Hat, Inc. - - ARA is free software: you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation, either version 3 of the License, or - (at your option) any later version. - - ARA is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with ARA. If not, see . +This project's code and code review is now on GitHub: https://github.com/ansible-community/ara-infra diff --git a/playbooks/ansible.cfg b/playbooks/ansible.cfg deleted file mode 100644 index 33d5108..0000000 --- a/playbooks/ansible.cfg +++ /dev/null @@ -1,10 +0,0 @@ -[defaults] -forks = 25 -gathering = smart -fact_caching = jsonfile -fact_caching_connection = /tmp/ -fact_caching_timeout = 3600 -inventory = hosts - -[ssh_connection] -pipelining = True diff --git a/playbooks/hosts b/playbooks/hosts deleted file mode 100644 index cf3ffca..0000000 --- a/playbooks/hosts +++ /dev/null @@ -1 +0,0 @@ -demo.recordsansible.org ansible_host=139.178.83.37 ansible_user=fedora ansible_python_interpreter=/usr/bin/python3 diff --git a/playbooks/live-demo.yaml b/playbooks/live-demo.yaml deleted file mode 100644 index 99606ee..0000000 --- a/playbooks/live-demo.yaml +++ /dev/null @@ -1,26 +0,0 @@ -- name: Provision demo.recordsansible.org - hosts: demo.recordsansible.org - gather_facts: yes - vars: - # ara_api - ara_api_write_login_required: true - ara_api_fqdn: api.demo.recordsansible.org - ara_api_frontend_server: nginx - ara_api_frontend_vhost: api.demo.recordsansible.org.conf.j2 - ara_api_wsgi_server: gunicorn - ara_api_allowed_hosts: - - api.demo.recordsansible.org - # TODO: Figure out if it's possible to set ara-web to use demo from localhost:3000 - ara_api_cors_origin_allow_all: true - ara_api_cors_origin_whitelist: - - https://web.demo.recordsansible.org - - http://logs.openstack.org - - http://localhost:3000 - # ara_web - ara_web_fqdn: web.demo.recordsansible.org - ara_web_api_endpoint: "https://api.demo.recordsansible.org" - ara_web_frontend_server: nginx - ara_web_frontend_vhost: web.demo.recordsansible.org.conf.j2 - roles: - - ara_api - - ara_web diff --git a/playbooks/roles b/playbooks/roles deleted file mode 120000 index d8c4472..0000000 --- a/playbooks/roles +++ /dev/null @@ -1 +0,0 @@ -../roles \ No newline at end of file diff --git a/playbooks/templates/api.demo.recordsansible.org.conf.j2 b/playbooks/templates/api.demo.recordsansible.org.conf.j2 deleted file mode 100644 index d2078b6..0000000 --- a/playbooks/templates/api.demo.recordsansible.org.conf.j2 +++ /dev/null @@ -1,46 +0,0 @@ -upstream ara_api { - # fail_timeout=0 means we always retry an upstream even if it failed - # to return a good HTTP response - server {{ ara_api_wsgi_bind }} fail_timeout=0; -} - -server { - listen 80; - server_name {{ ara_api_fqdn }}; - return 301 https://{{ ara_api_fqdn }}$request_uri; -} - -server { - listen 443; - server_name {{ ara_api_fqdn }}; - access_log /var/log/nginx/{{ ara_api_fqdn }}_access.log; - error_log /var/log/nginx/{{ ara_api_fqdn }}_error.log; - - ssl on; - ssl_certificate /etc/letsencrypt/live/{{ ara_api_fqdn }}/fullchain.pem; - ssl_certificate_key /etc/letsencrypt/live/{{ ara_api_fqdn }}/privkey.pem; - ssl_session_cache shared:SSL:10m; - ssl_session_timeout 10m; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_ciphers HIGH:!aNULL:!MD5; - - location /static { - expires 7d; - add_header Cache-Control "public"; - } - - # Everything, including static files, is served by the backend - location ~ { - # checks if the file exists, if not found proxy to app - try_files $uri @proxy_to_app; - } - - location @proxy_to_app { - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - proxy_set_header X-Forwarded-Proto $scheme; - proxy_set_header Host $http_host; - - proxy_redirect off; - proxy_pass http://ara_api; - } -} diff --git a/playbooks/templates/web.demo.recordsansible.org.conf.j2 b/playbooks/templates/web.demo.recordsansible.org.conf.j2 deleted file mode 100644 index 05fa2ee..0000000 --- a/playbooks/templates/web.demo.recordsansible.org.conf.j2 +++ /dev/null @@ -1,46 +0,0 @@ -{% if ara_web_dev_server %} -upstream ara_web { - # fail_timeout=0 means we always retry an upstream even if it failed - # to return a good HTTP response - server {{ ara_web_dev_server_bind_address }}:{{ ara_web_dev_server_bind_port }} fail_timeout=0; -} -{% endif %} - -server { - listen 80; - server_name {{ ara_web_fqdn }}; - return 301 https://{{ ara_web_fqdn }}$request_uri; -} - -server { - listen 443; - server_name {{ ara_web_fqdn }}; - root {{ ara_web_static_dir }}; - access_log /var/log/nginx/{{ ara_web_fqdn }}_access.log; - error_log /var/log/nginx/{{ ara_web_fqdn }}_error.log; - - ssl on; - ssl_certificate /etc/letsencrypt/live/{{ ara_web_fqdn }}/fullchain.pem; - ssl_certificate_key /etc/letsencrypt/live/{{ ara_web_fqdn }}/privkey.pem; - ssl_session_cache shared:SSL:10m; - ssl_session_timeout 10m; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_ciphers HIGH:!aNULL:!MD5; - - {% if ara_web_dev_server %} - location ~ { - # checks for static file, if not found proxy to server - try_files $uri @proxy_to_app; - } - - location @proxy_to_app { - # Redefine the header fields that NGINX sends to the upstream server - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - - # Define the location of the proxy server to send the request to - proxy_pass http://ara_web; - } - {% endif %} -} diff --git a/playbooks/website.yaml b/playbooks/website.yaml deleted file mode 100644 index 42c9627..0000000 --- a/playbooks/website.yaml +++ /dev/null @@ -1,7 +0,0 @@ -- name: Set up the ara.recordsansible.org website - hosts: ara.recordsansible.org - gather_facts: yes - tasks: - - name: Include the website role - include_role: - name: website diff --git a/roles/website/defaults/main.yaml b/roles/website/defaults/main.yaml deleted file mode 100644 index 33ede98..0000000 --- a/roles/website/defaults/main.yaml +++ /dev/null @@ -1,22 +0,0 @@ ---- -# Copyright (c) 2018 Red Hat, Inc. -# -# This file is part of ARA Records Ansible. -# -# ARA is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# ARA is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with ARA. If not, see . - -hugo_release: https://github.com/gohugoio/hugo/releases/download/v0.51/hugo_0.51_Linux-64bit.tar.gz -hugo_directory: /opt/hugo -hugo_theme: https://github.com/jpescador/hugo-future-imperfect -hugo_theme_directory: "{{ hugo_directory }}/themes/hugo-future-imperfect" diff --git a/roles/website/files/ara.recordsansible.org.conf b/roles/website/files/ara.recordsansible.org.conf deleted file mode 100644 index 2f092a1..0000000 --- a/roles/website/files/ara.recordsansible.org.conf +++ /dev/null @@ -1,52 +0,0 @@ -upstream hugo { - server 127.0.0.1:1313; -} - -server { - listen 80; - server_name www.getara.org getara.org www.recordsansible.org ara.recordsansible.org; - return 301 https://ara.recordsansible.org$request_uri; -} - -server { - listen 443; - server_name ara.recordsansible.org; - - ssl on; - ssl_certificate /etc/letsencrypt/live/ara.recordsansible.org/fullchain.pem; - ssl_certificate_key /etc/letsencrypt/live/ara.recordsansible.org/privkey.pem; - ssl_session_cache shared:SSL:10m; - ssl_session_timeout 10m; - ssl_protocols TLSv1 TLSv1.1 TLSv1.2; - ssl_ciphers HIGH:!aNULL:!MD5; - - access_log /var/log/nginx/ara.recordsansible.org_access.log; - error_log /var/log/nginx/ara.recordsansible.org_error.log; - - # Media: images, icons, video, audio, HTC - location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|mp4|ogg|ogv|webm|htc)$ { - access_log off; - add_header Cache-Control "max-age=2592000"; - } - - # CSS and Javascript - location ~* \.(?:css|js)$ { - add_header Cache-Control "max-age=2592000"; - access_log off; - } - - location ^~ { - # checks for static file, if not found proxy to server - try_files $uri @proxy_to_server; - } - - location @proxy_to_server { - # Redefine the header fields that NGINX sends to the upstream server - proxy_set_header Host $host; - proxy_set_header X-Real-IP $remote_addr; - proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; - - # Define the location of the proxy server to send the request to - proxy_pass http://hugo; - } -} diff --git a/roles/website/handlers/main.yaml b/roles/website/handlers/main.yaml deleted file mode 100644 index e3bc54e..0000000 --- a/roles/website/handlers/main.yaml +++ /dev/null @@ -1,23 +0,0 @@ ---- -# Copyright (c) 2018 Red Hat, Inc. -# -# This file is part of ARA Records Ansible. -# -# ARA is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# ARA is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with ARA. If not, see . - -- name: Restart nginx - become: yes - service: - name: nginx - state: restarted diff --git a/roles/website/tasks/hugo.yaml b/roles/website/tasks/hugo.yaml deleted file mode 100644 index f75e8ee..0000000 --- a/roles/website/tasks/hugo.yaml +++ /dev/null @@ -1,81 +0,0 @@ ---- -# Copyright (c) 2018 Red Hat, Inc. -# -# This file is part of ARA Records Ansible. -# -# ARA is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# ARA is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with ARA. If not, see . - -- become: yes - block: - - name: Install git - package: - name: git - state: present - - - name: Create user for Hugo - become: true - user: - name: hugo - comment: User for Hugo - shell: /sbin/nologin - home: "{{ hugo_directory }}" - - - name: Download Hugo release tarball - get_url: - url: "{{ hugo_release }}" - dest: "{{ hugo_directory }}" - register: hugo_download - - - name: Extract Hugo release tarball - unarchive: - src: "{{ hugo_download.dest }}" - dest: "{{ hugo_directory }}" - remote_src: yes - when: hugo_download is changed - - - name: Symlink Hugo in PATH - file: - src: "{{ hugo_directory }}/hugo" - dest: /usr/local/bin/hugo - owner: hugo - group: hugo - state: link - - - name: Clone Hugo theme - become_user: hugo - git: - repo: "{{ hugo_theme }}" - dest: "{{ hugo_theme_directory }}" - force: yes - update: yes - - - name: Configure Hugo server systemd service - template: - src: hugo.service.j2 - dest: /etc/systemd/system/hugo.service - owner: root - group: root - mode: 0644 - register: hugo_systemd - - - when: hugo_systemd is changed - block: - - name: Reload systemctl - systemd: - daemon_reload: yes - - - name: Restart Hugo - service: - name: hugo - state: restarted diff --git a/roles/website/tasks/main.yaml b/roles/website/tasks/main.yaml deleted file mode 100644 index ff28c45..0000000 --- a/roles/website/tasks/main.yaml +++ /dev/null @@ -1,42 +0,0 @@ ---- -# Copyright (c) 2018 Red Hat, Inc. -# -# This file is part of ARA Records Ansible. -# -# ARA is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# ARA is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with ARA. If not, see . - -# Zuul already prepares the src repository on the remote node and -# Zuul doesn't let you run pipe lookups on executors for security purposes -- name: Symlink ara-infra to persistent location with Zuul - become: yes - file: - src: "{{ ansible_user_dir }}/{{ zuul.project.src_dir }}" - dest: /opt/ara-infra - state: link - when: zuul is defined - -# git rev-parse --show-toplevel returns the root git directory -- name: Copy ara-infra to persistent location - become: yes - synchronize: - src: "{{ lookup('pipe', 'git rev-parse --show-toplevel') }}" - dest: /opt/ - delete: yes - when: zuul is not defined - -- name: Set up Hugo - include_tasks: hugo.yaml - -- name: Set up nginx - include_tasks: nginx.yaml diff --git a/roles/website/tasks/nginx.yaml b/roles/website/tasks/nginx.yaml deleted file mode 100644 index d25a53c..0000000 --- a/roles/website/tasks/nginx.yaml +++ /dev/null @@ -1,43 +0,0 @@ ---- -# Copyright (c) 2018 Red Hat, Inc. -# -# This file is part of ARA Records Ansible. -# -# ARA is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# ARA is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with ARA. If not, see . - -- become: yes - block: - - name: Install nginx - package: - name: nginx - state: present - - - name: Set selinux boolean to allow nginx to reverse proxy - seboolean: - name: httpd_can_network_connect - state: yes - persistent: yes - - - name: Set up nginx vhost - copy: - src: ara.recordsansible.org.conf - dest: /etc/nginx/conf.d/ara.recordsansible.org.conf - notify: - - Restart nginx - - - name: Start and enable nginx - service: - name: nginx - state: started - enabled: yes diff --git a/roles/website/templates/hugo.service.j2 b/roles/website/templates/hugo.service.j2 deleted file mode 100644 index df93420..0000000 --- a/roles/website/templates/hugo.service.j2 +++ /dev/null @@ -1,16 +0,0 @@ -[Unit] -Description=Hugo internal server -After=network.target - -[Service] -User=hugo -Group=hugo -Type=simple -ExecStart=/usr/local/bin/hugo server --source /opt/ara-infra/website --themesDir {{ hugo_directory }}/themes --baseURL "https://ara.recordsansible.org/" --appendPort=false -ProtectSystem=yes -ProtectHome=no -NoNewPrivileges=yes -PrivateTmp=yes - -[Install] -WantedBy=multi-user.target diff --git a/tests/website-post.yaml b/tests/website-post.yaml deleted file mode 100644 index c8b1ef8..0000000 --- a/tests/website-post.yaml +++ /dev/null @@ -1,42 +0,0 @@ ---- -# Copyright (c) 2018 Red Hat, Inc. -# -# This file is part of ARA Records Ansible. -# -# ARA is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# ARA is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with ARA. If not, see . - -- name: Website post-run - hosts: ara.recordsansible.org - gather_facts: yes - tasks: - - name: Create artifact directories - file: - path: "{{ ansible_user_dir }}/workspace/logs/build" - state: directory - recurse: yes - - - name: Generate static version of the website - become: yes - command: | - /usr/local/bin/hugo \ - --source /opt/ara-infra/website \ - --themesDir /opt/hugo/themes \ - --destination {{ ansible_user_dir }}/workspace/logs/build - - - name: Upload log artifacts - synchronize: - src: "{{ ansible_user_dir }}/workspace/logs" - dest: "{{ zuul.executor.log_root }}" - mode: pull - verify_host: true diff --git a/tests/website-pre.yaml b/tests/website-pre.yaml deleted file mode 100644 index d04b7a0..0000000 --- a/tests/website-pre.yaml +++ /dev/null @@ -1,53 +0,0 @@ ---- -# Copyright (c) 2018 Red Hat, Inc. -# -# This file is part of ARA Records Ansible. -# -# ARA is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# ARA is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with ARA. If not, see . - -- name: Website pre-run - hosts: ara.recordsansible.org - vars: - domain: ara.recordsansible.org - ansible_python_interpreter: /usr/bin/python3 - tasks: - - become: true - block: - - name: Install python3-pyOpenSSL - package: - name: python3-pyOpenSSL - state: present - - - name: Create expected letsencrypt directories - file: - path: "/etc/letsencrypt/live/{{ domain }}" - state: directory - recurse: yes - - - name: Generate an OpenSSL private key - openssl_privatekey: - path: "/etc/letsencrypt/live/{{ domain }}/privkey.pem" - - - name: Generate an OpenSSL CSR - openssl_csr: - path: /etc/letsencrypt/live/{{ domain }}/request.csr - privatekey_path: "/etc/letsencrypt/live/{{ domain }}/privkey.pem" - common_name: "{{ domain }}" - - - name: Generate a self signed SSL certificate - openssl_certificate: - path: "/etc/letsencrypt/live/{{ domain }}/fullchain.pem" - privatekey_path: "/etc/letsencrypt/live/{{ domain }}/privkey.pem" - csr_path: /etc/letsencrypt/live/{{ domain }}/request.csr - provider: selfsigned diff --git a/website/README.rst b/website/README.rst deleted file mode 100644 index 6bcbefe..0000000 --- a/website/README.rst +++ /dev/null @@ -1,25 +0,0 @@ -ARA's website -============= - -Hosted at `ara.recordsansible.org `_, the -website is built with `Hugo `_ and installed with the -``website`` role from this project's repository. - -The website uses the Hugo theme called `future-imperfect `_ -and it must be checked out as part of building the site. - -Development ------------ - -Clone repositories:: - - # This repo - git clone https://git.openstack.org/openstack/ara-infra /opt/ara-infra - # The Hugo theme (future-imperfect) - git clone https://github.com/jpescador/hugo-future-imperfect /opt/hugo/themes/hugo-future-imperfect - -Install Hugo following instructions from their `documentation `_. - -Run Hugo's standalone server:: - - hugo server --source /opt/ara-infra/website --themesDir /opt/hugo/themes diff --git a/website/config.toml b/website/config.toml deleted file mode 100644 index e528aed..0000000 --- a/website/config.toml +++ /dev/null @@ -1,96 +0,0 @@ -# https://gohugo.io/getting-started/configuration/ - -languageCode = "en-us" -title = "ARA Records Ansible | ara.recordsansible.org" -theme = "hugo-future-imperfect" -preserveTaxonomyNames = true -paginate = 10 -pluralizeListTitles = false -enableRobotsTXT = true -enableGitInfo = true -relativeURLs = true - -[params] - categoriesByCount = true - cssFiles = ["default"] - description = "ARA Records Ansible and makes it easier to understand and troubleshoot." - dynamicTitles = false - enableCDN = false - faviconVersion = "" - hideEmptyStats = false - highlightjsLang = ["bash", "python", "yaml"] - highlightjsTheme = "github" - highlightjs = true - imageStretch = "" - jsFiles = ["default"] - loadFavicon = true - navbarTitle = "ARA Records Ansible" - readingTime = true - removeBlur = false - rssAppearAtBottom = true - rssAppearAtTop = true - showSidebarCategories = true - socialShare = [] - viewMorePostsLink = "/blog/" - -[permalinks] - blog = "/blog/:year/:month/:day/:slug/" - -[params.intro] - paragraph = "Records Ansible and makes it easier to understand and troubleshoot." - [params.intro.pic] - src = "static/logo.png" - circle = false - imperfect = false - width = "" - alt = "ara.recordsansible.org" - -# https://discourse.gohugo.io/t/raw-html-getting-omitted-in-0-60-0/22032 -[markup.goldmark.renderer] -unsafe = true - -[params.postAmount] -# Sets the number of recent posts to show in the sidebar. The default value is 5. - sidebar = 8 - -# Sets the menu items in the navigation bar -# Identifier prepends a Font Awesome icon to the menu item -[[menu.main]] - name = "About" - url = "/" - identifier = "fa fa-question-circle" - weight = 1 - -[[menu.main]] - name = "Blog" - url = "/blog/" - identifier = "fa fa-newspaper-o" - weight = 2 - -[[menu.main]] - name = "Code" - url = "https://github.com/ansible-community/ara" - identifier = "fa fa-github" - weight = 3 - -[[menu.main]] - name = "Documentation" - url = "https://ara.readthedocs.io" - identifier = "fa fa-book" - weight = 4 - -[[menu.main]] - name = "Demo" - url = "https://demo.recordsansible.org" - identifier = "fa fa-external-link" - weight = 5 - -[[menu.main]] - name = "Community & Help" - url = "/community" - identifier = "fa fa-users" - weight = 6 - -[social] - github = "ansible-community/?q=ara" - twitter = "RecordsAnsible" diff --git a/website/content/_index.md b/website/content/_index.md deleted file mode 100644 index fd5762c..0000000 --- a/website/content/_index.md +++ /dev/null @@ -1,144 +0,0 @@ ---- -kind: home ---- - -ARA Records Ansible and makes it easier to understand and troubleshoot. - -It's another recursive acronym. - -![ara-full-logo](/static/ara-full-logo.png) - -## What it does - -Simple to install and get started, ara provides reporting by saving detailed and -granular results of ``ansible`` and ``ansible-playbook`` commands wherever you run them: - -- by hand or from a script -- from a laptop, a desktop, a container or a server -- for development, CI or production -- from a linux distribution or even on OS X (as long as you have ``python >= 3.5``) -- from tools such as AWX or Tower, Jenkins, GitLab CI, Rundeck, Zuul, Molecule, ansible-pull, ansible-test or ansible-runner - -By default, ara's Ansible callback plugin will record data to a local sqlite -database without requiring you to run a server or a service: - -![quickstart-default](/static/ara-quickstart-default.gif) - -ara can also provide a single pane of glass when recording data from multiple -locations by pointing the callback plugin to a running API server: - -![quickstart-server](/static/ara-quickstart-server.gif) - -The data is then made available for browsing, searching and querying over the -included reporting interface, a CLI client as well as a REST API. - -## How it works - -ARA Records Ansible execution results to sqlite, mysql or postgresql databases -by using an [Ansible callback plugin](https://docs.ansible.com/ansible/latest/plugins/callback.html). - -This callback plugin leverages built-in python API clients to send data to a -REST API server: - -![recording-workflow](/static/recording-workflow.png) - -## What it looks like - -### API browser - -Included by the API server with django-rest-framework, the API browser allows -users to navigate the different API endpoints and query recorded data. - -![ui-api-browser](/static/ui-api-browser.png) - -### Reporting interface - -A simple reporting interface built-in to the API server without any extra dependencies. - -![ui-playbook-details](/static/ui-playbook-details.png) - -### ara CLI - -A built-in CLI client for querying and managing playbooks and their recorded data. - -![cli-playbook-list](/static/cli-playbook-list.png) - -The full list of commands, their arguments as well as examples can be found in -the [CLI documentation](https://ara.readthedocs.io/en/latest/cli.html#cli-ara-api-client). - -## Getting started - -### Requirements - -- Any recent Linux distribution or Mac OS with python >=3.5 available -- The ara Ansible plugins must be installed for the same python interpreter as Ansible itself - -For RHEL 7 and CentOS 7 it is recommended to run the API server in a container due to missing or outdated dependencies. -See this [issue](https://github.com/ansible-community/ara/issues/99) for more information. - -### Recording playbooks without an API server - -With defaults and using a local sqlite database: - -```bash -# Install Ansible and ARA (with API server dependencies) for the current user -python3 -m pip install --user ansible "ara[server]" - -# Configure Ansible to use the ARA callback plugin -export ANSIBLE_CALLBACK_PLUGINS="$(python3 -m ara.setup.callback_plugins)" - -# Run an Ansible playbook -ansible-playbook playbook.yaml - -# Use the CLI to see recorded playbooks -ara playbook list - -# Start the built-in development server to browse recorded results -ara-manage runserver -``` - -### Recording playbooks with an API server - -You can get an API server deployed using the [ara Ansible collection](https://github.com/ansible-community/ara-collection) -or get started quickly using the container images from [DockerHub](https://hub.docker.com/r/recordsansible/ara-api) and -[quay.io](https://quay.io/repository/recordsansible/ara-api): - -```bash -# Create a directory for a volume to store settings and a sqlite database -mkdir -p ~/.ara/server - -# Start an API server with podman from the image on DockerHub: -podman run --name api-server --detach --tty \ - --volume ~/.ara/server:/opt/ara:z -p 8000:8000 \ - docker.io/recordsansible/ara-api:latest - -# or with docker from the image on quay.io: -docker run --name api-server --detach --tty \ - --volume ~/.ara/server:/opt/ara:z -p 8000:8000 \ - quay.io/recordsansible/ara-api:latest -``` - -Once the server is running, ara's Ansible callback plugin must be installed and configured to send data to it: - -```bash -# Install Ansible and ARA (without API server dependencies) for the current user -python3 -m pip install --user ansible ara - -# Configure Ansible to use the ARA callback plugin -export ANSIBLE_CALLBACK_PLUGINS="$(python3 -m ara.setup.callback_plugins)" - -# Set up the ARA callback to know where the API server is located -export ARA_API_CLIENT="http" -export ARA_API_SERVER="http://127.0.0.1:8000" - -# Run an Ansible playbook -ansible-playbook playbook.yaml - -# Use the CLI to see recorded playbooks -ara playbook list -``` - -Data will be available on the API server in real time as the playbook progresses and completes. - -You can read more about how container images are built and how to run them in the -[documentation](https://ara.readthedocs.io/en/latest/container-images.html). \ No newline at end of file diff --git a/website/content/blog/2016-05-21_an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/index.md b/website/content/blog/2016-05-21_an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/index.md deleted file mode 100644 index 977cc97..0000000 --- a/website/content/blog/2016-05-21_an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/index.md +++ /dev/null @@ -1,182 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development -tags: - - ansible - - openstack -date: 2016-05-21 -title: "ARA: An idea to store, browse and troubleshoot Ansible Playbook runs" -slug: an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs -type: post ---- - -# The context -[Ansible](https://www.ansible.com/) can be used for a lot of things and it's -grown pretty popular for managing servers and their configuration. - -In the [RDO](https://www.rdoproject.org/) and -[OpenStack](https://www.openstack.org/) communities, Ansible is heavily used to -deploy or test OpenStack through Continuous Integration (CI). Projects like -[TripleO-Quickstart](https://github.com/openstack/tripleo-quickstart), -[WeIRDO](https://github.com/redhat-openstack/weirdo), -[OpenStack-Ansible](https://github.com/openstack/openstack-ansible) or -[Zuul v3](https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#ansible) -are completely driven by Ansible. - -In the world of automated continuous integration, it's not uncommon to have -hundreds, if not thousands of jobs running every day for testing, building, -compiling, deploying and so on. - -Keeping up with a large amount of Ansible runs and their outcome, not -just in the context of CI, is challenging. - -# The idea -[ARA](https://github.com/dmsimard/ara) is an idea I came up with to try -and make Ansible runs easier to visualize, understand and troubleshoot. - -ARA is three things: - -1. An [Ansible callback plugin](https://ara.readthedocs.io/en/latest/configuration.html#ansible) to record playbook runs into a local or remote database -2. A [CLI client](https://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli) to query the database -3. A [web interface](https://ara.readthedocs.io/en/latest/faq.html#what-does-the-web-interface-look-like) to visualize the database - -ARA organizes recorded playbook data in a way to make it intuitive for you to -search and find what you're interested for as fast and as easily as possible. - -It provides summaries of task results per host or per playbook. - -It allows you to filter task results by playbook, play, host, task or by the -status of the task. - -With ARA, you're able to easily drill down from the summary view for the results -you're interested in, whether it's a particular host or a specific task. - -Beyond browsing a single ansible-playbook run, ARA supports recording and -viewing multiple runs in the same database. - -This allows you to, for example, recognize patterns (ex: this particular host -is always failing this particular task) since you have access to data from -multiple runs. - -ARA is an open source project available on [Github](https://github.com/dmsimard/ara) under the Apache v2 license. -[Documentation](https://ara.readthedocs.io/en/latest/) and -[frequently asked questions](https://ara.readthedocs.io/en/latest/faq.html) are available on readthedocs.io. - -# Why ? -As I mentioned before, the vast majority of the RDO CI is powered by Ansible. -When a job build fails, I have to look at one of these Jenkins -[console logs](https://dmsimard.com/files/ansible-jenkins.txt) that's >8000 -lines long. If it doesn't crash my browser, I get to dig across the large -amount of output to try and figure out what went wrong in the job build. - -When you're testing OpenStack trunk, you're going to be troubleshooting a *lot* -of those large failed jobs and it's painful. -Over time, I've (*unfortunately*) gotten used to it and got pretty good, actually. -However, it still takes me a non negligible amount of time just to find where -Ansible failed to know where to start searching for in the logs. - -It's also definitely a nightmare when someone else wants to look at the jobs -to try and understand what happened. - -ARA solves that painpoint - and many others - by making it easier to -browse the results of a playbook. - -### Other attempts -To try and help us before ARA was born, we leveraged two callbacks to -try and help us parse the Ansible Playbook output. - -The first is [human_log.py](https://gist.github.com/cliffano/9868180) which -helps pretty-printing output from tasks like "command" or "yum". -We also have [profile_tasks](https://github.com/jlafon/ansible-profile/blob/master/callback_plugins/profile_tasks.py) -that is built-in and helps by showing how much time each task took. - -These callbacks are definitely helpful for small playbooks or playbooks that -contain small or short-running tasks. -On long-running playbooks with a large amount of output, they almost make matters -worse by adding even more output into the task results. - -# How do I get started with ARA ? -I've tried to do simple, yet effective documentation on how to get started -with ARA. - -### 1) Install ARA -Documentation: [https://ara.readthedocs.io/en/latest/installation.html](https://ara.readthedocs.io/en/latest/installation.html) - -First, you'll need to install some packaged dependencies and then you -can install ARA from source or from pip. - -For example on a CentOS server: - - yum -y install gcc python-devel libffi-devel openssl-devel - pip install ara - -### 2) Configure the callback -Documentation: [https://ara.readthedocs.io/en/latest/configuration.html#ansible](https://ara.readthedocs.io/en/latest/configuration.html#ansible) -([What's an Ansible Callback ?](https://ara.readthedocs.io/en/latest/faq.html#what-s-an-ansible-callback)) - -The configuration of the callback is simple and seemless. You want to -add the following to your ansible.cfg file: - - [defaults] - callback_plugins = /usr/lib/python2.7/site-packages/ara/callback - - # Or, if using a virtual environment, for example - - [defaults] - callback_plugins = $VIRTUAL_ENV/lib/python2.7/site-packages/ara/callback - -### 3) Run a playbook with ansible-playbook -Run your favorite playbook ! - -### 4.1) Browse your data through the CLI -Documentation: [https://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli](https://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli) - - $ ara result list - +--------------------------------------+-------------+----------------+--------+---------------+----------------------------+----------------------------+ - | ID | Host | Task | Status | Ignore Errors | Time Start | Time End | - +--------------------------------------+-------------+----------------+--------+---------------+----------------------------+----------------------------+ - | a73efa33-0d1e-4a7d-8e28-a76fa93b9377 | localhost | Debug thing | ok | False | 2016-05-21 14:42:24.794070 | 2016-05-21 14:42:24.837268 | - +--------------------------------------+-------------+----------------+--------+---------------+----------------------------+----------------------------+ - - $ ara result show a73efa33-0d1e-4a7d-8e28-a76fa93b9377 - +---------------+----------------------------------------------------+ - | Field | Value | - +---------------+----------------------------------------------------+ - | ID | a73efa33-0d1e-4a7d-8e28-a76fa93b9377 | - | Host | localhost | - | Task | Debug thing (d04a5828-d32f-4349-89f1-39d7400b328f) | - | Status | ok | - | Ignore Errors | False | - | Time Start | 2016-05-21 14:42:24.794070 | - | Time End | 2016-05-21 14:42:24.837268 | - +---------------+----------------------------------------------------+ - -### 4.2) Browse your data through the web interface -Documentation: [https://ara.readthedocs.io/en/latest/usage.html#browsing-the-web-interface](https://ara.readthedocs.io/en/latest/usage.html#browsing-the-web-interface) -([What does the web UI look like ?](https://ara.readthedocs.io/en/latest/faq.html#what-does-the-web-interface-look-like)) - -Fire off the bundled webserver: - - $ ara-manage runserver - * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) - -And use your favorite browser. - -# There's no step five ! -We're all done here. That's the gist of it. - -A lot of effort was made towards making ARA as simple to install, -configure and use as possible. It is meant to be able to run from start -to finish locally but it is also powerful enough if you'd like to -aggregate runs into a central server. - -### Discussing or contributing to ARA -If you'd like to use ARA or contribute to the project, definitely feel -free ! Feedback, comments, ideas and suggestions are quite welcomed as -well. - -I hang out in the **#ara** channel on freenode IRC if you want to come chat about ARA ! - -Special thanks to [Lars Kellogg-Stedman](http://blog.oddbit.com/) for the early feedback on the project, ideas and code contributions. -He was very helpful in fleshing and maturing the idea into something better. diff --git a/website/content/blog/2016-06-07_ara-one-month-and-200-commits-later/index.md b/website/content/blog/2016-06-07_ara-one-month-and-200-commits-later/index.md deleted file mode 100644 index 581367a..0000000 --- a/website/content/blog/2016-06-07_ara-one-month-and-200-commits-later/index.md +++ /dev/null @@ -1,103 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development - - changelog -tags: - - ansible - - openstack -date: 2016-06-07 -title: "One month and 200 commits later" -slug: one-month-and-200-commits-later -type: post ---- - -On May 21st, I wrote a blog post about [ARA, an idea to store, browse and troubleshoot Ansible playbook runs](https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/). -Let's rewind a bit further back in time. - -On May 6th, I got tired of trying to make our [human_log](https://github.com/rdo-infra/weirdo/blob/master/playbooks/library/human_log.py) callback write user friendly HTML files. -I simply wasn't happy with my [attempts](https://review.gerrithub.io/#/q/topic:human_log_html)... - -I'm a big fan of the UNIX philosophy: Do one thing and do it well. -Trying to hack HTML writing into human_log didn't feel like that at all. - -I got frustrated and took a completely different direction. -What if, instead of writing and appending to a html file throughout a playbook run, I'd write to a database and make a dynamic, database-driven interface. -This would allow to organize playbook run data and also provide supported for aggregated run visualization. - -Not a lot of sleep was had that weekend. I [created a repository](https://github.com/dmsimard/ara/commit/ad09488bc291e6006f79110f903be962ab0d0a39) on friday and set out to have a prototype out by the following monday, - -And what a wild ride it's been since then. -I first shared the idea with an [alpha prototype preview](https://www.youtube.com/watch?v=K3jTqgm2YuY). -An awesome collaborator, [Lars Kellog-Stedman](http://blog.oddbit.com/), started contributing code, ideas and feedback. -I [announced the project](https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/) to the greater public with a [beta preview](https://www.youtube.com/watch?v=k3qtgSFzAHI). - -ARA is one month old and has more than 200 commits today. Since the prototype, ARA has gained quite a few features. Amongst other things: - -- Static generation of the web application, including feature parity with the dynamic web application -- Host fact recording and browsing -- A CLI client interface -- Support for aborted playbook runs and overall improvements around failure and exception handling -- A lot of unit and integration tests to ensure stability and prevent regressions ([we can even run Ansible's own integration test suite!](https://github.com/dmsimard/ara/commit/c1883013e8352eef68420e9136964547eb6a2e0c)) - -The OpenStack and the Ansible communities are excited with ARA's potential and opportunities to help them. -This fuels my motivation even further and I was already passionate about the project to help me in my own job to begin with. - -If ARA can help other Ansible users make sense of their playbook runs, saying that I'm very happy about it would be an understatement. - -### The next steps for ARA: To infinity and beyond -#### A proper development, contribution and issue tracking workflow. -The [OpenStack](http://www.openstack.org/) project has awesome [tooling, hosting, and testing infrastructure](http://docs.openstack.org/infra/system-config/). - -Every day, developers send hundreds of commits for review across more than a hundred projects in [Gerrit](https://review.openstack.org/#/). -These will go through the [Zuul](https://github.com/openstack-infra/zuul) pipeline manager and eventually tested with [Jenkins](https://jenkins.io/) on ephemeral virtual machines. These virtual machines are managed by -[nodepool](http://docs.openstack.org/infra/system-config/nodepool.html) and hosted on OpenStack cloud resources [donated by different contributing companies](http://docs.openstack.org/infra/system-config/contribute-cloud.html). - -OpenStack provides these resources under a program that was previously known as [StackForge](http://docs.openstack.org/infra/system-config/stackforge.html) for projects that are relevant to the OpenStack community and want to use the same development workflow. -It does not imply governance of the projects by the OpenStack foundation or the -OpenStack technical committee. The self-appointed core contributor -team continues to drive the direction of the project and decides what goes in (or -not). - -OpenStack happens to host several projects that are not OpenStack -specific, a good example of which is [Jenkins Job Builder](https://github.com/openstack-infra/jenkins-job-builder) (JJB). -While JJB was created to scratch an itch in the OpenStack community, the -tool's development remained agnostic and today sees development and -usage from outside the OpenStack community. - -There was significant interest to have ARA be part of the Ansible community umbrella as well. -It was a hard decision, but we ended up choosing the OpenStack community as it's home. - -ARA, if all goes well, should be joining the OpenStack ecosystem [soon](https://review.openstack.org/#/c/321226/) and we'll update documentation with the contribution workflow once that is done. - -#### A stable release -The break-neck pace of development in ARA until now has prevented us from declaring any sort of stable release beyond tagged development releases. - -This doesn't mean ARA has a lot of bugs, it means the database schema is still changing quite a bit. Because of this, we have chosen to not develop automated database migrations and upgrades. -This helps us merge new features and fixes faster without database migration overhead while the project is not yet considered stable. - -We have started identifying features which might require a database schema change. -Once we have ironed these out and the schema hasn't changed in a bit, it will be a good time to declare the schema stable and draft the first stable release of ARA. -When that first stable release is out, we will try to make sure that ARA can be upgraded without having to start your database from scratch. - -#### An even better user experience -I'd love ARA's web application to have it's own distinct identity, look and feel. It's a fairly generic web application with very basic html, javascript and bootstrap CSS right now. - -I'm really not a frontend guy. I sincerely hope that, over time, we can have skilled contributors help on that front. - -#### More features and improvments based on what users need -I'm a demanding Ansible user and I'll be using ARA a lot, that's for sure. -The feature development has been mostly driven by what I've needed so far. - -We have already received a lot of valuable feedback and comments about ARA. -I'd love this trend to continue so that we can improve ARA by adding or changing things we didn't think about. - -If ARA doesn't do something you'd need, I want to know about it. -If ARA could do something better, I want to know about it. - -We use [GitHub Issues](https://github.com/dmsimard/ara/issues) to help us keep track of things for the time being. - -### See you at the stable release -The first stable release will be a great milestone for ARA. Let's do another recap once it's out ! - -Until then, come chat with us on #ara on freenode IRC and thanks for all the interest and support ! diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/index.md b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/index.md deleted file mode 100644 index b58d9e3..0000000 --- a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/index.md +++ /dev/null @@ -1,92 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - community -tags: - - ansible - - continuous integration - - openstack -date: 2016-11-09 -title: Visualizing Kolla's Ansible playbooks with ARA -slug: visualizing-kolla-ansible-playbooks-with-ara -type: post ---- - -[Kolla](https://github.com/openstack/kolla) is an [OpenStack](http://www.openstack.org/) deployment tool that's growing -in popularity right now. - -An OpenStack installation by Kolla was even showcased by [Chris Hoge](https://twitter.com/hogepodge) and -[Mark Collier](https://twitter.com/sparkycollier) in one of the main [keynotes](https://www.youtube.com/watch?v=GGxzlLwt6WA&t=1325) -at the recent [Barcelona OpenStack Summit](https://www.openstack.org/summit/barcelona-2016/). - -{{< tweet 790816907746213888 >}} - -## Installing OpenStack is complex - -Installing and configuring OpenStack is no easy task -- Kolla tackles this challenge with the help of [Docker](https://www.docker.com/) -containers that are deployed with [Ansible](https://www.ansible.com/). - -This translates into quite a few playbooks, lots of plays, many more tasks and especially lots of data to parse through -when trying to understand what is going on. - -I was recently pleased to learn that the Kolla project implemented [ARA](https://github.com/ansible-community/ara) to help them -troubleshoot their CI jobs. - -## Installing ARA is simple - -How hard was it to integrate ARA with Kolla ? Eight lines of code. -8 lines of code, that's what's what it took and all you'll see in the [code review](https://review.openstack.org/#/c/368345/) -to implement it. - -![setup_gate.sh](kolla-1.png) - -![deploy_aio.sh](kolla-2.png) - -## I'm happy - -I'm very happy about this because this is exactly what I have been targetting with ARA. -A tool that provides a lot of value while staying very simple, requires little configuration, is seemless and above all, -doesn't get in the way of your existing workflows. - -## What it looks like - -So, now that Kolla has integrated ARA, what does it look like ? - -Let's look at a typical code review in Gerrit, we can see that one of the gate jobs failed: - -![gate job](kolla-gate.png) - -Clicking on the job link will send us to the logs where Kolla generated the ARA report inside the playbook_reports folder: - -![gate job](kolla-logs.png) - -Entering that folder gets us directly to a statically generated version of the ARA web application. -We can see that two playbooks have been run and one of them has a failed task: - -![web app](kolla-webapp.png) - -Clicking on that particular playbook brings up the playbook summary page: - -![playbook](kolla-playbook.png) - -You can then drill down directly to the failed tasks by clicking on the appropriate column: - -![failed task](kolla-failed.png) - -Drilling down further into the particular task that failed, we see that ARA records everything from it and makes it available: - -![task details](kolla-task.png) - -If we're interested in seeing what the actual Ansible task looks like, clicking on the file line card gets us to a saved -copy of the whole file, highlighting the line number where the task took place: - -![file details](kolla-file.png) - -## This is great, let's make it better - -It's great that people are using ARA and finding it useful. -I like the current state of ARA and the direction it's going but I'm also very excited about the awesome opportunities -some new features landing in the next release will open up. - -If you'd like to come chat with me about ARA, ask questions or would love to get involved, come join us on IRC. -We hang out on the #ara channel on freenode ! diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-1.png b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-1.png deleted file mode 100644 index 5be35f1..0000000 Binary files a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-1.png and /dev/null differ diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-2.png b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-2.png deleted file mode 100644 index a7e7f16..0000000 Binary files a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-2.png and /dev/null differ diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-failed.png b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-failed.png deleted file mode 100644 index bb5d294..0000000 Binary files a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-failed.png and /dev/null differ diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-file.png b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-file.png deleted file mode 100644 index 69815f2..0000000 Binary files a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-file.png and /dev/null differ diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-gate.png b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-gate.png deleted file mode 100644 index e37e233..0000000 Binary files a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-gate.png and /dev/null differ diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-logs.png b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-logs.png deleted file mode 100644 index 999d7d6..0000000 Binary files a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-logs.png and /dev/null differ diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-playbook.png b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-playbook.png deleted file mode 100644 index 59ac9a2..0000000 Binary files a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-playbook.png and /dev/null differ diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-task.png b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-task.png deleted file mode 100644 index 5e29edd..0000000 Binary files a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-task.png and /dev/null differ diff --git a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-webapp.png b/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-webapp.png deleted file mode 100644 index cefb126..0000000 Binary files a/website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/kolla-webapp.png and /dev/null differ diff --git a/website/content/blog/2016-12-01_0.10-the-biggest-release-yet-is-out/index.md b/website/content/blog/2016-12-01_0.10-the-biggest-release-yet-is-out/index.md deleted file mode 100644 index 9e2561b..0000000 --- a/website/content/blog/2016-12-01_0.10-the-biggest-release-yet-is-out/index.md +++ /dev/null @@ -1,98 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development - - changelog -tags: - - ansible - - openstack -date: 2016-12-01 -title: 0.10, the biggest release yet, is out ! -slug: 0.10-the-biggest-release-yet-is-out -type: post ---- - -19 commits, 59 changed files, 2,404 additions and 588 deletions... and more than a month's on and off work. - -[0.10 is out](https://github.com/ansible-community/ara/releases/tag/0.10.0) ! - -Where to get it ? Get started easily by [installing](http://ara.readthedocs.io/en/latest/installation.html) and [configuring](http://ara.readthedocs.io/en/latest/configuration.html) ARA. - -I'm excited to tell you about this new release ! Here's a few highlights ! - -## An improved web application browsing experience - -A lot of work has gone into the browsing experience: less clicks, more information, faster. - -I am by no means a professional frontend developer and I would definitely love help on this front but I think things are definitely starting to look good ! -I've added updated previews to the documentation: - -- In [screenshots](https://ara.readthedocs.io/en/latest/faq.html#what-does-the-web-interface-look-like) -- In [video](https://www.youtube.com/watch?v=zT1l-rFne-Q&index=3&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty) where we I showcase some playbook runs by the [OpenStack-Ansible](https://github.com/openstack/openstack-ansible) project. - -It's awesome to see how much the interface has improved over time. - -Curious ? Look at the [Alpha](https://www.youtube.com/watch?v=K3jTqgm2YuY&index=1&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty) and [Beta](https://www.youtube.com/watch?v=k3qtgSFzAHI&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty&index=2) video previews to see how far the project has come ! - -## Two new Ansible modules - -This new release features two new Ansible modules, [ara_record](http://ara.readthedocs.io/en/latest/usage.html#using-the-ara-record-module) and [ara_read](http://ara.readthedocs.io/en/latest/usage.html#using-the-ara-read-module). - -These allow you to respectively write and read persistent data that is made available throughout your playbook run and in the web interface. - -For example: -``` ---- -- name: Test playbook - hosts: localhost - tasks: - - name: Get git version of playbooks - command: git rev-parse HEAD - register: git_version - - - name: Record git version - ara_record: - key: "git_version" - value: "{{ git_version.stdout }}" -``` - -That would make the git_version key/value available in the web interface and on the CLI. - -And you can record different kind of data, too, see: - -``` ---- -- ara_record: - key: "{{ item.key }}" - value: "{{ item.value }}" - type: "{{ item.type }}" - with_items: - - { key: "log", value: "error", type: "text" } - - { key: "website", value: "http://domain.tld", type: "url" } - - { key: "data", value: '{ "key": "value" }', type: "json" } -``` - -Setting a type will make it so it's displayed accordingly in the interface. -More types will eventually be made available ! - -## Improved unit and integration testing coverage - -In order to make sure ARA works well -- and keeps working well -- we need to have a good amount of unit and integration testing. - -This coverage was improved since the previous release and additionnally, we now test against different versions of Ansible on both CentOS and Ubuntu. - -## Database schema finally declared stable - -Was ARA unstable before ? Not really. -However, at the rapid pace of development we were having with the project, we decided to avoid managing SQL migrations to avoid unnecessary overhead as the database schema was moving a lot. - -I feel the database is fairly fleshed out at this point at any new modifications will be handled automatically when running ARA. - -Any version of ARA >= 0.9.0 is considered a stable and managed database schema. - -## This is great, let's make it better ! - -The feedback around ARA has been awesome, keep it coming ! -A lot of the improvements that are part of this release is directly from ideas and comments provided by users. - -If you want to come chat, learn or discuss about ARA, you'll find us on #ara on IRC in the freenode server ! diff --git a/website/content/blog/2017-02-13_announcing-the-release-of-0.11/ara-with-icon.png b/website/content/blog/2017-02-13_announcing-the-release-of-0.11/ara-with-icon.png deleted file mode 100644 index 6795325..0000000 Binary files a/website/content/blog/2017-02-13_announcing-the-release-of-0.11/ara-with-icon.png and /dev/null differ diff --git a/website/content/blog/2017-02-13_announcing-the-release-of-0.11/ara.png b/website/content/blog/2017-02-13_announcing-the-release-of-0.11/ara.png deleted file mode 100644 index 71b130f..0000000 Binary files a/website/content/blog/2017-02-13_announcing-the-release-of-0.11/ara.png and /dev/null differ diff --git a/website/content/blog/2017-02-13_announcing-the-release-of-0.11/index.md b/website/content/blog/2017-02-13_announcing-the-release-of-0.11/index.md deleted file mode 100644 index f39233e..0000000 --- a/website/content/blog/2017-02-13_announcing-the-release-of-0.11/index.md +++ /dev/null @@ -1,113 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development - - changelog -tags: - - ansible - - openstack -date: 2017-02-13 -title: Announcing the release of 0.11 -slug: announcing-the-release-of-0.11 -type: post ---- - -We're on the road to version 1.0.0 and we're getting closer: introducing the release of version 0.11! - -[Four new contributors](https://github.com/ansible-community/ara/graphs/contributors) (!), [55 commits](https://github.com/ansible-community/ara/compare/0.10.0...0.11.0) since 0.10 and 112 files changed for a total of 2,247 additions and 939 deletions. - -New features, more stability, better documentation and better test coverage. - -## The changelog since 0.10.5 - -- New feature: ARA UI and Ansible version (ARA UI is running with) are now shown at the top right -- New feature: The Ansible version a playbook was run is now stored and displayed in the playbook reports -- New feature: New command: "ara generate junit": generates a junit xml stream of all task results -- New feature: ara_record now supports two new types: "list" and "dict", each rendered appropriately in the UI -- UI: Add ARA logo and favicon -- UI: Left navigation bar was removed (top navigation bar will be further improved in future versions) -- Bugfix: CLI commands could sometimes fail when trying to format as JSON or YAML -- Bugfix: Database and logs now properly default to ARA_DIR if ARA_DIR is changed -- Bugfix: When using non-ascii characters (ex: äëö) in playbook files, web application or static generation could fail -- Bugfix: Trying to use ara_record to record non strings (ex: lists or dicts) could fail -- Bugfix: Ansible config: 'tmppath' is now a 'type_value' instead of a boolean -- Deprecation: The "ara generate" command was deprecated and moved to "ara generate html" -- Deprecation: The deprecated callback location, ara/callback has been removed. Use ara/plugins/callbacks. -- Misc: Various unit and integration testing coverage improvements and optimization -- Misc: Slowly started working on full python 3 compatibility -- Misc: ARA now has a logo - -### ARA now has a logo ! - -Thanks [Jason Rist](https://twitter.com/knowncitizen) for the contribution, really appreciate it ! - -[With the icon](https://github.com/ansible-community/ara/blob/master/doc/source/_static/ara-with-icon.png): -![icon](ara-with-icon.png) - -[Without the icon](https://github.com/ansible-community/ara/blob/master/doc/source/_static/ara.png): -![full](ara.png) - -## Taking the newest version of ARA out for a spin - -Want to give this new version a try ? It's out on pypi! - -Install [dependencies and ARA](https://ara.readthedocs.io/en/latest/installation.html), configure the [Ansible callback location](https://ara.readthedocs.io/en/latest/configuration.html#ansible) and ansible-playbook your stuff ! - -Once ARA has recorded your playbook, you'll be able to fire off and browse the [embedded server](https://ara.readthedocs.io/en/latest/usage.html#browsing-the-web-interface) or generate a [static version](https://ara.readthedocs.io/en/latest/usage.html#generating-a-static-html-version-of-the-web-application) of the report. - -## The road ahead: version 1.0 - -What is coming in version 1.0 ? Let me ask you this question: what would *you* like in 1.0 ? -The development of ARA has mostly been driven by it's user's needs and I'm really excited with what we already have. - -I'd like to finish a few things before releasing 1.0... let's take a sneak peek. - -### New web user interface - -I've been working slowly but surely on a complete UI refactor, you can look at an early prototype preview here: - -{{< youtube h3vY87_EWHw >}} - -Some ideas and concepts have evolved since then but the general idea is to try and display more information in less pages, while not going overboard and have your browser throw up due to the weight of the pages. - -Some ARA users are running playbooks involving hundreds of hosts or thousands of tasks and it makes the static generation very slow, large and heavy. -While I don't think I'll be able to make the static generation work well at any kind of scale, I think we can make this better. - -There will have to be a certain point in terms of scale where users will be encouraged to leverage the dynamic web application instead. - -### Python 3 support - -ARA isn't gating against python3 right now and is actually failing unit tests when running python3. - -As Ansible is working towards python3 support, ARA needs to be there too. - -### More complex use case support (stability/maturity) - -There are some cases where it's unclear if ARA works well or works at all. -This is probably a matter of stability and maturity. - -For example, ARA currently might not behave well when running concurrent ansible-playbook runs from the same node or if a remote database server happens to be on vacation. - -More complex use case support might also mean providing users documentation on how to best leverage all the data that ARA records and provides: elasticsearch implementation, junit reports and so on. - -If ARA is useful to you, I'd be happy to learn about your use case. Get in touch and let's chat. - -### Implement support for ad-hoc ansible run logging - -ARA will by default record anything and everything related to ansible-playbook runs. -It also needs to support ad-hoc ansible commands as well. I want this before tagging 1.0. - -### Other features - -There's some other features I'd like to see make the cut for version 1.0: - -- [Fully featured Ansible role](https://github.com/openstack/ansible-role-ara) for ARA -- Store variables and extra variables -- Provide some level of support for data on a role basis (filter tasks by role, metrics, duration, etc.) -- Support generating a html or junit report for a specific playbook (rather than the whole thing) -- Packaging for Debian/Ubuntu and Fedora/CentOS/RHEL - -A stretch goal would be to re-write ARA to be properly split between client, server, UI and API -- however I'm okay to let that slip for 2.0! - -What else would you like to see in ARA ? Let me know in the comments, on IRC in #ara on freenode or on [twitter](https://twitter.com/dmsimard)! - diff --git a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/facts.png b/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/facts.png deleted file mode 100644 index 7df0010..0000000 Binary files a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/facts.png and /dev/null differ diff --git a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/file.png b/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/file.png deleted file mode 100644 index 3bb16ee..0000000 Binary files a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/file.png and /dev/null differ diff --git a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/home.png b/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/home.png deleted file mode 100644 index 56045c2..0000000 Binary files a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/home.png and /dev/null differ diff --git a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/index.md b/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/index.md deleted file mode 100644 index 0ea11e2..0000000 --- a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/index.md +++ /dev/null @@ -1,146 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development - - changelog -tags: - - ansible - - openstack -date: 2017-03-12 -title: An even better Ansible reporting interface with ARA 0.12 -slug: an-even-better-ansible-reporting-interface-with-ara-0-12 -type: post ---- - -Not even a month ago, I announced the release of [ARA 0.11](https://dmsimard.com/2017/02/13/announcing-the-ara-0.11-release/) with a bunch of new features and improvements. - -Today, I'm back with some more great news and an awesome new release, ARA 0.12(.3) ! - -That's right, 0.12.3! - -Due to the nature of this new release, I wanted to be sure to get feedback from the users before getting the word out. - -We got a lot of great input! This allowed us to fix some bugs and significantly improve the performance. - -0.12 features a completely re-written and re-designed web application user interface. Let's look at some of the highlights ! - -## A new web application interface - -I know what you're most interested in is... WHAT DOES IT LOOK LIKE !? - -### What it looks like - -Here's some highlights of the new user interface -- it doesn't end here so please -read on ! - -The home page now features the data recorded by ARA: - -![home](home.png) - -The core of the user interface now revolves around one and single page where you'll be able to find all the information about your playbooks: - -![reports](reports.png) - -Quickly have a glance at your playbook host summary: - -![summary](summary.png) - -Or dig into the host details to look at all the facts Ansible gathered for you: - -![facts](facts.png) - -Figure out which tasks took the longest just by sorting the table accordingly: - -![sort](sort.png) - -Or search to figure out which tasks failed: - -![search](search.png) - -Click on the action to get context on where this task ran: - -![file](file.png) - -Or click on the status to take a look at all the details Ansible has to offer: - -![result](result.png) - -## The logic behind the UI changes - -There were three main objectives with this refactor of the web interface. - -### Improve UX - -A lot of effort was spent on the user experience. -You need to be able to find what you want: intuitively, quickly and easily. - -Data and result tables are now sortable and searchable by default and browsing -tips were added to the interface to help you make the most of what it has to -offer. - -### Scalability and performance - -The interface must be fast, responsive, clutter-free, make sense and behave - consistently across your use case scenarios, whether you are looking at -reports which contains five tasks or ten thousand. - -Pagination settings have been introduced in order to customize your browsing -experience according to your needs. - -### Static report generation time and weight - -Another objective of this user interface work was to optimize the static report -generation performance and weight. - -Static generation is one of the great features of ARA which is very heavily used -in the context of continuous integration where the report is generated and -attached to the artifacts of the job. - -Here's a real-life example of the same database being generated on ARA 0.11 and -ARA 0.12: - -ARA integration tests (5 playbooks, 59 tasks, 69 results): - -* Before: 5.4 seconds, 1.6MB (gzipped), 217 files -* After: 2 seconds, 1.2MB (gzipped), 119 files - -OpenStack-Ansible (1 playbook, 1547 tasks, 1667 results): - -* Before: 6m21 seconds, 31MB (gzipped), 3710 files -* After: 20 seconds, 8.9MB (gzipped), 1916 files - -For larger scale playbooks, we're looking at a generation performance that is -over 19 times faster. I'm really happy about the results. - -## But wait, there's more - -If you thought the UI work was enough to warrant it's own release, you're right ! - -Some other changes also sneaked their way into this release as well. - -### First party WSGI support - -A lot of ARA users were interested in scaling their centralized deployment. -This meant helping users deploy the ARA web interface through WSGI with a web server. - -To help people get going, we now ship a WSGI script bundled with ARA and documented -how you can set it up with Apache and mod_wsgi. -The documentation is available [here](http://ara.readthedocs.io/en/latest/webserver.html). - -### Other things - -* Fixed syntax highlighting when viewing files -* Preparations for supporting the upcoming Ansible 2.3 release -* Started working on full python 3 support -* Various performance improvements - -## Well, that's it for now - -That was certainly a lot of stuff in one release ! - -I hope you're enjoying ARA - if you're not using it yet, it's easy ! - -Have a look at the documentation to learn how to [install ARA](http://ara.readthedocs.io/en/latest/installation.html) -and how to [configure Ansible](http://ara.readthedocs.io/en/latest/configuration.html) to use it. - -If you have any questions, feel free to drop by on IRC in #ara on the freenode server or hit me up on twitter: [@dmsimard](https://twitter.com/dmsimard). diff --git a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/reports.png b/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/reports.png deleted file mode 100644 index ffafe5a..0000000 Binary files a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/reports.png and /dev/null differ diff --git a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/result.png b/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/result.png deleted file mode 100644 index 32f902f..0000000 Binary files a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/result.png and /dev/null differ diff --git a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/search.png b/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/search.png deleted file mode 100644 index 972b533..0000000 Binary files a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/search.png and /dev/null differ diff --git a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/sort.png b/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/sort.png deleted file mode 100644 index a6a86a5..0000000 Binary files a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/sort.png and /dev/null differ diff --git a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/summary.png b/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/summary.png deleted file mode 100644 index b68e0a6..0000000 Binary files a/website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/summary.png and /dev/null differ diff --git a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/index.md b/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/index.md deleted file mode 100644 index 827105e..0000000 --- a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/index.md +++ /dev/null @@ -1,148 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development - - changelog -tags: - - ansible - - openstack -date: 2017-05-05 -title: "ARA 0.13 is out and it's awesome !" -slug: ara-0-13-is-out-and-its-awesome -type: post ---- - -I'm excited to announce the release of ARA: Ansible Run Analysis 0.13.0! - -ARA 0.13.0 is available on [PyPi](https://pypi.python.org/pypi/ara) or from source on [GitHub](https://github.com/ansible-community/ara). -I'm also happy to announce that ARA 0.13.0 will be the first version of ARA packaged for Fedora and CentOS EPEL. - -Stay tuned in the near future to hear when the packages will be available. - -## Wait, what's ARA ? - -ARA is an Ansible callback plugin that records your playbook runs, wherever it is. - -Whether you're running Ansible from your personal laptop or from a server, -you basically just need to [install ARA](http://ara.readthedocs.io/en/latest/installation.html), -configure [Ansible](https://ara.readthedocs.io/en/latest/configuration.html#ansible) to use ARA and you're good to go. - -ARA organizes the data in a way to help you visualize, understand and troubleshoot -what happened throughout your playbook. - -So, what does it look like ? -Here's a video demo of the interface where I explain the different features it -offers: - -{{< youtube k3i8VPCanGo >}} - -ARA also provides a [command line interface (CLI)](http://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli) -as well for use in your different scripts or implementations. - -## What's new in 0.13 - -I'm very surprised with the amount of improvements that we managed to land in -0.13, I hope you'll be as happy as I am ! - -The full changelog is available on [GitHub](https://github.com/ansible-community/ara/releases/tag/0.13.0) -but let's highlight the really cool stuff. - -### Permanent links ! - -If you've been using ARA for a while, you know that before the recent UI re-design, -almost the entirety of the content had their own links. - -This, in fact, was quite problematic as it meant creating thousands and thousands -of files when generating the static HTML version of the ARA web application. - -With the new UI, we're leveraging a lot of fun hacks in order to optimize the -time it takes to generate a static report, it's size and it's weight. - -In any case, permanent links are back and this time, without a significant impact -on the static generation so I'm very happy with that. - -Find the blue chain icon to get your permanent links ! - -![permalinks](permalinks.png) - -### Playbook parameters and task tags - -When you run the ``ansible-playbook`` command, you can pass options to it. - -Whether that's an inventory file (``-i /path/hosts``), tags (``--tags production``) -or maybe extra-vars (``--extra-vars var=value``). - -These are now all recovered by ARA and attached to your playbook report: - -![parameters](parameters.png) - -Extra vars are not saved by default as a security precaution since it is often -used for passing things like passwords. You can make it so ARA saves extra vars -or does not save another key you feel might contain sensitive data in the -[configuration](https://ara.readthedocs.io/en/latest/configuration.html#ara-ignore-parameters). - -Additionally, ARA now records task tags which allows to highlight when tasks -are tagged and to search for them in the task list: - -![parameters](tags.png) - -### Get to your reports faster with more content in less space - -We've slimmed down the width requirements of the web interface while also making -more room inside the panels for content. - -I am not interested in restricting the width of the application so that users -with larger resolutions can fully take advantage of their width. - -It's a delicate balance to maintain so that larger resolutions don't feel like -there's too much whitespace and smaller resolutions are still functional. I think -we've struck an acceptable middle ground. - -To that end, the browsing tips were useful but were taking too much screen -real estate and have been folded into "?" icons in each panel. - -You'll also notice that the "Home" page has been relocated to "About" and that -the playbook reports page is now the default home page. - -The "About" page serves it's purpose at explaining what ARA is and what it is -doing so it's actual content remains unchanged. - -### A better view of your recorded files - -After a lot of headaches, we've finally been able to land a proper file -panel to make your files easier to browse. - -This is what is looked like before and after: - -![file](new-file-tab.png) - -I'm pretty happy with the way it turned out and the headaches were worth it ! - -### Bug fixes - -What good software does not have any bugs ? We fixed a few things here and -there: - -- Include tasks could be recorded twice on Ansible >= 2.2 -- Tasks using loops (ex: ``with_items``) now record each item result -- Performance improvements - -## That's it for now - -0.13 was supposed to be a small release in preparation of packaging ARA for -RHEL-based distributions. Turns out I was wrong, there's a bunch of stuff in -there and now that it's out, I can sleep better ! - -For the road to the next version of ARA, I'd really love to target full python 3 -compatibility and I could definitely use some help. - -If you'd like to contribute to ARA's development, you can find documentation -on how to do it [here](https://ara.readthedocs.io/en/latest/contributing.html). - -Otherwise, keep the feedback coming ! - -ARA functionality is in large part driven by users' needs and feedback. -Come chat with us on IRC on the freenode server in ``#ara``. - -In celebration of this new release, I'm also hosting an AMA (Ask me anything) -on [Reddit](https://www.reddit.com/r/ansible/comments/69gkpz/hi_ransible_a_new_version_of_ara_ansible_run/) -- feel free to join ! diff --git a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/new-file-tab.png b/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/new-file-tab.png deleted file mode 100644 index 92db9f5..0000000 Binary files a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/new-file-tab.png and /dev/null differ diff --git a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/parameters.png b/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/parameters.png deleted file mode 100644 index 078f087..0000000 Binary files a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/parameters.png and /dev/null differ diff --git a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/permalinks.png b/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/permalinks.png deleted file mode 100644 index 842cfd4..0000000 Binary files a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/permalinks.png and /dev/null differ diff --git a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/tags.png b/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/tags.png deleted file mode 100644 index d01e87f..0000000 Binary files a/website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/tags.png and /dev/null differ diff --git a/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/ara-with-icon.png b/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/ara-with-icon.png deleted file mode 100644 index 6795325..0000000 Binary files a/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/ara-with-icon.png and /dev/null differ diff --git a/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/ara.png b/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/ara.png deleted file mode 100644 index 71b130f..0000000 Binary files a/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/ara.png and /dev/null differ diff --git a/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/first-commit.png b/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/first-commit.png deleted file mode 100644 index 7de93dd..0000000 Binary files a/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/first-commit.png and /dev/null differ diff --git a/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/index.md b/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/index.md deleted file mode 100644 index 5fa9f3d..0000000 --- a/website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/index.md +++ /dev/null @@ -1,255 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - community -tags: - - ansible - - openstack -date: 2017-05-08 -title: "ARA is one year old! A look back at the past year." -slug: ara-is-one-year-old-a-look-back-at-the-past-year -type: post ---- - -ARA is one year old, happy birthday ARA ! - -![first-commit](first-commit.png) - -ARA's come a long way since the early prototypes. - -The latest version, [0.13](https://dmsimard.com/2017/05/05/ara-ansible-run-analysis-0-13-is-out-and-its-awesome/), looks pretty awesome. -It even caught the eye of Michael DeHaan, the author of Ansible ! - -{{< tweet 861007096803995648 >}} - -Let's go back in time to look at the humble beginnings of the project and some -of the important milestones that marked it's history this past year. - -## An idea and a first prototype - -I was beyond frustrated from trying to understand and troubleshoot Ansible -playbook runs at scale. - -In the [OpenStack](https://www.openstack.org/) and -[RDO](https://www.rdoproject.org/) communities, we heavily leverage Ansible for, -amongst other things, continuous integration and testing jobs. - -This meant part of my days were spent looking at tens of thousands of lines of -Ansible playbook output. The [human_log](https://gist.github.com/cliffano/9868180) callback -did not really help, it almost made matters worse by making the output even larger. - -I got the idea for ARA on Friday **May 6th 2016**. -I started hacking that evening and after what was probably the most time I've -spent coding in a weekend, I had a prototype to share with close colleagues -and friends to validate the idea with the very first demo video. - -The family was unfortunately neglected during that weekend ! - -{{< youtube K3jTqgm2YuY >}} - -
-## A very, very important contributor - -One of those colleagues that ended up finding out about this prototype was -Lars Kellogg-Stedman ([@larsks](https://twitter.com/larsks)) and I owe a lot of -ARA's success to him today. - -I don't consider myself a professional programmer by any stretch. Just a system -administrator passionate with building tools to help him in his job. - -Lars came around with [code contributions](https://github.com/ansible-community/ara/commits?author=larsks) -for things I admittedly wasn't very good at. - -A much better database schema, a better structure and framework for the flask web -application, a lot of great improvements to the callback plugin, UNIT TESTS -(of which there was probably none at the time) and I could keep going. - -Lars was pretty busy with other projects at the time, which made me appreciate -so much more the time I spent discussing and developing ARA with him. - -## ARA's first public appearance - -On **May 21st 2016**, I shared ARA for the first time to the public in a blog post: -[ARA: An idea to store, browse and troubleshoot Ansible playbook runs](https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/). - -With this post came a second demo video: - -{{< youtube k3qtgSFzAHI >}} - -
-## ARA becomes an OpenStack community project - -We're **June 7th 2016**, [ARA is one month old and we've broken 200 commits already](https://dmsimard.com/2016/06/07/ara-one-month-and-200-commits-later/). - -**June 8th 2016**, the [patch](https://review.openstack.org/#/c/321226/) to make ARA -an OpenStack community project merges and [github.com/ansible-community/ara](https://github.com/ansible-community/ara) is born. - -To this day, I still keep the [github.com/dmsimard/ara](https://github.com/ansible-community/ara) repository around for -the sake of keeping the history of issues and pull requests that were created before ARA -joined the OpenStack community. - -## A first OpenStack project starts using ARA - -This came to me as a surprise, **[September 10th 2016](https://review.openstack.org/#/c/368345/)**, -a Kolla contributor implemented ARA to their integration tests in order to help them -figure out what's going on. - -It was the [first official OpenStack project to use ARA](https://dmsimard.com/2016/11/09/visualizing-kolla-ansible-playbooks-with-ara/). -It all snowballed from there. - -## A second OpenStack project tries ARA - -After I [shared that Kolla used ARA](https://dmsimard.com/2016/11/09/visualizing-kolla-ansible-playbooks-with-ara/), -the OpenStack-Ansible project submitted a [patch](https://review.openstack.org/#/c/396565/) -to add it to their project on **November 11th 2016**. - -[OpenStack-Ansible](https://github.com/openstack/openstack-ansible) was a very important early adopter of ARA, I owe them a lot. -OpenStack-Ansible simply operated at a much larger scale than we did at RDO. - -They had thousands of tasks, many more results and this highlighted a lot of the assumptions -and flaws that ARA had at the time on operating at a large scale. - -The UI and the performance were simply not suited for this kind of scale and we were able -to improve ARA a lot thanks to their feedback. - -## ARA 0.10: A stable database and a first UI redesign - -**December 2016**: [ARA 0.10 was the biggest release of ARA at the time](https://dmsimard.com/2016/12/01/ara-0.10-the-biggest-release-yet-is-out/). - -It marked the first version where we felt the database schema would not move as much -and we could manage to not break users by implementing SQL migrations without too much -development overhead. - -ARA 0.10 also shipped a first major UI redesign that would later be the foundation -of the interface ARA still uses today: - -{{< youtube zT1l-rFne-Q >}} - -
-## ARA 0.11: ARA gets a logo and a new UI prototype - -**[January 2017](https://github.com/ansible-community/ara/commit/0d5d0939a6b7a319d99acc1fb20d4ca282bd76ab)**, -Jason Rist ([@knowncitizen](https://twitter.com/knowncitizen)) came up with a logo for ARA ! - -With the icon: -![icon](ara-with-icon.png) - -Without the icon: -![full](ara.png) - -It looked awesome, I was really happy about it and it first made it's way in -[version 0.11](https://dmsimard.com/2017/02/13/announcing-the-ara-0.11-release/) of **February 2017**. - -This is also around the time where I started experimenting with yet another redesign -for the user interface: - -{{< youtube h3vY87_EWHw >}} -
-I feel like version 0.11 was the tipping point where users outside of the OpenStack -community *really* started getting interested in ARA. - -I got plenty of interest over social media and started getting a lot of questions from users. - -{{< tweet 831617879804088325 >}} - -
-## ARA 0.12: A second UI redesign lands - -**March 2017**, [ARA 0.12](https://dmsimard.com/2017/03/12/an-even-better-ansible-reporting-interface-with-ara-0-12/) is released -with a first iteration of the new user interface. - -0.12 was the first version I felt confident recommending to users running Ansible at a large scale. -The UI redesign had those large scale users in mind while staying efficient and intuitive for small scale users. - -It made browsing large amount of contents easy and straightforward and a lot of considerations went into making the interface as responsive as possible. It also very significantly improved one of the key features of ARA, the -[static HTML generation of the web application](https://ara.readthedocs.io/en/latest/usage.html#generating-a-static-html-version-of-the-web-application). - -0.12 was also the first time I did a voiced demo of the ARA web interface to highlight its different features: - -{{< youtube aQiN5wBXZ4g >}} -
-The amount of interest in ARA was growing from all over the place. -It was exciting, motivating but also a bit overwhelming. -This is about the time where I [reached out](http://lists.openstack.org/pipermail/openstack-dev/2017-March/114509.html) -to the OpenStack community to ask if they wanted to help developing ARA. - -This call for help brought a few contributions but not as much as I would have hoped. -The fact that ARA is not my full time job means I have to spend a lot my -personal time on the project but in the end, the user feedback is so rewarding -that I don't mind too much. - -That said, I have to say I am extremely grateful to my employer, Red Hat, and my -managers who don't mind me spending time on ARA. Red Hat is awesome. - -## An Ansible meetup talk and STICKERS ! - -**April 2017**, I was invited to speak at the [Ansible Montreal](https://www.meetup.com/Ansible-Montreal/events/238557821/) -meetup to talk about how Ansible users can extend Ansible with the help of plugins. - -I totally had to make ARA stickers to bring to the meetup ! - -{{< tweet 844173191627116545 >}} - -I did not bring enough stickers, though. It was a limited print and I wanted -to keep some for the next occasion. The stickers were gone in minutes ! - -The meetup was a great opportunity for me to highlight how ARA was taking advantage -of the plugin system in Ansible. - -{{< tweet 852318379406962688 >}} - -You can find the slides for that talk (*in french*) [here](http://redhat.slides.com/dmsimard/comment-personnaliser-et-etendre-ansible-module-actions-et-callbacks). - -## ARA 0.13: Where we are today - -**May 4th 2017**, there's lightsaber battles at the Red Hat Summit and [ARA 0.13 is released](https://dmsimard.com/2017/05/05/ara-ansible-run-analysis-0-13-is-out-and-its-awesome/). - -{{< tweet 860111867552944128 >}} - -ARA 0.13 improved on the foundation that 0.12 brought: bug fixes, new features, better performance -and a better and more streamlined UI. - -There were so many improvements in 0.13 that I had to create yet another video demo -because the demo from 0.12 felt outdated. - -{{< youtube k3i8VPCanGo >}} -
-The amount of interest in ARA with this new release is absolutely nuts. - -Lots of comments on [Reddit](https://www.reddit.com/r/ansible/comments/69gkpz/hi_ransible_a_new_version_of_ara_ansible_run/) and -so many Twitter mentions, likes and retweets that I had to create a filter for -all the email notifications... My phone wouldn't stop ringing! - -## The future: what's next - -I could have never predicted what would become of this idea I had a year ago and -with that in mind, I definitely can't predict what the next year holds for ARA. - -On the technical side of things, there are clear things we need to be doing to -further improve ARA. - -ARA needs full python 3 compatibility, we need to have a better split of the -components (web application, API, client, CLI) and things like that. -I'm not a frontend developer and I'm convinced a professional UX developer could -make the UI so much better. - -There's other very interesting opportunities to make the life of Ansible users -easier, not just with Ansible itself but through it's implementation in other -tools. - -There are [questions](https://www.reddit.com/r/ansible/comments/69gkpz/hi_ransible_a_new_version_of_ara_ansible_run/dh6tiyt/) -with the eventual Open Source release of [Ansible Tower](https://www.ansible.com/tower) -and how we might see Tower and ARA working together. - -ARA might also end up integrated in [some shape or form](https://review.openstack.org/#/c/444088/) -with [Zuul](http://status.openstack.org/zuul/), OpenStack's famous gate and pipelining -system. - -Some people have also been requesting ARA to be a native plugin in [Molecule](https://github.com/metacloud/molecule/issues/728). - -What's next ? Only time will tell ! - -Thanks to everyone that was involved with ARA so far, even as a user, your feedback -and questions have contributed to shaping ARA into what it is today. - -See you next year :) diff --git a/website/content/blog/2017-08-16_whats-coming-in-ara-1.0/index.md b/website/content/blog/2017-08-16_whats-coming-in-ara-1.0/index.md deleted file mode 100644 index 71bebf5..0000000 --- a/website/content/blog/2017-08-16_whats-coming-in-ara-1.0/index.md +++ /dev/null @@ -1,240 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development -tags: - - ansible - - openstack -date: 2017-08-16 -title: "What's coming in ARA 1.0 ?" -slug: whats-coming-in-ara-1.0 -type: post ---- - -Not long ago, I wrote that ARA: Ansible Run Analysis had it's first [birthday](https://dmsimard.com/2017/05/08/ara-is-one-year-old-a-look-back-at-the-past-year/). - -It was an important milestone and it was a great opportunity to reflect back on where -the project was coming from and think about what we needed to do in the future. - -Just for fun, let's look at what I had written back in May to summarize what -was probably coming: - -- ~~Python 3 compatibility~~ - - This is done and was shipped in [ARA 0.14](https://github.com/ansible-community/ara/releases/tag/0.14.0)! - It was a lot of work but ARA is now working and is fully tested against both python2.7 and python3.5. -- ~~An implementation in Zuul~~ - - This is in [progress](https://review.openstack.org/#/c/487853/) ! It's no longer a question of "if" but rather "how". - A lot of improvements in ARA 1.0 is geared towards making this integration easier (amongst other things) and we'll - have a lot of discussions about it at the upcoming [OpenStack project team gathering](https://www.openstack.org/ptg/). -- ARA and [Ansible Tower](https://www.ansible.com/tower) working together ? - - I'm excited to see the opportunities with Tower but it's not open source yet ! - We know it's coming soon and [latest estimates](https://www.reddit.com/r/ansible/comments/6jat04/my_review_of_ansiblefest_london_2017/djdrjgk/) - put us at a release within the next year. - -## ARA 1.0: An important version - -ARA 1.0 will be the largest and most significant release ever since the project -was created. - -For me, 1.0 is not just any number. I attach a lot of weight and importance to -it. - -First, it signals a very clear message that I feel ARA is battle tested and stable -enough for production use. -Users and deployments, both small and large, have proven me that ARA can be relied on. - -Second, it means that we are making backwards incompatible changes, especially -around the database schema: more on that later. - -And finally, we are completely re-writing the backend and it's logic, both of which -had been implemented in the very early days when prototyping ARA. -This new foundation will help ARA move forward faster and make integration -easier for users and systems. - -Development of ARA 1.0 is not finished yet. -You can follow the progress and the code reviews [here](https://review.openstack.org/#/q/project:openstack/ara+branch:feature/1.0) -or by hanging out in IRC in #ara on the freenode server. - -Here's some highlights about things that have already landed or will land in this next release. - -## What's coming: decoupling components and a revamped backend - -ARA will have lived until 1.0 on a backend implementation that was born out of the -prototype and proof of concept more than a year ago. It has served us well but it was -no longer appropriate for the scope of the project. - -One of the biggest flaws of the previous backend implementation was that the -different components were coupled and also imported and queried the database model directly. - -For example, when using ARA in a centralized fashion, this meant: - -- Installing the callback, module, CLI and Ansible on the web server(s) hosting the web application -- Installing the web application (and all it's dependencies) when only requiring the CLI, callback or modules -- Putting/sharing a username/password for a database connection string to send data back to a central database - -From a developer perspective, it also meant that we had no abstraction to the database and needed -to know and understand how the different components worked to introduce new features. -It also meant understanding how to query the database directly and code duplication -because every feature had their own database queries. - -The backend is being essentially re-written from scratch in order to leverage a -new python and REST API. -This layer of abstraction will make things easier to maintain and develop against. - -## What's coming: a python and REST API - -What is probably the biggest endeavour of the 1.0 release is the introduction of python and REST APIs. - -We are doing this for the project's own consumption in the first place to decouple the -components and re-write the backend. -However, I'm curious to see what users and developers might come up with ! - -Something that was important for me when designing the API was to keep ARA simple. -The introduction of an API should not mean that users suddenly have to run a web server in order to collect data. -It shouldn't also be a requirement when running the web application because the reports can be -generated to static HTML. - -Before 1.0, all you had to do to get started was to [install ARA](http://ara.readthedocs.io/en/latest/installation.html) and configure the [Ansible callback path](http://ara.readthedocs.io/en/latest/configuration.html). -This will not change and the default use case remains local and offline data collection with no additional -set up or configuration steps. - -The approach I am taking with the two APIs is to make them identical with the exact -same interface. In fact, the python API is simply an internal passthrough to the REST API -with no HTTP calls involved. This will make it easier to use, develop, test and maintain because of the -feature parity that is available by default. - -Here's some examples to show the APIs are the same -- these examples do and return the exact same thing. - -**For retrieving data**: - -``` -# Get the playbook with 'id' 1 over http -curl -H "Content-Type: application/json" -X GET /api/v1/playbooks/1 - -# Get the playbook with 'id' 1 with the python API -from ara.api.playbooks import PlaybookApi -playbook = PlaybookApi().get(id=1) -``` - -**For creating data**: - -``` -# Create a play in a playbook over http -curl -H "Content-Type: application/json" -X POST -d '{ "playbook_id": 1, "name": "Play name", "started": "1970-08-14T00:52:49.570031" }' /api/v1/plays/ - -# Create a play in a playbook with the python API -from ara.api.plays import PlayApi - -data = { - "playbook_id": 1, - "name": "Play name", - "started": "1970-08-14T00:52:49.570031" -} -play = PlayApi().post(data) -``` - -**For updating data**: - -``` -# Update a play in a playbook over http -curl -H "Content-Type: application/json" -X PATCH -d '{ "play_id": 1, "ended": "1970-08-14T00:52:49.570031" }' /api/v1/plays - -# Update a play in a playbook with the python API -from ara.api.plays import PlayApi - -data = { - "play_id": 1, - "ended": "1970-08-14T00:52:49.570031" -} -play = PlayApi().patch(data) -``` - -There might eventually be some "wrapper" methods for the Python API (i.e, "create_playbook") -but in the end, it will use the same API behind the scene. - -## What's coming: input and output drivers - -Currently, there is only one supported way to send data to ARA -- it's through the -Ansible callback. -However, some users have also requested different means of sending data to ARA. - -An example that was given is that events could be dropped on a message bus (ex: mqtt, rabbitmq) -instead of being sent directly to the API or database and then ARA could pick up and -process the events asynchronously. - -The input will be made modular and generic to allow the notion of "input drivers" so that -we can implement these new methods easily. - -For output, in a similar fashion, it's currently possible to export data from ARA to -static HTML, junit or subunit. - -There's also been interest in sending data to other backends such as graphite, elasticsearch or influxdb. - -The appeal in using ARA to export data to these backends instead of using different Ansible callbacks -that already do that comes from having the data aggregated, organized and indexed in one place and -then being able to export the data asynchronously. - -The current export methods will be folded back as "output drivers" and will use a common -interface to export data. We will be able to leverage this to make it easier to implement -other output drivers. - -## What's coming: database improvements and backwards incompatibility - -A significant amount of work has been done on the database model. -The magnitude of these changes have made me consider backwards incompatibility -for the first time since ARA introduced database upgrades in [ARA 0.10 from December 2016](https://github.com/ansible-community/ara/releases/tag/0.10.0). - -The decision to move forward without supporting migrations and upgrades was to make this work possible and drastically easier. - -I think the database layout is almost ready to go for 1.0, here's some examples of what has been done so far: - -- Primary keys and identifiers are now integers instead of UUIDs - (Thanks [Monty Taylor](http://lists.openstack.org/pipermail/openstack-dev/2017-April/115013.html) for the tip) -- The ``Stats`` and ``HostFacts`` tables were merged with the ``Hosts`` table -- The ``TaskResults`` table was renamed to ``Results`` because there's no other kind of results -- The ``Data`` table was renamed to ``Records`` because "Data" was an awful name and it better matches it's intended use case with the ``ara_record`` [module](http://ara.readthedocs.io/en/latest/usage.html#using-the-ara-record-module). -- Different fields have been removed or renamed to make their names more accurate and their meaning consistent throughout the application -- Some relationships between components were added, removed or changed - -## What's coming: new features - -All of the previous changes do not have much of an impact on end users. They are -all mostly under the hood and does not add much immediate value for users. - -Here's some features I would like to land in 1.0: - -- Add support for recording ad-hoc command execution (ex: ``ansible -m service -a "name=foo state=stopped``) -- Add support check/dry mode (``ansible-playbook --check``) -- Add support diff mode (``ansible-playbook --diff``) -- Add support for grouping or searching playbooks - -**Edit**: The following features will also be landing in 1.0! - -- Inventory, vars, host_vars and group_vars files will be saved with the report -- Role handlers, meta and defaults files will also be saved with the report -- Hosts will now display which groups they are a member of -- Playbook reports will be able to be named for displaying in the UI. For example - you will be able to name a playbook "deploy mysql" and it will display that instead - of the playbook file name ".yml". - -## When is all of that coming ? - -So, when is all of that coming, you ask ? - -Work is progressing very well on the API implementation and there should not be -too many changes remaining to do in the database model. - -{{< tweet 897641159681593344 >}} - -There has not been any work so far to refactor the backend to use the API and there -are also some features I'd like to land. - -With all things considered, I would like to be able to release an alpha or a beta -release sometime during September. I'll reach out to ask users to test it out :) - -If you'd like to follow the progress, I sometime post updates on Twitter, you can -follow me there as [@dmsimard](https://twitter.com/dmsimard). - -Feel free to come hang around on IRC to chat about ARA as well ! -You'll find users and myself in #ara on the Freenode server. - diff --git a/website/content/blog/2017-08-27_new-ways-of-reaching-the-ara-community/index.md b/website/content/blog/2017-08-27_new-ways-of-reaching-the-ara-community/index.md deleted file mode 100644 index 85c55ea..0000000 --- a/website/content/blog/2017-08-27_new-ways-of-reaching-the-ara-community/index.md +++ /dev/null @@ -1,63 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - community -tags: - - ansible - - openstack - - irc - - slack - - twitter -date: 2017-08-27 -title: "New ways of reaching the ARA community" -slug: new-ways-of-reaching-the-ara-community -type: post ---- - -More and more users requested other ways of reaching the ARA community and I've -finally given in ! - -Until now, the only way of getting in touch was through IRC and I understand -that, in 2017, IRC is not for everyone. - -## Seamless communication across IRC, Slack ~~and Discord~~ - -That's right, you can now reach us through Slack and Discord. -Both are linked to IRC so messages sent to one will be relayed automatically -to the others. - -- To join us on Slack, use this [invitation](https://join.slack.com/t/ara-community/shared_invite/MjMxNzI4ODAxMDQxLTE1MDM4MDEzMTEtNzU1NTUwMTcyOQ) -- ~~To join us on Discord, use this [invitation](https://discord.gg/z2SGdc9)~~ (With over 30 people in the ARA slack and none in the Discord, we discontinued it.) - -To join us on **IRC**, you'll find us in the **#ara** channel of the **freenode** server. -A web chat application to connect to IRC is available [here](https://webchat.freenode.net/). - -### On Reddit - -When I'm not discussing Ansible or answering questions about it in -[/r/ansible](https://www.reddit.com/r/ansible/), you might come across noteworthy -announcements about ARA that I post there. - -There's a lot of discussion around ARA on /r/ansible and I will never get used -to users recommending other users that they use ARA, it's pretty awesome. - -### On Twitter (new!) - -You might have been following me on Twitter ([@dmsimard](https://twitter.com/dmsimard)) -to keep up with updates I post about ARA's developments or new announcements. - -That's cool but in case you'd like a curated feed that contains only worthwhile -updates about things relevant to Ansible and ARA, the new account to follow -is [@ara_community](https://twitter.com/ara_community). - -{{< tweet 901820543539798016 >}} - -## Other things - -No, I'm not going to stand up a WhatsApp, a Snapchat, an Instagram and a Facebook -page :) - -This is the best you've got for now ! - -Thanks for reading and see you online ! - diff --git a/website/content/blog/2017-11-22_status-update-ara-1.0/1.0-diff.png b/website/content/blog/2017-11-22_status-update-ara-1.0/1.0-diff.png deleted file mode 100644 index e63cc9d..0000000 Binary files a/website/content/blog/2017-11-22_status-update-ara-1.0/1.0-diff.png and /dev/null differ diff --git a/website/content/blog/2017-11-22_status-update-ara-1.0/1.0-files.png b/website/content/blog/2017-11-22_status-update-ara-1.0/1.0-files.png deleted file mode 100644 index 2d1d9a0..0000000 Binary files a/website/content/blog/2017-11-22_status-update-ara-1.0/1.0-files.png and /dev/null differ diff --git a/website/content/blog/2017-11-22_status-update-ara-1.0/db-layout-0.x.png b/website/content/blog/2017-11-22_status-update-ara-1.0/db-layout-0.x.png deleted file mode 100644 index c09506b..0000000 Binary files a/website/content/blog/2017-11-22_status-update-ara-1.0/db-layout-0.x.png and /dev/null differ diff --git a/website/content/blog/2017-11-22_status-update-ara-1.0/db-layout-1.x.png b/website/content/blog/2017-11-22_status-update-ara-1.0/db-layout-1.x.png deleted file mode 100644 index 6aec0eb..0000000 Binary files a/website/content/blog/2017-11-22_status-update-ara-1.0/db-layout-1.x.png and /dev/null differ diff --git a/website/content/blog/2017-11-22_status-update-ara-1.0/git-network.png b/website/content/blog/2017-11-22_status-update-ara-1.0/git-network.png deleted file mode 100644 index 0cdd612..0000000 Binary files a/website/content/blog/2017-11-22_status-update-ara-1.0/git-network.png and /dev/null differ diff --git a/website/content/blog/2017-11-22_status-update-ara-1.0/index.md b/website/content/blog/2017-11-22_status-update-ara-1.0/index.md deleted file mode 100644 index 9c89823..0000000 --- a/website/content/blog/2017-11-22_status-update-ara-1.0/index.md +++ /dev/null @@ -1,394 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development -tags: - - ansible - - openstack -date: 2017-11-22 -title: "Status update: ARA 1.0" -slug: status-update-ara-1.0 -type: post ---- - -Back in [August](https://dmsimard.com/2017/08/16/whats-coming-in-ara-1.0/), I -posted about what was the roadmap for ARA 1.0 and why it was a very important -milestone for the project. - -We're now almost in December and I said there would likely be a beta version -out by September. "What's going on ?", one might ask... A fair question. - -There's definitely been progress and I could've been doing a better job at -communicating updates other than the [tweet](https://twitter.com/ara_community) -from time to time. It's time for a lengthy status update ! - -If you don't know what's ARA yet, you should be excited and read about what's -coming. Looking at a [video demo](https://www.youtube.com/watch?v=k3i8VPCanGo&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty&t=12s&index=6) -will give you an idea of what the project can already do today. - -This is a companion blog post to a talk given at the -[Ansible Montreal](https://www.meetup.com/Ansible-Montreal/) meetup, you can see -the slides (in french) [here](https://redhat.slides.com/dmsimard/ara-10-update). - -If you'd like skip ahead and read the technical details about what's done -and what's not, feel free to scroll down a bit ! - -How is 1.0 being developed ? -============================ - -Time goes quickly when we're having fun, doesn't it ? 1.0 has been in progress -for a good four months already. - -1.0 is being developed 100% in the open. If you're on [Slack or IRC](https://dmsimard.com/2017/08/27/new-ways-of-reaching-the-ara-community/) -you might have seen one of the notifications that is sent every time there is -a new patch submitted for code review. - -![slack notification](slack-notification.png) - -You can also see all the patches as well -as the results for their unit and integration tests live on -[Gerrit](https://review.openstack.org/#/q/project:openstack/ara+branch:feature/1.0). - -There's currently two git branches, -[master](https://github.com/ansible-community/ara/tree/master) and -[feature/1.0](https://github.com/ansible-community/ara/tree/feature/1.0). This allows -the development of version 1.0 to move forward without impacting the master -branch from which the 0.15.x series of releases are currently tagged from. - -These branches have diverged a LONG time ago and are VERY different by now. - -![git network](git-network.png) - -Keeping these two separate branches is great, but it also means that bugfixes -done in a branch that is applicable to the other branch should be backported. - -Sometimes, this is easy, other times... it is not. Ever heard of merge -conflicts ? Maybe some code is gone, perhaps it has moved elsewhere or has -changed dramatically. - -While I am trying to keep the master branch frozen for new features in order to -reduce the maintenance cost associated with this model, there are some things -that absolutely need to go through.. for example Ansible 2.4 support. - -It was unrealistic to delay the support for Ansible 2.4 until ARA 1.0. - -Ansible 2.4 changed (and [broke](https://github.com/ansible/ansible/pull/31200)) -some internal methods ARA was relying on and there was no way to support it -properly unless we adapted to it. - -This is compounded by the fact that ARA supports all stable releases of Ansible -in both Python 2 and Python 3 which have not reached their -[end of life](http://docs.ansible.com/ansible/latest/release_and_maintenance.html) -yet. - -Hopefully we can ship this awesome 1.0 release soon so we no longer have to -maintain these two branches in parallel ! - -Why is this taking so long ? -============================ - -I'll be honest here: like probably many open source software maintainers, ARA -isn't my full time job. I'm not sure I would enjoy working on ARA full time -anyway. I enjoy system administration and architecture way too much. - -Don't get me wrong, I'm quite passionate about ARA but it's ultimately a project -I created to make my job easier. I'm not making money with ARA, the fact that -it is useful to other people is super rewarding for me and that's enough. - -I consider myself very privileged to be working at Red Hat which values -innovation driven by open source projects. While Red Hat grants me a certain -amount of freedom to work on ARA at my discretion, I recognize that my role is -very important for the [RDO](https://www.rdoproject.org/) and -[OpenStack](https://www.openstack.org/) communities and that's why I'll always -priorize spending the vast majority of my time there. - -So far, ARA has been largely developed on my own time and I had to take a step -back for a bit. - -I like to say that work isn't work when you're having fun -- I still believe -that, but I was just "working" too much for a while there. - -I'll always be happy to receive and review patches from contributors in ARA. -If you'd like to help, you can get started by looking at the -[contribution documentation](http://ara.readthedocs.io/en/latest/contributing.html). - -So when is it shipping ? -======================== - -I'd rather not try to plan for a date again like I did when I mentioned -September for a beta. It sucks because I failed to deliver and it sucks because -I am not managing the expectations of the users. - -1.0 will ship when it is ready. -However, the primary purpose of 1.0 is to be the sandbox where we allow -ourselves to break all backwards compatibility. - -If we're done merging all the features that we have an expectation would have -an impact on backwards (in)compatibility, it means we're nearing release. - -I'm not going to delay the release indefinitely, things without impact that -aren't ready for 1.0 will ship later, that's all. - -I'll stay vague on purpose: in the first half of 2018 ? If you end up having it -sooner, you can celebrate ! - -Let's move on to the status update of things. - -The API is done (kind of) -======================== - -I took an iterative approach and there was definitely some trial and error -involved. - -I decided to write a first version of the API by itself, only being validated -through purpose-built functional unit tests. When I started integrating it in -the different components of ARA.. there was some drawbacks. Me, as a user of -the API, did not feel comfortable leaving things as is. - -The experiment was still worthwhile, though. It allowed me to "bootstrap" the -API from end to end and have an endpoint represent each component. The big -picture was there and we only needed to tweak things. - -Some problems were [performance](https://github.com/flask-restful/flask-restful/issues/612) -related, others were around needing to do too many API calls to get one -particular bit of information and so on. - -{{< tweet 927199009147678721 >}} - -I drew some inspiration from the [Ansible AWX](https://github.com/ansible/awx) -API structure as well as from a Dropbox project which also uses Flask and -Flask-Restful, [Changes](https://github.com/dropbox/changes).. and went back to -the drawing board. - -Today, the API itself is mostly finished. - -The default callback, the [ara_record and ara_read modules](http://ara.readthedocs.io/en/latest/usage.html#using-the-ara-record-module) -have been re-written to consume the API rather than doing direct SQL queries. - -The next step is to refactor the [command line interface](http://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli) -as well as the web application interface do to the same thing. - -Using an API without compromising the user experience -===================================================== - -Creating an API for an application that wasn't designed "API first" and then -re-writing it to consume that API was fun and challenging. - -It was a learning experience and I'm pretty happy I was able to keep the result in line with the -[project core values](http://ara.readthedocs.io/en/latest/manifesto.html): - -- Simplicity is fundamental -- Do one thing and do it well -- Empower users to get their work done -- Don’t require users to change their workflows -- De-centralized, offline and standalone by default - -**De-centralized, offline and standalone by default** ? With an API ? Yes. - -I might end up explaining this implementation in it's own blog post eventually -but through a clever use of [Flask](http://flask.pocoo.org/), the built-in API -client in ARA can either communicate with an API endpoint over HTTP (REST) or -offline, locally through internal Flask calls. - -**Don't require users to change their workflows**: This means ARA continues -to ship without a requirement to run a daemon, a web server or an API server. - -``pip install ara``, set up the [Ansible callback](http://ara.readthedocs.io/en/latest/configuration.html#ansible), -run your Ansible playbook as you are used to and that's it. That's what it has -been since the project was created and I promise it will stay that way. - -**Simplicity is fundamental** is a very important one for me. Adding an API -shouldn't make things complicated for the users... or the developers. - -In fact, most of the work in ARA 1.0 isn't user-facing. It's a re-write of the -backend to move from the foundation the prototype had been built on more than -a year ago. The API will allow for faster development, make maintenance -easier and allow people to programmatically query ARA. - -Again, simplicity is a core theme here, even in how the built-in API client -was designed to work. - -Here's some quick examples that leverages the client: - -``` -# Import the "Playbook" API client -from ara.api.v1.playbooks import PlaybookApi - -# Get a list of all playbooks -response, playbooks = PlaybookApi().get() - -# Get the details of a specific playbook -response, playbook = PlaybookApi().get(id=1) - -# Create a new playbook in the database -response, playbook = PlaybookApi().post( - ansible_version="2.4.1.0", - path="/home/user/ansible/playbooks/playbook.yml", - parameters={ - "inventory": "/etc/ansible/inventory", - "become": True, - # ... - } - -# Import the "Result" API client -from ara.api.v1.results import ResultApi - -# Get all failed results for a specific playbook -response, results = ResultApi().get(playbook_id=1, - failed=True) -``` - -I mentioned earlier that the API had two ways of being exposed, one that was -offline (internal, no network or http involved) and one online (HTTP REST). - -What you're not seeing in the background for the examples above is that no -matter if we're relying on the internal API or the HTTP REST API, the code -*and* the return values don't change. -A new configuration parameter, ``ARA_API_CLIENT`` which defaults to ``offline``, -can be set by users to ``online`` with a configurable API endpoint. - -When using ``offline`` as the API client, you're limited to posting and querying -data on the host you are running Ansible from. This is perfect for most use -cases while keeping latency and overhead to a minimum. - -When using ``online`` as the API client, you can host the web application and -the API endpoint elsewhere and aggregate data from multiple machines running -Ansible to a single location without having to share SQL database credentials. - -Saving more files in ARA -======================== - -ARA currently saves playbook and role task files. That's pretty cool, but -what if we could also save group_vars, host_vars, defaults, meta, handlers -and other files ? - -Consider it done in 1.0: - -![files](1.0-files.png) - -There will be a feature in order to selectively exclude files from being saved -if you happen to store sensitive information in them and would rather leave -those out. - -An improved database schema -=========================== - -The main reason for breaking backwards compatibility and not providing an -upgrade path to 1.0 is the database schema. - -**Before 1.0** - -![database before](db-layout-0.x.png) - -**1.0 (currently)** - -![database after](db-layout-1.x.png) - -It would have been quite a burden to handle SQL migrations for every change -that landed. Tables have been deleted, some fields have been renamed or even -moved to other tables. - -Another issue is that the database model did not provide a -[metadata description](http://docs.sqlalchemy.org/en/latest/core/metadata.html). - -This made it outright impossible to properly upgrade models on certain engines -such as [sqlite](https://github.com/miguelgrinberg/Flask-Migrate/issues/61) -because indexes and foreign keys may have been created without a name that we -can refer back to. - -All in all, 1.0 is meant to be a clean slate with hopefully no "unfixable" -mistakes from a database perspective. If you're an expert at database models -and sqlalchemy, feel free to reach out, I'd love your input before this -version ships ! - -Still on the to-do list -======================= - -There's been a lot of work on 1.0 already, but that doesn't mean we're done -yet. - -![git network](1.0-diff.png) - -Here's a high level overview of the big items still on the to-do list: - -- **Input drivers**: The default callback will be folded back as an - input driver in a structure that will easily allow us to implement other means - of sending data to ARA. For example, we might want to send and process data - asynchronously through a message bus such as RabbitMQ, MQTT or a buffer like - Redis. - -- **Output drivers**: The current ways of exporting data from ARA like HTML, - junitXML and Subunit will be converted to output drivers in a generic - structure in order to make it straightforward to implement new ways of - exporting data. For example, this would allow us to create new drivers to - send task data from ARA to InfluxDB, Graphite or Logstash which could be a - nice addition to your Grafana or Kibana dashboards. - -- **Decoupling components**: Splitting ARA components into subpackages would - make it easier to install only what you need on the machines you want. It's - silly to install the web application dependencies if you only want to use - the callback. ``pip install ara`` should still reel everything in, but - if you need just the callback, for example, there should be a package that - does just that. - -- **Playbook labels**: These would allow users to label their playbooks in - order to display custom names in the web interface. For instance, you might - want your playbook to be named "Install MySQL" instead of - "/home/user/playbooks/playbook.yml". That's fair, let's do that. - -- **Playbook logical groups**: It would be useful to be able to group playbooks - together somehow. Pretend your deployment is really in fact 9 different - playbooks, you might want to group them together under a name like - "production deployment" so that you know you ran these together and look at - them as one logical report instead of 9 different reports. Or maybe those 9 - playbooks are actually 3 for the development environment, 3 for the test - environment and 3 for the production -- you will be able to group them as - such. - -On the wishlist -=============== - -I don't want to delay the release of ARA 1.0 for too long, but there are some -features I know would be very cool to add but are either complicated or would -delay the release further. - -What's probably on the top of that list are persistent hosts. Ironically, the -prototype of ARA more than a year ago shipped with persistent hosts. What are -they ? - -Right now, Ansible does not provide a way to uniquely identify each host and -this is a problem. - -In puppet, for example, there is the concept of certificate signature between -a puppet agent and a puppet master. If a puppet agent shows up with the same -hostname as an existing machine, the certificate won't match and there will be -an error. - -In the early days of ARA, we ran with the assumption that hosts were unique -across playbook runs. This means that if you had a server called "webserver" -and you ran 10 playbooks with that host, we could tally the results for that -particular host. You can see what it looked like in a very old video -[demo](https://www.youtube.com/watch?v=k3qtgSFzAHI&index=2&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty). - -The reason why we took that out is because your host, "webserver", can be -one server today and a completely different server tomorrow: Ansible won't -know the difference. -This kind of assumption ended up causing problems and we decided it was wise -to revisit this later. - -I have some ideas around how we could implement this for ARA but the -implementation of persistent hosts is a lot of work that would have repercussions -on the database schema, the backend, the API, the CLI, the web interface.. -everything. - -What's on your wishlist ? -========================= - -Do you have any items on your wishlist for ARA ? Let me know ! - -You can get in touch through [Twitter](https://twitter.com/dmsimard), [IRC -and Slack](https://dmsimard.com/2017/08/27/new-ways-of-reaching-the-ara-community/) -and you'll also find me lurking in [/r/ansible](https://www.reddit.com/r/ansible/). - -Thanks for reading and see you around ! - diff --git a/website/content/blog/2017-11-22_status-update-ara-1.0/slack-notification.png b/website/content/blog/2017-11-22_status-update-ara-1.0/slack-notification.png deleted file mode 100644 index 7db452a..0000000 Binary files a/website/content/blog/2017-11-22_status-update-ara-1.0/slack-notification.png and /dev/null differ diff --git a/website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/ara-reports.png b/website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/ara-reports.png deleted file mode 100644 index a99df45..0000000 Binary files a/website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/ara-reports.png and /dev/null differ diff --git a/website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/gnu-is-not-unix.png b/website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/gnu-is-not-unix.png deleted file mode 100644 index bc8c254..0000000 Binary files a/website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/gnu-is-not-unix.png and /dev/null differ diff --git a/website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/index.md b/website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/index.md deleted file mode 100644 index 340fefa..0000000 --- a/website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/index.md +++ /dev/null @@ -1,100 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development -tags: - - ansible - - openstack -date: 2018-02-23 -title: "Rebranding Ansible Run Analysis to ARA Records Ansible" -slug: rebranding-ansible-run-analysis-to-ara-records-ansible -type: post ---- - -So I got an idea recently... Let's rebrand Ansible Run Analysis to ARA records -Ansible. - -![review](gnu-is-not-unix.png) - -[![asciicast](https://asciinema.org/a/164917.png)](https://asciinema.org/a/164917) - -If you'd like to review and comment on the code change, you can do -so here: [https://review.openstack.org/#/c/547245/](https://review.openstack.org/#/c/547245/). - -# Why ? - -I watched the last season of [Sillicon Valley](https://www.hbo.com/silicon-valley) recently. -The series, while exaggerated, provides a humorous look at the world of startups. - -I don't have any plans on creating a startup but I love that it makes you think -about things like needing a clever name or how you would do a proper "elevator" -pitch to get funding. - -I'll take this opportunity to practice for fun. - -ARA Records Ansible is Another Recursive Acronym. - -{{< tweet 966896714274631685 >}} - -## The "elevator" pitch - -ARA Records Ansible is a project from the OpenStack community that makes it -easier to understand and troubleshoot your Ansible roles and playbooks. - -- Do you run ``ansible-playbook -vv`` by default ? -- 50 000 lines of console output to look at ? -- How do you tell what changed ? Where ? Which task failed ? -- What parameters did you use in that ansible-playbook command yesterday ? -- What was the value of your host facts last week ? -- What code or Ansible version did your playbook run with a month ago ? - -With ARA, you don't need to look at your 50 000 lines. -ARA tells you everything about your entire playbook execution history through an -intuitive self-hosted web interface, command line interface and soon, a full REST API. - -![ara reports](ara-reports.png) - -## The "we're stuck in the elevator" pitch - -ARA is a native [Ansible callback plugin](http://docs.ansible.com/ansible/devel/plugins/callback.html) -that transparently save everything about your playbook executions. - -No matter if you're running Ansible from your laptop or from a server, this data -ends up in a database -- offline to sqlite by default. -If you prefer MySQL or PostgreSQL, that's cool too. - -ARA makes your playbook execution history database available through: - -- A web dashboard: Python [flask](http://flask.pocoo.org/) and [Patternfly](http://www.patternfly.org/) CSS (pretty!) -- A CLI client: Python [cliff](https://github.com/openstack/cliff) -- same client interface as the "[openstack](https://docs.openstack.org/python-openstackclient/latest/)" command (json or yaml output, etc.!) -- An API: Available in version 1.0 ([work in progress](https://dmsimard.com/categories/ara/)!) - -{{< tweet 920666123674075137 >}} - -# Learn more about the project - -One of ARA's core features and values is its simplicity. -You can read about the other core values of ARA in the [project manifesto](https://ara.readthedocs.io/en/latest/manifesto.html). - -To learn more about ARA, you can find the [open source code on GitHub](https://github.com/ansible-community/ara) or -watch this short video demo on YouTube: - -{{< youtube k3i8VPCanGo >}} - -# Chat with us - -If you want to chat with us, we're on IRC in the #ara channel on [Freenode](https://webchat.freenode.net/) -and on [Slack](https://join.slack.com/t/ara-community/shared_invite/MjMxNzI4ODAxMDQxLTE1MDM4MDEzMTEtNzU1NTUwMTcyOQ). - -The two are bridged with [slack-irc](https://github.com/ekmartin/slack-irc) so -everyone can talk together. It's pretty awesome. - -# Stay up to date - -If you'd like to stay up to date on what's coming in ARA 1.0, you can follow -the project on [Twitter: @ARecordsAnsible](https://twitter.com/ARecordsAnsible) -and [this blog](https://dmsimard.com/categories/ara/). - -Thanks for reading, feel free to let me know if you have any questions ! - -- [@dmsimard](https://twitter.com/dmsimard) diff --git a/website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/index.md b/website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/index.md deleted file mode 100644 index e4e981d..0000000 --- a/website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/index.md +++ /dev/null @@ -1,118 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - community -tags: - - ansible - - openstack - - zuul -date: 2018-04-09 -title: "Scaling ARA to a million Ansible playbooks a month" -slug: scaling-ara-to-a-million-ansible-playbooks-a-month -type: post ---- - -The [OpenStack](https://www.openstack.org/) community runs over 300 000 CI jobs -with [Ansible](https://www.ansible.com/) every month with the help of the -awesome [Zuul](https://zuul-ci.org/). - -![Zuul Pipelines](zuul-pipelines.png) - -It even provides ARA reports for ARA's [integration test jobs](https://github.com/ansible-community/ara#contributing-testing-issues-and-bugs) -in a sort-of nested way. Zuul's Ansible ends up installing Ansible and ARA. -It makes my brain hurt sometimes... but in an awesome way. - -![Zuul ARA Report](zuul-ci.png) - -As a core contributor of the infrastructure team there, I get to witness issues -and get a lot of feedback directly from the users. - -[Static HTML report generation](https://ara.readthedocs.io/en/latest/usage.html#generating-a-static-html-version-of-the-web-application) -in ARA is simple but didn't scale very well for us. One day, I was randomly -chatting with Ian Wienand and he pointed out an -[attempt](https://review.openstack.org/#/c/120317/) at a WSGI middleware that -would serve extracted logs. - -That inspired me to write something similar but for dynamically loading ARA -sqlite databases instead... This resulted in an awesome feature that I had not -yet taken the time to explain very well... until now. - -*Excerpt from the [documentation](https://ara.readthedocs.io/en/latest/advanced.html#serving-ara-sqlite-databases-over-http)* - -> To put this use case into perspective, it was “benchmarked” against a single job from the [OpenStack-Ansible](https://github.com/openstack/openstack-ansible) project: -> -> - 4 playbooks -> - 4647 tasks -> - 4760 results -> - 53 hosts, of which 39 had gathered host facts -> - 416 saved files -> -> Generating a static report from that database takes ~1min30s on an average machine. -> The result contains 5321 files and 5243 directories for an aggregate size of 63MB (or 27MB recursively gzipped). -> -> This middleware allows you to host the exact same report on your web server just by storing the sqlite database which is just one file and weighs 5.6MB. -> - -This middleware can be useful if you're not interested in aggregating data in -a central database server like MySQL or PostgreSQL. - -The OpenStack CI use case is decentralized: each of the >300 000 Zuul CI jobs -have their own sqlite database uploaded as part of the log and artifact collection. - -There's a lot of benefits of doing things this way: - -- There's no network latency to a remote database server: the first bottleneck is your local disk speed. - - Even if it's a 5ms road trip, this adds up over hundreds of hosts and thousands of tasks. - - Oh, and contrary to popular belief, [sqlite is pretty damn fast](https://sqlite.org/speed.html). -- There's no risk of a network interruption or central database server crash which would make ARA (and your sysadmins) panic. -- Instead of one large database with lots of rows, you have more databases ("shards") with fewer rows. -- Instead of generating thousands of files and directories, you're dealing with one small sqlite file. -- There's no database cluster to maintain, just standard file servers with a web server in front. - -Another benefit is that you can easily have as many individual reports as -you'd like, all you have to do is to configure ARA to use a custom database -location. - -When I announced that we'd be switching to the sqlite middleware on -[openstack-dev](http://lists.openstack.org/pipermail/openstack-dev/2018-March/128902.html), -I mentioned that projects could leverage this within their jobs and -OpenStack-Ansible was the first to take a stab at it: -[https://review.openstack.org/#/c/557921/](https://review.openstack.org/#/c/557921/). - -Their job's logs now look like this: - -``` -ara-report/ansible.sqlite # ARA report for this Zuul job -logs/ # Job's logs -└── ara-report/ # ARA report for this OpenStack-Ansible deployment - └── ansible.sqlite # Database for this OpenStack-Ansible deployment -``` - -The performance improvements for the OpenStack community at large are -significant. - -Even if we're spending 1 minute generating and transferring thousands of HTML -files... That's >300 000 minutes worth of compute that could be spent running -other jobs. - -How expensive are 300 000 minutes (or 208 days!) of compute ? -What about bandwidth and storage ? - -## Unfreezing ARA's stable release for development - -The latest version of ARA is currently 0.14.6 and ARA was more or less in -feature-freeze mode while all the work was focused on the next major release, -"[1.0](https://dmsimard.com/2017/11/22/status-update-ara-1.0/)". - -However, there is a growing amount of large scale users (me included!) that are -really pushing the current limitations of ARA and 1.0 (or 2.0!) won't be ready -for a while still. - -I couldn't afford to leave performance issues and memory leaks ruin the -experience of a tool that would otherwise be very useful to them. - -These improvement opportunities have convinced me that there will be a 0.15.0 -release for ARA. - -Stay tuned for the 0.15.0 release notes and another update about 2.0 in the -near future :) diff --git a/website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/zuul-ci.png b/website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/zuul-ci.png deleted file mode 100644 index d4bb9a4..0000000 Binary files a/website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/zuul-ci.png and /dev/null differ diff --git a/website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/zuul-pipelines.png b/website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/zuul-pipelines.png deleted file mode 100644 index b842811..0000000 Binary files a/website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/zuul-pipelines.png and /dev/null differ diff --git a/website/content/blog/2018-05-03_ara-records-ansible-0.15-has-been-released/index.md b/website/content/blog/2018-05-03_ara-records-ansible-0.15-has-been-released/index.md deleted file mode 100644 index 9a0ba0a..0000000 --- a/website/content/blog/2018-05-03_ara-records-ansible-0.15-has-been-released/index.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development - - changelog -tags: - - ansible - - openstack -date: 2018-05-03 -title: "ARA Records Ansible 0.15 has been released" -slug: ara-records-ansible-0.15-has-been-released -type: post ---- - -I was recently writing that ARA was open to limited development for the stable -release in order to improve the performance for larger scale users. - -This limited development is the result of this 0.15.0 release. - -{{< tweet 986675784478752770 >}} - -# Changelog for ARA Records Ansible 0.15.0 - -The following are highlights from the 0.15.0 release. -The full list of changes between 0.14.6 and 0.15.0 are available on [GitHub](https://github.com/ansible-community/ara/compare/0.14.6...0.15.0). - -- ARA: Ansible Run Analysis has been [rebranded](https://dmsimard.com/2018/02/23/rebranding-ansible-run-analysis-to-ara-records-ansible/) - to ARA Records Ansible (Another Recursive Acronym) -- Significant improvements to memory usage and performance when running ARA as - a WSGI application with ``ara-wsgi`` or ``ara-wsgi-sqlite``. -- Resolved an issue where the ``ara-wsgi-sqlite`` middleware could serve a - cached report instead of the requested one -- Added support for configuring the ``SQLALCHEMY_POOL_SIZE``, - ``SQLALCHEMY_POOL_TIMEOUT`` and ``SQLALCHEMY_POOL_RECYCLE`` parameters. - See [configuration documentation](https://ara.readthedocs.io/en/latest/configuration.html#parameters-and-their-defaults) for more information. -- Logging was fixed and improved to provide better insight when in DEBUG level. -- Vastly improved the default [logging configuration](https://github.com/ansible-community/ara/blob/3c91da40871e50fa2854231d54f298ed996d1da6/ara/config/logger.py#L27-L78). - ARA will create a default logging configuration file in ``~/.ara/logging.yml`` that you can customize, if need be. - Deleting this file will make ARA create a new one with updated defaults. -- Added python modules to help configure Ansible to use ARA, for example, - ``python -m ara.setup.callback_plugins`` will print the path to ARA's callback plugins. - You can find more examples in the [configuration documentation](https://ara.readthedocs.io/en/latest/configuration.html). -- Implemented a workaround for fixing a race condition where an ``ansible-playbook`` command - may be interrupted after the playbook was recorded in the database but before playbook file was saved. -- Flask 0.12.3 was [blacklisted](https://github.com/ansible-community/ara/commit/87272840bfc8b4c5db10593e47884e33a0f4af40) - from ARA's requirements, this was a broken release. -- The ARA CLI can now be called with "python -m ara". This can be useful if you - need to specify a specific python interpreter, for example. -- Updated and improved integration tests across different operating systems, - python2 and python3 with different versions of Ansible. The full test matrix - is available in the [README](https://github.com/ansible-community/ara#contributing-testing-issues-and-bugs). - -# Known issue: memory usage - -Please note that there is a known issue regarding high memory usage when -browsing reports in the web application for **very large** reports. - -For the time being, we have put mitigations in place in order to prevent very -high memory usage but this might be insufficient for large scale users. - -{{< tweet 984265830052651008 >}} - -If you'd like to help and contribute to resolve this, please reach out. - -# A disclaimer - -As with previous releases of the 0.x series, this new version comes with a -warning that we are not currently planning to provide backwards/forwards -compatibility with the next major release of ARA, 1.0. - -In practice, this means that we are trying to focus all of our current -contribution efforts towards making 1.0 as great as possible without burdening -our limited development resources with keeping backwards and forwards -compatibility. -This means your existing databases will not be upgradable to the new version so -you will need to start from a new database. - -One thing you don't have to worry about is that we are serious about keeping -the project as simple as possible, in line with the project's [manifesto and core values](https://ara.readthedocs.io/en/latest/manifesto.html). -ARA 1.0 will work the same way as it did before: ``pip install ara``, ``export ANSIBLE_CALLBACK_PLUGINS=$ara_location/plugins/callbacks`` and you're good to go. - -1.0 will provide additional features such as a revamped database, a backend -rewritten from scratch, an improved frontend and a full REST API. - -You can look at some of the work we have been doing for this new version [here](https://dmsimard.com/categories/ara/). - -There are a lot exciting opportunities where you can contribute to the development -of 1.0, please reach out to me (dmsimard) if you're interested in helping ! - -You can come chat with us on [IRC or on Slack](https://github.com/ansible-community/ara#contributing-testing-issues-and-bugs). diff --git a/website/content/blog/2018-10-08_ansiblefest-2018-community-project-highlights/community-projects-ansiblefest.jpg b/website/content/blog/2018-10-08_ansiblefest-2018-community-project-highlights/community-projects-ansiblefest.jpg deleted file mode 100644 index 4aea15b..0000000 Binary files a/website/content/blog/2018-10-08_ansiblefest-2018-community-project-highlights/community-projects-ansiblefest.jpg and /dev/null differ diff --git a/website/content/blog/2018-10-08_ansiblefest-2018-community-project-highlights/index.md b/website/content/blog/2018-10-08_ansiblefest-2018-community-project-highlights/index.md deleted file mode 100644 index 8e71f0f..0000000 --- a/website/content/blog/2018-10-08_ansiblefest-2018-community-project-highlights/index.md +++ /dev/null @@ -1,120 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - community -tags: - - ansible - - ansiblefest - - openstack - - zuul - - molecule - - ansible-lint -date: 2018-10-08 -title: "AnsibleFest 2018: Community project highlights" -slug: ansiblefest-2018-community-project-highlights -type: post ---- - -With two days of [AnsibleFest](https://www.ansible.com/ansiblefest) instead of -one this time around, we had 100% more time to talk about Ansible things ! - -I got to attend great sessions, learn a bunch of things, chat and exchange -war stories about Ansible, ARA, Zuul, Tower and many other things. - -It was awesome and I wanted to take the time to share a bit about some of the -great Ansible community projects that were featured during the event. - -At one point during the keynotes, the Ansible community lead, [Robyn Bergeron](https://twitter.com/robynbergeron), -talked about the community around Ansible with this slide: - -![Ansible Community Projects](community-projects-ansiblefest.jpg) - -It's possible this was the first time you were hearing about these projects. -Let me tell you about how awesome they are. - -## ansible-lint and molecule - -Before AnsibleFest, Ansible announced their intention to [adopt the ansible-lint and Molecule projects](https://www.reddit.com/r/ansible/comments/9j4de4/ansible_to_adopt_molecule_and_ansiblelint_projects/) -as part of a commitment to improve [Ansible Galaxy](https://galaxy.ansible.com/). - -{{< tweet 1047535123254665216 >}} - -[ansible-lint](https://github.com/willthames/ansible-lint) helps you write consistent Ansible code for your roles and playbooks while adhering to best practices. -I think the first project I really started using ansible-lint on was on WeIRDO, a CI framework, as far back as [2015](https://github.com/rdo-infra/weirdo/commit/839b8a2e022b6a6f675dc9dcd40b6cf334a60741#diff-b91f3d5bd63fcd17221b267e851608e8) ! - -It's simple and it just works: - -``` -$ ansible-lint examples/example.yml -[ANSIBLE0004] Git checkouts must contain explicit version -examples/example.yml:15 -Task/Handler: git check - -[ANSIBLE0002] Trailing whitespace -examples/example.yml:35 - with_items: - -[ANSIBLE0006] git used in place of git module -examples/example.yml:24 -Task/Handler: executing git through command -``` - -[Molecule](https://github.com/metacloud/molecule) provides a simple framework for easily and repetedly testing your roles and playbooks against different environments and operating systems. -There was a great talk last year about it by Elana Hashman from Rackspace: [Infrastructure Testing with Molecule](https://www.ansible.com/infrastructure-testing-with-molecule). - -I haven't had the chance to try Molecule yet but here's what it looks like in practice on [Asciinema](https://asciinema.org/a/161977?speed=2&autoplay=1&loop=1): - - - -## Zuul - -[Zuul](https://zuul-ci.org/) drives continuous integration, delivery, and deployment systems with a focus on project gating and interrelated projects. - -The recordings aren't available yet but there were two sessions about Zuul at AnsibleFest last week: - -- [Using Zuul CI/CD system to test Ansible Networking content](https://agenda.fest.ansible.com/SessionDetail.aspx?id=482140) by [Paul Belanger](https://twitter.com/pabelanger) and [Ricardo Carrillo Cruz](https://twitter.com/rcarrillocruz) from Red Hat -- [The Build is Never Broken](https://agenda.fest.ansible.com/SessionDetail.aspx?id=482019) by [Clint Byrum](https://twitter.com/spamaps) from GoDaddy - -Zuul eventually replaced Jenkins for the [OpenStack](https://www.openstack.org/) community and if you're curious why, you can read more about that story in a short interview on the [SuperUser blog](http://superuser.openstack.org/articles/zuul-case-study-the-openstack-foundation/). - -Zuul is awesome and let me try to give you a (long) elevator pitch of some of my favorite Zuul features: - -- By design, Zuul is meant to gate patches against your projects. Zuul will not merge a patch to your project until all the required jobs are green. -- Your jobs in Zuul are written as Ansible roles and playbooks, letting you re-use your existing deployment code and leverage thousands of built-in Ansible modules and plugins natively out of the box. -- Zuul takes care of providing you with an Ansible inventory for your jobs by abstracting the logic of creating and destroying single-use virtualized or containerized environments. -- Every project managed by Zuul is independant and can manage their own jobs, inventories and pipeline configuration in tree. -- When evaluating a change or a pull request, Zuul will speculatively load and run the new configuration coming from the change. This means you can create a new job and iterate on it until it works without even merging any code or commits. -- Horizontally scalable for the vast majority of it's workloads, Zuul is highly available and is designed to consume multiple clouds and regions concurrently with the expectation that they will eventually fail. -- Zuul provides an implementation of [ARA Records Ansible](https://github.com/ansible-community/ara) for providing host and task granular results and metrics through an intuitive reporting web interface. - -{{< tweet 1032799030131142657 >}} - -If you'd like to learn more about how [Zuul](https://zuul-ci.org/) works and what it does, I like to recommend the following two presentations by core developers of the project: - -- [Zuul: Testing the future](https://www.youtube.com/watch?v=KXh0sh3ETkQ) by James Blair -- [Zuul v3 for gating](https://www.youtube.com/watch?v=6177329H4Tg) by Monty Taylor - -You can chat with the Zuul users and developers on [IRC or on mailing lists](https://zuul-ci.org/community.html). - -## ARA Records Ansible - -[ARA Records Ansible](https://getara.org/) playbook runs and makes the recorded data available and intuitive for users and systems. - -I created ARA a little over two years ago to make it easy for Ansible users to understand and troubleshoot what occurred during their playbook runs. - -In a nutshell, ARA provides a native [Ansible callback plugin](https://docs.ansible.com/ansible/latest/plugins/callback.html) which saves everything to a local or remote database. -It then provides a command line interface as well as a web interface to query that database and provide detailed metrics and reporting for your Ansible playbooks. - -ARA is notably implemented in Zuul and is used for reporting on more than [a million playbooks every month](http://superuser.openstack.org/articles/scaling-ara-ansible/) for the OpenStack community. -You can find the ARA source code mirrored on [GitHub](https://github.com/openstack?q=ara). - -If you've never seen an ARA report before, I've recorded a live demonstration of the web interface a while back which is still accurate: - -{{< youtube k3i8VPCanGo >}} -
- -The upcoming major release, ARA 1.0, is all but a complete rewrite and has been more than a year in the making. -1.0 will feature a brand new REST API which will let users query and integrate their Ansible execution data in their tools and workflows. - -The to-do list for 1.0 is shrinking and I'm really excited that we're getting to a point where people can start using it for testing purposes. -If you'd like to test 1.0 or contribute to ARA, please reach out to me on [IRC, Slack](https://github.com/ansible-community/ara#community-and-getting-help) or [Twitter](https://twitter.com/dmsimard)! \ No newline at end of file diff --git a/website/content/blog/2019-11-06_announcing-the-release-of-ara-1.2/index.md b/website/content/blog/2019-11-06_announcing-the-release-of-ara-1.2/index.md deleted file mode 100644 index 224a4c0..0000000 --- a/website/content/blog/2019-11-06_announcing-the-release-of-ara-1.2/index.md +++ /dev/null @@ -1,86 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - news -tags: - - ansible -date: 2019-11-06 -title: "Announcing the release of ARA Records Ansible 1.2" -slug: announcing-the-release-of-ara-records-ansible-1.2 -type: post ---- - -ARA 1.2.0 has been released ! - -You can refer to GitHub repository for the -[changelog](https://github.com/ansible-community/ara/releases/tag/1.2.0) as well -as the [full list of changes](https://github.com/ansible-community/ara/compare/1.1.0...1.2.0). - -Here are some of the highlights in this new version: - -## A simple built-in interface - -Back by popular demand, ARA 1.2.0 re-introduces a built-in web interface -included by default with the API server. - -![new-built-in](new-built-in.png) - -This new lightweight and simple interface is designed to work without any -additional dependencies. - -It is still very much a work in progress but we would love to hear about what -you think about it ! - -You can find a live demo available at [api.demo.recordsansible.org](https://api.demo.recordsansible.org/). - -[ara-web](https://github.com/ansible-community/ara-web) will continue to evolve -over time but is in need of contributors knowledgeable in javascript to live up -to it's potential. - -## Static HTML generation - -It is once again possible to generate a static HTML version of the built-in -interface. -This static HTML version contains all the files you need to serve reports from -any web server. - -The layout of the files has been slightly improved over ARA 0.x but it is still -unefficient at scale due to the amount of files generated. - -Static reports can be created using ``ara-manage generate ``. - -## Performance improvements - -ARA has a performance impact when recording your playbooks but we hope to keep -this overhead as small as possible. - -This release contains fixes to avoid doing unnecessary calls to the API during -the execution of Ansible playbooks and we're on the lookout to find other -improvement opportunities in the future. - -## 1.3 preview: searching, sorting, filtering - -The next version will feature improved search and add support for sorting and -filtering results through the API. - -This means we will be able to implement these features in the built-in web -interface as well as ara-web. - -For example, here is an early iteration of a search dialog in the built-in -interface: - -![search](search.png) - -## Want to try ARA ? - -Have a look at the [quickstart](https://github.com/ansible-community/ara#quickstart) or -read the [installation](https://ara.readthedocs.io/en/latest/installation.html) -and [configuration](https://ara.readthedocs.io/en/latest/ansible-configuration.html) -documentation for more information. - -## Want to contribute, chat or need help ? - -ARA could use your help and we can also help you get started. -Please reach out ! - -The project community hangs out on [IRC and Slack](https://ara.recordsansible.org/community/). diff --git a/website/content/blog/2019-11-06_announcing-the-release-of-ara-1.2/new-built-in.png b/website/content/blog/2019-11-06_announcing-the-release-of-ara-1.2/new-built-in.png deleted file mode 100644 index d58eb82..0000000 Binary files a/website/content/blog/2019-11-06_announcing-the-release-of-ara-1.2/new-built-in.png and /dev/null differ diff --git a/website/content/blog/2019-11-06_announcing-the-release-of-ara-1.2/search.png b/website/content/blog/2019-11-06_announcing-the-release-of-ara-1.2/search.png deleted file mode 100644 index 4d6dcad..0000000 Binary files a/website/content/blog/2019-11-06_announcing-the-release-of-ara-1.2/search.png and /dev/null differ diff --git a/website/content/blog/2019-11-18_ara-0.16-maintenance-release-and-eventual-end-of-life/index.md b/website/content/blog/2019-11-18_ara-0.16-maintenance-release-and-eventual-end-of-life/index.md deleted file mode 100644 index ac17dd1..0000000 --- a/website/content/blog/2019-11-18_ara-0.16-maintenance-release-and-eventual-end-of-life/index.md +++ /dev/null @@ -1,51 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - news -tags: - - ansible -date: 2019-11-18 -title: "ARA 0.16 maintenance release and eventual end of life" -slug: ara-0.16-maintenance-release-and-eventual-end-of-life -type: post ---- - -ARA 0.16.6 is a maintenance release for users that are not yet using ARA 1.x. - -Changes since 0.16.5: - -- Fixed web application crash due to encoding/decoding of binary non-ascii - content in task results -- The [sqlite middleware](https://ara.readthedocs.io/en/stable-0.x/advanced.html) - was adapted to support running under gunicorn as well as mod_wsgi -- ``python -m ara.setup.env`` now returns commands that use bash expansion to - take into account existing environment variables - -## Eventual end of life for ARA 0.x - -All new feature and development effort for more than a year has been spent on -the master branch of ARA which is the basis of version 1.x releases. - -In fact, 0.x and 1.x are vastly different code bases at this point. -They are diverging by more than 400 commits and 1.x features a completely -re-written backend which includes many new features, notably a new REST API. - -Users are encouraged to try the latest release of ARA, currently [1.2.0](https://ara.recordsansible.org/blog/2019/11/06/announcing-the-release-of-ara-records-ansible-1.2/), -and create issues on [GitHub](https://github.com/ansible-community/ara/issues) -if they encounter any problems or missing features. - -ARA 0.16.6 could be the last release of ARA 0.x if no major issues are found. - -## Want to try the latest release ? - -Have a look at the [quickstart](https://github.com/ansible-community/ara#quickstart) or -read the [installation](https://ara.readthedocs.io/en/latest/installation.html) -and [configuration](https://ara.readthedocs.io/en/latest/ansible-configuration.html) -documentation for more information. - -## Want to contribute, chat or need help ? - -ARA could use your help and we can also help you get started. -Please reach out ! - -The project community hangs out on [IRC and Slack](https://ara.recordsansible.org/community/). diff --git a/website/content/blog/2019-11-22_ara-is-now-packaged-for-fedora/index.md b/website/content/blog/2019-11-22_ara-is-now-packaged-for-fedora/index.md deleted file mode 100644 index 2d071a3..0000000 --- a/website/content/blog/2019-11-22_ara-is-now-packaged-for-fedora/index.md +++ /dev/null @@ -1,57 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - news -tags: - - ansible -date: 2019-11-22 -title: "ARA is now packaged for Fedora !" -slug: ara-is-now-packaged-for-fedora -type: post ---- - -We are happy to announce that ARA has successfully been included in -[Fedora](https://getfedora.org/) ! - -The latest version ([1.2.0](https://ara.recordsansible.org/blog/2019/11/06/announcing-the-release-of-ara-records-ansible-1.2/)) -is available for Fedora 30, Fedora 31 as well as Rawhide (Fedora 32). - -You can find the source for the packaging files on [src.fedoraproject.org](https://src.fedoraproject.org/rpms/ara/tree/master) -and the builds on [koji.fedoraproject.org](https://koji.fedoraproject.org/koji/packageinfo?packageID=24394). - -Contributions to improve the packaging are welcome ! - -## How to get started - -If you are using Ansible from Fedora packages, you can start recording playbooks -with ARA easily: - -``` -# Install ARA's Ansible plugins and the API server dependencies -dnf install ara ara-server - -# Configure Ansible to load the ARA callback plugin -export ANSIBLE_CALLBACK_PLUGINS=$(python3 -m ara.setup.callback_plugins) - -# Run your playbooks as usual -ansible-playbook playbook.yml - -# Start the embedded server -ara-manage runserver - -# That's it ! -``` - -You can see what it looks like in this small recording: - - - -If you'd like to see the built-in web interface looks like, you can find a live -demo on [api.demo.recordsansible.org](https://api.demo.recordsansible.org). - -## Want to contribute, chat or need help ? - -If you would like to help with packaging ARA for other distributions, please -reach out ! - -The project community hangs out on [IRC and Slack](https://ara.recordsansible.org/community/). diff --git a/website/content/blog/2019-12-03_announcing-the-release-of-ara-1.3/index.md b/website/content/blog/2019-12-03_announcing-the-release-of-ara-1.3/index.md deleted file mode 100644 index 94f0710..0000000 --- a/website/content/blog/2019-12-03_announcing-the-release-of-ara-1.3/index.md +++ /dev/null @@ -1,90 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - news -tags: - - ansible -date: 2019-12-03 -title: "Announcing the release of ARA Records Ansible 1.3" -slug: announcing-the-release-of-ara-records-ansible-1.3 -type: post ---- - -ARA 1.3.0 has been released ! - -You can refer to GitHub repository for the -[changelog](https://github.com/ansible-community/ara/releases/tag/1.3.0) as well -as the [full list of changes](https://github.com/ansible-community/ara/compare/1.2.0...1.3.0) since 1.2. - -Here are some of the highlights in this new version: - -## Required python version - -Before ARA 1.3, it was required to run the API server under at least python 3.6 -and above. - -We've removed the requirement on python 3.6 by replacing a few -instances of [python f-strings](https://www.python.org/dev/peps/pep-0498/) by -a compatible string format. - -This change will make it easier to run the API server on a wider range of -operating systems which include python 3.5. - -## API improvements for searching, sorting and filtering - -1.3 improves on the existing search functionality of the API and makes it -possible to sort and filter the results of your search. - -Let's look at a few practical examples to demonstrate how this might be useful. - -Pretend you'd like to search for playbooks that failed in the last 30 days and -have them ordered with the most recent first, you could construct your query -like so: - - /api/v1/playbooks?started_after=2019-11-03T09:57:36&status=failed&order=-started - -If you want to find out which Ansible tasks took the longest to run, you can -sort by duration instead: - - /api/v1/tasks?order=duration - -Or if you want to see which hosts are failing the most tasks: - - /api/v1/hosts?order=-failed - -## Playbook index search - -Some of the improvements to the API for this release were implemented in the -playbook index of the built-in interface: - -![playbook-search](playbook-search.png) - -We hope to bring similar functionality in the future for searching through -plays, tasks, results and hosts. - -## Database pruning - -Depending on your use case, you might not want to keep playbook reports in your -database forever. - -1.3 introduces a new command, [ara-manage prune](https://ara.readthedocs.io/en/latest/ara-manage.html#ara-manage-prune), -which takes care of deleting playbooks and all of their resources if they are -older than a specified amount of days. - -For example, the ``ara-manage prune`` command could be set up to run in a cron -to control the size of your database while keeping the amount of rows to a -minimum for steady performance. - -## Want to try ARA ? - -Have a look at the [quickstart](https://github.com/ansible-community/ara#quickstart) or -read the [installation](https://ara.readthedocs.io/en/latest/installation.html) -and [configuration](https://ara.readthedocs.io/en/latest/ansible-configuration.html) -documentation for more information. - -## Want to contribute, chat or need help ? - -ARA could use your help and we can also help you get started. -Please reach out ! - -The project community hangs out on [IRC and Slack](https://ara.recordsansible.org/community/). diff --git a/website/content/blog/2019-12-03_announcing-the-release-of-ara-1.3/playbook-search.png b/website/content/blog/2019-12-03_announcing-the-release-of-ara-1.3/playbook-search.png deleted file mode 100644 index 6ce3225..0000000 Binary files a/website/content/blog/2019-12-03_announcing-the-release-of-ara-1.3/playbook-search.png and /dev/null differ diff --git a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/1.0-diff.png b/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/1.0-diff.png deleted file mode 100644 index e63cc9d..0000000 Binary files a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/1.0-diff.png and /dev/null differ diff --git a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/1.0-release.png b/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/1.0-release.png deleted file mode 100644 index d8cfa93..0000000 Binary files a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/1.0-release.png and /dev/null differ diff --git a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/1.1-release.png b/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/1.1-release.png deleted file mode 100644 index a87565d..0000000 Binary files a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/1.1-release.png and /dev/null differ diff --git a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/index.md b/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/index.md deleted file mode 100644 index cc088b4..0000000 --- a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/index.md +++ /dev/null @@ -1,174 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - news -tags: - - ansible -date: 2019-12-31 -title: "2019 in restrospective for ARA Records Ansible" -slug: 2019-in-retrospective-for-ara-records-ansible -type: post ---- - -ARA has come a long way since I created it as a side project -[back in 2016](https://ara.recordsansible.org/blog/2016/05/21/an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs). - -The end of 2019 is here and I thought this was a good opportunity to look back -at the progress of the project over the past year. - -# The journey to finally release 1.0 - -ARA 0.x has been successful at recording Ansible playbook runs and providing -reporting capabitilies. However, it lacked -- amongst other things -- an API to -make it easier to integrate with other systems. - -I've been talking about a version 1.0 for ARA as far back as -[August 2017](https://ara.recordsansible.org/blog/2017/08/16/whats-coming-in-ara-1.0) -and it has been quite an adventure since then. - -First, I tried to extend the existing flask backend and retrofit an API with -flask-restful. It was challenging but was a great learning experience. - -I ended up with something that worked for most of the resource endpoints by -[November 2017](https://ara.recordsansible.org/blog/2017/11/22/status-update-ara-1.0). - -![1.0 diff](1.0-diff.png) - -But then, nothing really happened for a while: I burned myself out. - -Between work, family and different personal issues, I wasn't able to keep up -with the growth and pace of development of ARA anymore. - -I maintained the project to keep it working with newer versions of Ansible with -the occasional bug fixes but that was it. - -## A fork in the road - -A fellow Red Hatter, Guillaume Vincent, eventually came up with a -[demonstration](https://github.com/ansible-community/ara/tree/a889a71bcd8d549b8c13d009a7ecb3266ad0b6d4) -of an API for ARA built on top of django and django-rest-framework, -coincidentally the same backend stack as AWX/Tower. - -It worked and it turned out easier and simpler than what I had been working -on with flask. - -This left me with a difficult question: - -> Should I throw away over a hundred commits that I burned myself out on to -> start from scratch on a more robust framework that I know little about ? - -After thinking about it and leaving the question lingering for a few months, I -eventually decided to bite the bullet and pursue development on top of django -and django-rest-framework instead. - -For historical purposes, I tagged the -[tip of the flask-restful branch](https://github.com/ansible-community/ara/releases/tag/old-flask-1.0) -and then deleted it. - -Guillaume and another contributor, Florian Apolloner, helped me along the -learning curve and little by little, we made progress towards what eventually -released as 1.0 more than 300 commits later. - -And so, on June 4th 2019, -[ARA 1.0 was announced](https://ara.recordsansible.org/blog/2019/06/04/announcing-the-release-of-ara-records-ansible-1.0). - -![1.0 release](1.0-release.png) - -## 1.0 wasn't even feature complete - -After almost two years of working part time on 1.0, throwing away more than a -hundred commits and working on some 300 more, it wasn't even feature complete -when it was released. - -At this rate, it was going to take at least a few months to attain feature -parity with 0.x and land what I felt was missing. - -Guillaume authored [ara-web](https://github.com/ansible-community/ara-web), a -javascript web interface to the API but couldn't keep contributing due to lack -of time and shifting priorities. - -The new backend, API and API clients were working and in a state where they -could be useful to people. - -Even if just for my own sanity, I couldn't keep those out of the hands of users -any longer while we worked on implementing missing features or made the web -interface pretty. - -## But then, it got better - -With 1.1 came iterative improvements and brought back the -[distributed sqlite database backend](https://ara.readthedocs.io/en/latest/distributed-sqlite-backend.html) -from [0.x](https://ara.readthedocs.io/en/stable-0.x/advanced.html). - -![1.1 release](1.1-release.png) - -Later, the [1.2 release](https://ara.recordsansible.org/blog/2019/11/06/announcing-the-release-of-ara-records-ansible-1.2) -brought back a simple built-in web reporting interface as well as static HTML -generation. - -![new-built-in](new-built-in.png) - -More recently, the -[1.3 release](https://ara.recordsansible.org/blog/2019/12/03/announcing-the-release-of-ara-records-ansible-1.3) -improved the API capabilities for searching, ordering and filtering queries -and brought these features to the built-in interface. - -![Playbook search](playbook-search.png) - -## About ara-web - -Bringing back a built-in interface to ARA was a bit controversial because the -original intent was for ara-web to be the one frontend interface for reporting: -now there's two with varying levels of features. - -ARA 0.x had a built-in interface but it also made the project depend on -many dependencies that weren't otherwise necessary. - -On the other hand, requiring ara-web means that users need to install npm/node -javascript dependencies and run an API server in order to allow the hosted web -interface to connect to it. - -It wasn't really a user experience improvement over what we had in 0.x. - -So, for the time being, we've struck somewhere in the middle and settled on a -built-in interface without any javascript dependencies using simple django -templating. - -I believe ara-web is an important project but requires the help of contributors -skilled in javascript to realize the project's potential. - -# Looking to 2020 - -There's still many features and improvements to work on and I'd like to -do a better job at tracking them in the project's -[GitHub issues](https://github.com/ansible-community/ara/issues). - -At a high level, here are some of the things I'd like to see implemented: - -- [Allow pull requests on GitHub](https://github.com/ansible-community/ara-infra/issues/4) -- Re-introduce a CLI for querying the API and get results back -- Host-level web reporting view: browse results for a single host across multiple recorded playbooks -- [Support for recording diffs](https://github.com/ansible-community/ara/issues/29) (for ex: template tasks) -- [Support for recording adhoc commands](https://github.com/ansible-community/ara/issues/27) -- [Formalize a profiling/benchmark approach to identify performance improvement opportunities](https://github.com/ansible-community/ara-infra/issues/6) -- [Proper documentation for integration with AWX/Tower](https://github.com/ansible-community/ara/issues/24) -- [ARA is now packaged for Fedora](https://ara.recordsansible.org/blog/2019/11/22/ara-is-now-packaged-for-fedora) but it would be nice to have packages for other distributions as well - -In the meantime, I've created a short [user survey](https://github.com/ansible-community/ara/issues/103) -and would appreciate hearing from you to help drive future development: - -{{< tweet 1212075055817150464 >}} - -## Want to try ARA ? - -Have a look at the [quickstart](https://github.com/ansible-community/ara#quickstart) or -read the [installation](https://ara.readthedocs.io/en/latest/installation.html) -and [configuration](https://ara.readthedocs.io/en/latest/ansible-configuration.html) -documentation for more information. - -## Want to contribute, chat or need help ? - -ARA could use your help and we can also help you get started. -Please reach out ! - -The project community hangs out on [IRC and Slack](https://ara.recordsansible.org/community/). diff --git a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/new-built-in.png b/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/new-built-in.png deleted file mode 100644 index d58eb82..0000000 Binary files a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/new-built-in.png and /dev/null differ diff --git a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/playbook-search.png b/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/playbook-search.png deleted file mode 100644 index 6ce3225..0000000 Binary files a/website/content/blog/2019-12-31_2019-in-restrospective-for-ara-records-ansible/playbook-search.png and /dev/null differ diff --git a/website/content/blog/2019_01_16_introducing-new-projects-for-the-upcoming-1.0-release/index.md b/website/content/blog/2019_01_16_introducing-new-projects-for-the-upcoming-1.0-release/index.md deleted file mode 100644 index 92ecf92..0000000 --- a/website/content/blog/2019_01_16_introducing-new-projects-for-the-upcoming-1.0-release/index.md +++ /dev/null @@ -1,235 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development -tags: - - ansible - - openstack -date: 2019-01-16 -title: "Introducing new projects for the upcoming 1.0 release" -slug: introducing-new-projects-for-the-upcoming-1.0-release -type: post ---- - -Attempting to release a version 1.0 for ARA Records Ansible has been quite an -adventure that started more than a year ago. - -The general vision hasn't changed since then and the status updates from 2017 -are still relevant from that perspective. - -Hundreds of commits later, the code underneath what will be released as ARA 1.0 -has been almost entirely re-written from scratch, though. - -We've been making steady progress with the help of two new core contributors -(thanks Florian and Guillaume!) and we're excited to share some of the things -we have been working on. - -We'll be providing updates on a more regular basis as we draw closer to the -official release of 1.0 but I first wanted to introduce how ARA 1.0 has been -split into different projects. - -# Different projects - -In 1.0, ARA will no longer be a single project requiring you to install -dependencies that you might not be interested in for a wide array of use cases. - -For example, in 0.x, you would need to install the web application dependencies -even though you might just be after the callback plugin. - -As things stand today, ARA 1.0 is composed of four individual projects which -lets you install them together or independently of one another depending on -your use case. - -You can find more information on the general workflow and interaction between -the components in the [documentation](https://ara-server.readthedocs.io/en/latest/arch.html). - -## ara-server - -ara-server ([github](https://github.com/ansible-community/ara-server)) is a new python 3 -application built with [django](https://www.djangoproject.com/) and -[django-rest-framework](https://www.django-rest-framework.org/). - -It provides the database models as well as a brand new REST API for -integrating ARA's playbook execution results into your workflows. - -If you want to see what the API looks like in practice, there is a live demo -available [here](https://api.demo.recordsansible.org/api/v1). - -This web interface, provided out of the box by django-rest-framework, allows you -to navigate or query the API directly from your web browser. - -We're not quite ready to call the API stable yet but we're getting there. -As we start deploying and using the API ourselves, we expect to improve and -fine tune things before the official release. - -An important consideration we had when developing this new API is that it should -not be necessary for users to stand up an API server. - -This brings us to the next component we'll talk about, ara-clients. - -## ara-clients - -ara-clients ([github](https://github.com/ansible-community/ara-clients)) is the project -that provides the python API clients for communicating with the API. - -It currently ships two clients, one that works without needing to stand up an -API server, ``AraOfflineClient`` and one that expects a remote HTTP API -endpoint, ``AraHttpClient``. - -The clients have the same interface and they take care of abstracting the -differences when running offline or online. - -In a nutshell, the current implementation of the offline client spawns an -ephemeral API server instance so you don't need to. - -Using these clients with the API is designed to be as straightforward as possible. -Here's a simple example from the -[documentation](https://ara-server.readthedocs.io/en/latest/using_api_clients.html): - -{{< highlight python >}} -#!/usr/bin/env python3 -# Import the client -from ara.clients.http import AraHttpClient - -# Instanciate the HTTP client with an endpoint where ara-server is listening -client = AraHttpClient(endpoint="https://api.demo.recordsansible.org") - -# Get a list of failed playbooks -# /api/v1/playbooks?status=failed -playbooks = client.get("/api/v1/playbooks", status="failed") - -# If there are any failed playbooks, retrieve their failed results -# and provide some insight. -for playbook in playbooks["results"]: - # Retrieve results for this playbook - # /api/v1/results?playbook=<:id>&status=failed - results = client.get("/api/v1/results", playbook=playbook["id"], status="failed") - - # Iterate over failed results to get meaningful data back - for result in results["results"]: - # Get the task that generated this result - # /api/v1/tasks/<:id> - task = client.get(f"/api/v1/tasks/{result['task']}") - - # Get the file from which this task ran from - # /api/v1/files/<:id> - file = client.get(f"/api/v1/files/{task['file']}") - - # Get the host on which this result happened - # /api/v1/hosts/<:id> - host = client.get(f"/api/v1/hosts/{result['host']}") - - # Print something useful - print(f"Failure on {host['name']}: '{task['name']}' ({file['path']}:{task['lineno']})") -{{< /highlight >}} - -Running this bit of code against the current demo API provides the following -output: - -``` -Failure on localhost: 'smoke-tests : Record with no key' (/home/zuul/src/git.openstack.org/openstack/ara-infra/tests/integration/roles/smoke-tests/tasks/ara-ops.yaml:166) -Failure on localhost: 'smoke-tests : Record with no value' (/home/zuul/src/git.openstack.org/openstack/ara-infra/tests/integration/roles/smoke-tests/tasks/ara-ops.yaml:177) -Failure on localhost: 'smoke-tests : Record with invalid type' (/home/zuul/src/git.openstack.org/openstack/ara-infra/tests/integration/roles/smoke-tests/tasks/ara-ops.yaml:188) -Failure on localhost: 'smoke-tests : Return false' (/home/zuul/src/git.openstack.org/openstack/ara-infra/tests/integration/roles/smoke-tests/tasks/test-ops.yaml:25) -Failure on localhost: 'fail' (/tmp/ara-integration-tests/ara-infra/tests/integration/failed.yaml:22) -``` - -This simple example illustrates how you can search for things with the API and -access every bit of data from your playbook executions. -Playbook and role files, host facts, task metrics, granular and detailed -results, everything is there and organized to make it easy to work with. - -These clients are the same ones used by the ARA Ansible callback plugin as well -as the ``ara_record`` Ansible module in order to communicate with the API. -These are part of the next project from 1.0 we'll talk about: ara-plugins. - -## ara-plugins - -ara-plugins ([github](https://github.com/ansible-community/ara-plugins)) is the project -that contains Ansible modules and plugins for ARA. - -Ansible provides a [callback plugin interface](https://docs.ansible.com/ansible/latest/plugins/callback.html) -in order to let users hook into certain events that occur during the playbook -execution such as ``v2_playbook_on_start``, ``v2_playbook_on_task_start``, -``v2_runner_on_ok``, ``v2_runner_on_failed``, etc. - -ara-plugins provides a callback plugin which leverages this interface to save -everything to a database by sending it to the API throughout the playbook run. - -The project also contains an Ansible module called ``ara_record`` which -can be used to record anything and associate it with the playbook report. - -We've seen users record things like git commit hashes, links to artifacts or -logs and other data relevant to their use cases. - -The callback and the ara_record module are not new to ARA 1.0 but they have been -updated to use the new API. -They are basically the only components that have not been re-written from scratch. - -## ara-web - -ara-web ([github](https://github.com/ansible-community/ara-web)) will be the new web -interface project in ARA 1.0. - -It is a modern javascript interface built with [React](https://reactjs.org/) -and [patternfly-next](https://github.com/patternfly/patternfly-next), the -upcoming release of [Patternfly](https://www.patternfly.org/). - -We are iterating on the actual web interface right now and a lot of technical -problems are solved. For example: - -- You can configure which API server the built-in client will connect to -- It has it's own API client implementation for querying ara-server and parsing it's replies -- It has jobs for running unit and build integration tests on every patch -- We even get drafts of the dashboard during code reviews (more on that in another blog post) - -Want to know what it looks like ? There's a [live demo](https://web.demo.recordsansible.org). -The live demo is stateless -- it relies entirely on the API live demo to retrieve -data for displaying the reports. - -## But wait, there's more ! - -There are two other projects that aren't about strictly recording Ansible, but -they're about helping us recording Ansible with Another Recursive Acronym. - -### ara-infra - -ara-infra ([github](https://github.com/ansible-community/ara-infra)) is a satellite -project for ARA's infrastructure related things. - -So far, it contains: - -- The source for this website and blog -- The Ansible playbooks and roles to deploy it from scratch -- The integration tests for the website deployment as well as the CI jobs to run them - -This is also where we will eventually find integration tests common to all the -ARA projects as well as the jobs to update the API and web demos after each commit. - -### ansible-role-ara - -ansible-role-ara ([github](https://github.com/openstack/ansible-role-ara)) wants -to be able to install and configure ARA in a few different standardized deployment -scenarios: - -- Inside a virtual environment (or not) -- From source or from distribution packages (when available) -- apache with mod_wsgi -- With nginx+gunicorn -- With nginx+uwsgi - -Supporting these deployment scenarios would be a very good asset for integration -testing ARA itself. - -# Coming soon (™) - -This is all coming soon. It's not a matter of days but it's not years either ! - -If you would like to contribute code, feedback, documentation or even help test -the upcoming release, please reach out ! It could allow us to ship sooner :) - -We're on [Twitter](https://twitter.com/ARecordsAnsible), -[#ara](http://webchat.freenode.net/?channels=%23ara) on the freenode IRC -network and on [Slack](https://join.slack.com/t/arecordsansible/shared_invite/enQtMjMxNzI4ODAxMDQxLWU4MmZhZTI4ZjRjOTUwZTM2MzM3MzcwNDU1YzFmNzRlMzI0NTUzNDY1MWJlNThhM2I4ZTViZjUwZTRkNTBiM2I). - -See you around ! diff --git a/website/content/blog/2019_03_11_ara-1.0-alpha3-back-to-basics/changelog.png b/website/content/blog/2019_03_11_ara-1.0-alpha3-back-to-basics/changelog.png deleted file mode 100644 index 8c44ec1..0000000 Binary files a/website/content/blog/2019_03_11_ara-1.0-alpha3-back-to-basics/changelog.png and /dev/null differ diff --git a/website/content/blog/2019_03_11_ara-1.0-alpha3-back-to-basics/index.md b/website/content/blog/2019_03_11_ara-1.0-alpha3-back-to-basics/index.md deleted file mode 100644 index faaca45..0000000 --- a/website/content/blog/2019_03_11_ara-1.0-alpha3-back-to-basics/index.md +++ /dev/null @@ -1,94 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development -tags: - - ansible -date: 2019-03-11 -title: "ARA 1.0 alpha3: back to basics" -slug: ara-1.0-alpha3-back-to-basics -type: post ---- - -The release of ARA 1.0 draws closer with a third alpha milestone ! - -![changelog](changelog.png) - -If there was a theme for this milestone it would be "back to basics". - -In the previous [blog post](https://ara.recordsansible.org/blog/2019/01/16/introducing-new-projects-for-the-upcoming-1.0-release/), -I explained that ARA 1.0 was composed of several projects such as ``ara-server``, -``ara-plugins``, ``ara-clients``, ``ara-web`` and ``ara-infra``. - -As a user, developer and maintainer of these projects myself, I realized that -the added complexity of keeping the projects separated did not provide enough -value to be worth it. - -Things had to be simple because simplicity is a feature in ARA and failing to -be simple meant that we were not staying true to the project's [core values](https://ara.readthedocs.io/en/stable/manifesto.html). - -## One repository instead of three - -The ``ara-server``, ``ara-plugins`` and ``ara-clients`` projects have been -merged into a single repository and you'll be able to find them in the -[feature/1.0 branch](https://github.com/ansible-community/ara/tree/feature/1.0) of the -main ARA repository. - -This means less burden for contributors and maintainers, for example: - -- We're managing a single repository instead of three -- We no longer need to manage dependencies between the three projects -- We have one project to test, tag, release and contribute to -- Tests and CI no longer need to account for the projects being separated - -By removing this overhead, the project becomes simpler to maintain which -frees up time to do other things. - -For users, this means that there's only a single package with everything -included on PyPi instead of three and 1.0 is already lightweight in terms of -dependencies when compared to 0.x. - -[ara-web](https://github.com/ansible-community/ara-web) will remain a standalone project -that provides a javascript web client to the API. - -[ara-infra](https://github.com/ansible-community/ara-infra) will also remain since it's -for managing the infrastructure for the project -- including this blog post and -the Ansible playbooks to deploy it ! - -## Two built-in Ansible roles - -ARA 1.0 will ship with two built-in Ansible roles: - -- [ansible-role-ara-api](https://ara.readthedocs.io/en/latest/ansible-role-ara-api.html) -- [ansible-role-ara-web](https://ara.readthedocs.io/en/latest/ansible-role-ara-web.html) - -These roles work are already in use to deploy -[api.demo.recordsansible.org](https://api.demo.recordsansible.org) as -well as [web.demo.recordsansible.org](https://web.demo.recordsansible.org) from -playbooks in [ara-infra](https://github.com/ansible-community/ara-infra/commit/a797094b61d3dd5f7ccb25849499489ed40cafea). - -These roles are rough around the edges right now but they will evolve over -time to provide a framework for integration testing different ways of deploying -the new ARA API server as well as the web client interface. - -# Beta ? - -There's still some work to do before moving forward with what we'd call a beta. - -For example, we need to iterate on [ara-web](https://github.com/ansible-community/ara-web) -and make sure the API is able to give it the information it needs. - -Also, while the API and the API clients are nearing completion, we currently do -not yet have a CLI client which would allow users to query the API easily from -the command line (ex: ``ara playbook list``). - -There's other things but I think those are definitely the ones that stand out -the most right now. - -## In the meantime... - -If you would like to contribute code, feedback, documentation or help test the -alpha milestones with us, please reach out ! - -Find us on [#ara](http://webchat.freenode.net/?channels=%23ara) on the freenode IRC -network and on [Slack](https://join.slack.com/t/arecordsansible/shared_invite/enQtMjMxNzI4ODAxMDQxLWU4MmZhZTI4ZjRjOTUwZTM2MzM3MzcwNDU1YzFmNzRlMzI0NTUzNDY1MWJlNThhM2I4ZTViZjUwZTRkNTBiM2I). diff --git a/website/content/blog/2019_05_16_1.0-beta-help-wanted/index.md b/website/content/blog/2019_05_16_1.0-beta-help-wanted/index.md deleted file mode 100644 index 8069a2b..0000000 --- a/website/content/blog/2019_05_16_1.0-beta-help-wanted/index.md +++ /dev/null @@ -1,149 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - development -tags: - - ansible -date: 2019-05-16 -title: "ARA 1.0 beta: help wanted" -slug: ara-1.0-beta-help-wanted -type: post ---- - -The first beta milestone for ARA 1.0 is out and ready for testing ! - -## Main changes since 1.0.0a4 - -- API: ``/api/v1/info`` has moved to ``/`` and ``/`` was improved with a link to the API index -- API: ``CORS_ORIGIN_WHITELIST`` now requires the scheme (http/https) as per [django-cors-headers](https://github.com/ottoyiu/django-cors-headers/blob/master/HISTORY.rst#300-2019-05-10) -- API: Playbook arguments are now provided when listing playbooks at ``/api/v1/playbooks`` -- Clients: API clients now support authenticating with specified credentials -- Ansible roles: The ara_api role now supports using postgresql for the server database - -## Packaging - -The API server dependencies are now optional when installing ARA 1.0. -They can be installed by specifying the ``[server]`` extra requirement: - - # Note: --pre is required when installing an ARA 1.0 pre-release - pip install [--pre] ara[server] - -Local or offline usage of ARA 1.0 still requires the API server dependencies -installed but the server does not need to be running. - -In addition, when using postgresql, the psycopg2 library can be installed by -specifying the ``[postgresql]`` extra requirement: - - pip install [--pre] ara[server,postgresql] - -## Miscellaneous - -- The code review and CI infrastructure was rebranded from OpenStack to [OpenDev](https://opendev.org/) -- The GitHub mirror is now available at https://github.com/ansible-community/ara -- Bug, issue and feature tracking have been moved to https://github.com/ansible-community/ara/issues - -# Want to try it out ? - -The live demos on [api.demo.recordsansible.org](https://api.demo.recordsansible.org/) and -[web.demo.recordsansible.org](https://web.demo.recordsansible.org/) have been -updated with this latest beta release. - -If you want a quick start, you can have a look at the -[README](https://github.com/ansible-community/ara/tree/feature/1.0#quickstart) -or there is otherwise plenty of documentation available to get started: - -- [Installing the 1.0 pre-release](https://ara.readthedocs.io/en/latest/installation.html) -- [Configuring Ansible to use ARA](https://ara.readthedocs.io/en/latest/ansible-configuration.html) -- [Configuring the ARA Ansible plugins](https://ara.readthedocs.io/en/latest/ara-plugin-configuration.html) -- [Customizing the API server configuration](https://ara.readthedocs.io/en/latest/api-configuration.html) -- [Setting up authentication and security considerations](https://ara.readthedocs.io/en/latest/api-security.html) -- [API endpoint documentation and object relationships](https://ara.readthedocs.io/en/latest/api-documentation.html) -- [How to use the API with the built-in API clients](https://ara.readthedocs.io/en/latest/api-usage.html) -- [Architecture and workflows: how ARA records data](https://ara.readthedocs.io/en/latest/architecture.html) - -There are even built-in Ansible roles to help you set up an API server as well -as the new web interface: - -- [ansible-role-ara-api](https://ara.readthedocs.io/en/latest/ansible-role-ara-api.html) -- [ansible-role-ara-web](https://ara.readthedocs.io/en/latest/ansible-role-ara-web.html) - -In fact, these roles are the ones used to deploy the live demos on -[api.demo.recordsansible.org](https://api.demo.recordsansible.org/) and -[web.demo.recordsansible.org](https://web.demo.recordsansible.org/): - -![live-demo-playbook](live-demo-playbook.png) - -You'll find the Ansible roles in the [source repository](https://github.com/ansible-community/ara/tree/feature/1.0/roles). - -# Help wanted - -ARA is a free and open source community project and it needs help from users and -contributors for maintenance, new features and improvements. - -As a user, your feedback is invaluable to know if the project is living up to -your expecations. - -Was the documentation clear ? Did you encounter an issue when trying ARA ? -Do you have an idea for a new feature ? -Join the community and chat with us on -[IRC or on Slack](https://ara.recordsansible.org/community/) to tell us about it ! - -As a contributor, there is a wide range of things we could use your help for. - -Issues and features are now tracked on GitHub for [ara](https://github.com/ansible-community/ara/issues?q=is%3Aopen+is%3Aissue+-label%3A0.x), -[ara-web](https://github.com/ansible-community/ara-web/issues) as well as -[ara-infra](https://github.com/ansible-community/ara-infra/issues). - -The API server is based on [Django](https://www.djangoproject.com/) and -[django-rest-framework](https://www.django-rest-framework.org/) while the web -interface is a stateless javascript application built with -[patternfly](https://www.patternfly.org/) and [react](https://reactjs.org/). - -With this beta release, the API should be mostly settled although we are -interested in feedback for the database model if you are familiar with Django. - -The web interface needs the most love but hey, it works and it is able to query -the API successfully. - -All commits are code reviewed, unit and integration tested before being -merged to the project. - -{{< tweet 1128458564945756161 >}} - -If you have time to contribute, I can help point you in the right direction to -get started. - -You can find me as ``dmsimard`` on IRC, Slack and [Twitter](https://twitter.com/dmsimard). - -# After the beta: releasing ARA 1.0 - -Installing and configuring ARA 1.0 is very similar to ARA 0.x on purpose. - -However, there will be no support for upgrading an existing 0.x database to 1.0. -The backend and database model is now based on Django instead of Flask and -everything was essentially re-written from scratch. - -If you are currently using ARA 0.x, now might be a good time to make sure you -don't upgrade unexpectedly: - -{{< tweet 1120780543954751489 >}} - -It is important to note that 1.0 will be released without full feature parity -with 0.x. It was a tough decision to make but I'm convinced the new API provides -too much value to keep it unreleased while we implement the missing features. - -Namely, these are some of the things that will be missing from the 1.0 release: - -- A command-line interface (ex: ``ara playbook list``) -- Generating and exporting data (ex: ``ara generate html``, ``ara generate junit``) -- An approach for large scale distributed environments similar to [sqlite over http in 0.x](https://superuser.openstack.org/articles/scaling-ara-ansible/) - -If these features are important to you, we could use your help to port them to -use the new API or you can hold out until they are eventually shipped in a -future release. - -Otherwise, the plan is to have a sufficient amount of users try out the beta -to see if there are any bugs or gaps we might have missed and then tag the -release once these have been resolved. - -Soon ™ diff --git a/website/content/blog/2019_05_16_1.0-beta-help-wanted/live-demo-playbook.png b/website/content/blog/2019_05_16_1.0-beta-help-wanted/live-demo-playbook.png deleted file mode 100644 index d81a43f..0000000 Binary files a/website/content/blog/2019_05_16_1.0-beta-help-wanted/live-demo-playbook.png and /dev/null differ diff --git a/website/content/blog/2019_06_04-announcing-the-release-of-ara-1.0/index.md b/website/content/blog/2019_06_04-announcing-the-release-of-ara-1.0/index.md deleted file mode 100644 index 193e6d9..0000000 --- a/website/content/blog/2019_06_04-announcing-the-release-of-ara-1.0/index.md +++ /dev/null @@ -1,89 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - news -tags: - - ansible -date: 2019-06-04 -title: "Announcing the release of ARA Records Ansible 1.0" -slug: announcing-the-release-of-ara-records-ansible-1.0 -type: post ---- - -After more than three years since the creation of the project, we are excited -to announce that the version 1.0 of ARA Records Ansible is now available. - -This new release marks the deprecation of ARA 0.x and while full feature parity -has not yet been achieved, we are moving forward and we will iterate to add -missing features in future releases. - -## Main changes from ARA 0.x - -- The backend has been re-written from Flask to Django/Django-rest-framework -- A new API as well as built-in API clients are available to record and query playbook results -- The project's dependencies have been decoupled: the Ansible plugins, API backend and web interface can be installed independently from one another -- The web interface has been re-written as a standalone project: [ara-web](https://github.com/ansible-community/ara-web) - -In summary, all the different components before 1.0, including the web interface, -would communicate directly with the database model. - -After 1.0, these components communicate with the new REST API which results in -easier development, maintenance and integration. - -# Want to try it out ? - -If you want a quick start, you can have a look at the -[README](https://github.com/ansible-community/ara/tree/feature/1.0#quickstart) -or there is otherwise plenty of documentation available to get started: - -- [Frequently asked questions](https://ara.readthedocs.io/en/latest/faq.html) -- [Installing ARA](https://ara.readthedocs.io/en/latest/installation.html) -- [Configuring Ansible to use ARA](https://ara.readthedocs.io/en/latest/ansible-configuration.html) -- [Configuring the ARA Ansible plugins](https://ara.readthedocs.io/en/latest/ara-plugin-configuration.html) -- [Customizing the API server configuration](https://ara.readthedocs.io/en/latest/api-configuration.html) -- [Setting up authentication and security considerations](https://ara.readthedocs.io/en/latest/api-security.html) -- [API endpoint documentation and object relationships](https://ara.readthedocs.io/en/latest/api-documentation.html) -- [How to use the API with the built-in API clients](https://ara.readthedocs.io/en/latest/api-usage.html) - -You might also want to take a look at ARA features such as -[playbook names and labels](https://ara.readthedocs.io/en/latest/playbook-names-and-labels.html) -and the [ara_record action module to record arbitrary data](https://ara.readthedocs.io/en/latest/ara-record.html). - -There are even built-in Ansible roles to help you set up an API server as well -as the new web interface: - -- [ansible-role-ara-api](https://ara.readthedocs.io/en/latest/ansible-role-ara-api.html) -- [ansible-role-ara-web](https://ara.readthedocs.io/en/latest/ansible-role-ara-web.html) - -In fact, these roles are the ones used to deploy the live demos on -[api.demo.recordsansible.org](https://api.demo.recordsansible.org/) and -[web.demo.recordsansible.org](https://web.demo.recordsansible.org/): - -![live-demo-playbook](live-demo-playbook.png) - -You'll find the Ansible roles in the [source repository](https://github.com/ansible-community/ara/tree/feature/1.0/roles). - -# What comes after 1.0 - -In the near future, what is now the default master branch in the ara git -repository will be relocated to stable/0.x and future dot releases to ARA 0.x, -if necessary, will be tagged from that branch. - -The feature/1.0 branch will be relocated to the master branch and will be where -development will happen moving forward. - -A similar switch will happen for the documentation on -[ara.readthedocs.org](https://ara.readthedocs.org) where the default -documentation will now be for 1.0 and the 0.x documentation will remain available. - -Afterwards, the plan is to keep iterating on ara as well as ara-web and resume -a regular pace of release. - -# Questions ? Issues ? Feedback ? - -Please reach out ! - -You can find us on [IRC and Slack](https://ara.recordsansible.org/community/) -and issues are tracked on GitHub for both -[ara](https://github.com/ansible-community/ara/issues) and -[ara-web](https://github.com/ansible-community/ara-web/issues). diff --git a/website/content/blog/2019_06_04-announcing-the-release-of-ara-1.0/live-demo-playbook.png b/website/content/blog/2019_06_04-announcing-the-release-of-ara-1.0/live-demo-playbook.png deleted file mode 100644 index d81a43f..0000000 Binary files a/website/content/blog/2019_06_04-announcing-the-release-of-ara-1.0/live-demo-playbook.png and /dev/null differ diff --git a/website/content/blog/2020-04-14-ara-0.16.7-and-end-of-life-for-0.x/index.md b/website/content/blog/2020-04-14-ara-0.16.7-and-end-of-life-for-0.x/index.md deleted file mode 100644 index 12b1f2d..0000000 --- a/website/content/blog/2020-04-14-ara-0.16.7-and-end-of-life-for-0.x/index.md +++ /dev/null @@ -1,45 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - news -tags: - - ansible -date: 2020-04-14 -title: "ARA 0.16.7 and end of life for 0.x" -slug: ara-0.16.7-and-end-of-life-for-0.x -type: post ---- - -0.16.7 is a maintenance release for ARA 0.x. - -Changes since 0.16.6: - -- Fix typo in ara.setup.env for ANSIBLE_ACTION_PLUGINS ([pull request](https://github.com/ansible-community/ara/pull/97)) -- Pin pyfakefs to <4 in order to avoid breaking python2 usage ([issue](https://github.com/ansible-community/ara/issues/118)) -- Pin junit-xml to <=1.8 in order to avoid deprecation warnings in unit tests - -## ARA 0.x end of life - -The code base for ARA 0.x has not been actively maintained and developed -since 2018 and will officially reach end of life June 4th, 2019, one year -after the release of ARA 1.0. - -Unless critical bugs are found between this release and June 4th, 0.16.7 -will be the last supported release of the [0.x branch](https://github.com/ansible-community/ara/tree/stable/0.x). - -Please use the latest version of ARA to benefit from the -new features and fixes. - -## Want to try ARA ? - -Have a look at the [quickstart](https://github.com/ansible-community/ara#quickstart) or -read the [installation](https://ara.readthedocs.io/en/latest/installation.html) -and [configuration](https://ara.readthedocs.io/en/latest/ansible-configuration.html) -documentation for more information. - -## Want to contribute, chat or need help ? - -ARA could use your help and we can also help you get started. -Please reach out ! - -The project community hangs out on [IRC and Slack](https://ara.recordsansible.org/community/). diff --git a/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/host_search.png b/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/host_search.png deleted file mode 100644 index 4a235c1..0000000 Binary files a/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/host_search.png and /dev/null differ diff --git a/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/index.md b/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/index.md deleted file mode 100644 index f776564..0000000 --- a/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/index.md +++ /dev/null @@ -1,189 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - news -tags: - - ansible -date: 2020-04-15 -title: "Announcing the upcoming release of ARA 1.4.0" -slug: announcing-the-upcoming-release-of-ara-1.4.0 -type: post ---- - -1.4 is a significant milestone for the project and adds a number of new features, bugfixes and improvements. -The first release candidate, 1.4.0rc1, is available for testing from PyPi with ``pip install --pre ara[server]``. - -Here you can find some of the highlights for this upcoming release. - -For the full list of changes, see the [changelog](https://github.com/ansible-community/ara/releases/tag/1.4.0rc1) or the -[list of commits since 1.3.2](https://github.com/ansible-community/ara/compare/1.3.2...1.4.0rc1). - -## The ara_api Ansible role now supports EL8 - -The [ara_api Ansible role](https://github.com/ansible-community/ara/tree/master/roles/ara_api) can be used to deploy -and configure a production-ready ARA API server and, in fact, this is the role used to deploy the -[live demo](https://github.com/ansible-community/ara-infra/blob/master/playbooks/live-demo.yaml) on [api.demo.recordsansible.org](https://api.demo.recordsansible.org). - -The role is also used to integration test ARA itself and we've added support for deploying on EL 8 so we can have -test coverage on the new release of CentOS and RHEL. - -## Added support for searching plays, tasks and hosts by name - -Ansible doesn't have the concept of playbook names but ARA provides the ability to [give them one](https://ara.readthedocs.io/en/latest/playbook-names-and-labels.html). -This allows users to make playbook reports easier to find later. - -Plays and tasks will usually have a name, not just to make [ansible-lint](https://github.com/ansible/ansible-lint/blob/master/lib/ansiblelint/rules/TaskHasNameRule.py) -happy but because it improves readability and helps explain and describe what your roles and playbooks do. - -And then, you have hosts which, of course, will have a hostname or inventory name. - -In 1.4, you can now use the API to search for those names and the implementation supports partial matches so you can -search for only part of the name if need be. - -In practice, it's as simple as the following: - -- ``/api/v1/playbooks?name=production`` -- ``/api/v1/plays?name=deploy`` -- ``/api/v1/tasks?name=apache`` -- ``/api/v1/hosts?name=localhost`` - -So, for example, if you have a task that suddenly started failing, you can search for the task and compare it easily -with previous results of the same task: - -![task_search](task_search.png) - -Searching for a host name instead would allow you to see the results for this host across different Ansible playbook -runs: - -![host_search](host_search.png) - -## Playbook labels - -ARA allows users to configure labels for their Ansible playbook runs. - -For example, perhaps you run playbooks on different environments (prod, dev, staging) or maybe you run them on different -clouds or for different customers. -Another use case might be to tag your playbook after the control node that ran the playbook if you run playbooks from -multiple locations and want to categorize them. - -Tagging your playbooks with labels is meant to make your life easier if you want to find specific playbook runs later on. - -It was always intended for playbook labels to be searchable but support for it didn't land until now. -Now, you can search playbook reports matching only a specific label, making it simple for you to find what you're -looking for: - -- ``/api/v1/playbooks?label=production`` - -In addition, it is now possible to specify default labels which results in all playbooks being run to be tagged after -a set of default labels on top of the labels that might already be specified by the playbooks. - -You can learn more about how to use labels from the [documentation](https://ara.readthedocs.io/en/latest/playbook-names-and-labels.html). - -## Reversed default sort order for playbooks, plays, tasks and results - -Although it might be logical to return data and display elements in a chronological order, it also means that what -users are interested in is probably at the bottom because a failure typically means the end of a playbook. - -When browsing a list of playbooks, you'd also want the most recent playbook at the top rather than buried at the end -of the last page. - -So, in 1.4, we've reversed the default sort order so that the most recent items will be returned at the top instead -of at the bottom. - -## Querying the ARA API from inside playbooks with new plugins: ara_playbook and ara_api - -``ara_playbook`` is a new action plugin which queries the ARA API and returns data for the playbook that is currently running. -``ara_api`` is a new free-form lookup plugin to query anything and everything from the API. - -These new plugins are still somewhat experimental and might be rough around the edges but they are designed to be -convenient and flexible. -They do not require users to specify API endpoints or credentials since they are retrieved by the callback plugin to -record playbooks already. - -You can use these for anything you'd like but to inspire your creativity, a common use case from users is to craft -web URLs that lead directly to the playbook report. -However, in order to be able to do that, you need the playbook ID and this is where ``ara_playbook`` can help. - -Another example: ARA is using these plugins to test itself during [integration tests](https://github.com/ansible-community/ara/blob/master/tests/integration/lookups.yaml): -{{< highlight yaml >}} -- name: Assert playbook properties - hosts: localhost - gather_facts: yes - vars: - ara_playbook_name: ARA self tests - ara_playbook_labels: - - lookup-tests - tasks: - - name: Retrieve the current playbook so we can get the ID - ara_playbook: - register: playbook_query - - - name: Recover data from ARA - vars: - playbook_id: "{{ playbook_query.playbook.id | string }}" - set_fact: - playbook: "{{ lookup('ara_api', '/api/v1/playbooks/' + playbook_id) }}" - tasks: "{{ lookup('ara_api', '/api/v1/tasks?playbook=' + playbook_id) }}" - results: "{{ lookup('ara_api', '/api/v1/results?playbook=' + playbook_id) }}" - - - name: Assert playbook properties - assert: - that: - - playbook.name == 'ARA self tests' - - "playbook.labels | selectattr('name', 'search', 'lookup-tests') | list | length == 1" - - playbook.ansible_version == ansible_version.full - - playbook_dir in playbook.path - - "'tests/integration/lookups.yaml' in playbook.path" - - "playbook.files | length == playbook['items']['files']" - - "playbook.hosts | length == playbook['items']['hosts']" - - "playbook.plays | length == playbook['items']['plays']" - - "tasks.results | length == playbook['items']['tasks']" - - "results.results | length == playbook['items']['results']" -{{< /highlight >}} - -You can read more about these new plugins in the [documentation](https://ara.readthedocs.io/en/latest/ara-api-lookup.html). - -## Preventing sensitive files from being saved - -ARA will keep a unique, compressed copy of your playbook, role and variable files so that contextual data is readily -available when you browse and troubleshoot your playbooks. - -If you happen to have sensitive information in those files, you might not want ARA to pick them up so your credentials -or secrets are not exposed. - -In 1.4, you can now use the ``ARA_IGNORED_FILES`` or the ``ignored_files`` setting in the ansible.cfg configuration for -ARA in order to do just that. -This is a list of patterns and if a file matches a pattern, it will not be saved. - -So, if your Ansible vault files are named something like ``secrets.vault.yaml``, you could use ``secrets`` or ``vault`` as the pattern. - -This is a configuration for the callback plugin, not the API server. -For more information on how to configure the callback plugin behavior, see the [documentation](https://ara.readthedocs.io/en/latest/ara-plugin-configuration.html). - -## Improvements to the bundled web reporting interface - -[ara-web](https://github.com/ansible-community/ara-web) has unfortunately not been worked on but the bundled simple -reporting interface has been improving slowly but surely. - -In 1.4, the playbook report list is now paginated and not only has the label display been improved but you can also -search for playbooks by labels now that it is supported by the API: - -![ui_index](ui_index.png) - -# That's it for now ! - -There's plenty of work left to do but it will need to be in a future release ! - -## Want to try ARA ? - -Have a look at the [quickstart](https://github.com/ansible-community/ara#quickstart) or -read the [installation](https://ara.readthedocs.io/en/latest/installation.html) -and [configuration](https://ara.readthedocs.io/en/latest/ansible-configuration.html) -documentation for more information. - -## Want to contribute, chat or need help ? - -ARA could use your help and we can also help you get started. -Please reach out ! - -The project community hangs out on [IRC and Slack](https://ara.recordsansible.org/community/). diff --git a/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/task_search.png b/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/task_search.png deleted file mode 100644 index 7771c0a..0000000 Binary files a/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/task_search.png and /dev/null differ diff --git a/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/ui_index.png b/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/ui_index.png deleted file mode 100644 index e20f6d5..0000000 Binary files a/website/content/blog/2020-04-15-announcing-the-upcoming-release-of-ara-1.4.0/ui_index.png and /dev/null differ diff --git a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/host_search.png b/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/host_search.png deleted file mode 100644 index a513ad0..0000000 Binary files a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/host_search.png and /dev/null differ diff --git a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/index.md b/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/index.md deleted file mode 100644 index f3dddc1..0000000 --- a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/index.md +++ /dev/null @@ -1,153 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - news -tags: - - ansible -date: 2020-09-23 -title: "Announcing the release of ARA 1.5.0" -slug: announcing-the-release-of-ara-1.5.0 -type: post ---- - -ARA 1.5.0 as well as 1.5.1 (due to a release issue with PyPi) have been released ! - -For the full list of changes, see the [changelog on GitHub](https://github.com/ansible-community/ara/releases/tag/1.5.0) -as well as the list of [commits since 1.4.3](https://github.com/ansible-community/ara/compare/1.4.3...1.5.0). - -The changelogs and release notes for every version are now included in the [documentation](https://ara.readthedocs.io/en/latest/changelog-release-notes.html) as well. - -Check out the refreshed [getting started](https://ara.readthedocs.io/en/latest/getting-started.html) guide to try it out. - -Here you can find some of the highlights for this new release. - -## A new CLI - -1.5.0 ships with a brand new CLI client included in the main ara package. -The ``ara`` command allows you to query and browse playbooks and their results right from your terminal or shell scripts. - -You can see the full list of commands and their arguments in the [documentation](https://ara.readthedocs.io/en/latest/cli.html#cli-ara-api-client) -but here's a brief demonstration: - -*Note: [View directly on asciinema for full-width resolution](https://asciinema.org/a/361348)* - - - -## Search and pagination for playbook results - -Before 1.5.0, clicking on a playbook to view its details would return all of its task results which could -take a long time to load and render for playbooks with thousands of results. - -Results are now paginated which vastly improves performance and a search form was added to the task results panel: - -![result_search](result_search.png) - -The new search feature was implemented in the hosts summary table so clicking on a host or it's number of tasks for a -status will automatically search for it: - -![host_search](host_search.png) - -We're also leveraging this new search feature in the playbook list in order to send users to the most relevant results. - -For failed playbooks, we'll search for "failed" and "unreachable" results: - -![playbook_link_failed](playbook_link_failed.png) - -And for successful playbooks, we'll search for "changed" results: - -![playbook_link_ok](playbook_link_ok.png) - -## Playbook expiration - -Over time, ara could end up with a lot of playbooks stuck in the "running" status because they were interrupted or -would otherwise never complete for a variety of reasons. - -In 1.5.0, this is addressed by the implementation of a new status: "expired". -Combined with the new [ara expire](https://ara.readthedocs.io/en/latest/cli.html#ara-expire) command, playbooks stuck -in the "running" status can now be set to be expired after a specified amount of hours. - -ara doesn't have the machinery for doing or scheduling tasks automatically (yet?) but ``ara expire`` can be set up to run -from a cron like [ara playbook prune](https://ara.readthedocs.io/en/latest/cli.html#ara-playbook-prune) to keep the database tidy. - -## Argument labels - -A new feature for the callback plugin, [argument labels](https://ara.readthedocs.io/en/latest/ansible-plugins-and-use-cases.html#cli-argument-labels), -will automatically label playbooks after specified CLI arguments if they are used. - -For example, if ``--check`` is used and set up as an argument label, the playbook will be tagged with ``check:True`` or ``check:False`` if it wasn’t. - -In practice, this could be used to filter out dry-run playbooks where ``--check`` was used by searching for the ``check:False`` label. - -## Improved accuracy of task result durations - -Before 1.5.0, ara would use the start date of a task as the start date of a result which is accurate when running -against a single host but isn't when running against many hosts. - -Now, ara will leverage a new callback hook introduced by Ansible in 2.8: ``v2_runner_on_start``. -This new hook gives ara the ability to set an accurate start time for each individual result instead of the task start date. - -This is probably best explained by data and so given a list of 16 hosts with a simple ``sleep 5`` command task: - -Before, notice how the duration increments over time: - -``` -> ara result list --playbook 139 -+------+---------+----------+------+------+----------------------------------+-----------------+ -| id | status | playbook | task | host | started | duration | -+------+---------+----------+------+------+----------------------------------+-----------------+ -| 8930 | changed | 139 | 2249 | 3618 | 2020-09-22T18:31:45.341429-04:00 | 00:00:05.375164 | -| 8931 | changed | 139 | 2249 | 3619 | 2020-09-22T18:31:45.341429-04:00 | 00:00:05.461543 | -| 8932 | changed | 139 | 2249 | 3620 | 2020-09-22T18:31:45.341429-04:00 | 00:00:05.543938 | -| 8933 | changed | 139 | 2249 | 3621 | 2020-09-22T18:31:45.341429-04:00 | 00:00:05.591372 | -| 8934 | changed | 139 | 2249 | 3622 | 2020-09-22T18:31:45.341429-04:00 | 00:00:05.680345 | -| 8935 | changed | 139 | 2249 | 3623 | 2020-09-22T18:31:45.341429-04:00 | 00:00:10.729962 | -| 8936 | changed | 139 | 2249 | 3624 | 2020-09-22T18:31:45.341429-04:00 | 00:00:10.814804 | -| 8937 | changed | 139 | 2249 | 3625 | 2020-09-22T18:31:45.341429-04:00 | 00:00:10.898037 | -| 8938 | changed | 139 | 2249 | 3626 | 2020-09-22T18:31:45.341429-04:00 | 00:00:10.945747 | -| 8939 | changed | 139 | 2249 | 3627 | 2020-09-22T18:31:45.341429-04:00 | 00:00:11.625386 | -| 8940 | changed | 139 | 2249 | 3628 | 2020-09-22T18:31:45.341429-04:00 | 00:00:16.018695 | -| 8941 | changed | 139 | 2249 | 3629 | 2020-09-22T18:31:45.341429-04:00 | 00:00:16.105170 | -| 8942 | changed | 139 | 2249 | 3630 | 2020-09-22T18:31:45.341429-04:00 | 00:00:16.142404 | -| 8943 | changed | 139 | 2249 | 3631 | 2020-09-22T18:31:45.341429-04:00 | 00:00:16.218316 | -| 8944 | changed | 139 | 2249 | 3632 | 2020-09-22T18:31:45.341429-04:00 | 00:00:16.871900 | -| 8945 | changed | 139 | 2249 | 3633 | 2020-09-22T18:31:45.341429-04:00 | 00:00:21.239833 | -+------+---------+----------+------+------+----------------------------------+-----------------+ -``` - -After, each result is just about 5 seconds as we'd expect: -``` -> ara result list --playbook 138 -+------+---------+----------+------+------+----------------------------------+-----------------+ -| id | status | playbook | task | host | started | duration | -+------+---------+----------+------+------+----------------------------------+-----------------+ -| 8928 | changed | 138 | 2247 | 3616 | 2020-09-22T18:31:28.360197-04:00 | 00:00:05.253263 | -| 8927 | changed | 138 | 2247 | 3615 | 2020-09-22T18:31:23.371466-04:00 | 00:00:05.242033 | -| 8926 | changed | 138 | 2247 | 3614 | 2020-09-22T18:31:23.282524-04:00 | 00:00:05.250794 | -| 8925 | changed | 138 | 2247 | 3613 | 2020-09-22T18:31:23.233829-04:00 | 00:00:05.261368 | -| 8924 | changed | 138 | 2247 | 3612 | 2020-09-22T18:31:23.149980-04:00 | 00:00:05.263708 | -| 8923 | changed | 138 | 2247 | 3611 | 2020-09-22T18:31:23.097555-04:00 | 00:00:05.265748 | -| 8922 | changed | 138 | 2247 | 3610 | 2020-09-22T18:31:18.100408-04:00 | 00:00:05.276012 | -| 8921 | changed | 138 | 2247 | 3609 | 2020-09-22T18:31:18.013179-04:00 | 00:00:05.272808 | -| 8920 | changed | 138 | 2247 | 3608 | 2020-09-22T18:31:17.966802-04:00 | 00:00:05.271878 | -| 8919 | changed | 138 | 2247 | 3607 | 2020-09-22T18:31:17.883465-04:00 | 00:00:05.271284 | -| 8918 | changed | 138 | 2247 | 3606 | 2020-09-22T18:31:17.834228-04:00 | 00:00:05.269352 | -| 8915 | changed | 138 | 2247 | 3603 | 2020-09-22T18:31:12.522211-04:00 | 00:00:05.448203 | -| 8917 | changed | 138 | 2247 | 3605 | 2020-09-22T18:31:12.518245-04:00 | 00:00:05.587271 | -| 8914 | changed | 138 | 2247 | 3602 | 2020-09-22T18:31:12.514695-04:00 | 00:00:05.372740 | -| 8913 | changed | 138 | 2247 | 3601 | 2020-09-22T18:31:12.510951-04:00 | 00:00:05.328318 | -| 8916 | changed | 138 | 2247 | 3604 | 2020-09-22T18:31:12.505742-04:00 | 00:00:05.512212 | -+------+---------+----------+------+------+----------------------------------+-----------------+ -``` - -# That's it for now ! - -There's plenty of work left to do but it will need to be in a future release ! - -## Want to contribute, chat or need help ? - -ARA could use your help and we can also help you get started. -Please reach out ! - -The project community hangs out on [IRC and Slack](https://ara.recordsansible.org/community/). - -You can also stay up to date on the latest news by following [@RecordsAnsible](https://twitter.com/RecordsAnsible) on Twitter. diff --git a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/playbook_link_failed.png b/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/playbook_link_failed.png deleted file mode 100644 index 689d3e8..0000000 Binary files a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/playbook_link_failed.png and /dev/null differ diff --git a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/playbook_link_ok.png b/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/playbook_link_ok.png deleted file mode 100644 index 8710604..0000000 Binary files a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/playbook_link_ok.png and /dev/null differ diff --git a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/result_search.png b/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/result_search.png deleted file mode 100644 index f8aef59..0000000 Binary files a/website/content/blog/2020-09-23-announcing-the-release-of-ara-1.5.0/result_search.png and /dev/null differ diff --git a/website/content/blog/2020-11-01-benchmarking-ansible-and-ara-for-fun-and-science/index.md b/website/content/blog/2020-11-01-benchmarking-ansible-and-ara-for-fun-and-science/index.md deleted file mode 100644 index 51f430e..0000000 --- a/website/content/blog/2020-11-01-benchmarking-ansible-and-ara-for-fun-and-science/index.md +++ /dev/null @@ -1,215 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - experiments -tags: - - ansible -date: 2020-11-01 -title: "Benchmarking ansible and ara for fun and science" -slug: benchmarking-ansible-and-ara-for-fun-and-science -type: post ---- - -We've had issues opened about benchmarking and improving the performance of the -[ara callback plugin](https://github.com/ansible-community/ara/issues/171) as -well as the [API server](https://github.com/ansible-community/ara/issues/170) for -a while now and I only recently took a bit of time to tackle the callback. - -It paid off because we can already see significant performance benefits in -the 1.5.3 release of ara and it builds a foundation for future improvement -opportunities. - -If you'd like to see the raw unformatted data that was used for this post, check -out this [gist on GitHub](https://gist.github.com/dmsimard/ab6d53ac2edf3b4ab6076e20dcad8fe9). - -## A benchmarking playbook - -Whenever you want to improve something, it's important to measure it first so you -know how better (or worse!) things become as a result of your changes. - -The first step was to create a standardized benchmarking playbook that we could -run across a variety of configurations and parameters. - -The playbook was designed to run a specified number of tasks against a specified -number of hosts. - -It's available in the [git repository](https://github.com/ansible-community/ara/blob/master/tests/integration/benchmark.yaml) -but it's simple and small enough to include here: - -```yaml -# Copyright (c) 2020 The ARA Records Ansible authors -# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) - -- name: Create many hosts - hosts: localhost - gather_facts: no - vars: - benchmark_host_count: 25 - tasks: - - name: Add a host to the inventory - add_host: - ansible_connection: local - hostname: "host-{{ item }}" - groups: benchmark - with_sequence: start=1 end={{ benchmark_host_count }} - -- name: Run tasks on many hosts - hosts: benchmark - vars: - benchmark_task_file: "{{ playbook_dir }}/benchmark_tasks.yaml" - # Run N tasks per host - benchmark_task_count: 50 - # Off by default to prevent accidental load spike on localhost - benchmark_gather_facts: no - gather_facts: "{{ benchmark_gather_facts }}" - tasks: - - name: Include a task file - include_tasks: "{{ benchmark_task_file }}" - with_sequence: start=1 end={{ benchmark_task_count }} -``` - -and then the [benchmark_tasks.yaml](https://github.com/ansible-community/ara/blob/master/tests/integration/benchmark_tasks.yaml) file: - -```yaml -# Copyright (c) 2020 The ARA Records Ansible authors -# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) - -# These are tasks meant to be imported by benchmark.yaml - -- name: Run a task - debug: - msg: "{{ inventory_hostname }} running task {{ item }}/{{ benchmark_task_count }}" -``` - -## Methodology - -All tests ran under Ansible 2.10.2 and ``ANSIBLE_FORKS=50`` using the default sqlite database backend for ara. - -The benchmark playbook was run three times: - -```bash -# 25 hosts and 50 tasks: 1276 results -ansible-playbook -i 'localhost,' -c local tests/integration/benchmark.yaml - -# 100 hosts and 50 tasks: 5101 results -ansible-playbook -i 'localhost,' -c local tests/integration/benchmark.yaml \ - -e benchmark_host_count=100 - -# 200 hosts and 200 tasks: 40201 results -ansible-playbook -i 'localhost,' -c local tests/integration/benchmark.yaml \ - -e benchmark_host_count=200 \ - -e benchmark_task_count=200 -``` - -*Note: localhost and two bootstrap tasks are included in the results below* - -## ansible without ara - -| tasks | hosts | results | duration | -|-------|-------|---------|----------| -| 52 | 26 | 1276 | 0m 11s | -| 52 | 101 | 5101 | 0m 41s | -| 202 | 201 | 40201 | 6m 01s | - -Our control: basically how much time these playbooks take to run without ara enabled so we can calculate the overhead -and performance when enabling the ara callback. - -## ansible with ara 1.5.1 - -| api client | api server | tasks | hosts | results | duration | -|------------|------------|-------|-------|---------|----------| -| offline | django | 52 | 26 | 1276 | 0m 56s | -| http | django | 52 | 26 | 1276 | 0m 32s | -| http | gunicorn | 52 | 26 | 1276 | 0m 36s | -| offline | django | 52 | 101 | 5101 | 3m 31s | -| http | django | 52 | 101 | 5101 | 1m 39s | -| http | gunicorn | 52 | 101 | 5101 | 2m 19s | -| offline | django | 202 | 201 | 40201 | 30m22s | -| http | django | 202 | 201 | 40201 | 17m28s | -| http | gunicorn | 202 | 201 | 40201 | 21m38s | - -1.5.1 is the latest version that didn't implement threading inside the callback plugin. - -It's curious that the django built-in webserver outperformed running with gunicorn when using the http client. -I was not able to reproduce this result in 1.5.3. - -I was aware that there was an overhead when enabling the callback but never realized -the performance hit was this much until now, taking the time to accurately measure it: - -| results | without ara | 1.5.1 | overhead | -|---------|-------------|--------|----------| -| 1276 | 11s | 32s | 21s | -| 5101 | 41s | 1m39s | 58s | -| 40201 | 6m01s | 17m28s | ~11.4m | - -## ansible with ara 1.5.3 - -| api client | api server | tasks | hosts | results | duration | -|------------|------------|-------|-------|---------|----------| -| offline | django | 52 | 26 | 1276 | 0m 52s | -| http | django | 52 | 26 | 1276 | 0m 30s | -| http | gunicorn | 52 | 26 | 1276 | 0m 20s | -| offline | django | 52 | 101 | 5101 | 3m 22s | -| http | django | 52 | 101 | 5101 | 1m 37s | -| http | gunicorn | 52 | 101 | 5101 | 1m 09s | -| offline | django | 202 | 201 | 40201 | 29m25s | -| http | django | 202 | 201 | 40201 | 17m24s | -| http | gunicorn | 202 | 201 | 40201 | 13m47s | - -1.5.2 introduced threading in the callback and then 1.5.3 was subsequently released to workaround -an [issue when using the offline client](https://github.com/ansible-community/ara/issues/183), forcing it to use a single thread for now. - -From the table above, we can tell: - -- Running a single thread with the offline client is just about the same performance as 1.5.1 without threading, if only a little bit faster. -- There is a significant improvement in performance due to the multi-threading when using the http client -- Running the API server with gunicorn outperforms using the built-in django development server - -1.5.3 reduced the overhead of the callback plugin quite a bit when comparing to 1.5.1: - -| results | without ara | 1.5.3 | overhead | -|---------|-------------|--------|----------| -| 1276 | 11s | 20s | 9s | -| 5101 | 41s | 1m 09s | 28s | -| 40201 | 6m 01s | 13m47s | ~7.8m | - -## For science: ara 0.16.8 - -| tasks | hosts | results | duration | -|-------|-------|---------|----------| -| 52 | 26 | 1276 | 0m 28s | -| 52 | 101 | 5101 | 1m 56s | -| 202 | 201 | 40201 | 19m05s | - -Although ara 0.x is no longer supported, it turns out it still works if you use the [stable/0.x](https://github.com/ansible-community/ara/tree/stable/0.x) git branch.. even with Ansible 2.10! - -It was interesting to run the same benchmark against 0.x because it runs a completely different backend. -It uses flask instead of django and doesn't provide an API: the callback talks directly to the database through flask-sqlalchemy. - -## Putting it all together - -Tallying up the numbers, we can see that we're on the right track and performance is improving: - -| tasks | hosts | results | without ara | 0.16.8 | 1.5.1 | 1.5.3 | -|-------|-------|---------|-------------|--------|---------|---------| -| 52 | 26 | 1276 | 11s | 28s | 32s | 20s | -| 52 | 101 | 5101 | 41s | 1m 56s | 1m 39s | 1m 09s | -| 202 | 201 | 40201 | 6m 01s | 19m05s | 17m28s | 13m47s | - -There is definitely more work to do and more opportunities to improve performance to find. -There will unfortunately always be an overhead but it needs to be low enough that it's worth it without sacrificing simplicity. - -In the future, it could be interesting the measure the impact of other parameters on performance like: - -- Ansible forks -- what difference does having 25, 100 or 200 forks make ? -- Callback threads -- is there a benefit when running more threads in the threadpool ? -- Version of Python -- is there any difference between python 3.5 and 3.9 ? -- Version of Ansible -- was there any performance improvements or regressions between 2.8 and 2.10 ? -- Database backend -- is sqlite faster than mysql ? what about postgresql ? -- Application backend -- is gunicorn faster than uwsgi ? what about apache mod_wsgi ? -- Latency: what's the impact on performance of adding a jump box ? what about 50ms ? 250ms ? - -If you'd like to help, have a look at the issues on GitHub or come chat with us -on [Slack or IRC](https://ara.recordsansible.org/community/) ! - -See you around o/ diff --git a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/index.md b/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/index.md deleted file mode 100644 index 1d7bf8f..0000000 --- a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/index.md +++ /dev/null @@ -1,161 +0,0 @@ ---- -author: "David Moreau Simard" -categories: - - experiments -tags: - - ansible -date: 2021-01-30 -title: "Benchmarking Ansible and python versions for fun and science" -slug: benchmarking-ansible-and-python-versions-for-fun-and-science -type: post ---- - -In the last [blog post](https://ara.recordsansible.org/blog/2020/11/01/benchmarking-ansible-and-ara-for-fun-and-science/), -we benchmarked the overhead of recording Ansible playbooks with ara. - -There's still room for improvement but we measured that it was on the right track and getting better. -The data is useful so we can tell whether we're improving things or making them worse ! - -ara 1.5.5 adds support for searching playbooks by Ansible versions and I was curious what kind of data we would get if we ran our benchmark with different versions of Ansible. -I also want to see if there is a difference when running ansible and ara under different python versions. - -Feel free to skip to the end for the numbers but if you're interested in the setup and how we'll run the benchmark, read on ! - -## About the API server - -By default, you don't need to run an API server and ara records Ansible playbooks to a local sqlite database. -It looks like this: - -```bash -# Install Ansible and ARA (with API server dependencies) for the current user -python3 -m pip install --user ansible "ara[server]" - -# Configure Ansible to use the ARA callback plugin -export ANSIBLE_CALLBACK_PLUGINS="$(python3 -m ara.setup.callback_plugins)" - -# Run an Ansible playbook -ansible-playbook playbook.yaml - -# Use the CLI to see recorded playbooks -ara playbook list - -# Start the built-in server to browse recorded results -ara-manage runserver -``` - -For the purpose of our tests, we'll record the results to a running API server where you can also browse the results: [demo.recordsansible.org](https://demo.recordsansible.org). - -It is a single VM with 8 cores, 16GB of RAM and backed by SSD storage running: - -- CentOS 8 with python 3.6.8 -- ara 1.5.5rc1 -- gunicorn backend (django app, ara api server) -- nginx reverse proxy with ssl termination (thanks [letsencrypt](https://letsencrypt.org/)!) -- mysql (mariadb 10.3) backend: so we are less prone to [sqlite database lock contention](https://github.com/ansible-community/ara/issues/195) (threading is hard) - -It's deployed with the [ara collection](https://github.com/ansible-community/ara-collection) and the [geerlingguy.mysql](https://github.com/geerlingguy/ansible-role-mysql) role for the mysql backend. - -## Running our playbooks - -Keeping latency to a minimum, we'll run our ``ansible-playbook`` commands from a Fedora 33 VM on the same hypervisor. -The goal this time around isn't to measure performance with remote hosts so we'll run a specified number of Ansible tasks to run against "fake" hosts which are, in fact, localhost. - -This is what it looks like: - -```bash -# Install Ansible and ARA (without API server dependencies) for the current user -python3 -m pip install --user ansible==$ANSIBLE_VERSION ara - -# Set a modest amount of forks and configure Ansible to use the ARA callback plugin -export ANSIBLE_FORKS=50 -export ANSIBLE_CALLBACK_PLUGINS="$(python3 -m ara.setup.callback_plugins)" - -# Configure the ARA callback to know where the API server is located and enable threading -export ARA_API_CLIENT="http" -export ARA_API_SERVER="https://demo.recordsansible.org" -export ARA_API_USERNAME="*****" -export ARA_API_PASSWORD="*****" -export ARA_CALLBACK_THREADS=4 - -# Put our version of python into an ara label so we can easily search for this playbook later -export python_version=$(python3 -c "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}.{sys.version_info.micro}')") -export ARA_DEFAULT_LABELS="benchmarking,python_version:${python_version}" - -# Retrieve the benchmark playbook and run it against localhost -git clone https://github.com/ansible-community/ara ara-src -cd ara-src -time ansible-playbook -i "localhost," -c local tests/integration/benchmark.yaml \ - -e "benchmark_host_count=100" \ - -e "benchmark_task_count=50" -``` - -## Ansible versions with python 3.9.1 - -I ran the playbook three times for each of the following Ansible versions: - -- ``ansible-base==2.10.5`` (provided when installing ``ansible==2.10.6``) -- ``ansible==2.9.17`` -- ``ansible==2.8.18`` -- ``ansible==2.7.17`` - -I haven't put effort into testing ansible<=2.6 for now because it was broken and has been out of maintenance since 2019. - -In total, ara recorded 12 playbooks and ran 624 tasks across 1212 hosts for a total of 61212 results: - -![playbook-metrics](playbook-metrics.png) - -All versions performed quite comparably, though we can see a trend from 2.7.17 through 2.10.5: - -![version-metrics](version-metrics.png) - -More specifically: - -![playbook-list](playbook-list.png) - -It's an interesting trend: with everything else being equal, it's getting a little bit slower for each series of releases. -We'll have to find out later if this continues when [ansible-core 2.11](https://docs.ansible.com/ansible/devel/roadmap/ROADMAP_2_11.html) succeeds to ansible-base 2.10. - -## Python versions with ansible-base 2.10.5 - -What about if we run the same experiment by switching python versions instead ? - -In a similar pattern, I ran the playbook three times for each of the following Python versions: - -- 3.9.1 (default on Fedora 33 and already done when we tested different Ansible versions) -- 3.8.7 -- 3.7.9 -- 3.6.12 -- 3.5.10 - -I did this by installing ara and ansible under specific interpreters, for example: -``/usr/bin/python3.8 -m pip install --user ansible==2.10.6 ara`` - -![python-versions](python-versions.png) - -# Putting it all together - -I haven't tested all possible combinations but we have a lot of data already with the 24 playbooks we've run. -Picking the middle result out of three tests because the truth probably lies somewhere between the best and the worst run, we have the following table: - -| ansible | python 3.9.1 | python 3.8.7 | python 3.7.9 | python 3.6.12 | python 3.5.10 | -|---------|--------------|--------------|--------------|---------------|---------------| -| 2.10.5 | 01:24.77 | 01:23.92 | 01:27.04 | 01:28.30 | 01:28.75 | -| 2.9.17 | 01:23.03 | | | | | -| 2.8.18 | 01:22.42 | | | | | -| 2.7.17 | 01:21.93 | | | | | - -Again, these results are quite comparable but there is a small difference. - -Considering this is a synthetic benchmark that only iterates over 5000 debug tasks, it isn't representative of real-life scenarios where the performance can vary on a number of factors including but not limited to: -- the latency between hosts -- whether you're tunneling through a jump box or bastion host (instead of running from the jump box/bastion host itself) -- the performance of the tasks themselves (like how long it takes to install a package from a mirror, start a service, etc) -- tuning ansible forks, ssh pipelining, etc - -More things to benchmark in the future maybe ! - -In the meantime, if you'd like to explore the data from these benchmarks, they're available on [demo.recordsansible.org](https://demo.recordsansible.org/?label=benchmarking). - -Until then, if you have any questions or would like to chat, come join us on [Slack or IRC](https://ara.recordsansible.org/community/) ! - -o/ \ No newline at end of file diff --git a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/playbook-list.png b/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/playbook-list.png deleted file mode 100644 index e62b97d..0000000 Binary files a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/playbook-list.png and /dev/null differ diff --git a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/playbook-metrics.png b/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/playbook-metrics.png deleted file mode 100644 index 9f56adc..0000000 Binary files a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/playbook-metrics.png and /dev/null differ diff --git a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/python-versions.png b/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/python-versions.png deleted file mode 100644 index 8998748..0000000 Binary files a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/python-versions.png and /dev/null differ diff --git a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/version-metrics.png b/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/version-metrics.png deleted file mode 100644 index 03e0f5a..0000000 Binary files a/website/content/blog/2021-01-30-benchmarking-ansible-and-python-versions-for-fun-and-science/version-metrics.png and /dev/null differ diff --git a/website/content/community.md b/website/content/community.md deleted file mode 100644 index 6a44e81..0000000 --- a/website/content/community.md +++ /dev/null @@ -1,17 +0,0 @@ ---- -title: "Community & Help" -slug: community -type: page ---- - -Need help, want to contribute or just chat about ARA with the community ? - -![community](/static/ansiblefest-community.png) - -There's multiple ways to stay up to date and get in touch: - -- IRC: [#ara](https://webchat.freenode.net/?channels=#ara) on the freenode network -- Slack: [arecordsansible.slack.com](https://join.slack.com/t/arecordsansible/shared_invite/enQtMjMxNzI4ODAxMDQxLTU2NTU3YjMwYzRlYmRkZTVjZTFiOWIxNjE5NGRhMDQ3ZTgzZmQyZTY2NzY5YmZmNDA5ZWY4YTY1Y2Y1ODBmNzc) -- Twitter: [@RecordsAnsible](https://twitter.com/RecordsAnsible) - -See you there ! diff --git a/website/content/static/ansiblefest-community.png b/website/content/static/ansiblefest-community.png deleted file mode 100644 index e754eb2..0000000 Binary files a/website/content/static/ansiblefest-community.png and /dev/null differ diff --git a/website/content/static/ara-full-logo.png b/website/content/static/ara-full-logo.png deleted file mode 100644 index 1bcc084..0000000 Binary files a/website/content/static/ara-full-logo.png and /dev/null differ diff --git a/website/content/static/ara-quickstart-default.gif b/website/content/static/ara-quickstart-default.gif deleted file mode 100644 index 051519e..0000000 Binary files a/website/content/static/ara-quickstart-default.gif and /dev/null differ diff --git a/website/content/static/ara-quickstart-server.gif b/website/content/static/ara-quickstart-server.gif deleted file mode 100644 index 85d06e8..0000000 Binary files a/website/content/static/ara-quickstart-server.gif and /dev/null differ diff --git a/website/content/static/cli-playbook-list.png b/website/content/static/cli-playbook-list.png deleted file mode 100644 index e6c7fa6..0000000 Binary files a/website/content/static/cli-playbook-list.png and /dev/null differ diff --git a/website/content/static/favicon.ico b/website/content/static/favicon.ico deleted file mode 100644 index 3755e79..0000000 Binary files a/website/content/static/favicon.ico and /dev/null differ diff --git a/website/content/static/logo.png b/website/content/static/logo.png deleted file mode 100644 index 1eda4d3..0000000 Binary files a/website/content/static/logo.png and /dev/null differ diff --git a/website/content/static/recording-workflow.png b/website/content/static/recording-workflow.png deleted file mode 100644 index 78af02c..0000000 Binary files a/website/content/static/recording-workflow.png and /dev/null differ diff --git a/website/content/static/reports.png b/website/content/static/reports.png deleted file mode 100644 index a99df45..0000000 Binary files a/website/content/static/reports.png and /dev/null differ diff --git a/website/content/static/ui-api-browser.png b/website/content/static/ui-api-browser.png deleted file mode 100644 index 59435f0..0000000 Binary files a/website/content/static/ui-api-browser.png and /dev/null differ diff --git a/website/content/static/ui-playbook-details.png b/website/content/static/ui-playbook-details.png deleted file mode 100644 index bc39b4a..0000000 Binary files a/website/content/static/ui-playbook-details.png and /dev/null differ diff --git a/website/layouts/index.html b/website/layouts/index.html deleted file mode 100644 index 338afea..0000000 --- a/website/layouts/index.html +++ /dev/null @@ -1,13 +0,0 @@ -{{ partial "header" . }} - {{ $.Scratch.Set "shareNav" true }} - {{ partial "navbar" . }} - -
-
-
- {{ .Content }} -
-
-
- {{ partial "sidebar" . }} -{{ partial "footer" . }} diff --git a/website/layouts/partials/favicon.html b/website/layouts/partials/favicon.html deleted file mode 100644 index c6956d1..0000000 --- a/website/layouts/partials/favicon.html +++ /dev/null @@ -1,4 +0,0 @@ -{{ if .Site.Params.loadFavicon }} - - -{{ end }} diff --git a/website/layouts/partials/gitinfo.html b/website/layouts/partials/gitinfo.html deleted file mode 100644 index 64b8c83..0000000 --- a/website/layouts/partials/gitinfo.html +++ /dev/null @@ -1,8 +0,0 @@ -{{ with .GitInfo }} - -
-

Page source

-

This page was last updated from source: {{ .AuthorDate }}

-

Git commit: {{ .Hash }}

-
-{{ end }} diff --git a/website/layouts/partials/sidebar.html b/website/layouts/partials/sidebar.html deleted file mode 100644 index 9db94d6..0000000 --- a/website/layouts/partials/sidebar.html +++ /dev/null @@ -1,139 +0,0 @@ - -