Refactor "emit-ara-html" into a new role: "ara-report"

We are going to provide support the ARA sqlite middleware which
provides the ability for an ARA web application to dynamically load
databases, eliminating the need for HTML generation.

HTML generation does not scale very well: it's orders of magnitude
larger than the database file containing the data and it also takes
a significant amount of time to generate the report.

It would be awkward to add dynamic database things into a role called
"emit-ara-html" and so I took the opportunity to refactor things a
little bit and make the role less prone to failures.

Note that this role was developed with the nested Ansible use case in
mind -- a job running it's own copy of Ansible with ARA should be able
to use this role to generate the report or save the database.

This role will be tested with 'base-test' first and it is expected to
make the 'emit-ara-html' role obsolete if everything works well.

Change-Id: Idedc0bfa1f0f89356b795fb9e2a16f9421a2dc18
Depends-On: I3b10c93b4902a9b45e23c227863e472697f662ef

roles/ara-report/README.rst

@@ -0,0 +1,59 @@
+If ARA is enabled, generates a report or saves a copy of the ARA database.
+
+**Role Variables**
+
+.. zuul:rolevar:: ara_report_run
+   :default: ``true``
+
+   Whether to run this role or not.
+   Possible values:
+
+   - ``true`` (always run)
+   - ``false`` (never run)
+   - ``failure`` (only run when there has been a failure)
+
+.. zuul:rolevar:: ara_database_path
+   :default: ``{{ zuul.executor.work_root }}/.ara/ansible.sqlite``
+
+   Absolute path where the ARA database is expected on the control node.
+   This should be where the ansible-playbook execution had ARA save the host,
+   task and result data if you provided a custom location through
+   ``ARA_DATABASE`` or an ``ansible.cfg`` file.
+
+.. zuul:rolevar:: ara_report_type
+   :default: ``html``
+
+   Possible values:
+
+   - ``html``
+   - ``database``
+
+   ``html`` will have ARA generate and save a statically generated HTML report
+   inside ``ara_report_path``.
+
+   ``database`` will only save the raw ARA sqlite database inside
+   ``ara_report_path``. The database can then be downloaded by users or loaded
+   dynamically by the ``ara-wsgi-sqlite`` middleware.
+
+   See the `ARA documentation`_ for details.
+
+.. _ARA documentation: https://ara.readthedocs.io/en/latest/advanced.html
+
+.. zuul:rolevar:: ara_compress_html
+   :default: ``true``
+
+   When report_type is 'html', whether to compress the ARA HTML output or not.
+
+.. tip::
+   Make sure the web server is configured to set the required mimetypes_ in
+   order to serve gzipped content properly.
+
+.. _mimetypes: https://git.openstack.org/cgit/openstack-infra/puppet-openstackci/tree/templates/logs.vhost.erb?id=5fe1f3d2d5e40c2458721e7dcf8631d62ea2525f#n24
+
+.. zuul:rolevar:: ara_report_path
+   :default: ``ara``
+
+   This path is relative to the root of the log directory.
+
+   When report_type is 'html' directory where the HTML report will be generated.
+   When report_type is 'database', directory where the database is saved.

roles/ara-report/defaults/main.yaml

@@ -0,0 +1,26 @@
+# Whether to run this role or not
+# Possible values:
+#   - true (always)
+#   - false (never)
+#   - 'failure' (only on failure)
+ara_report_run: true
+
+# Absolute path where the ARA database is expected on the control node.
+ara_database_path: "{{ zuul.executor.work_root }}/.ara/ansible.sqlite"
+
+# Whether the report is in html format or in database format
+# Possible values:
+#  - html
+#  - database
+ara_report_type: 'html'
+
+# When report_type is 'html' directory where the HTML report will be generated.
+# When report_type is 'database', directory where the database is saved.
+# Note that when using 'database', an ARA web application can load the database
+# dynamically. See the ARA documentation for details:
+# https://ara.readthedocs.io/en/latest/advanced.html#serving-ara-sqlite-databases-over-http
+# This path is relative to the root of the log directory.
+ara_report_path: 'ara'
+
+# When report_type is 'html', whether to compress the ARA HTML output or not
+ara_compress_html: true

roles/ara-report/tasks/main.yaml

@@ -0,0 +1,81 @@
+- block:
+    - name: Check that ARA is installed
+      command: bash -c "type -p ara"
+      ignore_errors: yes
+      register: ara_installed
+
+    - name: Warn if ARA is not installed
+      debug:
+        msg: |
+          ARA is not installed on the executor node, no report will be available.
+      when: ara_installed.rc != 0
+
+    - name: Check that the ARA database exists
+      stat:
+        path: "{{ ara_database_path }}"
+      register: ara_db
+
+    - name: Warn if no database is found
+      debug:
+        msg: |
+          No report will be available because the ARA database was not found.
+      when: not ara_db.stat.exists
+
+    - name: Validate role configuration
+      assert:
+        that:
+          - ara_report_type in ['html', 'database']
+          - ara_compress_html in [true, false]
+          - ara_report_run in [true, false, 'failure']
+  rescue:
+    - name: Role validation rescue
+      debug:
+        msg: |
+          Something failed during the validation of the role configuration
+          and pre-requirements.
+          It is likely that no report will be available, please verify the
+          execution and the parameters of the role for details.
+
+- name: Prefix the log path with the log root
+  set_fact:
+    ara_report_path: "{{ zuul.executor.log_root }}/{{ ara_report_path }}"
+
+- when:
+    - ara_installed.rc == 0
+    - ara_db.stat.exists
+    - ara_report_type == 'html'
+  block:
+    # Always generate (true), never (false) or only on failure ('failure')
+    # Additionally cover for edge cases where zuul_success might be undefined
+    - name: Generate ARA HTML output
+      command: "ara generate html {{ ara_report_path }}"
+      environment:
+        ARA_DATABASE: "sqlite:///{{ ara_database_path }}"
+      when: ara_report_run | bool or
+            (ara_report_run == 'failure' and not zuul_success | default(false) | bool)
+      register: ara_generated
+
+    - name: Compress ARA HTML output
+      command: gzip --recursive --best {{ ara_report_path }}
+      when:
+        - ara_compress_html | bool
+        - not ara_generated | skipped
+  rescue:
+    - name: HTML generation rescue
+      debug:
+        msg: |
+          Something failed during the generation of the HTML report.
+          Please verify the execution of the role for details.
+
+- when:
+    - ara_installed.rc == 0
+    - ara_db.stat.exists
+    - ara_report_type == 'database'
+  block:
+    - name: Create the ARA database report directory
+      file:
+        path: "{{ ara_report_path }}"
+        state: directory
+
+    - name: Save the ARA database
+      command: cp {{ ara_database_path }} {{ ara_report_path }}