Add allow-nonvoting key

By default we add 'voting:1' to all queries so they are filtered
from the graph.

There are cases where we want to track bugs for non-voting jobs
in the graph, so add an 'allow-nonvoting' key for special bugs
that we want to track while we're trying to stabilize a job.

Updates the docs, adds a test query and unit test for the query
loader, fixes a bad query that was setting voting:1 already, and
removes the TODO in the query for bug 1539271.

Change-Id: I61b1d4b0faaa3f20fc5b04aa3392c68c61990866

README.rst

@@ -114,6 +114,11 @@ stack that can cause many tempest tests to fail.
      tracking race failures in non-voting jobs since they are considered
      unstable by definition (since they don't vote).
 
+     - There is, however, a special 'allow-nonvoting' key that can be added
+       to a query yaml file to allow tracking non-voting job bug failures in
+       the graph. They won't show up in the bot though (IRC or Gerrit
+       comments).
+
 #. Go to `logstash.openstack.org <http://logstash.openstack.org/>`_ and create
    an elastic search query to find the log message from step 1. To see the
    possible fields to search on click on an entry. Lucene query syntax is

elastic_recheck/loader.py

@@ -30,6 +30,10 @@ def load(directory='queries'):
         bugnum = os.path.basename(fname).rstrip('.yaml')
         query = yaml.load(open(fname).read())
         query['bug'] = bugnum
-        query['query'] = "%s AND voting:1" % query['query'].rstrip()
+        # By default we filter out non-voting jobs, but in certain cases we
+        # want to show failures for non-voting jobs in the graph while we
+        # stabilize a job, so check for a special 'allow-nonvoting' key.
+        if not query.get('allow-nonvoting', False):
+            query['query'] = "%s AND voting:1" % query['query'].rstrip()
         data.append(query)
     return data

elastic_recheck/tests/unit/queries/1539271.yaml

@@ -0,0 +1,4 @@
+query: >
+  message:"Migration running for 30 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0)"
+  AND tags:"screen-n-cpu.txt"
+allow-nonvoting: true

elastic_recheck/tests/unit/test_load_queries.py

@@ -29,6 +29,11 @@ class TestLoadQueries(tests.TestCase):
         for q in queries:
             self.assertIsNotNone(q['bug'])
             self.assertIsNotNone(q['query'])
+            # check for the allow-nonvoting flag
+            if q['bug'] == '1539271':
+                self.assertNotIn('voting:1', q['query'])
+            else:
+                self.assertIn('voting:1', q['query'])
 
     def test_grenade_compat(self):
         # grenade logs are in logs/new/ and logs/old, while devstack is in

queries/1532809.yaml

@@ -1,3 +1,3 @@
 query: >-
-  message:"No lease, failing" AND voting:1
+  message:"No lease, failing" AND
   tags:"console"

queries/1539271.yaml

@@ -1,4 +1,5 @@
-# TODO(sdague): add a flag to not ignore non-voting jobs so this shows up in the status page
+# This is used to track live migration failures in the non-voting multinode job
 query: >
   message:"Migration running for 30 secs, memory 100% remaining; (bytes processed=0, remaining=0, total=0)"
   AND tags:"screen-n-cpu.txt"
+allow-nonvoting: true