db01ca6d16
As can be seen in logs of the periodic generation job, our cgit does a weird thing where sometimes it returns a 404 page with content, and sometimes a zero response (see [1] for example, the last number is response size). This appears to be an openstack CI issue; possibly due to cgit caching or similar (see [2] for manual test). It will have to be investigated with the host apache logs. This is resulting in a lot of projects incorrectly being picked up as having plugins (I7116571d2a2b1fc3a61e5f1ed46ac2cbc244775a). I'm not sure if this problem is also releated to the original status-code issues mentioned in the code, but testing shows that cgit is correctly returning 404's for missing files (you can see in the logs [1]). Thus switch the logic to examine the return code which avoids this issue. [1] http://logs.openstack.org/periodic/propose-devstack-plugins-list/e55790c/console.html.gz#_2016-05-04_06_46_51_660 [2] http://paste.openstack.org/show/496434/ Change-Id: I6a06347d91d091441f6f7b70f99aba6d8e9add4b
62 lines
1.9 KiB
Python
62 lines
1.9 KiB
Python
#! /usr/bin/env python
|
|
|
|
# Copyright 2016 Hewlett Packard Enterprise Development Company, L.P.
|
|
#
|
|
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
|
# not use this file except in compliance with the License. You may obtain
|
|
# a copy of the License at
|
|
#
|
|
# http://www.apache.org/licenses/LICENSE-2.0
|
|
#
|
|
# Unless required by applicable law or agreed to in writing, software
|
|
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
|
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
|
# License for the specific language governing permissions and limitations
|
|
# under the License.
|
|
|
|
# This script is intended to be run as part of a periodic proposal bot
|
|
# job in OpenStack infrastructure.
|
|
#
|
|
# In order to function correctly, the environment in which the
|
|
# script runs must have
|
|
# * network access to the review.openstack.org Gerrit API
|
|
# working directory
|
|
# * network access to https://git.openstack.org/cgit
|
|
|
|
import logging
|
|
import json
|
|
import requests
|
|
|
|
logging.basicConfig(level=logging.DEBUG)
|
|
|
|
url = 'https://review.openstack.org/projects/'
|
|
|
|
# This is what a project looks like
|
|
'''
|
|
"openstack-attic/akanda": {
|
|
"id": "openstack-attic%2Fakanda",
|
|
"state": "READ_ONLY"
|
|
},
|
|
'''
|
|
|
|
def is_in_openstack_namespace(proj):
|
|
# only interested in openstack namespace (e.g. not retired
|
|
# stackforge, etc)
|
|
return proj.startswith('openstack/')
|
|
|
|
# Check if this project has a plugin file
|
|
def has_devstack_plugin(proj):
|
|
r = requests.get("https://git.openstack.org/cgit/%s/plain/devstack/plugin.sh" % proj)
|
|
return r.status_code == 200
|
|
|
|
logging.debug("Getting project list from %s" % url)
|
|
r = requests.get(url)
|
|
projects = sorted(filter(is_in_openstack_namespace, json.loads(r.text[4:])))
|
|
logging.debug("Found %d projects" % len(projects))
|
|
|
|
found_plugins = filter(has_devstack_plugin, projects)
|
|
|
|
for project in found_plugins:
|
|
# strip of openstack/
|
|
print(project[10:])
|