Centralize Release Tagging
Story:2000292 Change-Id: Ie40ea6e7538bf4e5f6b67d8604bc3b524ca8e9e6
This commit is contained in:
Doug Hellmann
parent
e7fc00f2de
commit
4923b34de1
@ -35,6 +35,8 @@ permits.
|
|||||||
:maxdepth: 1
|
:maxdepth: 1
|
||||||
|
|
||||||
specs/ansible_puppet_apply
|
specs/ansible_puppet_apply
|
||||||
|
specs/apps-site
|
||||||
|
specs/centralize-release-tagging
|
||||||
specs/code-search
|
specs/code-search
|
||||||
specs/doc-publishing
|
specs/doc-publishing
|
||||||
specs/infra-cloud
|
specs/infra-cloud
|
||||||
|
|||||||
322
specs/centralize-release-tagging.rst
Normal file
322
specs/centralize-release-tagging.rst
Normal file
@ -0,0 +1,322 @@
|
|||||||
|
::
|
||||||
|
|
||||||
|
Copyright 2015, Hewlett-Packard Development Company, L.P.
|
||||||
|
|
||||||
|
This work is licensed under a Creative Commons Attribution 3.0
|
||||||
|
Unported License.
|
||||||
|
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||||
|
|
||||||
|
..
|
||||||
|
This template should be in ReSTructured text. Please do not delete
|
||||||
|
any of the sections in this template. If you have nothing to say
|
||||||
|
for a whole section, just write: "None". For help with syntax, see
|
||||||
|
http://sphinx-doc.org/rest.html To test out your formatting, see
|
||||||
|
http://www.tele3.cz/jbar/rest/rest.html
|
||||||
|
|
||||||
|
============================
|
||||||
|
Centralize Release Tagging
|
||||||
|
============================
|
||||||
|
|
||||||
|
Include the URL of your StoryBoard story:
|
||||||
|
|
||||||
|
https://storyboard.openstack.org/#!/story/2000292
|
||||||
|
|
||||||
|
We would like to manage release tags through reviews, just as we do
|
||||||
|
with other changes. Unfortunately, gerrit does not yet support
|
||||||
|
reviewing tags directly. This spec describes an alternate solution
|
||||||
|
using a separate git repository, with some manual tools for short-term
|
||||||
|
use and some ideas about automation for later.
|
||||||
|
|
||||||
|
Problem Description
|
||||||
|
===================
|
||||||
|
|
||||||
|
Until now we have encouraged project teams to prepare their own
|
||||||
|
library releases as new versions of projects were needed. We've
|
||||||
|
started running into a couple of problems with that, with releases
|
||||||
|
not coming often enough, or at a bad time in the release cycle, or
|
||||||
|
with version numbering not being applied consistently, or without
|
||||||
|
proper announcements. To address these issues, the release management
|
||||||
|
team is proposing to create a small team of library release managers
|
||||||
|
to handle the process around all library releases (clients,
|
||||||
|
non-application projects, middleware, Oslo libraries, etc.). This
|
||||||
|
will give us a chance to collaborate and review the version numbers
|
||||||
|
for new releases, as well as stay on top of "stale" libraries with
|
||||||
|
fixes or features that sit unreleased for a period of time. It will
|
||||||
|
also be the first step to creating an automated review process for
|
||||||
|
release tags.
|
||||||
|
|
||||||
|
Proposed Change
|
||||||
|
===============
|
||||||
|
|
||||||
|
Create a new git repository with a name like
|
||||||
|
``openstack/releases``.
|
||||||
|
|
||||||
|
For each set of deliverables, we will use one YAML file in
|
||||||
|
``openstack/releases`` per release series to hold all of the metadata
|
||||||
|
for all releases of that deliverable. For each release, we need to
|
||||||
|
track:
|
||||||
|
|
||||||
|
* the launchpad project name (such as ``oslo.config``)
|
||||||
|
* the series (Kilo, Liberty, etc.)
|
||||||
|
* for each repository
|
||||||
|
* the name (such as ``openstack/oslo.config``)
|
||||||
|
* the hash of the commit to be tagged
|
||||||
|
* the version number to use
|
||||||
|
* highlights for the release notes email (optional)
|
||||||
|
|
||||||
|
We will track this metadata for the history of all releases of the
|
||||||
|
deliverable, so we can use the same data to render a set of release
|
||||||
|
history documentation.
|
||||||
|
|
||||||
|
The release tools need the series name to update launchpad, and they
|
||||||
|
also need the branch name to check the history of tags visible from
|
||||||
|
that branch to do some validation. Since git tags apply to a commit,
|
||||||
|
and are not branch-aware, we can use the series name in the filename
|
||||||
|
and have the tagging script either assume some defaults (if it does
|
||||||
|
not find a stable branch of that name use master) or recognize which
|
||||||
|
series is currently on master.
|
||||||
|
|
||||||
|
The file will be named based on the deliverable to be tagged, so
|
||||||
|
releases for ``liberty`` from the ``openstack/oslo.config`` repository
|
||||||
|
will have a file in ``openstack/releases`` called
|
||||||
|
``liberty/oslo.config.yaml``. Releases of the same deliverable from
|
||||||
|
the ``stable/kilo`` branch will be described by
|
||||||
|
``kilo/oslo.config.yaml``.
|
||||||
|
|
||||||
|
For example, one version of
|
||||||
|
``liberty/oslo.config.yaml`` might contain::
|
||||||
|
|
||||||
|
---
|
||||||
|
launchpad: oslo.config
|
||||||
|
releases:
|
||||||
|
- version: 1.12.0
|
||||||
|
projects:
|
||||||
|
- repo: openstack/oslo.config
|
||||||
|
hash: 02a86d2eefeda5144ea8c39657aed24b8b0c9a39
|
||||||
|
|
||||||
|
and then for the subsequent release it would be updated to contain::
|
||||||
|
|
||||||
|
---
|
||||||
|
launchpad: oslo.config
|
||||||
|
releases:
|
||||||
|
- version: 1.12.1
|
||||||
|
projects:
|
||||||
|
- repo: openstack/oslo.config
|
||||||
|
hash: 0c9113f68285f7b55ca01f0bbb5ce6cddada5023
|
||||||
|
highlights: >
|
||||||
|
This release includes the change to stop importing
|
||||||
|
from the 'oslo' namespace package.
|
||||||
|
|
||||||
|
For deliverables with multiple repositories, the list of projects
|
||||||
|
would contain all of them. For example, the Neutron deliverable might
|
||||||
|
be described by ``liberty/neutron.yaml`` containing:
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
---
|
||||||
|
launchpad: neutron
|
||||||
|
releases:
|
||||||
|
- version: 7.0.0
|
||||||
|
projects:
|
||||||
|
- repo: openstack/neutron
|
||||||
|
hash: somethingunique
|
||||||
|
- repo: openstack/neutron-fwaas
|
||||||
|
hash: somethingunique
|
||||||
|
- repo: openstack/neutron-lbaas
|
||||||
|
hash: somethingunique
|
||||||
|
- repo: openstack/neutron-vpnaas
|
||||||
|
hash: somethingunique
|
||||||
|
|
||||||
|
For Phase I, we won't have much true automation using these files, and
|
||||||
|
someone from the release team will need to run a tool to read the file
|
||||||
|
and apply the appropriate tags. That tool should be a straightforward
|
||||||
|
modification to the existing ``release_postversion.sh`` script.
|
||||||
|
|
||||||
|
For future phases we can investigate having enough signed data in the
|
||||||
|
YAML file to let an automated job apply the tag when the request is
|
||||||
|
approved.
|
||||||
|
|
||||||
|
Alternatives
|
||||||
|
------------
|
||||||
|
|
||||||
|
Allow Project Owners to Continue Tagging Releases
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
We will continue to have issues with incorrect semver use and poorly
|
||||||
|
timed releases.
|
||||||
|
|
||||||
|
Have Project Owners Request Releases via Email/IRC
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Rather than going through the bureaucracy of managing the requests via
|
||||||
|
git we could just use email and IRC as we have been doing. However,
|
||||||
|
that would not bring us closer to automating releases after the
|
||||||
|
requests are reviewed, which is our ultimate goal.
|
||||||
|
|
||||||
|
Update Gerrit to Support Reviewing Tags
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Apparently the Gerrit project team is interested in the feature, but
|
||||||
|
it isn't a high priority. We could consider this a Phase III for the
|
||||||
|
project if someone from our community becomes available to work on
|
||||||
|
it. On the other hand, we would need to find another way to track
|
||||||
|
releases after-the-fact, and tags in one repository do not handle
|
||||||
|
multi-repo deliverables such as neutron.
|
||||||
|
|
||||||
|
Use Branches
|
||||||
|
~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Earlier drafts of this proposal suggested using different branches of
|
||||||
|
the ``openstack/releases`` repository to manage releases from
|
||||||
|
different branches of the upstream projects. That forces us to create
|
||||||
|
all the same branches in the new repository that are needed in any
|
||||||
|
repository for which releases are being managed. Since not all of them
|
||||||
|
will use the same branching structure, this is not optimal.
|
||||||
|
|
||||||
|
One File Per Repository
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Use a file named based on the git repository to be tagged, so
|
||||||
|
releases from the ``master`` branch of the ``openstack/oslo.config``
|
||||||
|
repository would have a file in ``openstack/releases`` called
|
||||||
|
``openstack/oslo.config/master/releases.yaml``. Releases for the same
|
||||||
|
repository from the ``stable/kilo`` branch will be described by
|
||||||
|
``openstack/oslo.config/stable/kilo/releases.yaml``
|
||||||
|
|
||||||
|
For example, one version of
|
||||||
|
``openstack/oslo.config/master/releases.yaml`` might contain::
|
||||||
|
|
||||||
|
--
|
||||||
|
series: liberty
|
||||||
|
hash: 02a86d2eefeda5144ea8c39657aed24b8b0c9a39
|
||||||
|
version: 1.12.0
|
||||||
|
|
||||||
|
and then for the subsequent release it would be updated to contain::
|
||||||
|
|
||||||
|
--
|
||||||
|
series: liberty
|
||||||
|
hash: 0c9113f68285f7b55ca01f0bbb5ce6cddada5023
|
||||||
|
version: 1.12.1
|
||||||
|
highlights: >
|
||||||
|
This release includes the change to stop importing
|
||||||
|
from the 'oslo' namespace package.
|
||||||
|
|
||||||
|
Multi-repo deliverables such as Neutron could use separate files,
|
||||||
|
submitted together.
|
||||||
|
|
||||||
|
This scheme does not allow us to easily product web pages showing the
|
||||||
|
release histories.
|
||||||
|
|
||||||
|
Single File With All Branches
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Rather than maintaining a separate file for each branch, we could use
|
||||||
|
a single file and list all branches in it. This makes it a little more
|
||||||
|
complicated to detect new changes, though, and has the same problem as
|
||||||
|
appending all releases to a single file -- the tool that applies the
|
||||||
|
tags needs to check all of them, and the list will only grow over
|
||||||
|
time.
|
||||||
|
|
||||||
|
Branch After Project in the Path
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
We could use file names like ``oslo.config/kilo.yaml`` instead of
|
||||||
|
``kilo/oslo.config.yaml``. That would place all of the files from the
|
||||||
|
same deliverable in a directory together. However, it is more likely
|
||||||
|
that we will focus on the contents of a series rather than historical
|
||||||
|
releases of an individual project.
|
||||||
|
|
||||||
|
Record Launchpad Names in the Governance Repository
|
||||||
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
There is a separate list of projects in the governance repository, and
|
||||||
|
we could list some of the data about projects that doesn't change
|
||||||
|
there. That would require the tool download the relevant files,
|
||||||
|
though, and would not help us with scripting releases for projects not
|
||||||
|
under TC governance.
|
||||||
|
|
||||||
|
Implementation
|
||||||
|
==============
|
||||||
|
|
||||||
|
Assignee(s)
|
||||||
|
-----------
|
||||||
|
|
||||||
|
Primary assignee:
|
||||||
|
doug-hellmann
|
||||||
|
|
||||||
|
I'm comfortable setting up a new repository and building in-tree
|
||||||
|
tools. I may need help with some of the validation job work.
|
||||||
|
|
||||||
|
Gerrit Topic
|
||||||
|
------------
|
||||||
|
|
||||||
|
Use Gerrit topic "centralize-release-tagging" for all patches related to this spec.
|
||||||
|
|
||||||
|
.. code-block:: bash
|
||||||
|
|
||||||
|
git-review -t centralize-release-tagging
|
||||||
|
|
||||||
|
Work Items
|
||||||
|
----------
|
||||||
|
|
||||||
|
1. Create the ``openstack/releases`` repository, with a README and
|
||||||
|
template YAML file.
|
||||||
|
2. Create a new tool (or update an existing script) in
|
||||||
|
``openstack-infra/release-tools`` to read the YAML files from
|
||||||
|
``openstack/releases`` and run the interactive release script we
|
||||||
|
use now.
|
||||||
|
3. Create a basic validation tool to read the YAML files and provide a
|
||||||
|
check job. We can't do a lot to validate the requested tag, beyond
|
||||||
|
noticing that it already exists, but we can make sure all of the
|
||||||
|
needed parts are there and can be parsed properly, and we can run a
|
||||||
|
report showing the unreleased changes, what pbr thinks the version
|
||||||
|
should be, and whether the version means a major jump in the series
|
||||||
|
to help the reviewer (these are all things I do by hand right now).
|
||||||
|
4. Make ``release_postversion.sh`` smarter about figuring out the
|
||||||
|
branch for validating the proposed version number.
|
||||||
|
|
||||||
|
Repositories
|
||||||
|
------------
|
||||||
|
|
||||||
|
``openstack/releases`` will hold the release request files.
|
||||||
|
|
||||||
|
Servers
|
||||||
|
-------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
DNS Entries
|
||||||
|
-----------
|
||||||
|
|
||||||
|
None
|
||||||
|
|
||||||
|
Documentation
|
||||||
|
-------------
|
||||||
|
|
||||||
|
We will document the process in the README in ``openstack/releases``
|
||||||
|
to start, and then in the Project Driver's Guide portion of
|
||||||
|
infra-manual.
|
||||||
|
|
||||||
|
Security
|
||||||
|
--------
|
||||||
|
|
||||||
|
During Phase I releases will still be tagged by people with
|
||||||
|
established trust rings. For future phases where the tagging is
|
||||||
|
handled by a post-merge job we will want to do some validation of
|
||||||
|
signed data in the request file.
|
||||||
|
|
||||||
|
Testing
|
||||||
|
-------
|
||||||
|
|
||||||
|
We have fairly robust release tools now, but we will want to test some
|
||||||
|
of the new tools for working with the YAML files.
|
||||||
|
|
||||||
|
Dependencies
|
||||||
|
============
|
||||||
|
|
||||||
|
- https://review.openstack.org/189856 -- Creating a library-release
|
||||||
|
team with Gerrit ACLs to push tags to repositories containing
|
||||||
|
libraries.
|
||||||
|
|
||||||
|
- http://lists.openstack.org/pipermail/openstack-dev/2015-June/066346.html --
|
||||||
|
Mailing list thread initiating the discussion.
|
||||||
Loading…
Reference in New Issue
Block a user