Centralize Release Tagging
Story:2000292 Change-Id: Ie40ea6e7538bf4e5f6b67d8604bc3b524ca8e9e6
This commit is contained in:
Doug Hellmann
parent
e7fc00f2de
commit
4923b34de1
@ -35,6 +35,8 @@ permits.
|
||||
:maxdepth: 1
|
||||
|
||||
specs/ansible_puppet_apply
|
||||
specs/apps-site
|
||||
specs/centralize-release-tagging
|
||||
specs/code-search
|
||||
specs/doc-publishing
|
||||
specs/infra-cloud
|
||||
|
||||
322
specs/centralize-release-tagging.rst
Normal file
322
specs/centralize-release-tagging.rst
Normal file
@ -0,0 +1,322 @@
|
||||
::
|
||||
|
||||
Copyright 2015, Hewlett-Packard Development Company, L.P.
|
||||
|
||||
This work is licensed under a Creative Commons Attribution 3.0
|
||||
Unported License.
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
..
|
||||
This template should be in ReSTructured text. Please do not delete
|
||||
any of the sections in this template. If you have nothing to say
|
||||
for a whole section, just write: "None". For help with syntax, see
|
||||
http://sphinx-doc.org/rest.html To test out your formatting, see
|
||||
http://www.tele3.cz/jbar/rest/rest.html
|
||||
|
||||
============================
|
||||
Centralize Release Tagging
|
||||
============================
|
||||
|
||||
Include the URL of your StoryBoard story:
|
||||
|
||||
https://storyboard.openstack.org/#!/story/2000292
|
||||
|
||||
We would like to manage release tags through reviews, just as we do
|
||||
with other changes. Unfortunately, gerrit does not yet support
|
||||
reviewing tags directly. This spec describes an alternate solution
|
||||
using a separate git repository, with some manual tools for short-term
|
||||
use and some ideas about automation for later.
|
||||
|
||||
Problem Description
|
||||
===================
|
||||
|
||||
Until now we have encouraged project teams to prepare their own
|
||||
library releases as new versions of projects were needed. We've
|
||||
started running into a couple of problems with that, with releases
|
||||
not coming often enough, or at a bad time in the release cycle, or
|
||||
with version numbering not being applied consistently, or without
|
||||
proper announcements. To address these issues, the release management
|
||||
team is proposing to create a small team of library release managers
|
||||
to handle the process around all library releases (clients,
|
||||
non-application projects, middleware, Oslo libraries, etc.). This
|
||||
will give us a chance to collaborate and review the version numbers
|
||||
for new releases, as well as stay on top of "stale" libraries with
|
||||
fixes or features that sit unreleased for a period of time. It will
|
||||
also be the first step to creating an automated review process for
|
||||
release tags.
|
||||
|
||||
Proposed Change
|
||||
===============
|
||||
|
||||
Create a new git repository with a name like
|
||||
``openstack/releases``.
|
||||
|
||||
For each set of deliverables, we will use one YAML file in
|
||||
``openstack/releases`` per release series to hold all of the metadata
|
||||
for all releases of that deliverable. For each release, we need to
|
||||
track:
|
||||
|
||||
* the launchpad project name (such as ``oslo.config``)
|
||||
* the series (Kilo, Liberty, etc.)
|
||||
* for each repository
|
||||
* the name (such as ``openstack/oslo.config``)
|
||||
* the hash of the commit to be tagged
|
||||
* the version number to use
|
||||
* highlights for the release notes email (optional)
|
||||
|
||||
We will track this metadata for the history of all releases of the
|
||||
deliverable, so we can use the same data to render a set of release
|
||||
history documentation.
|
||||
|
||||
The release tools need the series name to update launchpad, and they
|
||||
also need the branch name to check the history of tags visible from
|
||||
that branch to do some validation. Since git tags apply to a commit,
|
||||
and are not branch-aware, we can use the series name in the filename
|
||||
and have the tagging script either assume some defaults (if it does
|
||||
not find a stable branch of that name use master) or recognize which
|
||||
series is currently on master.
|
||||
|
||||
The file will be named based on the deliverable to be tagged, so
|
||||
releases for ``liberty`` from the ``openstack/oslo.config`` repository
|
||||
will have a file in ``openstack/releases`` called
|
||||
``liberty/oslo.config.yaml``. Releases of the same deliverable from
|
||||
the ``stable/kilo`` branch will be described by
|
||||
``kilo/oslo.config.yaml``.
|
||||
|
||||
For example, one version of
|
||||
``liberty/oslo.config.yaml`` might contain::
|
||||
|
||||
---
|
||||
launchpad: oslo.config
|
||||
releases:
|
||||
- version: 1.12.0
|
||||
projects:
|
||||
- repo: openstack/oslo.config
|
||||
hash: 02a86d2eefeda5144ea8c39657aed24b8b0c9a39
|
||||
|
||||
and then for the subsequent release it would be updated to contain::
|
||||
|
||||
---
|
||||
launchpad: oslo.config
|
||||
releases:
|
||||
- version: 1.12.1
|
||||
projects:
|
||||
- repo: openstack/oslo.config
|
||||
hash: 0c9113f68285f7b55ca01f0bbb5ce6cddada5023
|
||||
highlights: >
|
||||
This release includes the change to stop importing
|
||||
from the 'oslo' namespace package.
|
||||
|
||||
For deliverables with multiple repositories, the list of projects
|
||||
would contain all of them. For example, the Neutron deliverable might
|
||||
be described by ``liberty/neutron.yaml`` containing:
|
||||
|
||||
::
|
||||
|
||||
---
|
||||
launchpad: neutron
|
||||
releases:
|
||||
- version: 7.0.0
|
||||
projects:
|
||||
- repo: openstack/neutron
|
||||
hash: somethingunique
|
||||
- repo: openstack/neutron-fwaas
|
||||
hash: somethingunique
|
||||
- repo: openstack/neutron-lbaas
|
||||
hash: somethingunique
|
||||
- repo: openstack/neutron-vpnaas
|
||||
hash: somethingunique
|
||||
|
||||
For Phase I, we won't have much true automation using these files, and
|
||||
someone from the release team will need to run a tool to read the file
|
||||
and apply the appropriate tags. That tool should be a straightforward
|
||||
modification to the existing ``release_postversion.sh`` script.
|
||||
|
||||
For future phases we can investigate having enough signed data in the
|
||||
YAML file to let an automated job apply the tag when the request is
|
||||
approved.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
Allow Project Owners to Continue Tagging Releases
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
We will continue to have issues with incorrect semver use and poorly
|
||||
timed releases.
|
||||
|
||||
Have Project Owners Request Releases via Email/IRC
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Rather than going through the bureaucracy of managing the requests via
|
||||
git we could just use email and IRC as we have been doing. However,
|
||||
that would not bring us closer to automating releases after the
|
||||
requests are reviewed, which is our ultimate goal.
|
||||
|
||||
Update Gerrit to Support Reviewing Tags
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Apparently the Gerrit project team is interested in the feature, but
|
||||
it isn't a high priority. We could consider this a Phase III for the
|
||||
project if someone from our community becomes available to work on
|
||||
it. On the other hand, we would need to find another way to track
|
||||
releases after-the-fact, and tags in one repository do not handle
|
||||
multi-repo deliverables such as neutron.
|
||||
|
||||
Use Branches
|
||||
~~~~~~~~~~~~
|
||||
|
||||
Earlier drafts of this proposal suggested using different branches of
|
||||
the ``openstack/releases`` repository to manage releases from
|
||||
different branches of the upstream projects. That forces us to create
|
||||
all the same branches in the new repository that are needed in any
|
||||
repository for which releases are being managed. Since not all of them
|
||||
will use the same branching structure, this is not optimal.
|
||||
|
||||
One File Per Repository
|
||||
~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Use a file named based on the git repository to be tagged, so
|
||||
releases from the ``master`` branch of the ``openstack/oslo.config``
|
||||
repository would have a file in ``openstack/releases`` called
|
||||
``openstack/oslo.config/master/releases.yaml``. Releases for the same
|
||||
repository from the ``stable/kilo`` branch will be described by
|
||||
``openstack/oslo.config/stable/kilo/releases.yaml``
|
||||
|
||||
For example, one version of
|
||||
``openstack/oslo.config/master/releases.yaml`` might contain::
|
||||
|
||||
--
|
||||
series: liberty
|
||||
hash: 02a86d2eefeda5144ea8c39657aed24b8b0c9a39
|
||||
version: 1.12.0
|
||||
|
||||
and then for the subsequent release it would be updated to contain::
|
||||
|
||||
--
|
||||
series: liberty
|
||||
hash: 0c9113f68285f7b55ca01f0bbb5ce6cddada5023
|
||||
version: 1.12.1
|
||||
highlights: >
|
||||
This release includes the change to stop importing
|
||||
from the 'oslo' namespace package.
|
||||
|
||||
Multi-repo deliverables such as Neutron could use separate files,
|
||||
submitted together.
|
||||
|
||||
This scheme does not allow us to easily product web pages showing the
|
||||
release histories.
|
||||
|
||||
Single File With All Branches
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Rather than maintaining a separate file for each branch, we could use
|
||||
a single file and list all branches in it. This makes it a little more
|
||||
complicated to detect new changes, though, and has the same problem as
|
||||
appending all releases to a single file -- the tool that applies the
|
||||
tags needs to check all of them, and the list will only grow over
|
||||
time.
|
||||
|
||||
Branch After Project in the Path
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
We could use file names like ``oslo.config/kilo.yaml`` instead of
|
||||
``kilo/oslo.config.yaml``. That would place all of the files from the
|
||||
same deliverable in a directory together. However, it is more likely
|
||||
that we will focus on the contents of a series rather than historical
|
||||
releases of an individual project.
|
||||
|
||||
Record Launchpad Names in the Governance Repository
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
There is a separate list of projects in the governance repository, and
|
||||
we could list some of the data about projects that doesn't change
|
||||
there. That would require the tool download the relevant files,
|
||||
though, and would not help us with scripting releases for projects not
|
||||
under TC governance.
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
doug-hellmann
|
||||
|
||||
I'm comfortable setting up a new repository and building in-tree
|
||||
tools. I may need help with some of the validation job work.
|
||||
|
||||
Gerrit Topic
|
||||
------------
|
||||
|
||||
Use Gerrit topic "centralize-release-tagging" for all patches related to this spec.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git-review -t centralize-release-tagging
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
1. Create the ``openstack/releases`` repository, with a README and
|
||||
template YAML file.
|
||||
2. Create a new tool (or update an existing script) in
|
||||
``openstack-infra/release-tools`` to read the YAML files from
|
||||
``openstack/releases`` and run the interactive release script we
|
||||
use now.
|
||||
3. Create a basic validation tool to read the YAML files and provide a
|
||||
check job. We can't do a lot to validate the requested tag, beyond
|
||||
noticing that it already exists, but we can make sure all of the
|
||||
needed parts are there and can be parsed properly, and we can run a
|
||||
report showing the unreleased changes, what pbr thinks the version
|
||||
should be, and whether the version means a major jump in the series
|
||||
to help the reviewer (these are all things I do by hand right now).
|
||||
4. Make ``release_postversion.sh`` smarter about figuring out the
|
||||
branch for validating the proposed version number.
|
||||
|
||||
Repositories
|
||||
------------
|
||||
|
||||
``openstack/releases`` will hold the release request files.
|
||||
|
||||
Servers
|
||||
-------
|
||||
|
||||
None
|
||||
|
||||
DNS Entries
|
||||
-----------
|
||||
|
||||
None
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
We will document the process in the README in ``openstack/releases``
|
||||
to start, and then in the Project Driver's Guide portion of
|
||||
infra-manual.
|
||||
|
||||
Security
|
||||
--------
|
||||
|
||||
During Phase I releases will still be tagged by people with
|
||||
established trust rings. For future phases where the tagging is
|
||||
handled by a post-merge job we will want to do some validation of
|
||||
signed data in the request file.
|
||||
|
||||
Testing
|
||||
-------
|
||||
|
||||
We have fairly robust release tools now, but we will want to test some
|
||||
of the new tools for working with the YAML files.
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
- https://review.openstack.org/189856 -- Creating a library-release
|
||||
team with Gerrit ACLs to push tags to repositories containing
|
||||
libraries.
|
||||
|
||||
- http://lists.openstack.org/pipermail/openstack-dev/2015-June/066346.html --
|
||||
Mailing list thread initiating the discussion.
|
||||
Loading…
Reference in New Issue
Block a user