Doug Hellmann df8e26261b cleanups for centralize-release-tagging

Fix a typo and markup issue in the existing spec

Change-Id: I789384daa14ceaecab709595acec2a7b37aee1fe

2015-07-07 19:08:43 +00:00

11 KiB

Raw Permalink Blame History

Copyright 2015, Hewlett-Packard Development Company, L.P.

This work is licensed under a Creative Commons Attribution 3.0
Unported License.
http://creativecommons.org/licenses/by/3.0/legalcode

Centralize Release Tagging

Include the URL of your StoryBoard story:

https://storyboard.openstack.org/#!/story/2000292

We would like to manage release tags through reviews, just as we do with other changes. Unfortunately, gerrit does not yet support reviewing tags directly. This spec describes an alternate solution using a separate git repository, with some manual tools for short-term use and some ideas about automation for later.

Problem Description

Until now we have encouraged project teams to prepare their own library releases as new versions of projects were needed. We've started running into a couple of problems with that, with releases not coming often enough, or at a bad time in the release cycle, or with version numbering not being applied consistently, or without proper announcements. To address these issues, the release management team is proposing to create a small team of library release managers to handle the process around all library releases (clients, non-application projects, middleware, Oslo libraries, etc.). This will give us a chance to collaborate and review the version numbers for new releases, as well as stay on top of "stale" libraries with fixes or features that sit unreleased for a period of time. It will also be the first step to creating an automated review process for release tags.

Proposed Change

Create a new git repository with a name like openstack/releases.

For each set of deliverables, we will use one YAML file in openstack/releases per release series to hold all of the metadata for all releases of that deliverable. For each release, we need to track:

the launchpad project name (such as oslo.config)
the series (Kilo, Liberty, etc.)
for each repository
- the name (such as openstack/oslo.config)
- the hash of the commit to be tagged
the version number to use
highlights for the release notes email (optional)

We will track this metadata for the history of all releases of the deliverable, so we can use the same data to render a set of release history documentation.

The release tools need the series name to update launchpad, and they also need the branch name to check the history of tags visible from that branch to do some validation. Since git tags apply to a commit, and are not branch-aware, we can use the series name in the filename and have the tagging script either assume some defaults (if it does not find a stable branch of that name use master) or recognize which series is currently on master.

The file will be named based on the deliverable to be tagged, so releases for liberty from the openstack/oslo.config repository will have a file in openstack/releases called liberty/oslo.config.yaml. Releases of the same deliverable from the stable/kilo branch will be described by kilo/oslo.config.yaml.

For example, one version of liberty/oslo.config.yaml might contain:

---
launchpad: oslo.config
releases:
  - version: 1.12.0
    projects:
      - repo: openstack/oslo.config
        hash: 02a86d2eefeda5144ea8c39657aed24b8b0c9a39

and then for the subsequent release it would be updated to contain:

---
launchpad: oslo.config
releases:
  - version: 1.12.1
    projects:
      - repo: openstack/oslo.config
        hash: 0c9113f68285f7b55ca01f0bbb5ce6cddada5023
    highlights: >
       This release includes the change to stop importing
       from the 'oslo' namespace package.

For deliverables with multiple repositories, the list of projects would contain all of them. For example, the Neutron deliverable might be described by liberty/neutron.yaml containing:

---
launchpad: neutron
releases:
  - version: 7.0.0
    projects:
      - repo: openstack/neutron
        hash: somethingunique
      - repo: openstack/neutron-fwaas
        hash: somethingunique
      - repo: openstack/neutron-lbaas
        hash: somethingunique
      - repo: openstack/neutron-vpnaas
        hash: somethingunique

For Phase I, we won't have much true automation using these files, and someone from the release team will need to run a tool to read the file and apply the appropriate tags. That tool should be a straightforward modification to the existing release_postversion.sh script.

For future phases we can investigate having enough signed data in the YAML file to let an automated job apply the tag when the request is approved.

Alternatives

Allow Project Owners to Continue Tagging Releases

We will continue to have issues with incorrect semver use and poorly timed releases.

Have Project Owners Request Releases via Email/IRC

Rather than going through the bureaucracy of managing the requests via git we could just use email and IRC as we have been doing. However, that would not bring us closer to automating releases after the requests are reviewed, which is our ultimate goal.

Update Gerrit to Support Reviewing Tags

Apparently the Gerrit project team is interested in the feature, but it isn't a high priority. We could consider this a Phase III for the project if someone from our community becomes available to work on it. On the other hand, we would need to find another way to track releases after-the-fact, and tags in one repository do not handle multi-repo deliverables such as neutron.

Use Branches

Earlier drafts of this proposal suggested using different branches of the openstack/releases repository to manage releases from different branches of the upstream projects. That forces us to create all the same branches in the new repository that are needed in any repository for which releases are being managed. Since not all of them will use the same branching structure, this is not optimal.

One File Per Repository

Use a file named based on the git repository to be tagged, so releases from the master branch of the openstack/oslo.config repository would have a file in openstack/releases called openstack/oslo.config/master/releases.yaml. Releases for the same repository from the stable/kilo branch will be described by openstack/oslo.config/stable/kilo/releases.yaml

For example, one version of openstack/oslo.config/master/releases.yaml might contain:

--
series: liberty
hash: 02a86d2eefeda5144ea8c39657aed24b8b0c9a39
version: 1.12.0

and then for the subsequent release it would be updated to contain:

--
series: liberty
hash: 0c9113f68285f7b55ca01f0bbb5ce6cddada5023
version: 1.12.1
highlights: >
   This release includes the change to stop importing
   from the 'oslo' namespace package.

Multi-repo deliverables such as Neutron could use separate files, submitted together.

This scheme does not allow us to easily produce web pages showing the release histories.

Single File With All Branches

Rather than maintaining a separate file for each branch, we could use a single file and list all branches in it. This makes it a little more complicated to detect new changes, though, and has the same problem as appending all releases to a single file -- the tool that applies the tags needs to check all of them, and the list will only grow over time.

Branch After Project in the Path

We could use file names like oslo.config/kilo.yaml instead of kilo/oslo.config.yaml. That would place all of the files from the same deliverable in a directory together. However, it is more likely that we will focus on the contents of a series rather than historical releases of an individual project.

Record Launchpad Names in the Governance Repository

There is a separate list of projects in the governance repository, and we could list some of the data about projects that doesn't change there. That would require the tool download the relevant files, though, and would not help us with scripting releases for projects not under TC governance.

Implementation

Assignee(s)

Primary assignee:: doug-hellmann

I'm comfortable setting up a new repository and building in-tree tools. I may need help with some of the validation job work.

Gerrit Topic

Use Gerrit topic "centralize-release-tagging" for all patches related to this spec.

git-review -t centralize-release-tagging

Work Items

Create the openstack/releases repository, with a README and template YAML file.
Create a new tool (or update an existing script) in openstack-infra/release-tools to read the YAML files from openstack/releases and run the interactive release script we use now.
Create a basic validation tool to read the YAML files and provide a check job. We can't do a lot to validate the requested tag, beyond noticing that it already exists, but we can make sure all of the needed parts are there and can be parsed properly, and we can run a report showing the unreleased changes, what pbr thinks the version should be, and whether the version means a major jump in the series to help the reviewer (these are all things I do by hand right now).
Make release_postversion.sh smarter about figuring out the branch for validating the proposed version number.

Repositories

openstack/releases will hold the release request files.

Servers

None

DNS Entries