859 Commits

Author SHA1 Message Date
Doug Hellmann
b469dc7260 add a --strict flag to the template generator
When --strict is passed warnings are treated as errors to stop the
build. This is useful for debugging when running tools/test.sh
locally, for example.

Change-Id: I1747f0e554d76ed85d8e00cf850f3413eef1949b
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-05-16 14:28:12 -04:00
Petr Kovar
f64ab3c235 [www] Mark Q as maintained
This will make Queens pages point to the correct branch.

Change-Id: If78961126a8daea2237ca4b98a4df8c1b4f2cb14
2018-02-16 17:58:37 +01:00
Petr Kovar
759e35b12b Open development for Rocky
* Update project data for P, Q and latest
* Mark N as EOL

Change-Id: I3a2dba6ad8f68ba043f9d7cb57f84ae4e6f01456
2018-02-13 18:11:14 +01:00
Doug Hellmann
d9d05a062c only start the link checkers when there are links to check
Do not start the ThreadPool when there are no links to check.

Change-Id: I355ea5af5d937e6b5d4cf481d9a29c6d3b123f09
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-01-26 09:53:57 -05:00
Doug Hellmann
7890d18566 include governed projects in output
In 0ba83e5 we started trying to ignore unofficial projects and
introduced a bug so that all projects were dropped from the project
list as part of that evaluation. This patch fixes the bug by copying
the valid data from the raw values read in to the list of values being
passed to the template.

Change-Id: I4aeafe6d8116f3b032a5d339660878a76a10e2f3
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-01-26 09:52:36 -05:00
Doug Hellmann
0ba83e5e36 ignore unofficial projects in latest.yaml
Update the template generator to ignore unofficial projects when
rendering for the current series. Ignoring, rather than throwing an
error, allows us to clean up the data without the build breaking.

To make the validation work for glance-store, a new field is needed to
set the 'deliverable-name', since it is different from the project
name used to build URLs.

Change-Id: I0876c93b49d57818d0b689fd03a0171a96e27547
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-01-24 14:41:55 -05:00
Monty Taylor
7c1662415f Distinguish between cloud-wide and per-service apis
The two are different use cases, and we have things that focus on the
cloud-wide story. Lead with those, then show docs for individual
services that users can fallback to if they need to.

Update docs and update this for all branches that your the project-data
for language-bindings (pike and queens only).

Change-Id: Ib1f7b7c77ec2b9536c4a03425694a6300025a6c3
2018-01-09 13:31:12 +01:00
Zane Bitter
cf1af853cf Fix broken diskimage-builder redirect
The diskimage-builder repo is listed in the governance repo as part of
the Infrastructure project, so
https://docs.openstack.org/diskimage-builder/ was redirecting to
https://docs.openstack.org/infra/diskimage-builder/ but the docs are
published at https://docs.openstack.org/diskimage-builder/latest/

Change-Id: I6a9390abf29a859c853b9a1107d52c74a8c48383
2017-11-30 10:33:19 -05:00
Andreas Jaeger
7d58709270 Fix typo in ZUUL infra variable
.zuul.yaml has NEWREF, so fix typo in tools/build-all-rst.sh.

Change-Id: Iae8414fbc5c6f087e2deb887e8215706034d8e41
2017-11-24 18:39:16 +01:00
Andreas Jaeger
21edbc931f Fix ZUUL_REFNAME
Set up the variables the same way the post job is set up:
Pass them in via environment in job definition.

Pass ZUUL_ environment variables through to jobs.

Note ZUUL_REFNAME is not set, use ZUUL_BRANCH instead.

Change-Id: I5863a05384c9a48445d5ce52ccc508519c7c00f8
2017-11-23 17:16:14 +01:00
Eric Fried
4f236a3f70 Raise on init, parse, render, or write fails
If jinja template initialization, parsing, rendering, or writing fails,
we used to sometimes log an error message, then sometimes continue
(ignoring the failure), sometimes reraise the exception, and sometimes
fail without raising the exception.  With this change, we always raise
the exception so we print the traceback and fail the build right away.

Change-Id: I04a36c4c8c69dbd08e9ea677983dbe00fa3a67a3
2017-10-22 16:18:30 -05:00
Frank Kloeker
0b1166756e Fix URL collision with new contributor-guide
Change-Id: I4a88d6147ad352c253c8652ba70f596e494a64c3
see: http://logs.openstack.org/59/512659/1/check/build-tox-manuals-checkbuild/dc92ba0/job-output.txt.gz#_2017-10-17_20_56_12_471615
2017-10-18 00:17:45 +02:00
Doug Hellmann
e6bf33566a parameterize the deploy guide list
Update the page template for the deployment guide list to use the
project-data.

Add data to pike for the existing deployment guides.

Update the latest project-data file for the available deployment
guides for queens.

Change-Id: Idc5923ee8b930c75594d42d91450b25be7ee4594
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-09-26 17:26:21 -04:00
Andreas Jaeger
a2f040015a [www] Don't publish redirect-tests.txt
Checking http://files.openstack.org/docs/ , we see that
redirect-tests.txt gets published - this is a test file, let's remove it
before publishing.

Change-Id: Id9b2fb0f7d196d53e64296e99ba959c029579b81
2017-09-22 21:25:08 +02:00
Alexandra Settle
85107272be [contrib-guide] Updating url to doc-contrib-guide
Based off conversation at the PTG, we agreed it would be
beneficial to ensure the contrib-guide is clearly marked
as the doc contrib guide outside of the title.

This change includes a redirect.

Change-Id: I5abf915f0b94a482afa961e6b86364c26aae5d79
2017-09-22 14:57:47 +00:00
Andreas Jaeger
945d2cad6e Sync id common files to security-doc
The security guide is translated into Indonesian, copy the common
translated files over whenever they change.

Change-Id: Ic34b01ac0f706e5649179a8f0b42018c1c5a46fb
2017-09-16 09:35:07 +02:00
Andreas Jaeger
822e7d695a [www] Handle pynotedb and subunitsql
These infra repos are publishing at the normal location (/REPO) and not
under /infra/REPO. Add them to our indices and avoid the redirection for
infra repos.

Change-Id: Ib2fa39725746c1b15bee1e2b57c58ff133e4187d
2017-09-09 07:49:48 +02:00
Doug Hellmann
dcd0ce9584 use whereto to test redirect rules
Add a file containing the tests for all of the redirect rules and set
up whereto to run during the check. The test file is a template, just
like the htaccess file, to ensure we auto-generate the rules for all
of the various project redirects.

Fix a few issues with the template generator that ended up producing
duplicate or useless rules.

Change-Id: I0d306149dc33f64d44d9482fab02101bd1f6d7f5
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-08-25 18:56:14 -04:00
Doug Hellmann
413f6686f6 ignore some repositories when generating redirects
The releases and releasestatus repos end up producing redirects that
conflict with other rules. Since neither has documentation anyway, we
can ignore them when generating the redirect rules.

Change-Id: Ie1e0217685a9d1209b5265112c8fc1ca2a49f5b0
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-08-25 16:57:18 -04:00
Doug Hellmann
307bc75db2 ignore non-template files based on their directory
Rather than looking at the file extension, look at the directory where
the files exist.

Change-Id: I1a2b668a285437478b19c1e965e1efb943f4f598
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-08-25 15:47:51 -04:00
Monty Taylor
27ebdb01df Use os-service-types for service type validation
We wrote a library to encompass this data, and it knows a bit more about
how to handle things like placement and whatnot. Use it.

Canonical name for senlin is clustering, use that one to fix build.

Change-Id: I215fb11c1e7e6aac5e020f3f3da3a0a0632cc2eb
2017-08-18 19:57:33 +02:00
Doug Hellmann
ad6dbc2966 open queens doc development
This patch marks pike as being "maintained" and creates the queens
landing page templates.

Change-Id: I8efa2706f340e3f6ae7815d0a30e025c5352b555
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-08-13 11:14:22 +02:00
Andreas Jaeger
7fdc0e2569 Build separate glossary
Build glossary as separate document.

Change-Id: I2592d68d6770d66ceaab87a774522ab1671b2296
2017-08-08 14:38:38 +01:00
Doug Hellmann
a85dab5668 only check links associated with the project type
When using www-generator.py with --check-all-links we don't want to
look for links that we do not expect to find or want to use. Limit the
URLs checked by the project type in that case.

Change-Id: I7560ec5d57d564730ebe60957153ea63f58002f8
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-08-05 15:17:33 -04:00
Doug Hellmann
e7bdd8b3a7 move series dates into generator
Change-Id: If4b7ea23b0a52f863d0770a8b460e07f609a0f25
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-25 16:48:59 -04:00
Doug Hellmann
37c83af5a5 add landing pages for historical versions
Add landing pages with references to all of the historical versions
for completeness. For the versions where we do have guides still
published, include links.

Change-Id: I62211adab1628f10e88f4892dd85a50170da1d96
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-25 16:48:58 -04:00
Doug Hellmann
fd2481a391 remove series pages for EOL series
Change-Id: I7ce2104474fcbfb45196bbf95d7b68158c400943
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-24 16:25:08 -04:00
Doug Hellmann
f8c4422317 add options to template generator to make it easier to only test links
Change-Id: I3862985ae188234df0fdd5a2fc156e12a73b1441
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-24 16:24:55 -04:00
Andreas Jaeger
e852bb5626 add SERIES_TITLE template variable
Add a variable to hold the title of the series for a given
template. This is set to the name, regardless of whether the series is
the "latest" or not.

Change-Id: I3ca9b19b9e46478107f4c689f85772f15863557a
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-24 16:24:43 -04:00
Doug Hellmann
f77fb23c63 run the link checks in parallel
Use a ThreadPool to check the links in parallel so the check runs
faster. Results are still reported in order.

Also only get the HEAD and not the whole body of the page.

Change-Id: Ie4fbb97dd3393ac050b36057aee3b939dea58de2
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-24 16:24:40 -04:00
Andreas Jaeger
33e278602f Update RST building
No need for build-install-guides-rst.sh, move to build-all-rst.sh.
Simplify build-all-rst.sh since we now do not have versioned or draft
docs anymore.

Change-Id: I213eff811d83ddfb1da3bb15dd8c99adeec93eb9
2017-07-21 20:25:56 +02:00
Doug Hellmann
11baa9130a update instructions for EOL for a series
Change-Id: I7e13d429f3d076e13882797432a8ee569ae90a82
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-21 11:21:48 -04:00
Doug Hellmann
c1bbe3f43e use all upper case variables for template vars from generator
Standardize on all upper case variable names so it is easy to tell
local variables in a template apart from global variables set by the
generator.

Change-Id: I69a73b580f1b32547287616c768887732553f9c5
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-21 11:21:48 -04:00
Doug Hellmann
ba9e59454a build the dropdown list of releases from data
Use the list of series names in the template generator to build the
dropdown list of series names so we don't have to update that template
by hand every release cycle.

Change-Id: I8f7950f25cb16b0bc5ffbd8e7d257157432b11f6
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-21 11:21:48 -04:00
Doug Hellmann
b88b95bd53 move settings about the released series to template generator
Move the setting for the released and in-development series to the
template generator so that we only have to track that information in
one place.

Change-Id: Ibd66e3ca9dea7451647b8d8289f0f9c11c002477
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-21 11:21:29 -04:00
Doug Hellmann
987b7aaf5c publish from master to the working site
Publish changes to this repo directly from the master branch to the
working site. Do not move changes into a "draft" folder before
publishing.

"Docs like code"

Change-Id: Iac54bd70ec587ba27019f5377dcf99a3c9460dcb
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-21 11:21:29 -04:00
Doug Hellmann
fed641fe66 have the page generator set the series for each page
Instead of setting the series for each page within the page by hand,
have the template rendering code derive the value based on the path to
the template.

Change-Id: I366e809b710e5766c6345d817afc2717eff5e0a8
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-21 11:21:29 -04:00
Doug Hellmann
6ea7250c93 be more accurate when checking for guide links
Use URLs with index.html to explicitly look for content being
published, not just an apache directory listing.

Closes-Bug: #1705250
Change-Id: Iae116ee81210489d9de2a1833d341c3842ac95e5
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-19 08:29:59 -04:00
Jenkins
014d719910 Merge "Removing the ops-guide from openstack-manuals" 2017-07-19 10:25:42 +00:00
Alexandra Settle
2479ff50ac Removing the ops-guide from openstack-manuals
The ops guide is intended to be moved to the os wiki.
This document can be sourced to migrate in the before-migration
tag in the repo.

Change-Id: I1ca840909438521d0d1da3469f7fa3f2927a6225
TODO: Add landing page detailing the above
2017-07-19 06:51:54 +00:00
Doug Hellmann
32a20ae990 make the test script smarter about its output directory
If we ran "tox -e publishdocs" locally, there won't be a
publish-docs/www/ directory and we don't want to create one when the
template test script is run. Have it check the existing directories to
derive where it should write its output.

Change-Id: I4ee8495bc0fb8c6a59424332d498956ea7e7e870
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-18 18:19:51 -04:00
Doug Hellmann
f0f66ebd28 track the list of templates being rendered
Rather than having a list of pages that needs to be checked and
updated by hand, keep track of the pages we write and pass the list to
the template to render a complete list automatically.

Change-Id: Ia3f5acb4ce850de5f64b66e93409860d4bbf2af7
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-18 21:09:17 +02:00
Doug Hellmann
2ad1d522cc rename templateFile variable to template_file
More pep8-y.

Change-Id: I76a6b218a30c65b15593336a29c4da0e916ebbb9
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-18 13:56:10 -04:00
Doug Hellmann
2dd080ebb7 move template rendering to its own function
Move the rendering logic to its own function so we can reuse it.

Also add a switch to disable URL checking to expedite testing.

Change-Id: I7fe6c56c3c204c3a9080996e2c41b5e18999ded0
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-18 13:55:47 -04:00
Doug Hellmann
4017c3e019 let the template generator compute relative paths
Instead of hard-coding paths to the css, js, and images directories
let the template generator compute those and pass them to the template
context.

Change-Id: I27a55deeb68136fa826496ec3d0223a303346724
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-18 13:52:31 -04:00
Doug Hellmann
d550f7d517 require all projects to have published docs
Before we add projects to the project list page, ensure that they have
already published documentation for the relevant series.

Change-Id: Ia81a775c95caa1b3a32640abb86c1beb56697d6e
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-18 11:42:16 -04:00
Doug Hellmann
1fd07e6e31 Remove the user-guide
The useful content in the user guide is all being moved to
project-specific docsets. We can remove the guide here, and replace it
with a /user/ landing page that provides links to all of the known user
guides for services and clients.

Change-Id: I7005b4288b94e755f406fd6a8e3273265b643042
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-14 17:05:46 +02:00
Doug Hellmann
60c0a3ebbb add a new landing page to list all admin guides
Add an admin-guide.html page that lists all admin guides for all
projects that have them.

Replace the link to the central admin guide on the main page with a link
to the new templated page.

Add a flag to the project data schema to indicate when a project has an
admin guide.

Update the project data to set the has_admin_guide flag for projects
that already have them published.

Change-Id: I514fe5034c26a42cfb4225d521691c32c0ba9ef4
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-11 22:06:58 +02:00
Doug Hellmann
0d07be764f add flag to www-generator to look for links to enable
Add --check-all-links option to www-generator.py and have it report a
warning for links that do exist for a project but that are not enabled
in the project data file. This can be used to update that data file to
turn on flags and link to additional documentation.

Change-Id: I7941c6fd8ef53c37bd309542d5938699fa6fb475
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-11 20:05:06 +00:00
Doug Hellmann
8b760d9bae add validation requiring clients to have a description value
Change-Id: I91e219b5e61015eb1369c6271a7c00a32f91caa5
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2017-07-11 13:47:16 -04:00