openstack/openstack-ansible-lxc_hosts
Code Issues Proposed changes

Convert lxc_hosts role to use simple download URL

Browse Source 
For a very long time we've been parsing and using the lxc images as
provided by upstream lxc. While these images are functional there are by
no means optimal. In general they're quite a bit larger than they need
to be and contian a lot of little sharp edges that have cut us over
the years. This change removes all of the lxc image cache parsing and
meta-data linking and simply downloads the rootfs a given url. To
maintain compatibility with the legacy images a script has been created
to parse the image index and return the legacy image url.

The result of this change:

* Access to smaller more optimal base image which is well known by the
  corresponding communities.

* Deployers now have the ability to set and forget the download url for an
  internal image instead of having to create a cache infrastructure
  compatible with the lxc download template.

* Any rootfs tarball will work as an image.

* Fewer tasks are executed and less memory is consumed resulting in faster
  deployment times.

* The base cache has a uniform meta-data setup giving all container
  types the same access to config, devices, and templating.

Change-Id: I1775e775bbb7fe86bdffdd8296c2cff5ebc5bac8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This commit is contained in:
Kevin Carter 2018-03-17 12:18:49 -05:00 committed by Kevin Carter (cloudnull)
parent d8bca1f440
commit 7e98da3d0f
16 changed files with 231 additions and 108 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
defaults/main.yml
View File

@ -154,25 +154,22 @@ lxc_cache_distro_packages: "{{ _lxc_cache_distro_packages }}"
# The value is specified in seconds, with the default being 20 minutes.
lxc_cache_prep_timeout: 1200
# Set the servers to download LXC images from
# NOTE(mhayden): The main images.linuxcontainers.org site will redirect
# requests to (us|uk).images.linuxcontainers.org upon the first request. We
# add the mirrors here to get around some HTTP 400 errors and allow aria2 to
# download from both mirrors at the same time.
# Set the server to download LXC images from.
lxc_image_cache_server_mirrors:
- https://us.images.linuxcontainers.org
- https://uk.images.linuxcontainers.org
- https://images.linuxcontainers.org
# The DNS name of the LXD server to source the base container cache from
# NOTE(cloudnull): This var should be removed in R.
lxc_image_cache_server: "{{ lxc_image_cache_server_mirrors[0].strip('http(?s)://') }}"
# URL for the image to build our containers
lxc_hosts_container_image_url: "{{ _lxc_hosts_container_image_url }}"
# If enabled the image will be fetched from the legacy [images.linuxcontainers.org].
lxc_hosts_container_image_download_legacy: false
# Local path to cached image
lxc_image_cache_path: "/var/lib/machines/{{ lxc_container_base_name }}"
# Mode to pull image. This is used to pull the image from a remote source.
# Valid options are [import-tar, import-raw]
lxc_image_cache_pull_mode: import-tar
lxc_image_cache_pull_mode: "import-tar"
# Set this option to true to pull a new cached image.
lxc_image_cache_refresh: false
@ -198,7 +195,7 @@ lxc_cache_download_template_options: >-
--release {{ lxc_cache_map.release }}
--arch {{ lxc_cache_map.arch }}
--force-cache
--server {{ lxc_image_cache_server }}
--server localhost
--variant {{ lxc_cache_default_variant }}
{{ lxc_cache_download_template_extra_options }}

2
handlers/main.yml
View File

@ -77,7 +77,7 @@
- name: Remove rootfs archive
file:
path: "/tmp/rootfs.tar.xz"
path: "/tmp/{{ cache_basename }}"
state: "absent"
- name: Restart dnsmasq

27
releasenotes/notes/lxc_image_cache-f14701a7qw90al21.yaml Normal file
View File

@ -0,0 +1,27 @@
---
features:
- The option ``lxc_hosts_container_image_url`` has been added allowing
deployers to define their base image url to whatever it needs to be
removing the requirement for operators to maintain an internal LXC
index in the event they want to host a private repository.
- The option ``lxc_hosts_container_image_download_legacy`` has been
added allowing a deployer to enable the use of the legacy lxc image
repository. This option is a Boolean and has a default of **false**.
deprecations:
- The variable ``lxc_image_cache_server_mirrors`` has been deprecated
in the "lxc_hosts" role. This option has been replaced by the static
variable ``lxc_hosts_container_image_url``. This variable will continue
to function as a single element list allowing existing automation to
function when in legacy image mode but should not be considered in use
by default.
- The variable ``lxc_image_cache_server`` has been deprecated in the
``lxc_hosts`` role. This option has been replaced by the static
variable ``lxc_hosts_container_image_url``.
other:
- The use of **images.linuxcontainers.org** is no longer required. While
the images provided by that build system are perfectly functional they
have been less than optimal in a lot ways for a very long time. The
``lxc_hosts`` role will now pull a base image from the upstream distro
being deployed. If a deployer wishes to continue using the images from
**images.linuxcontainers.org** they are welcome to but it is no longer
forced.

2
tasks/lxc_cache_preparation.yml
View File

@ -13,7 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Pull SystemD Version
- name: Pull systemd version
command: "systemctl --version"
changed_when: false
register: systemd_version

3
tasks/lxc_cache_preparation_systemd_new.yml
View File

@ -50,7 +50,7 @@
machinectl
--verify=no
{{ lxc_image_cache_pull_mode }}
/tmp/rootfs.tar.xz
/tmp/{{ cache_basename }}
{{ lxc_container_base_name }}
register: pull_image
until: pull_image | success
@ -60,6 +60,5 @@
failed_when:
- pull_image.rc != 0
- "'failed' in pull_image.stderr | lower"
with_items: "{{ lxc_images }}"
notify:
- Remove rootfs archive

2
tasks/lxc_cache_preparation_systemd_old.yml
View File

@ -136,7 +136,7 @@
- name: Place container rootfs
unarchive:
src: "/tmp/rootfs.tar.xz"
src: "/tmp/{{ cache_basename }}"
dest: "/var/lib/machines/{{ lxc_container_base_name }}"
remote_src: True
notify:

67
tasks/lxc_cache_prestage.yml
View File

@ -13,65 +13,50 @@
# See the License for the specific language governing permissions and
# limitations under the License.
- name: Set LXC cache path fact
- name: Set LXC cache fact(s)
set_fact:
cache_path_fact: "{{ lxc_container_cache_path }}/{{ lxc_cache_map.distro }}/{{ lxc_cache_map.release }}/{{ lxc_cache_map.arch }}/{{ lxc_cache_default_variant }}"
cache_index_item: "{{ lxc_cache_map.distro }};{{ lxc_cache_map.release }};{{ lxc_cache_map.arch }};{{ lxc_cache_default_variant }};"
cache_time: "{{ ansible_date_time.epoch }}"
- name: Fetch LXC image-index
shell: >
aria2c
--max-connection-per-server=4
--allow-overwrite=true
--dir=/tmp
--out=index-system
--check-certificate={{ (lxc_hosts_validate_certs | bool) | lower }}
{% for server in lxc_image_cache_server_mirrors %}{{ server }}/meta/1.0/index-system {% endfor %}
args:
warn: no
tags:
- skip_ansible_lint
- name: Legacy image url fetch
block:
- name: Create legacy image URL fetch
template:
src: "lxc-image-fetch-url.py.j2"
dest: "/usr/local/bin/lxc-image-fetch-url"
owner: "root"
group: "root"
mode: "0755"
- name: Retrieve the expiry object
slurp:
src: "/tmp/index-system"
changed_when: false
register: image_index
- name: Fetch legacy container image url
command: "/usr/local/bin/lxc-image-fetch-url {{ lxc_cache_map.distro }} {{ lxc_cache_map.release }} {{ lxc_cache_map.arch }} {{ lxc_cache_default_variant }}"
register: legacy_image_url
retries: 3
delay: 1
until: legacy_image_url | success
# Example index lines:
# ubuntu;xenial;amd64;default;20180123_08:05;/images/ubuntu/xenial/amd64/default/20180123_08:05/
# As there may be multiple images, and they use a timestamp, we can sort
# the resulting list in reverse order and use the first item in the list
# as it will be the latest available.
- name: Set image index fact
- name: Set LXC cache fact(s) (legacy)
set_fact:
lxc_hosts_container_image_url: "{{ legacy_image_url.stdout.strip() }}"
when:
- lxc_hosts_container_image_download_legacy | bool
- name: Set LXC cache basename
set_fact:
lxc_images: >
{%- set images = [] %}
{%- for image in (image_index.content | b64decode).splitlines() %}
{%- if image | match("^" + cache_index_item) %}
{%- set _ = images.append(image) %}
{%- endif %}
{%- endfor %}
{{- images | sort(reverse=True) -}}
cache_basename: "{{ lxc_hosts_container_image_url | basename }}"
# We only want to download a single image, rather than downloading
# each of them in turn and they overwrite each other as they all
# download to the same file name and path.
- name: Pre-stage the LXC image on the system
shell: >
aria2c
--max-connection-per-server=4
--allow-overwrite=true
--dir=/tmp
--out=rootfs.tar.xz
--out={{ cache_basename }}
--check-certificate={{ (lxc_hosts_validate_certs | bool) | lower }}
{% for server in lxc_image_cache_server_mirrors %}{{ server }}{{ lxc_images[0].split(';')[-1] }}rootfs.tar.xz {% endfor %}
{{ lxc_hosts_container_image_url }}
> /var/log/aria2c-image-prestage.log 2>&1
args:
warn: no
when:
- lxc_images is defined
register: prestage_image
async: 300
poll: 0

120
templates/lxc-image-fetch-url.py.j2 Normal file
View File

@ -0,0 +1,120 @@
#!/usr/bin/env python
# Copyright 2017, Rackspace US, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
try:
import httplib
except ImportError:
import http.client as httplib
import ssl
try:
import urlparse
except ImportError:
import urllib.parse as urlparse
import sys
__DOC__ = """
USAGE:
This script will take arguments passed into it to discover the url
of a given container image using the distro, release, architecture,
and variant as the parameters.
EXAMPLE:
# python lxc-image-fetch-url ubuntu xenial amd64 default
"""
LXC_CACHE_SERVER = '{{ lxc_image_cache_server_mirrors[0].strip() }}'
LXC_INDEX = '{0}/meta/1.0/index-system'.format(LXC_CACHE_SERVER)
def get_image_url(url, depth=0):
if depth > 10:
raise SystemExit('Too many redirects')
url_path = urlparse.urlparse(url, allow_fragments=True)
if url_path.scheme == 'https':
conn = httplib.HTTPSConnection(
host=url_path.netloc,
context=ssl._create_unverified_context()
)
else:
conn = httplib.HTTPConnection(host=url_path.netloc)
try:
conn.request('GET', url_path.path)
except httplib.BadStatusLine:
raise SystemExit('Connection Failure')
else:
resp = conn.getresponse()
headers = dict(resp.getheaders())
check_redirect = headers.get('location', None)
if not check_redirect:
check_redirect = headers.get('Location', None)
if check_redirect:
depth += 1
return get_image_url(
url=check_redirect,
depth=depth
)
else:
return (
str(url),
resp.read().decode('UTF-8').splitlines()
)
finally:
conn.close()
def main():
try:
distro = sys.argv[1]
release = sys.argv[2]
arch = sys.argv[3]
variant = sys.argv[4]
except IndexError:
print('Missing argument, Please see the documentation.')
raise SystemExit(__DOC__)
_, meta = get_image_url(url=LXC_INDEX, depth=0)
image_hint = '{0};{1};{2};{3}'.format(
distro,
release,
arch,
variant
)
images = sorted([i for i in meta if i.strip().startswith(image_hint)])
if not images:
print(__DOC__ + '\nAvailable options:')
for line in meta:
print(';'.join(line.split(';')[:-2]))
raise SystemExit(
'No Image found with image hint "{0}"'.format(image_hint)
)
container_url = urlparse.urljoin(
urlparse.urljoin(
LXC_CACHE_SERVER,
images[0].split(';')[-1]
),
'rootfs.tar.xz'
)
print(container_url)
if __name__ == '__main__':
main()

21
templates/prep-scripts/_container_sys_setup.sh.j2 Normal file
View File

@ -0,0 +1,21 @@
# System containers will require mount points for the following
mkdir -p /dev
mkdir -p /proc
mkdir -p /sys/fs/cgroup
# In order for the package manager to function /dev/null must exist. This is
# is being run here because some images do not create /dev/null by default.
if [[ ! -e /dev/null ]]; then
mknod /dev/null c 1 3
chmod 0666 /dev/null
fi
# Create the directory where local facts will be stored
mkdir -p /etc/ansible/facts.d
# Create the cache resolvers
rm /etc/resolv.conf || true
{% for resolver in lxc_cache_prep_dns %}
echo "nameserver {{ resolver }}" >> /etc/resolv.conf
{% endfor %}

7
templates/prep-scripts/centos_7_prep.sh.j2
View File

@ -3,12 +3,7 @@ set -e -x
{{ lxc_cache_prep_pre_commands }}
mkdir -p /etc/ansible/facts.d/
rm /etc/resolv.conf || true
{% for resolver in lxc_cache_prep_dns %}
echo "nameserver {{ resolver }}" >> /etc/resolv.conf
{% endfor %}
{% include 'templates/prep-scripts/_container_sys_setup.sh.j2' %}
rpm --import /etc/pki/rpm-gpg/*

7
templates/prep-scripts/opensuse_leap_42_prep.sh.j2
View File

@ -3,12 +3,7 @@ set -e -x
{{ lxc_cache_prep_pre_commands }}
mkdir -p /etc/ansible/facts.d/
rm /etc/resolv.conf || true
{% for resolver in lxc_cache_prep_dns %}
echo "nameserver {{ resolver }}" >> /etc/resolv.conf
{% endfor %}
{% include 'templates/prep-scripts/_container_sys_setup.sh.j2' %}
# We have (tried to!) copied repo-oss and repo-update from the host so wipe everything else.
find /etc/zypp/repos.d/ -type f ! -name "repo-oss.repo" -a ! -name "repo-update.repo" -delete

7
templates/prep-scripts/ubuntu_16_prep.sh.j2
View File

@ -3,12 +3,7 @@ set -e -x
{{ lxc_cache_prep_pre_commands }}
mkdir -p /etc/ansible/facts.d/
rm /etc/resolv.conf || true
{% for resolver in lxc_cache_prep_dns %}
echo "nameserver {{ resolver }}" >> /etc/resolv.conf
{% endfor %}
{% include 'templates/prep-scripts/_container_sys_setup.sh.j2' %}
apt-key add /root/repo.keys
rm /root/repo.keys

29
tests/test.yml
View File

@ -44,17 +44,6 @@
set_fact:
lxc_container_ssh_key: "{{ _root_ssh_key['content'] | b64decode }}"
# This is a very dirty hack due to images.linuxcontainers.org
# constantly failing to resolve in openstack-infra.
- name: Implement hard-coded hosts entries for consistently failing name
lineinfile:
path: "/etc/hosts"
line: "{{ item }}"
state: present
with_items:
- "91.189.91.21 images.linuxcontainers.org us.images.linuxcontainers.org"
- "91.189.88.37 images.linuxcontainers.org uk.images.linuxcontainers.org"
# This is a temporary hack to override the LXC image source to
# the reverse proxy if the test is run in OpenStack-Infra.
- name: Check if this is an OpenStack-CI nodepool instance
@ -62,24 +51,6 @@
path: /etc/nodepool/provider
register: nodepool
- name: Discover the lxc_image_cache_server value when in nodepool
shell: |
source /etc/ci/mirror_info.sh
echo "${NODEPOOL_MIRROR_HOST}:8080/images.linuxcontainers"
args:
executable: /bin/bash
register: lxc_reverse_proxy
when:
- nodepool.stat.exists | bool
tags:
- skip_ansible_lint
- name: Set a fact to override lxc_image_cache_server value when in nodepool
set_fact:
lxc_image_cache_server_mirrors: ["http://{{ lxc_reverse_proxy.stdout.strip('/') }}"]
when:
- nodepool.stat.exists | bool
roles:
- role: "lxc_hosts"
lxc_net_address: 10.100.100.1

8
vars/redhat-7.yml
View File

@ -16,6 +16,8 @@
system_config_dir: "/etc/sysconfig"
systemd_utils_prefix: "/lib/systemd"
_lxc_hosts_container_image_url: "https://github.com/CentOS/sig-cloud-instance-images/raw/CentOS-7/docker/centos-7-docker.tar.xz"
# Required rpm packages.
lxc_hosts_distro_packages:
- aria2
@ -47,7 +49,7 @@ lxc_xz_bin: xz
lxc_cache_map:
distro: centos
arch: amd64
arch: "{{ lxc_architecture_mapping.get( ansible_architecture ) }}"
release: 7
copy_from_host:
- /etc/environment
@ -63,6 +65,8 @@ _lxc_cache_distro_packages:
- cronie
- epel-release
- gcc
- iproute
- iputils
- libffi-devel
- openssh-server
- openssl
@ -71,6 +75,8 @@ _lxc_cache_distro_packages:
- python-devel
- rsync
- sudo
- systemd
- systemd-sysv
- systemd-networkd
- tar
- wget

10
vars/suse-42.yml
View File

@ -18,6 +18,8 @@ lxc_hosts_external_repo:
- name: "OBS:Virtualization:containers"
uri: "{{ lxc_hosts_opensuse_mirror_obs_url }}/repositories/Virtualization:/containers/openSUSE_Leap_{{ ansible_distribution_version }}/"
_lxc_hosts_container_image_url: "https://github.com/openSUSE/docker-containers-build/raw/openSUSE-Leap-42.3/{{ ansible_architecture }}/openSUSE-Leap-42.3.base.{{ ansible_architecture }}.tar.xz"
system_config_dir: "/etc/sysconfig"
systemd_utils_prefix: "/usr/lib/systemd"
@ -41,8 +43,8 @@ lxc_xz_bin: xz
lxc_cache_map:
distro: opensuse
arch: amd64
release: "{{ ansible_distribution_version }}"
arch: "{{ lxc_architecture_mapping.get( ansible_architecture ) }}"
release: "42.3"
copy_from_host:
- /etc/environment
- /etc/localtime
@ -56,6 +58,8 @@ _lxc_cache_distro_packages:
- cronie
- dbus-1
- gcc
- iproute2
- iputils
- libffi-devel
- libopenssl-devel
- openssh
@ -65,6 +69,8 @@ _lxc_cache_distro_packages:
- python-xml
- rsync
- sudo
- systemd
- systemd-sysvinit
- tar
- wget
- which

6
vars/ubuntu-16.04.yml
View File

@ -16,6 +16,8 @@
## APT Cache Options
cache_timeout: 600
_lxc_hosts_container_image_url: "http://cdimage.ubuntu.com/ubuntu-base/releases/16.04/release/ubuntu-base-16.04.4-base-{{ lxc_cache_map.arch }}.tar.gz"
# Required apt packages.
lxc_hosts_distro_packages:
- apparmor
@ -74,6 +76,8 @@ _lxc_cache_distro_packages:
- cron # xenial doesn't have cronie
- dbus
- gcc
- iproute2
- iputils-ping
- libffi-dev
- libssl-dev
- openssh-server
@ -83,6 +87,8 @@ _lxc_cache_distro_packages:
- python3-dev
- rsync
- sudo
- systemd
- systemd-sysv
- tar
- wget
- debianutils # for 'which' executable