StarlingX Application Generation Tool

The purpose of this tool is to generate a StarlingX App from a workload/app in an easy way without the complete StarlingX build environment. The tool is ulimately a python package, but is also wrapped as a debian package because this tool is included in StarlingX. After a full install users may use the generator to help develop apps directly on platform.

Below you will find the steps to deploy an application as a StarlingX App.

• StarlingX Application Generation Tool
  • Why deploy an application as a StarlingX application?
  • Tools requirements
  • Prerequisites
  • Generate the StarlingX Application package
    • App manifest configuration
    • Metadata File Configuration
    • App Setup configuration
  • Run the StarlingX App Generator
    • FluxCD Packaging
      • FluxCD Manifest
      • Plugins
      • Metadata
      • Tarballs
  • Customizing the application
    • FluxCD Manifest
      • FluxCD
      • Plugins
      • Other files

Why deploy an application as a StarlingX application?

It's important to understand that any user workload can be deployed in many ways to the Kubernetes cluster(s) that StarlingX manages:

• with the most common Kubernetes package manager, Helm;
• with Flux, to enjoy all the benefits that come with it ; and finally
• as a StarlingX Application, which benefits from tight integration with the StarlingX system.

Deploying applications offers a number of benefits in a deployed environment:

1. Applications are managed throughout an upgrade from one version of STX to another.
2. Appliations are managed throughout an upgrade of Kubernetes.
3. Deployment of applications is managed automatically in a distributed cloud environment.

TODO Elaborate on additional vantages of deploying an app as a StarlingX app

Software Requirements

• Helm version 3+
• Python version 3.8+
• Python packages tracked in ./stx-app-generator/stx-app-generator/requirements.txt
• For Testing, Python packages tracked in ./test-requirements.txt

Prerequisites

As the StarlingX Platform manages a distributed Kubernetes cluster, for an application to be deployed as a StarlingX App it needs to be designed so it can run on Kubernetes.

Additionally, it needs to provide a Helm Chart which will be managed via FluxCD by StarlingX itself.

Standalone Installation

If one desires to install the python package in an environment other than StarlingX, the use one of the following two options.

Option 1 - Install in same folder

git clone https://opendev.org/starlingx/app-gen-tool.git
cd ./app-gen-tool/
python -m venv venv
source ./venv/bin/activate
pip install -e ./stx-app-generator/stx-app-generator

Option 2 - Install From Url

python -m venv venv
source ./venv/bin/activate
pip install git+https://opendev.org/starlingx/app-gen-tool.git#subdirectory=./stx-app-generator/stx-app-generator

Generate the StarlingX Application package

This is what you'll find in the app-gen-tool repository:

.
├── LICENSE
├── README.md
├── app_manifest.yaml
├── bandit.yaml
├── debian_build_layer.cfg
├── debian_iso_image.inc
├── debian_pkg_dirs
├── debian_stable_wheels.inc
├── example
│   ├── adminer-0.2.1.tgz
│   └── app-test-adminer-1.yaml
├── playbooks
│   └── app-gen-tool-tox-coverage
│       └── pre.yaml
├── scripts
│   └── fetch_chart_info.sh
├── stx-app-generator
│   ├── debian
│   │   ├── deb_folder
│   │   │   ├── changelog
│   │   │   ├── control
│   │   │   ├── copyright
│   │   │   ├── rules
│   │   │   ├── source
│   │   │   │   ├── format
│   │   │   │   └── options
│   │   │   ├── stx-app-generator-wheels.install
│   │   │   ├── stx-app-generator.dirs
│   │   │   └── stx-app-generator.install
│   │   └── meta_data.yaml
│   └── stx-app-generator
│       ├── app_gen_tool
│       │   ├── __init__.py
│       │   ├── application.py
│       │   ├── cmd
│       │   │   ├── __init__.py
│       │   │   └── generator.py
│       │   ├── common.py
│       │   ├── constants.py
│       │   ├── fluxcd.py
│       │   ├── generator.py
│       │   ├── templates_flux
│       │   │   ├── base
│       │   │   │   ├── helmrepository.template
│       │   │   │   ├── kustomization.template
│       │   │   │   └── namespace.template
│       │   │   ├── fluxcd-manifest
│       │   │   │   ├── helmrelease.template
│       │   │   │   └── kustomization.template
│       │   │   └── kustomization.template
│       │   └── templates_plugins
│       │       ├── common.template
│       │       ├── helm.template
│       │       ├── kustomize.template
│       │       └── lifecycle.template
│       ├── requirements.txt
│       ├── setup.cfg
│       ├── setup.py
│       ├── stx_app_generator.egg-info
│       │   ├── PKG-INFO
│       │   ├── SOURCES.txt
│       │   ├── dependency_links.txt
│       │   ├── entry_points.txt
│       │   ├── not-zip-safe
│       │   ├── pbr.json
│       │   ├── requires.txt
│       │   └── top_level.txt
│       └── tests
│           ├── __init__.py
│           ├── conftest.py
│           └── unit
│               ├── __init__.py
│               ├── resources
│               │   ├── adminer-0.2.1.tgz
│               │   └── app-test.yaml
│               ├── test_application_class.py
│               └── test_camel_case.py
├── test-requirements.txt
└── tox.ini

The app_manifest.yaml is the most important configuration step since it specifies everything necessary to build the StarlingX application.

The app manifest configuration can be divided into three parts, which will have their own dedicated section below:

• App manifest configuration
• Metadata file configuration
• App Setup Configuration

App manifest configuration

In this stage the section appManifestFile-config from the app_manifest.yaml will be configured.

Below you will find a brief explanation of every one of the required fields which will help you fill them out for you application:

• appName field: desired StarlingX application name, referenced throughout the whole system.
• appVersion field: the version of the application that the generated package will represent.
• namespace field: desired StarlingX application namespace (note that this namespace is not the same as the Kubernetes namespace).
• chart
  • name field: your Helm chart name as it is in the chart metadata.
  • version field: your chart version as it is in the chart metadata.
  • path field: relative path to the Helm chart directory, Helm repo or Helm package file.

  Note

  : Currently only Helm charts in directories have been tested.

  • chartGroup field: chartgroup in which the helm-chart belong
• chartGroup
  • name: Name of the chartgroup
  • description: description of chart set
  • sequenced: enables sequenced chart deployment in a group
  • chart_names: a list of the names of the charts from your application.
• manifest:
  • name: manifest name
  • releasePrefix: appends to the front of all charts released by the manifest in order to manage releases throughout their lifecycle

Note that the minimum required fields that will need to be filled in order for the StarlingX App Generator to work properly will depend on the intended type of packaging.

Metadata File Configuration

In this stage the section metadataFile-config from the app_manifest.yaml will be configured.

This section's objective is to configure the generation/creation of a metadata.yaml file, which is the core metadata file for a StarlingX App package.

This metadata.yaml file is very flexible, hence the metadataFile-config section is also very flexible. Other values may be passed in order to enable advanced features within the StarlingX platform. For a better understanding of each attribute in this section please refer to this link.

App Setup configuration

In this stage the section setupFile-config from the app_manifest.yaml will be configured.

Below you will find a brief explanation of every one of the required fields which will help you fill them out for you application:

• metadata section:
  • author/author-email/url fields: authorship information.
  • classifier section: an array of additional information.

This section is related to the setup-cfg file that will be generated. For more advanced use cases you may want to refer to the documentation.

Run the StarlingX App Generator

One must install via pip install method described in the Installation section.

To get command line options:

stx-app-generator -h

Recommend reviewing the '-h' output for a full list of options.

Here is an example.

stx-app-generator -i app_manifest.yaml -o ./output

With the command above, the StarlingX App Generator will create a set of files and package everything in the chosen StarlingX format, and then finally place the file in an output folder.

Supported Packaging Methods

The following sections explain in high-level the most important parts of the package.

Flux Packaging

FluxCD Manifest

The generator will first create the FluxCD Manifest following the structure below:

fluxcd-manifests/
├── base
│   ├── helmrepository.yaml
│   ├── kustomization.yaml
│   └── namespace.yaml
├── kustomization.yaml
└── CHART-NAME
    ├── helmrelease.yaml
    ├── kustomization.yaml
    ├── CHART-NAME-static-overrides.yaml
    └── CHART-NAME-system-overrides.yaml

For every Helm chart configured in the app_manifest.yaml file, a folder with the name of the chart will be created.

Note

: The CHART-NAME-static-overrides.yaml file will be empty.

Plugins

After the creation of the FluxCD Manifest, the generator will also create a set of plugins with an empty implementation.

The Structure of the plugins created will be:

plugins/
├── k8sapp_APP_NAME
│   ├── common
│   │   ├── __init__.py
│   │   └── constants.py
│   ├── helm
│   │   ├── __init__.py
│   │   └── CHART_NAME.py
│   ├── kustomize
│   │   ├── __init__.py
│   │   └── kustomize_APP_NAME.py
│   └── lifecycle
│       ├── __init__.py
│       └── lifecycle_APP_NAME.py
├── __init__.py
├── setup.cfg
└── setup.py

The setup.cfg file will be created according to the setupFile-config section in the app_manifest.yaml.

Metadata

In the third step of the execution the metadata.yaml file will be generated with the information given in metadataFile-config section in the app_manifest.yaml.

Tarballs

After the main files have been created, the generator will start packaging everything.

Firstly it will package every helm-chart, that was given in the app_manifest.yaml file, into a .tgz file, saving these files into a folder named charts.

The generator, then, will package the plugins with the wheel format.

Lastly, creates a checksum sha256 signature file for the output tarball and the output tarball itself, which will be called

<APPNAME>-<APPVERSION>.tgz

The structure of the app inside the tarball will be the following:

 APPNAME-APPVERSION.tgz/
 ├── charts/
 ├── fluxcd-manifests/
 ├── plugins/
 ├── checksum.sha256
 └── metadata.yaml

Warning: At this point, the generated package is a working StarlingX App, however it contains empty templates for some files. The following sections will describe how to further enhance your StarlingX App.

If you wish to add customization for the particularities of your application, it is important to modify some of the generated files.

In order to allow such customization, the generator provides additional functions to modify specific files in the package.

stx-app-generator -i app_manifest.yaml [-o ./output] [--overwrite] [--no-package]|[--package-only]

Where:

• -i/--input: path to the app_manifest.yaml configuration file.
• -o/--output: output folder. Defaults to a new folder with the app name in the current directory.
• --overwrite: deletes existing output folder before starting.
• --no-package: only creates the FluxCD manifest, plugins and the metadata file, without compressing them in a tarball.
• --package-only: create the plugins wheels, sha256 file, helm-chart tarball and package the entire application into a tarball.

This means that, in order to be able to make additional configuration, one must:

• first run the App Generator with --no-package;
• then do the changes (described in the following sections);
• finally, run the App Generator again with --package-only.

Flux Manifest

FluxCD

Note

: this section needs improvement.

Most of the generated manifest won't need any modification, but for every Helm chart in the app_manifest.yaml, a static-overrides file will be created. The static-overrides file contains all information that is not to be overwritten inside the values.yaml of the Helm chart.

Plugins

The StarlingX App Generator will create 3 main plugins: the Helm, the Kustomize and the Lifecycle plugins.

• The helm/APP_NAME.py file is responsible for the overriding methods that will be used to create the Helm overrides for the StarlingX App.

• The kustomize_APP_NAME.py is a plugin that is used to make changes to the top-level kustomization resource list based on the platform mode.

• The lifecycle_APP_NAME.py is responsible for performing lifecycle actions on the application using the lifecycle hooks of the StarlingX Platform.

The files created by the generator will have an empty implementation and is up to the developer to implement everything that is necessary for the application to run as intended.

The sysinv folder in the StarlingX config repository contains a multitude of functions and variables that may be helpful in the development of application plugins.

Other files

For the customization of the application the modifications above, in the FluxCD and the plugins, should be enough for the application to run as expected in the StarlingX platform.

With that in mind, it is recommended to check if the metadata and the setup.cfg have been created as they should. Particularly, the setup.cfg may need careful attention if the modifications on the plugin file should be reflected in it.