Add Debian Packaging support.

To include app-gen-tool as a built-in tool to STX at
ISO build time, this python package must be wrapped
into a debian package. The debian package will install
this repo as a python package, so ultimately this repo
shall remain a python package.

The readme was updated to reflect the new directory
structure.

Note - Currently removing debian_iso_image.inc until
Armada content is deprecated.

Test Plan:
PASS: Test that this package builds properly in the build
      process.
PASS: Build an ISO and install in AIO-SX config. Verify that
      the package functions as expected with the example
      application.

Story: 2010937
Task: 49156
Task: 49158

Depends-On: https://review.opendev.org/c/starlingx/manifest/+/902093

Change-Id: I3d1981fe4742f12f5802b77e883cb6a55afdc76c
Signed-off-by: Reed, Joshua <Joshua.Reed@windriver.com>

.vscode/launch.json

@@ -21,11 +21,11 @@
         },
         {
             // First pip install this repo
-            // starlingx-app-generator --input=./TEST/app-test-adminer-1.yaml.yaml --output=./TEMP/output --overwrite
+            // stx-app-generator --input=./example/app-test-adminer-1.yaml --output=./example/output --type=fluxcd --overwrite
             "name": "Python: Installed Generator",
             "type": "python",
             "request": "launch",
-            "program": "${workspaceFolder}/venv/bin/starlingx-app-generator",
+            "program": "${workspaceFolder}/venv/bin/stx-app-generator",
             "cwd": "${workspaceFolder}",
             "console": "integratedTerminal",
             "justMyCode": true,

.vscode/settings.json

@@ -3,7 +3,7 @@
         100
     ],
     "python.testing.pytestArgs": [
-        "tests"
+        "./stx-app-generator/stx-app-generator/tests"
     ],
     "python.testing.unittestEnabled": false,
     "python.testing.pytestEnabled": true

README.md

@@ -1,7 +1,9 @@
 # StarlingX Application Generation Tool
 
 The purpose of this tool is to generate a StarlingX App from a workload/app
-in an easy way without the complete StarlingX build environment.
+in an easy way without the complete StarlingX build environment.  The tool is ulimately a python
+package, but is also wrapped as a debian package because this tool is included in StarlingX.  After
+a full install users may use the generator to help develop apps directly on platform.
 
 Below you will find the steps to deploy an application as a **StarlingX App**.
 
@@ -13,7 +15,7 @@ Below you will find the steps to deploy an application as a **StarlingX App**.
     - [App manifest configuration](#app-manifest-configuration)
     - [Metadata File Configuration](#metadata-file-configuration)
     - [App Setup configuration](#app-setup-configuration)
-  - [Run the StarlingX App Generator](#run-the-starlingx-app-generator)
+  - [Run the StarlingX App Generator](#run-the-stx-app-generator)
     - [FluxCD Packaging](#fluxcd-packaging)
       - [FluxCD Manifest](#fluxcd-manifest)
       - [Plugins](#plugins)
@@ -41,13 +43,19 @@ ways to the Kubernetes cluster(s) that StarlingX manages:
 - as a StarlingX Application, which benefits from tight integration with the
   [StarlingX system](https://opendev.org/starlingx/config).
 
-  ```TODO Elaborate on the vantages of deploying an app as a StarlingX app```
+Deploying applications offers a number of benefits in a deployed environment:
+
+1. Applications are managed throughout an upgrade from one version of STX to another.
+2. Appliations are managed throughout an upgrade of Kubernetes.
+3. Deployment of applications is managed automatically in a distributed cloud environment.
+
+  ```TODO Elaborate on additional vantages of deploying an app as a StarlingX app```
 
 ## Software Requirements
 
 - Helm version 2+
 - Python version 3.8+
-- Python packages tracked in ./requirements.txt
+- Python packages tracked in ./stx-app-generator/stx-app-generator/requirements.txt
 - For Testing, Python packages tracked in ./test-requirements.txt
 
 ## Prerequisites
@@ -59,7 +67,10 @@ run on [Kubernetes](https://kubernetes.io/).
 Additionally, it needs to provide a [Helm Chart](https://helm.sh/)
 which will be managed via [FluxCD](https://fluxcd.io/) by StarlingX itself.
 
-## Installation
+## Standalone Installation
+
+If one desires to install the python package in an environment other than StarlingX, the use one of
+the following two options.
 
 ### Option 1 - Install in same folder
 
@@ -68,7 +79,7 @@ git clone https://opendev.org/starlingx/app-gen-tool.git
 cd ./app-gen-tool/
 python -m venv venv
 source ./venv/bin/activate
-pip install -e .
+pip install -e ./stx-app-generator/stx-app-generator
 ```
 
 ### Option 2 - Install From Url
@@ -76,7 +87,7 @@ pip install -e .
 ```shell
 python -m venv venv
 source ./venv/bin/activate
-pip install git+https://opendev.org/starlingx/app-gen-tool.git
+pip install git+https://opendev.org/starlingx/app-gen-tool.git#subdirectory=./stx-app-generator/stx-app-generator
 ```
 
 ## Generate the StarlingX Application package
@@ -84,55 +95,91 @@ pip install git+https://opendev.org/starlingx/app-gen-tool.git
 This is what you'll find in the `app-gen-tool` repository:
 
 ```shell
+.
 ├── LICENSE
 ├── README.md
-├── app_gen_tool
-│   ├── __init__.py
-│   ├── __pycache__
-│   │   ├── __init__.cpython-39.pyc
-│   │   ├── application.cpython-39.pyc
-│   │   ├── common.cpython-39.pyc
-│   │   ├── constants.cpython-39.pyc
-│   │   └── generator.cpython-39.pyc
-│   ├── application.py
-│   ├── cmd
-│   │   ├── __init__.py
-│   │   ├── __pycache__
-│   │   │   ├── __init__.cpython-39.pyc
-│   │   │   └── generator.cpython-39.pyc
-│   │   └── generator.py
-│   ├── common.py
-│   ├── constants.py
-│   ├── generator.py
-│   ├── template_armada
-│   │   ├── armada-chart.template
-│   │   ├── armada-chartgroup.template
-│   │   └── armada-manifest.template
-│   ├── templates_flux
-│   │   ├── base
-│   │   │   ├── helmrepository.template
-│   │   │   ├── kustomization.template
-│   │   │   └── namespace.template
-│   │   ├── fluxcd-manifest
-│   │   │   ├── helmrelease.template
-│   │   │   └── kustomization.template
-│   │   └── kustomization.template
-│   └── templates_plugins
-│       ├── common.template
-│       ├── helm.template
-│       ├── kustomize.template
-│       └── lifecycle.template
 ├── app_manifest.yaml
+├── bandit.yaml
+├── debian_build_layer.cfg
+├── debian_iso_image.inc
+├── debian_pkg_dirs
+├── debian_stable_wheels.inc
 ├── example
-│   ├── adminer-0.2.1.tgz
-│   ├── app-test-adminer-1.yaml
-├── requirements.txt
+│   ├── adminer-0.2.1.tgz
+│   └── app-test-adminer-1.yaml
+├── playbooks
+│   └── app-gen-tool-tox-coverage
+│       └── pre.yaml
 ├── scripts
-│   └── fetch_chart_info.sh
-├── setup.py
+│   └── fetch_chart_info.sh
+├── stx-app-generator
+│   ├── debian
+│   │   ├── deb_folder
+│   │   │   ├── changelog
+│   │   │   ├── control
+│   │   │   ├── copyright
+│   │   │   ├── rules
+│   │   │   ├── source
+│   │   │   │   ├── format
+│   │   │   │   └── options
+│   │   │   ├── stx-app-generator-wheels.install
+│   │   │   ├── stx-app-generator.dirs
+│   │   │   └── stx-app-generator.install
+│   │   └── meta_data.yaml
+│   └── stx-app-generator
+│       ├── app_gen_tool
+│       │   ├── __init__.py
+│       │   ├── application.py
+│       │   ├── armada.py
+│       │   ├── cmd
+│       │   │   ├── __init__.py
+│       │   │   └── generator.py
+│       │   ├── common.py
+│       │   ├── constants.py
+│       │   ├── fluxcd.py
+│       │   ├── generator.py
+│       │   ├── templates_armada
+│       │   │   ├── armada-chart.template
+│       │   │   ├── armada-chartgroup.template
+│       │   │   └── armada-manifest.template
+│       │   ├── templates_flux
+│       │   │   ├── base
+│       │   │   │   ├── helmrepository.template
+│       │   │   │   ├── kustomization.template
+│       │   │   │   └── namespace.template
+│       │   │   ├── fluxcd-manifest
+│       │   │   │   ├── helmrelease.template
+│       │   │   │   └── kustomization.template
+│       │   │   └── kustomization.template
+│       │   └── templates_plugins
+│       │       ├── common.template
+│       │       ├── helm.template
+│       │       ├── kustomize.template
+│       │       └── lifecycle.template
+│       ├── requirements.txt
+│       ├── setup.cfg
+│       ├── setup.py
+│       ├── stx_app_generator.egg-info
+│       │   ├── PKG-INFO
+│       │   ├── SOURCES.txt
+│       │   ├── dependency_links.txt
+│       │   ├── entry_points.txt
+│       │   ├── not-zip-safe
+│       │   ├── pbr.json
+│       │   ├── requires.txt
+│       │   └── top_level.txt
+│       └── tests
+│           ├── __init__.py
+│           ├── conftest.py
+│           └── unit
+│               ├── __init__.py
+│               ├── resources
+│               │   ├── adminer-0.2.1.tgz
+│               │   └── app-test.yaml
+│               ├── test_application_class.py
+│               └── test_camel_case.py
 ├── test-requirements.txt
-├── tox.ini
-
+└── tox.ini
 ```
 
 The `app_manifest.yaml` is the most important configuration step since it
@@ -219,7 +266,7 @@ One must install via pip install method described in the Installation section.
 To get command line options:
 
 ```shell
-starlingx-app-generator -h
+stx-app-generator -h
 ```
 
 Recommend reviewing the '-h' output for a full list of options.
@@ -227,7 +274,7 @@ Recommend reviewing the '-h' output for a full list of options.
 Here is an example.
 
 ```shell
-starlingx-app-generator -i app_manifest.yaml -t [armada/fluxcd/both] -o ./output
+stx-app-generator -i app_manifest.yaml -t [armada/fluxcd/both] -o ./output
 ```
 
 With the command above, the StarlingX App Generator will create a set of files
@@ -357,7 +404,7 @@ In order to allow such customization, the generator provides additional
 functions to modify specific files in the package.
 
 ```shell
-starlingx-app-generator -i app_manifest.yaml -t <armada/fluxcd/both> [-o ./output] [--overwrite] [--no-package]|[--package-only]
+stx-app-generator -i app_manifest.yaml -t <armada/fluxcd/both> [-o ./output] [--overwrite] [--no-package]|[--package-only]
 ```
 
 Where:

debian_build_layer.cfg

@@ -0,0 +1 @@
+flock

debian_pkg_dirs

@@ -0,0 +1 @@
+stx-app-generator

debian_stable_wheels.inc

@@ -0,0 +1 @@
+stx-app-generator-wheel

stx-app-generator/debian/deb_folder/changelog

@@ -0,0 +1,5 @@
+stx-app-generator (1.0-1) unstable; urgency=medium
+
+  * Initial release.
+
+ -- Joshua Reed <joshua.reed@windriver.com>  Wed, 29 Nov 2023 16:22:25 -0400

stx-app-generator/debian/deb_folder/control

@@ -0,0 +1,31 @@
+Source: stx-app-generator
+Section: admin
+Priority: optional
+Maintainer: Starlingx Developers <StarlingX-discuss@lists.StarlingX.io>
+Build-Depends: debhelper-compat (= 13), dh-python, python3-setuptools, python3-all, python3-pbr, python3-wheel
+Build-Depends-Indep:
+ python3-fixtures,
+ python3-mock,
+ python3-testtools,
+ python3-testresources
+Standards-Version: 4.4.1
+Homepage: https://www.starlingx.io
+
+Package: stx-app-generator
+Architecture: all
+Depends: ${python3:Depends}, ${misc:Depends},
+ bash-completion,
+ python3-yaml
+Description: StarlingX - sysctem client and cli
+ system client and cli
+ .
+ This is the python3 version of the package
+
+Package: stx-app-generator-wheel
+Architecture: all
+Depends: ${python3:Depends}, ${misc:Depends}, python3-wheel
+Description: StarlingX - system client and cli
+ Starlingx Config Info
+ .
+ This is the support package for the PEP 427 wheel version of the package,
+ required for using pip inside a virtual environment.

stx-app-generator/debian/deb_folder/copyright

@@ -0,0 +1,42 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Upstream-Name: stx-app-generator
+Source: https://opendev.org/starlingx/config
+
+Files: *
+Copyright: (c) 2023 Wind River Systems, Inc
+           (c) Others (See individual files for more details)
+License: Apache-2
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ .
+    https://www.apache.org/licenses/LICENSE-2.0
+ .
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ .
+ On Debian-based systems the full text of the Apache version 2.0 license
+ can be found in `/usr/share/common-licenses/Apache-2.0'.
+
+# If you want to use GPL v2 or later for the /debian/* files use
+# the following clauses, or change it to suit. Delete these two lines
+Files: debian/*
+Copyright: 2023 Wind River Systems, Inc
+License: Apache-2
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+ .
+    https://www.apache.org/licenses/LICENSE-2.0
+ .
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+ .
+ On Debian-based systems the full text of the Apache version 2.0 license
+ can be found in `/usr/share/common-licenses/Apache-2.0'.

stx-app-generator/debian/deb_folder/rules

@@ -0,0 +1,19 @@
+#!/usr/bin/make -f
+export DH_VERBOSE = 1
+export PYBUILD_NAME=stx-app-generator
+export PBR_VERSION=1.0.0
+ROOT := $(CURDIR)/debian/tmp
+
+%:
+	dh $@ --with python3 --buildsystem=pybuild
+
+override_dh_install:
+
+	python3 setup.py install -f --install-layout=deb \
+		--root=$(CURDIR)/debian/tmp
+
+	python3 setup.py bdist_wheel \
+		--universal\
+		-d $(CURDIR)/debian/stx-app-generator-wheel/usr/share/python-wheels
+
+	dh_install

stx-app-generator/debian/deb_folder/source/format

@@ -0,0 +1 @@
+3.0 (quilt)

stx-app-generator/debian/deb_folder/source/options

@@ -0,0 +1 @@
+extend-diff-ignore = "^[^/]*[.]egg-info/"

stx-app-generator/debian/deb_folder/stx-app-generator-wheels.install

@@ -0,0 +1 @@
+usr/share/python-wheels/*

stx-app-generator/debian/deb_folder/stx-app-generator.dirs

@@ -0,0 +1 @@
+usr/bin

stx-app-generator/debian/deb_folder/stx-app-generator.install

@@ -0,0 +1,3 @@
+usr/bin/stx-app-generator
+usr/lib/python*/site-packages/app_gen_tool/*
+usr/lib/python*/dist-packages/*

stx-app-generator/debian/meta_data.yaml

@@ -0,0 +1,9 @@
+---
+debname: stx-app-generator
+debver: 1.0-1
+src_path: stx-app-generator
+revision:
+  dist: $STX_DIST
+  PKG_GITREVCOUNT: true
+#  SRC_GITREVCOUNT:
+#    SRC_BASE_SRCREV: c06abcda5761d9fa4af23671b93b5fa59136d47a

stx-app-generator/stx-app-generator/app_gen_tool/constants.py

@@ -9,9 +9,9 @@ ARMADA_CHARTGROUP_TEMPLATE = os.path.join('templates_armada', 'armada-chartgroup
 ARMADA_MANIFEST_TEMPLATE = os.path.join('templates_armada', 'armada-manifest.template')
 
 # Variables for FluxCD packaging
-FLUXCD_KUSTOMIZATION_TEMPLATE = os.path.join('templates_flux', 'kustomization.template')
-FLUXCD_BASE_TEMPLATES = os.path.join('templates_flux', 'base')
-FLUXCD_MANIFEST_TEMPLATE = os.path.join('templates_flux', 'fluxcd-manifest')
+FLUXCD_KUSTOMIZATION_TEMPLATE = os.path.join('templates_fluxcd', 'kustomization.template')
+FLUXCD_BASE_TEMPLATES = os.path.join('templates_fluxcd', 'base')
+FLUXCD_MANIFEST_TEMPLATE = os.path.join('templates_fluxcd', 'fluxcd-manifest')
 FLUXCD_COMMON_TEMPLATE = os.path.join('templates_plugins', 'common.template')
 FLUXCD_HELM_TEMPLATE = os.path.join('templates_plugins', 'helm.template')
 FLUXCD_KUSTOMIZE_TEMPLATE = os.path.join('templates_plugins', 'kustomize.template')

stx-app-generator/stx-app-generator/setup.cfg

@@ -0,0 +1,20 @@
+[metadata]
+name = stx-app-generator
+summary = StarlingX System Client and CLI
+author = WindRiver
+
+[global]
+setup-hooks =
+    pbr.hooks.setup_hook
+
+[files]
+packages =
+    stx-app-generator
+
+[pbr]
+autodoc_index_modules = False
+
+[egg_info]
+tag_build =
+tag_date = 0
+tag_svn_revision = 0

stx-app-generator/stx-app-generator/setup.py

@@ -6,7 +6,7 @@ import sysconfig
 from glob import glob
 from setuptools import setup, find_packages
 
-PACKAGE_NAME = 'starlingx-app-generator'
+PACKAGE_NAME = 'stx-app-generator'
 PACKAGE_DIRECTORY = 'app_gen_tool'
 SITE_PACKAGES_DIR = sysconfig.get_paths()["purelib"]
 
@@ -36,12 +36,14 @@ setup(
             f'{PACKAGE_NAME} = {PACKAGE_DIRECTORY}.cmd.generator:main',
         ],
     },
+    setup_requires=['pbr>=0.5'],
+    pbr=True,
     packages=find_packages(),
     include_package_data=True,
     data_files=[
         (
-            f'{SITE_PACKAGES_DIR}/{PACKAGE_DIRECTORY}/templates_flux',
-            _get_list_of_files(f'{PACKAGE_DIRECTORY}/templates_flux')
+            f'{SITE_PACKAGES_DIR}/{PACKAGE_DIRECTORY}/templates_fluxcd',
+            _get_list_of_files(f'{PACKAGE_DIRECTORY}/templates_fluxcd')
         ),
         (
             f'{SITE_PACKAGES_DIR}/{PACKAGE_DIRECTORY}/templates_plugins',

stx-app-generator/stx-app-generator/tests/unit/resources/app-test.yaml

@@ -7,7 +7,7 @@ appManifestFile-config:
   chart:
     - name: adminer
       version: 0.2.1
-      path: ./tests/unit/resources/adminer/adminer
+      path: ./stx-app-generator/stx-app-generator/tests/unit/resources/adminer/adminer
   chartGroup:
     - name:  adminer
       chart_names:

stx-app-generator/stx-app-generator/tests/unit/test_application_class.py

@@ -14,7 +14,7 @@ def _extract(tar_url, extract_path='.'):
             _extract(item.name, "./" + item.name[:item.name.rfind('/')])
 
 
-class TestCameCase:
+class TestFluxCDAppGen:
 
     def setup_class(cls):
 

tox.ini

@@ -5,7 +5,7 @@ skipsdist = True
 
 [testenv]
 install_command = pip install \
-    -c{env:UPPER_CONSTRAINTS_FILE:./requirements.txt} \
+    -c{env:UPPER_CONSTRAINTS_FILE:./stx-app-generator/stx-app-generator/requirements.txt} \
     {opts} {packages}
 setenv = VIRTUAL_ENV={envdir}
          OS_STDOUT_CAPTURE=1
@@ -151,7 +151,7 @@ usedevelop = False
 skip_install = True
 deps = -r{toxinidir}/test-requirements.txt
 commands =
-    flake8 ./app_gen_tool
+    flake8 ./stx-app-generator/stx-app-generator/app_gen_tool
 
 [testenv:venv]
 basepython = python3
@@ -161,32 +161,32 @@ commands = {posargs}
 basepython = python3
 description = Run Flake8 Linter
 commands =
-    flake8 ./app_gen_tool
+    flake8 ./stx-app-generator/stx-app-generator/app_gen_tool
 
 [testenv:pylint]
 basepython = python3
 description = Run pylint linter
 commands =
-    pylint ./app_gen_tool
+    pylint ./stx-app-generator/stx-app-generator/app_gen_tool
 
 [testenv:bandit]
 basepython = python3
 description = Run bandit check
 commands =
-    bandit -c ./bandit.yaml -r ./app_gen_tool
+    bandit -c ./bandit.yaml -r ./stx-app-generator/stx-app-generator/app_gen_tool
 
 [testenv:pytest]
 basepython = python3
 description = Run bandit check
 commands =
-    pytest tests/
+    pytest ./stx-app-generator/stx-app-generator/tests/
 
 
 [testenv:coverage-run]
 basepython = python3
 description = Run pytest with code coverage check
 commands =
-    coverage run -m pytest -s tests/
+    coverage run -m pytest -s ./stx-app-generator/stx-app-generator/tests
 
 [testenv:coverage-report]
 basepython = python3