openstack/devstack-plugin-ceph
Code Issues Proposed changes

Drop support for package based installation of Ceph

Browse Source 
This mode of deployment isn't supported by the Ceph
community, and was always a chimera that we were
feeding/maintaining.

Ceph's tool of choice to bootstrap and install a ceph
cluster is by using the Ceph Orchestrator (via the
cephadm tool).

We're also cleaning up the old/unused and poorly tested
"CONTAINERIZED_CEPH". When using ceph orchestrator,
ceph daemons are run within podman containers on the
devstack host.

Change-Id: I5f75cb829383d7acd536e24c70cc4418d93c13bc
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
This commit is contained in:
Goutham Pacha Ravi 2024-05-09 17:46:14 -07:00
parent c96d07cada
commit 784716d949
8 changed files with 33 additions and 1323 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.zuul.yaml
View File

@ -27,7 +27,6 @@
TEMPEST_RUN_VALIDATION: True
MYSQL_REDUCE_MEMORY: True
DISABLE_CEPHADM_POST_DEPLOY: True
CEPHADM_DEPLOY: true
devstack_plugins:
devstack-plugin-ceph: https://opendev.org/openstack/devstack-plugin-ceph
devstack_services:
@ -112,7 +111,6 @@
ENABLE_VOLUME_MULTIATTACH: true
TEMPEST_RUN_VALIDATION: true
USE_BLOCK_MIGRATION_FOR_LIVE_MIGRATION: false
CEPHADM_DEPLOY: True
DISABLE_CEPHADM_POST_DEPLOY: True
MYSQL_REDUCE_MEMORY: True
REMOTE_CEPH: False
@ -132,7 +130,6 @@
subnode:
devstack_localrc:
REMOTE_CEPH: True
CEPHADM_DEPLOY: True
CINDER_CEPH_UUID: d531d2d4-3937-429c-b0c2-658fe41e82aa
- project-template:

39
README.rst
View File

@ -12,23 +12,18 @@ services
As part of ``stack.sh``:
- Installs Ceph (client and server) packages
- Creates a Ceph cluster for use with openstack services
- Creates a Ceph cluster for use with openstack services using Ceph orchestrator
- Configures Ceph as the storage backend for Cinder, Cinder Backup,
Nova, Manila (not by default), and Glance services
Nova, Manila, and Glance services
- (Optionally) Sets up & configures Rados gateway (aka rgw or radosgw)
as a Swift endpoint with Keystone integration
- Set ``ENABLE_CEPH_RGW=True`` in your ``localrc``
as a Swift endpoint with Keystone integration. Set ``ENABLE_CEPH_RGW=True``
in your ``localrc``
- Supports Ceph cluster running local or remote to openstack services
As part of ``unstack.sh`` \| ``clean.sh``:
- Tears down the Ceph cluster and its related services
This plugin also gets used to configure Ceph as the storage backend for
the upstream Ceph CI job named
``gate-tempest-dsvm-full-devstack-plugin-ceph``
Usage
-----
@ -40,7 +35,7 @@ Usage
Run ``stack.sh`` in your devstack tree and boom! You're good to go.
- Ceph is setup as the default storage backend for Cinder, Cinder
Backup, Glance and Nova services. You have the ability to control
Backup, Glance, Manila and Nova services. You have the ability to control
each of the enabled services with the following configuration in your
``local.conf``:
@ -50,18 +45,12 @@ Run ``stack.sh`` in your devstack tree and boom! You're good to go.
ENABLE_CEPH_GLANCE=True # store images in ceph
ENABLE_CEPH_C_BAK=True # backup volumes to ceph
ENABLE_CEPH_NOVA=True # allow nova to use ceph resources
ENABLE_CEPH_MANILA=True # allow manila to use CephFS as backend (Native CephFS or CephFS via NFS)
Change any of the above lines to ``False`` to disable that feature
specifically.
- Ceph can be enabled as the storage backend for Manila with the
following setting in your ``local.conf``:
::
ENABLE_CEPH_MANILA=True
CephFS Native driver that supports native Ceph protocol is used by
Manila's CephFS Native driver that supports native Ceph protocol is enabled by
default. To use CephFS NFS-Ganesha driver that supports NFS protocol add
the setting:
@ -69,6 +58,13 @@ the setting:
MANILA_CEPH_DRIVER=cephfsnfs
If you'd like to use a standalone NFS Ganesha service in place of ceph orchestrator
deployed ``ceph-nfs`` service, set:
::
CEPHADM_DEPLOY_NFS=False
Make sure that the manila plugin is enabled before devstack-plugin-ceph
in the ``local.conf`` file.
@ -81,13 +77,6 @@ Known Issues / Limitations
- Tempest test failures when using RGW as swift endpoint
- Tempest fails due to verify-tempest-config erroring out, when using
RGW as swift endpoint
- Manila with CephFS - for Ubuntu, support only for Trusty Tahr (14.04
LTS) and beyond
TODOs
-----
- Fix Rados Gateway with Keystone for Swift on Fedora
Bugs
----

1103
devstack/lib/ceph
View File

File diff suppressed because it is too large Load Diff

2
devstack/lib/cephadm
View File

@ -383,7 +383,7 @@ function _install_and_configure_clustered_nfs {
}
function _install_and_configure_standalone_nfs {
source $CEPH_PLUGIN_DIR/lib/common
source $CEPH_PLUGIN_DIR/lib/nfs-ganesha
install_nfs_ganesha
configure_nfs_ganesha
start_nfs_ganesha

0
devstack/lib/common → devstack/lib/nfs-ganesha
View File

17
devstack/override-defaults
View File

@ -8,20 +8,9 @@ ENABLE_CEPH_NOVA=$(trueorfalse True ENABLE_CEPH_NOVA)
# Do not enable RGW by default as RGW is not tested in upstream CI.
ENABLE_CEPH_RGW=$(trueorfalse False ENABLE_CEPH_RGW)
if [[ $ENABLE_CEPH_CINDER == "True" ]]; then
if [[ $ENABLE_CEPH_ISCSI == "True" ]]; then
CINDER_DRIVER=${CINDER_DRIVER:-ceph_iscsi}
CINDER_ENABLED_BACKENDS=${CINDER_ENABLED_BACKENDS:-ceph_iscsi}
CINDER_ISCSI_HELPER=${CINDER_ISCSI_HELPER:-lioadm}
else
CINDER_DRIVER=${CINDER_DRIVER:-ceph}
CINDER_ENABLED_BACKENDS=${CINDER_ENABLED_BACKENDS:-ceph}
fi
fi
CINDER_DRIVER=${CINDER_DRIVER:-ceph}
CINDER_ENABLED_BACKENDS=${CINDER_ENABLED_BACKENDS:-ceph}
REMOTE_CEPH=$(trueorfalse False REMOTE_CEPH)
if [[ $ENABLE_CEPH_MANILA == "True" ]]; then
MANILA_CEPH_DRIVER=${MANILA_CEPH_DRIVER:-cephfsnative}
fi
CEPHADM_DEPLOY=$(trueorfalse False CEPHADM_DEPLOY)
REMOTE_CEPH=$(trueorfalse False REMOTE_CEPH)

172
devstack/plugin.sh
View File

@ -1,184 +1,38 @@
# ceph.sh - DevStack extras script to install Ceph
# cephadm.sh - DevStack extras script to install Ceph
if [[ "$1" == "source" ]]; then
# Initial source
if [[ "$CEPHADM_DEPLOY" = "True" ]]; then
source $TOP_DIR/lib/cephadm
else
source $TOP_DIR/lib/ceph
fi
source $TOP_DIR/lib/cephadm
elif [[ "$1" == "stack" && "$2" == "pre-install" ]]; then
if [[ "$ENABLE_CEPH_RGW" = "True" ]] && (is_service_enabled swift); then
die $LINENO \
"You cannot activate both Swift and Ceph Rados Gateway, \
please disable Swift or set ENABLE_CEPH_RGW=False"
fi
if [[ "$CEPHADM_DEPLOY" = "True" ]]; then
# Set up system services
echo_summary "[cephadm] Configuring system services ceph"
pre_install_ceph
else
echo_summary "Installing Ceph"
check_os_support_ceph
if [ "$REMOTE_CEPH" = "False" ]; then
if [ "$CEPH_CONTAINERIZED" = "True" ]; then
echo_summary "Configuring and initializing Ceph"
deploy_containerized_ceph
else
install_ceph
echo_summary "Configuring Ceph"
configure_ceph
# NOTE (leseb): we do everything here
# because we need to have Ceph started before the main
# OpenStack components.
# Ceph OSD must start here otherwise we can't upload any images.
echo_summary "Initializing Ceph"
start_ceph
fi
else
install_ceph_remote
fi
fi
echo_summary "[cephadm] Configuring system services ceph"
pre_install_ceph
elif [[ "$1" == "stack" && "$2" == "install" ]]; then
if [[ "$CEPHADM_DEPLOY" = "True" && "$REMOTE_CEPH" = "False" ]]; then
if [[ "$REMOTE_CEPH" = "False" ]]; then
# Perform installation of service source
echo_summary "[cephadm] Installing ceph"
install_ceph
set_min_client_version
elif [[ "$CEPHADM_DEPLOY" = "True" && "$REMOTE_CEPH" = "True" ]]; then
else
echo "[CEPHADM] Remote Ceph: Skipping install"
get_cephadm
else
# FIXME(melwitt): This is a hack to get around a namespacing issue with
# Paste and PasteDeploy. For stable/queens, we use the Pike UCA packages
# and the Ceph packages in the Pike UCA are pulling in python-paste and
# python-pastedeploy packages. The python-pastedeploy package satisfies the
# upper-constraints but python-paste does not, so devstack pip installs a
# newer version of it, while python-pastedeploy remains. The mismatch
# between the install path of paste and paste.deploy causes Keystone to
# fail to start, with "ImportError: cannot import name deploy."
# TODO(frickler): This is needed for all branches currently.
pip_install -U --force PasteDeploy
install_package python-is-python3
fi
elif [[ "$1" == "stack" && "$2" == "post-config" ]]; then
if [[ "$CEPHADM_DEPLOY" = "True" ]]; then
# Configure after the other layer 1 and 2 services have been configured
echo_summary "[cephadm] Configuring additional Ceph services"
configure_ceph
else
if is_ceph_enabled_for_service glance; then
echo_summary "Configuring Glance for Ceph"
configure_ceph_glance
fi
if is_ceph_enabled_for_service nova; then
echo_summary "Configuring Nova for Ceph"
configure_ceph_nova
fi
if is_ceph_enabled_for_service cinder; then
echo_summary "Configuring Cinder for Ceph"
configure_ceph_cinder
fi
if is_ceph_enabled_for_service nova || \
is_ceph_enabled_for_service cinder; then
# NOTE (leseb): the part below is a requirement
# to attach Ceph block devices
echo_summary "Configuring libvirt secret"
import_libvirt_secret_ceph
fi
if is_ceph_enabled_for_service manila; then
echo_summary "Configuring Manila for Ceph"
configure_ceph_manila
fi
if [ "$REMOTE_CEPH" = "False" ]; then
if is_ceph_enabled_for_service glance; then
echo_summary "Configuring Glance for Ceph"
configure_ceph_embedded_glance
fi
if is_ceph_enabled_for_service nova; then
echo_summary "Configuring Nova for Ceph"
configure_ceph_embedded_nova
fi
if is_ceph_enabled_for_service cinder; then
echo_summary "Configuring Cinder for Ceph"
configure_ceph_embedded_cinder
fi
if is_ceph_enabled_for_service manila; then
echo_summary "Configuring Manila for Ceph"
configure_ceph_embedded_manila
fi
if [ "$ENABLE_CEPH_RGW" = "True" ]; then
echo_summary "Configuring Rados Gateway with Keystone for Swift"
configure_ceph_embedded_rgw
if [ "$CEPH_CONTAINERIZED" = "False" ]; then
start_ceph_embedded_rgw
else
_configure_ceph_rgw_container
fi
fi
fi
fi
echo_summary "[cephadm] Configuring additional Ceph services"
configure_ceph
if [[ "$MDS_LOGS" == "True" ]]; then
enable_verbose_mds_logging
fi
elif [[ "$1" == "stack" && "$2" == "test-config" ]]; then
if is_service_enabled tempest; then
iniset $TEMPEST_CONFIG compute-feature-enabled swap_volume False
# Only enable shelve testing for branches which have the fix for
# nova bug 1653953.
if [[ "$TARGET_BRANCH" =~ stable/(ocata|pike) ]]; then
iniset $TEMPEST_CONFIG compute-feature-enabled shelve False
else
iniset $TEMPEST_CONFIG compute-feature-enabled shelve True
fi
# Attached volume extend support for rbd was introduced in Stein by
# I5698e451861828a8b1240d046d1610d8d37ca5a2
if [[ "$TARGET_BRANCH" =~ stable/(ocata|pike|queens|rocky) ]]; then
iniset $TEMPEST_CONFIG volume-feature-enabled extend_attached_volume False
else
iniset $TEMPEST_CONFIG volume-feature-enabled extend_attached_volume True
fi
# Volume revert to snapshot support for rbd was introduced in Ussuri by
# If8a5eb3a03e18f9043ff29f7648234c9b46376a0
if [[ "$TARGET_BRANCH" =~ stable/(ocata|pike|queens|rocky|stein|train) ]]; then
iniset $TEMPEST_CONFIG volume-feature-enabled volume_revert False
else
iniset $TEMPEST_CONFIG volume-feature-enabled volume_revert True
fi
fi
fi
if [[ "$1" == "unstack" ]]; then
if [[ "$CEPHADM_DEPLOY" = "True" ]]; then
cleanup_ceph
else
if [ "$CEPH_CONTAINERIZED" = "False" ]; then
if [ "$REMOTE_CEPH" = "True" ]; then
cleanup_ceph_remote
else
stop_ceph
cleanup_ceph_embedded
fi
else
cleanup_containerized_ceph
fi
cleanup_ceph_general
fi
fi
if [[ "$1" == "clean" ]]; then
if [[ "$CEPHADM_DEPLOY" = "True" ]]; then
cleanup_ceph
else
if [ "$REMOTE_CEPH" = "True" ]; then
cleanup_ceph_remote
else
cleanup_ceph_embedded
fi
cleanup_ceph_general
iniset $TEMPEST_CONFIG compute-feature-enabled shelve True
iniset $TEMPEST_CONFIG volume-feature-enabled extend_attached_volume True
iniset $TEMPEST_CONFIG volume-feature-enabled volume_revert True
fi
elif [[ "$1" == "unstack" || "$1" == "clean" ]]; then
cleanup_ceph
fi

20
devstack/settings
View File

@ -5,17 +5,7 @@
CEPH_PLUGIN_DIR=$(readlink -f $(dirname ${BASH_SOURCE[0]}))
# Add ceph plugin specific settings
# NOTE: Currently these are redundant since ceph
# plugin job defn defines them already, but
# once DEVSTACK_GATE_CEPH is removed, these
# won't be redundant, so its ok to have these
# anyways.
if [ "$ENABLE_CEPH_ISCSI" = "True" ]; then
TEMPEST_STORAGE_PROTOCOL=iSCSI
else
TEMPEST_STORAGE_PROTOCOL=ceph
fi
TEMPEST_STORAGE_PROTOCOL=ceph
# VOLUME_BACKING_FILE_SIZE should be sourced from devstack/stackrc but define
# a default here if not set already.
VOLUME_BACKING_FILE_SIZE=${VOLUME_BACKING_FILE_SIZE:-8GB}
@ -23,13 +13,7 @@ CEPH_LOOPBACK_DISK_SIZE=${CEPH_LOOPBACK_DISK_SIZE:-$VOLUME_BACKING_FILE_SIZE}
# Disable manage/unmanage snapshot tests on Tempest
TEMPEST_VOLUME_MANAGE_SNAPSHOT=False
# Source plugin's lib/cephadm or lib/ceph
# depending on chosen deployment method
if [[ "$CEPHADM_DEPLOY" = "True" ]]; then
source $CEPH_PLUGIN_DIR/lib/cephadm
else
source $CEPH_PLUGIN_DIR/lib/ceph
fi
source $CEPH_PLUGIN_DIR/lib/cephadm
# Set Manila related global variables used by Manila's DevStack plugin.
if (is_ceph_enabled_for_service manila); then