Troubleshoot the Block Storage
configurationThis section helps you solve some basic and common errors
that you might encounter during setup and configuration of the
Block Storage Service. The focus here is on failed
creation of volumes. The most important thing to know is where
to look in case of a failure.Two log files are especially helpful for solving volume
creation failures, the cinder-api log and the cinder-volume log. The
cinder-api log is
useful for determining if you have endpoint or connectivity
issues. If you send a request to create a volume and it fails,
review the cinder-api log to determine whether the request
made it to the Block Storage service. If the request is logged
and you see no errors or trace-backs, check the
cinder-volume log
for errors or trace-backs.Create commands are listed in the cinder-api log.These entries in the
cinder.openstack.common.log file can
be used to assist in troubleshooting your block storage
configuration.
# Print debugging output (set logging level to DEBUG instead
# of default WARNING level). (boolean value)
#debug=false
# Print more verbose output (set logging level to INFO instead
# of default WARNING level). (boolean value)
#verbose=false
# Log output to standard error (boolean value)
#use_stderr=true
# Default file mode used when creating log files (string
# value)
#logfile_mode=0644
# format string to use for log messages with context (string
# value)
#logging_context_format_string=%(asctime)s.%(msecs)03d %(levelname)s %(name)s [%(request_id)s %(user)s %(tenant)s] %(instance)s%(message)s
# format string to use for log mes #logging_default_format_string=%(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s
# data to append to log format when level is DEBUG (string
# value)
#logging_debug_format_suffix=%(funcName)s %(pathname)s:%(lineno)d
# prefix each line of exception output with this format
# (string value)
#logging_exception_prefix=%(asctime)s.%(msecs)03d %(process)d TRACE %(name)s %(instance)s
# list of logger=LEVEL pairs (list value)
#default_log_levels=amqplib=WARN,sqlalchemy=WARN,boto=WARN,suds=INFO,keystone=INFO,eventlet.wsgi.server=WARNsages without context
# (string value)
# If an instance is passed with the log message, format it
# like this (string value)
#instance_format="[instance: %(uuid)s]"
# If an instance UUID is passed with the log message, format
# it like this (string value)
# A logging.Formatter log message format string which may use
# any of the available logging.LogRecord attributes. Default:
# %(default)s (string value)
#log_format=%(asctime)s %(levelname)8s [%(name)s] %(message)s
# Format string for %%(asctime)s in log records. Default:
# %(default)s (string value)
#log_date_format=%Y-%m-%d %H:%M:%S
# (Optional) Name of log file to output to. If not set,
# logging will go to stdout. (string value)
#log_file=<None>
# (Optional) The directory to keep log files in (will be
# prepended to --log-file) (string value)
#log_dir=<None>
#instance_uuid_format="[instance: %(uuid)s]"
# If this option is specified, the logging configuration file
# specified is used and overrides any other logging options
# specified. Please see the Python logging module
# documentation for details on logging configuration files.
# (string value) # Use syslog for logging. (boolean value)
#use_syslog=false
# syslog facility to receive log lines (string value)
#syslog_log_facility=LOG_USER
#log_config=<None>These common issues might occur during configuration. To
correct, use these suggested solutions.Issues with state_path and
volumes_dir settings.The OpenStack Block Storage uses tgtd
as the default iscsi helper and implements persistent targets.
This means that in the case of a tgt restart or even a
node reboot your existing volumes on that node will be
restored automatically with their original IQN.In order to make this possible the iSCSI target
information needs to be stored in a file on creation
that can be queried in case of restart of the tgt
daemon. By default, Block Storage uses a
state_path variable, which if
installing with Yum or APT should be set to
/var/lib/cinder/. The next
part is the volumes_dir variable,
by default this just simply appends a
"volumes" directory to the
state_path. The result is a
file-tree
/var/lib/cinder/volumes/.While this should all be handled by the installer,
it can go wrong. If you have trouble creating volumes
and this directory does not exist you should see an
error message in the cinder-volume log indicating that
the volumes_dir does not exist, and
it should provide information about which path it was
looking for.The persistent tgt include file.Along with the option,
the iSCSI target driver also needs to be configured to
look in the correct place for the persist files. This
is a simple entry in the
/etc/tgt/conf.d file that you
should have set when you installed OpenStack. If
issues occur, verify that you have a
/etc/tgt/conf.d/cinder.conf
file.If the file is not present, create it with this
command:#echo 'include /var/lib/cinder/volumes/ *' >> /etc/tgt/conf.d/cinder.confNo sign of attach call in the cinder-api
log.This is most likely going to be a minor adjustment
to your nova.conf file. Make sure
that your nova.conf has this
entry:volume_api_class=nova.volume.cinder.APIFailed to create iscsi target error in the
cinder-volume.log
file.2013-03-12 01:35:43 1248 TRACE cinder.openstack.common.rpc.amqp ISCSITargetCreateFailed: Failed to create iscsi target for volume volume-137641b2-af72-4a2f-b243-65fdccd38780.You might see this error in
cinder-volume.log after
trying to create a volume that is 1 GB. To fix this
issue:Change content of the
/etc/tgt/targets.conf from
include /etc/tgt/conf.d/*.conf
to include
/etc/tgt/conf.d/cinder_tgt.conf, as
follows:include /etc/tgt/conf.d/cinder_tgt.conf
include /etc/tgt/conf.d/cinder.conf
default-driver iscsiRestart tgt
and cinder-*
services so they pick up the new configuration.