Troubleshooting ComputeCommon problems for Compute typically involve misconfigured networking or credentials that are not sourced properly in the environment. Also, most flat networking configurations do not enable ping or ssh from a compute node to the instances running on that node. Another common problem is trying to run 32-bit images on a 64-bit compute node. This section offers more information about how to troubleshoot Compute.Log files for ComputeCompute stores a log file for each service in
/var/log/nova. For example,
nova-compute.log is the log for the
nova-compute
service. You can set the following options to format log
strings for the nova.log module in
nova.conf:
logging_context_format_stringlogging_default_format_string
If the log level is set to debug, you can
also specify logging_debug_format_suffix
to append extra formatting. For information about what
variables are available for the formatter see:
http://docs.python.org/library/logging.html#formatter.
You have two options for logging for OpenStack Compute based on configuration
settings. In nova.conf, include the
logfile option to enable logging. Alternatively
you can set use_syslog=1, and then the nova
daemon logs to syslog.Common errors and fixes for ComputeThe ask.openstack.org site offers a place to ask and
answer questions, and you can also mark questions as
frequently asked questions. This section describes some
errors people have posted previously. We
are constantly fixing bugs, so online resources are a
great way to get the most up-to-date errors and
fixes.Credential errors, 401, 403 forbidden errorsA 403 forbidden error is caused by missing credentials.
Through current installation methods, there are basically
two ways to get the novarc file. The manual method
requires getting it from within a project zipfile, and the
scripted method just generates novarc out of the project
zip file and sources it for you. If you use the manual
method through a zip file, before sourcing novarc
be sure to save any credentials that were created previously, as they
can be overridden.
When you run nova-api the
first time, it generates the certificate authority information,
including openssl.cnf. If the CA components are
started prior to this, you may not be able to create your zip file.
Restart the services, then once your CA information is available,
you should be able to create your zip file.You may also need to check your http proxy settings to see if
they are causing problems with the novarc
creation.Instance errorsSometimes a particular instance shows "pending" or you
cannot SSH to it. Sometimes the image itself is the
problem. For example, when using flat manager networking,
you do not have a dhcp server, and certain images
don't support interface injection so you cannot connect
to them. The fix for this type of problem is to use an
image that does support this method, such as Ubuntu,
which should obtain an IP address correctly
with FlatManager network settings. To troubleshoot other
possible problems with an instance, such as one that stays
in a spawning state, first check the directory for the particular
instance under /var/lib/nova/instances
on the nova-compute
host and make sure it has the following files:libvirt.xmldiskdisk-rawkernelramdiskconsole.log (Once the instance actually starts you should
see a console.log.)Check the file sizes to see if they are reasonable. If
any are missing/zero/very small then nova-compute has
somehow not completed download of the images from
the Image service.Also check nova-compute.log for exceptions.
Sometimes they don't show up in the console output.Next, check the log file for the instance in the directory
/var/log/libvirt/qemu
to see if it exists and has any useful error messages
in it.Finally, from the directory for the instance under
/var/lib/nova/instances, try
#virsh create libvirt.xml and see if you
get an error when running this.Manually reset the state of an instanceIf an instance gets stuck in an intermediate state (e.g., "deleting"), you can
manually reset the state of an instance using the nova
reset-state command. This will reset it to an error state, which you
can then delete. For
example:$nova reset-state c6bbbf26-b40a-47e7-8d5c-eb17bf65c485$nova delete c6bbbf26-b40a-47e7-8d5c-eb17bf65c485You can also use the --active to
force the instance back into an active state instead of an
error state, for
example:$nova reset-state --active c6bbbf26-b40a-47e7-8d5c-eb17bf65c485Problems with injectionIf you are diagnosing problems with instances not booting,
or booting slowly, consider investigating file injection as a
cause. Setting libvirt_injection_partition
to -2 disables injection in libvirt. This can be required if you want to make user
specified files available from the metadata server (and config drive is not enabled),
for performance reasons, and also to avoid boot failure if injection itself fails.