openstack-manuals/doc/common/ch_support-compute.xml

<?xml version="1.0" encoding="UTF-8"?>
<chapter xmlns="http://docbook.org/ns/docbook"
    xmlns:xi="http://www.w3.org/2001/XInclude"
    xmlns:xlink="http://www.w3.org/1999/xlink" version="5.0" xml:id="troubleshooting-openstack-compute">
<title>Troubleshooting OpenStack Compute</title>
        <para>Common problems for Compute typically involve misconfigured networking or credentials that are not sourced properly in the environment. Also, most flat networking configurations do not enable ping or ssh from a compute node to the instances running on that node. Another common problem is trying to run 32-bit images on a 64-bit compute node. This section offers more information about how to troubleshoot Compute.</para>
        <section xml:id="log-files-for-openstack-compute"><title>Log files for OpenStack Compute</title><para>Log files are stored in /var/log/nova and there is a log file for each
            service, for example nova-compute.log. You can format the log
            strings using options for the nova.log module. The options used to
            set format strings are: logging_context_format_string and
            logging_default_format_string. If the log level is set to debug, you
            can also specify logging_debug_format_suffix to append extra
            formatting. For information about what variables are available for
            the formatter see:
            http://docs.python.org/library/logging.html#formatter.</para>
        <para>You have two options for logging for OpenStack Compute based on configuration
                settings. In nova.conf, include the logfile option to enable logging. Alternatively
                you can set use_syslog=1, and then the nova daemon logs to syslog.</para></section>
    <section xml:id="common-errors-and-fixes-for-openstack-compute">
        <title>Common Errors and Fixes for OpenStack Compute</title>
        <para>The ask.openstack.org site offers a place to ask and
            answer questions, and you can also mark questions as
            frequently asked questions. This section describes some
            errors people have posted previously. We
            are constantly fixing bugs, so online resources are a
            great way to get the most up-to-date errors and
            fixes.</para>
        <para>Credential errors, 401, 403 forbidden errors</para>
        <para>A 403 forbidden error is caused by missing credentials.
            Through current installation methods, there are basically
            two ways to get the novarc file. The manual method
            requires getting it from within a project zipfile, and the
            scripted method just generates novarc out of the project
            zip file and sources it for you. If you do the manual
            method through a zip file, then the following novarc
            alone, you end up losing the creds that are tied to the
            user you created with nova-manage in the steps
            before.</para>
        <para>When you run <systemitem class="service">nova-api</systemitem> the first time, it generates the
            certificate authority information, including openssl.cnf.
            If it gets started out of order, you may not be able to
            create your zip file. Once your CA information is
            available, you should be able to go back to nova-manage to
            create your zipfile.</para>
        <para>You may also need to check your proxy settings to see if
            they are causing problems with the novarc creation.</para>
        <para>Instance errors</para>
        <para>Sometimes a particular instance shows "pending" or you
            cannot SSH to it. Sometimes the image itself is the
            problem. For example, when using flat manager networking,
            you do not have a dhcp server, and an ami-tiny image
            doesn't support interface injection so you cannot connect
            to it. The fix for this type of problem is to use an
            Ubuntu image, which should obtain an IP address correctly
            with FlatManager network settings. To troubleshoot other
            possible problems with an instance, such as one that stays
            in a spawning state, first check your instances directory
            for i-ze0bnh1q dir to make sure it has the following
            files:</para>
        <itemizedlist>
            <listitem>
                <para>libvirt.xml</para>
            </listitem>
            <listitem>
                <para>disk</para>
            </listitem>
            <listitem>
                <para>disk-raw</para>
            </listitem>
            <listitem>
                <para>kernel</para>
            </listitem>
            <listitem>
                <para>ramdisk</para>
            </listitem>
            <listitem>
                <para>console.log (Once the instance actually starts
                    you should see a console.log.)</para>
            </listitem>
        </itemizedlist>
        <para>Check the file sizes to see if they are reasonable. If
            any are missing/zero/very small then <systemitem class="service">nova-compute</systemitem> has
            somehow not completed download of the images from
            objectstore.</para>
        <para>Also check nova-compute.log for exceptions. Sometimes
            they don't show up in the console output.</para>
        <para>Next, check the /var/log/libvirt/qemu/i-ze0bnh1q.log
            file to see if it exists and has any useful error messages
            in it.</para>

        <para>Finally, from the instances/i-ze0bnh1q directory, try
                <code>virsh create libvirt.xml</code> and see if you
            get an error there.</para>
    </section>
        <section xml:id="reset-state">
            <title>Manually reset the state of an instance</title>
            <para>If an instance gets stuck in an intermediate state (e.g., "deleting"), you can
                manually reset the state of an instance using the <command>nova
                    reset-state</command> command. This will reset it to an error state, which you
                can then delete. For
                example:<screen><prompt>$</prompt> <userinput>nova reset-state c6bbbf26-b40a-47e7-8d5c-eb17bf65c485</userinput>
<prompt>$</prompt> <userinput>nova delete c6bbbf26-b40a-47e7-8d5c-eb17bf65c485</userinput></screen></para>
            <para>You can also use the <literal>--active</literal> to force the instance back into
                an active state instead of an error state, for example:<screen><prompt>$</prompt> <userinput>nova reset-state --active c6bbbf26-b40a-47e7-8d5c-eb17bf65c485</userinput></screen>
            </para>
        </section>
       <section xml:id="problems-with-injection">
       <title>Problems with Injection</title>
       <para>If you are diagnosing problems with instances not booting,
       or booting slowly, consider investigating file injection as a
       cause. Setting <literal>libvirt_injection_partition</literal>
       to -2 disables injection in libvirt. This can be required if you want to make user
       specified files available from the metadata server (and config drive is not enabled),
       for performance reasons, and also to avoid boot failure if injection itself fails.</para>
       </section>
 </chapter>