openstack-manuals/doc/admin-guide-cloud/section_troubleshoot-cinder.xml

<?xml version="1.0" encoding="UTF-8"?>
<section xml:id="troubleshoot-cinder" xmlns="http://docbook.org/ns/docbook"
    xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink"
    version="1.0">
    <title>Troubleshoot your cinder installation</title>
    <para>This section is intended to help solve some basic and common errors that are encountered
        during setup and configuration of Cinder. The focus here is on failed creation of volumes.
        The most important thing to know is where to look in case of a failure. There are two log
        files that are especially helpful in the case of a volume creation failure. The first is the
        <systemitem class="service">cinder-api</systemitem> log, and the second is the <systemitem class="service">cinder-volume</systemitem> log.</para>
    <para>The <systemitem class="service">cinder-api</systemitem> log is useful in determining if you have
        endpoint or connectivity issues. If you send a request to
        create a volume and it fails, it's a good idea to look here
        first and see if the request even made it to the Cinder
        service. If the request seems to be logged, and there are no
        errors or trace-backs then you can move to the <systemitem class="service">cinder-volume</systemitem>
        log and look for errors or trace-backs there.</para>
    <para>There are some common issues to look out for. The following describes
	    some common issues hit during configuration and some suggested solutions.</para>
    <para><emphasis role="bold"><emphasis role="underline">Create commands are in <systemitem class="service">cinder-api</systemitem> log
                with no error</emphasis></emphasis></para>
    <para>
        <itemizedlist>
            <listitem>
                <para><emphasis role="bold"><literal>state_path</literal> and <literal>volumes_dir</literal> settings</emphasis></para>
                <para>Cinder uses <command>tgtd</command>
                    as the default iscsi helper and implements persistent targets.
                    This means that in the case of a tgt restart or
                    even a node reboot your existing volumes on that
                    node will be restored automatically with their
                    original IQN.</para>
                <para>In order to make this possible the iSCSI target information needs to be stored
                    in a file on creation that can be queried in case of restart of the tgt daemon.
                    By default, Cinder uses a <literal>state_path</literal> variable, which if installing via Yum or
                    APT should be set to <filename>/var/lib/cinder/</filename>. The next part is the <literal>volumes_dir</literal>
                    variable, by default this just simply appends a "<literal>volumes</literal>" directory to the
                    <literal>state_path</literal>. The result is a file-tree <filename>/var/lib/cinder/volumes/</filename>.</para>
                <para>While this should all be handled for you by you installer, it can go wrong. If
                    you're having trouble creating volumes and this directory does not exist you
                    should see an error message in the <systemitem class="service">cinder-volume</systemitem> log indicating that the
                    <literal>volumes_dir</literal> doesn't exist, and it should give you information to specify what
                    path exactly it was looking for.</para>
            </listitem>
            <listitem>
                <para><emphasis role="bold">persistent tgt include file</emphasis></para>
                <para>Along with the <literal>volumes_dir</literal> mentioned above, the iSCSI target driver also needs
                    to be configured to look in the correct place for the persist files. This is a
                    simple entry in <filename>/etc/tgt/conf.d</filename>, and you should have created this when you went
                    through the install guide. If you haven't or you're running into issues, verify
                    that you have a file <filename>/etc/tgt/conf.d/cinder.conf</filename>.</para>
                <para>If the file is not there, you can create it easily by doing the
                    following:<programlisting>
sudo sh -c "echo 'include /var/lib/cinder/volumes/*' >> /etc/tgt/conf.d/cinder.conf"
                    </programlisting></para>
            </listitem>
        </itemizedlist>
    </para>
    <para><emphasis role="bold"><emphasis role="underline">No sign of attach call in the <systemitem class="service">cinder-api</systemitem>
                log</emphasis></emphasis></para>
    <para>This is most likely going to be a minor adjustment to your
            <filename>nova.conf</filename> file. Make sure that your
            <filename>nova.conf</filename> has the following
        entry:<programlisting>
volume_api_class=nova.volume.cinder.API
        </programlisting></para>
    <para>And make certain that you EXPLICITLY set enabled_apis as the default will include
        osapi_volume:<programlisting>
enabled_apis=ec2,osapi_compute,metadata
    </programlisting>
    </para>
    <para><emphasis role="bold">Failed to create iscsi target error in the <filename>cinder-volume.log</filename></emphasis></para>

        <programlisting language="bash">2013-03-12 01:35:43 1248 TRACE cinder.openstack.common.rpc.amqp ISCSITargetCreateFailed: Failed to create iscsi target for volume volume-137641b2-af72-4a2f-b243-65fdccd38780.
</programlisting>
        <para>You may see this error in <filename>cinder-volume.log</filename> after trying to create a volume that is 1 GB. To fix this issue:
        </para>
    <para>Change content of the <filename>/etc/tgt/targets.conf</filename> from "include /etc/tgt/conf.d/*.conf" to:
    include /etc/tgt/conf.d/cinder_tgt.conf:</para>
        <programlisting language="bash">
            include /etc/tgt/conf.d/cinder_tgt.conf
            include /etc/tgt/conf.d/cinder.conf
            default-driver iscsi</programlisting>

        <para>Then restart tgt and <literal>cinder-*</literal> services so they pick up the new configuration.</para>
</section>