openstack-manuals/doc/config-reference/compute/section_compute-scheduler.xml

519 lines
27 KiB
XML

<section xml:id="section_compute-scheduler"
xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xi="http://www.w3.org/2001/XInclude"
xmlns:ns5="http://www.w3.org/1999/xhtml"
xmlns:ns4="http://www.w3.org/2000/svg"
xmlns:ns3="http://www.w3.org/1998/Math/MathML"
xmlns:ns="http://docbook.org/ns/docbook" version="5.0">
<?dbhtml stop-chunking?>
<title>Scheduling</title>
<para>Compute uses the <systemitem class="service"
>nova-scheduler</systemitem> service to determine how to
dispatch compute and volume requests. For example, the
<systemitem class="service">nova-scheduler</systemitem>
service determines which host a VM should launch on. The term
<firstterm>host</firstterm> in the context of filters
means a physical node that has a <systemitem class="service"
>nova-compute</systemitem> service running on it. You can
configure the scheduler through a variety of options.</para>
<para>Compute is configured with the following default scheduler
options:</para>
<programlisting language="ini">scheduler_driver=nova.scheduler.multi.MultiScheduler
compute_scheduler_driver=nova.scheduler.filter_scheduler.FilterScheduler
scheduler_available_filters=nova.scheduler.filters.all_filters
scheduler_default_filters=RetryFilter,AvailabilityZoneFilter,RamFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter</programlisting>
<para>By default, the scheduler_driver is configured as a filter
scheduler, as described in the next section. In the default
configuration, this scheduler considers hosts that meet all
the following criteria:</para>
<itemizedlist>
<listitem>
<para>Have not been attempted for scheduling purposes
(<literal>RetryFilter</literal>).</para>
</listitem>
<listitem>
<para>Are in the requested availability zone
(<literal>AvailabilityZoneFilter</literal>).</para>
</listitem>
<listitem>
<para>Have sufficient RAM available
(<literal>RamFilter</literal>).</para>
</listitem>
<listitem>
<para>Are capable of servicing the request
(<literal>ComputeFilter</literal>).</para>
</listitem>
<listitem>
<para>Satisfy the extra specs associated with the instance
type
(<literal>ComputeCapabilitiesFilter</literal>).</para>
</listitem>
<listitem>
<para>Satisfy any architecture, hypervisor type, or
virtual machine mode properties specified on the
instance's image properties.
(<literal>ImagePropertiesFilter</literal>).</para>
</listitem>
</itemizedlist>
<para>For information on the volume scheduler, refer the Block
Storage section of <link
xlink:href="http://docs.openstack.org/admin-guide-cloud/content/managing-volumes.html">
<citetitle>OpenStack Cloud Administrator
Guide</citetitle></link> for information.</para>
<section xml:id="filter-scheduler">
<title>Filter scheduler</title>
<para>The Filter Scheduler
(<literal>nova.scheduler.filter_scheduler.FilterScheduler</literal>)
is the default scheduler for scheduling virtual machine
instances. It supports filtering and weighting to make
informed decisions on where a new instance should be
created.</para>
</section>
<section xml:id="scheduler-filters">
<?dbhtml stop-chunking?>
<title>Filters</title>
<para>When the Filter Scheduler receives a request for a
resource, it first applies filters to determine which
hosts are eligible for consideration when dispatching a
resource. Filters are binary: either a host is accepted by
the filter, or it is rejected. Hosts that are accepted by
the filter are then processed by a different algorithm to
decide which hosts to use for that request, described in
the <link linkend="weights">Weights</link> section.
<figure xml:id="filter-figure">
<title>Filtering</title>
<mediaobject>
<imageobject>
<imagedata
fileref="../../common/figures/filteringWorkflow1.png"
scale="80"/>
</imageobject>
</mediaobject>
</figure>
</para>
<para>The <literal>scheduler_available_filters</literal>
configuration option in <filename>nova.conf</filename>
provides the Compute service with the list of the filters
that are used by the scheduler. The default setting
specifies all of the filter that are included with the
Compute service:</para>
<programlisting language="ini">scheduler_available_filters=nova.scheduler.filters.all_filters</programlisting>
<para>This configuration option can be specified multiple
times. For example, if you implemented your own custom
filter in Python called
<literal>myfilter.MyFilter</literal> and you wanted to
use both the built-in filters and your custom filter, your
<filename>nova.conf</filename> file would
contain:</para>
<programlisting language="ini">scheduler_available_filters=nova.scheduler.filters.all_filters
scheduler_available_filters=myfilter.MyFilter</programlisting>
<para>The <literal>scheduler_default_filters</literal>
configuration option in <filename>nova.conf</filename>
defines the list of filters that are applied by the
<systemitem class="service"
>nova-scheduler</systemitem> service. As mentioned,
the default filters are:</para>
<programlisting language="ini">scheduler_default_filters=AvailabilityZoneFilter,RamFilter,ComputeFilter</programlisting>
<para>The following sections describe the available
filters.</para>
<section xml:id="aggregatecorefilter">
<title>AggregateCoreFilter</title>
<para>Implements blueprint per-aggregate-resource-ratio.
AggregateCoreFilter supports per-aggregate
<literal>cpu_allocation_ratio</literal>. If the
per-aggregate value is not found, the value falls back
to the global setting.</para>
</section>
<section xml:id="aggregateinstanceextraspecsfilter">
<title>AggregateInstanceExtraSpecsFilter</title>
<para>Matches properties defined in an instance type's
extra specs against admin-defined properties on a host
aggregate. Works with specifications that are
unscoped, or are scoped with
<literal>aggregate_instance_extra_specs</literal>.
See the <link linkend="host-aggregates">host
aggregates</link> section for documentation on how
to use this filter.</para>
</section>
<section xml:id="aggregate-multi-tenancy-isolation">
<title>AggregateMultiTenancyIsolation</title>
<para>Isolates tenants to specific <link
linkend="host-aggregates">host aggregates</link>.
If a host is in an aggregate that has the metadata key
<literal>filter_tenant_id</literal> it only
creates instances from that tenant (or list of
tenants). A host can be in different aggregates. If a
host does not belong to an aggregate with the metadata
key, it can create instances from all tenants.</para>
</section>
<section xml:id="aggregate-ram-filter">
<title>AggregateRamFilter</title>
<para>Implements blueprint
<literal>per-aggregate-resource-ratio</literal>.
Supports per-aggregate
<literal>ram_allocation_ratio</literal>. If
per-aggregate value is not found, it falls back to the
default setting.</para>
</section>
<section xml:id="allhostsfilter">
<title>AllHostsFilter</title>
<para>This is a no-op filter, it does not eliminate any of
the available hosts.</para>
</section>
<section xml:id="availabilityzonefilter">
<title>AvailabilityZoneFilter</title>
<para>Filters hosts by availability zone. This filter must
be enabled for the scheduler to respect availability
zones in requests.</para>
</section>
<section xml:id="computecapabilitiesfilter">
<title>ComputeCapabilitiesFilter</title>
<para>Matches properties defined in an instance type's
extra specs against compute capabilities.</para>
<para>If an extra specs key contains a colon ":", anything
before the colon is treated as a namespace, and
anything after the colon is treated as the key to be
matched. If a namespace is present and is not
'capabilities', it is ignored by this filter.</para>
<note>
<para>Disable the ComputeCapabilitiesFilter when using
a Bare Metal configuration, due to <link
xlink:href="https://bugs.launchpad.net/nova/+bug/1129485"
>bug 1129485</link></para>
</note>
</section>
<section xml:id="computefilter">
<title>ComputeFilter</title>
<para>Passes all hosts that are operational and
enabled.</para>
<para>In general, this filter should always be enabled.
</para>
</section>
<section xml:id="corefilter">
<title>CoreFilter</title>
<para>Only schedule instances on hosts if there are
sufficient CPU cores available. If this filter is not
set, the scheduler may over provision a host based on
cores (for example, the virtual cores running on an
instance may exceed the physical cores).</para>
<para>This filter can be configured to allow a fixed
amount of vCPU overcommitment by using the
<option>cpu_allocation_ratio</option>
Configuration option in
<filename>nova.conf</filename>. The default setting
is:</para>
<programlisting language="ini">cpu_allocation_ratio=16.0</programlisting>
<para>With this setting, if 8 vCPUs are on a node, the
scheduler allows instances up to 128 vCPU to be run on
that node.</para>
<para>To disallow vCPU overcommitment set:</para>
<programlisting language="ini">cpu_allocation_ratio=1.0</programlisting>
<note>
<para>
The Compute API will always return the actual number of CPU
cores available on a compute node regardless of the value of
the <option>cpu_allocation_ratio</option> configuration key. As
a result changes to the <option>cpu_allocation_ratio</option>
are not reflected via the command line clients or the
dashboard. Changes to this configuration key are only taken
into account internally in the scheduler.
</para>
</note>
</section>
<section xml:id="differenthostfilter">
<title>DifferentHostFilter</title>
<para>Schedule the instance on a different host from a set
of instances. To take advantage of this filter, the
requester must pass a scheduler hint, using
<literal>different_host</literal> as the key and a
list of instance uuids as the value. This filter is
the opposite of the <literal>SameHostFilter</literal>.
Using the <command>nova</command> command-line tool,
use the <literal>--hint</literal> flag. For
example:</para>
<screen><prompt>$</prompt> <userinput>nova boot --image cedef40a-ed67-4d10-800e-17455edce175 --flavor 1 \
--hint different_host=a0cf03a5-d921-4877-bb5c-86d26cf818e1 \
--hint different_host=8c19174f-4220-44f0-824a-cd1eeef10287 server-1</userinput></screen>
<para>With the API, use the
<literal>os:scheduler_hints</literal> key. For
example:</para>
<programlisting language="json"><xi:include href="../../common/samples/server-scheduler-hints.json" parse="text"/></programlisting>
</section>
<section xml:id="diskfilter">
<title>DiskFilter</title>
<para>Only schedule instances on hosts if there is
sufficient disk space available for root and ephemeral
storage.</para>
<para>This filter can be configured to allow a fixed
amount of disk overcommitment by using the
<literal>disk_allocation_ratio</literal>
Configuration option in
<filename>nova.conf</filename>. The default setting
is:</para>
<programlisting language="ini">disk_allocation_ratio=1.0</programlisting>
<para>Adjusting this value to greater than 1.0 enables
scheduling instances while over committing disk
resources on the node. This might be desirable if you
use an image format that is sparse or copy on write
such that each virtual instance does not require a 1:1
allocation of virtual disk to physical storage.</para>
</section>
<section xml:id="groupaffinityfilter">
<title>GroupAffinityFilter</title>
<para>The GroupAffinityFilter ensures that an instance is
scheduled on to a host from a set of group hosts. To
take advantage of this filter, the requester must pass
a scheduler hint, using <literal>group</literal> as
the key and an arbitrary name as the value. Using the
<command>nova</command> command-line tool, use the
<literal>--hint</literal> flag. For
example:</para>
<screen><prompt>$</prompt> <userinput>nova boot --image cedef40a-ed67-4d10-800e-17455edce175 --flavor 1 \
--hint group=foo server-1</userinput></screen>
</section>
<section xml:id="groupantiaffinityfilter">
<title>GroupAntiAffinityFilter</title>
<para>The GroupAntiAffinityFilter ensures that each
instance in a group is on a different host. To take
advantage of this filter, the requester must pass a
scheduler hint, using <literal>group</literal> as the
key and an arbitrary name as the value. Using the
<command>nova</command> command-line tool, use the
<literal>--hint</literal> flag. For
example:</para>
<screen><prompt>$</prompt> <userinput>nova boot --image cedef40a-ed67-4d10-800e-17455edce175 --flavor 1 \
--hint group=foo server-1</userinput></screen>
</section>
<section xml:id="imagepropertiesfilter">
<title>ImagePropertiesFilter</title>
<para>Filters hosts based on properties defined on the
instance's image. It passes hosts that can support the
specified image properties contained in the instance.
Properties include the architecture, hypervisor type,
and virtual machine mode. for example, an instance
might require a host that runs an ARM-based processor
and QEMU as the hypervisor. An image can be decorated
with these properties by using:</para>
<screen><prompt>$</prompt> <userinput>glance image-update img-uuid --property architecture=arm --property hypervisor_type=qemu</userinput></screen>
<para>The image properties that the filter checks for
are:</para>
<itemizedlist>
<listitem>
<para><literal>architecture</literal>:
Architecture describes the machine
architecture required by the image. Examples
are i686, x86_64, arm, and ppc64.</para>
</listitem>
<listitem>
<para><literal>hypervisor_type</literal>:
Hypervisor type describes the hypervisor
required by the image. Examples are xen, kvm,
qemu, and xenapi.</para>
</listitem>
<listitem>
<para><literal>vm_mode</literal>: Virtual machine
mode describes the hypervisor application
binary interface (ABI) required by the image.
Examples are 'xen' for Xen 3.0 paravirtual
ABI, 'hvm' for native ABI, 'uml' for User Mode
Linux paravirtual ABI, exe for container virt
executable ABI.</para>
</listitem>
</itemizedlist>
</section>
<section xml:id="isolatedhostsfilter">
<title>IsolatedHostsFilter</title>
<para>Allows the admin to define a special (isolated) set
of images and a special (isolated) set of hosts, such
that the isolated images can only run on the isolated
hosts, and the isolated hosts can only run isolated
images. The flag
<literal>restrict_isolated_hosts_to_isolated_images</literal>
can be used to force isolated hosts to only run
isolated images.</para>
<para>The admin must specify the isolated set of images
and hosts in the <filename>nova.conf</filename> file
using the <literal>isolated_hosts</literal> and
<literal>isolated_images</literal> configuration
options. For example:
<programlisting language="ini">isolated_hosts=server1,server2
isolated_images=342b492c-128f-4a42-8d3a-c5088cf27d13,ebd267a6-ca86-4d6c-9a0e-bd132d6b7d09</programlisting>
</para>
</section>
<section xml:id="jsonfilter">
<title>JsonFilter</title>
<para>The JsonFilter allows a user to construct a custom
filter by passing a scheduler hint in JSON format. The
following operators are supported:<itemizedlist>
<listitem>
<para>=</para>
</listitem>
<listitem>
<para>&lt;</para>
</listitem>
<listitem>
<para>></para>
</listitem>
<listitem>
<para>in</para>
</listitem>
<listitem>
<para>&lt;=</para>
</listitem>
<listitem>
<para>>=</para>
</listitem>
<listitem>
<para>not</para>
</listitem>
<listitem>
<para>or</para>
</listitem>
<listitem>
<para>and</para>
</listitem>
</itemizedlist>The filter supports the following variables:<itemizedlist>
<listitem>
<para>$free_ram_mb</para>
</listitem>
<listitem>
<para>$free_disk_mb</para>
</listitem>
<listitem>
<para>$total_usable_ram_mb</para>
</listitem>
<listitem>
<para>$vcpus_total</para>
</listitem>
<listitem>
<para>$vcpus_used</para>
</listitem>
</itemizedlist>Using the <command>nova</command>
command-line tool, use the <literal>--hint</literal>
flag:</para>
<screen><prompt>$</prompt> <userinput>nova boot --image 827d564a-e636-4fc4-a376-d36f7ebe1747 \
--flavor 1 --hint query='[">=","$free_ram_mb",1024]' server1</userinput></screen>
<para>With the API, use the
<literal>os:scheduler_hints</literal> key:</para>
<programlisting language="json"><xi:include href="../../common/samples/server-scheduler-hints2.json" parse="text"/></programlisting>
</section>
<section xml:id="ramfilter">
<title>RamFilter</title>
<para>Only schedule instances on hosts that have
sufficient RAM available. If this filter is not set,
the scheduler may over provision a host based on RAM
(for example, the RAM allocated by virtual machine
instances may exceed the physical RAM).</para>
<para>This filter can be configured to allow a fixed
amount of RAM overcommitment by using the
<literal>ram_allocation_ratio</literal>
configuration option in
<filename>nova.conf</filename>. The default setting
is:</para>
<programlisting language="ini">ram_allocation_ratio=1.5</programlisting>
<para>With this setting, if there is 1GB of free RAM, the
scheduler allows instances up to size 1.5GB to be run
on that instance.</para>
</section>
<section xml:id="retryfilter">
<title>RetryFilter</title>
<para>Filter out hosts that have already been attempted
for scheduling purposes. If the scheduler selects a
host to respond to a service request, and the host
fails to respond to the request, this filter prevents
the scheduler from retrying that host for the service
request.</para>
<para>This filter is only useful if the
<literal>scheduler_max_attempts</literal>
configuration option is set to a value greater than
zero.</para>
</section>
<section xml:id="samehostfilter">
<title>SameHostFilter</title>
<para>Schedule the instance on the same host as another
instance in a set of instances. To take advantage of
this filter, the requester must pass a scheduler hint,
using <literal>same_host</literal> as the key and a
list of instance uuids as the value. This filter is
the opposite of the
<literal>DifferentHostFilter</literal>. Using the
<command>nova</command> command-line tool, use the
<literal>--hint</literal> flag:</para>
<screen><prompt>$</prompt> <userinput>nova boot --image cedef40a-ed67-4d10-800e-17455edce175 --flavor 1 \
--hint same_host=a0cf03a5-d921-4877-bb5c-86d26cf818e1 \
--hint same_host=8c19174f-4220-44f0-824a-cd1eeef10287 server-1</userinput></screen>
<para>With the API, use the
<literal>os:scheduler_hints</literal> key:</para>
<programlisting language="json"><xi:include href="../../common/samples/server-scheduler-hints3.json" parse="text"/></programlisting>
</section>
<section xml:id="simplecidraffinityfilter">
<title>SimpleCIDRAffinityFilter</title>
<para>Schedule the instance based on host IP subnet range.
To take advantage of this filter, the requester must
specify a range of valid IP address in CIDR format, by
passing two scheduler hints:</para>
<variablelist>
<varlistentry>
<term><literal>build_near_host_ip</literal></term>
<listitem>
<para>The first IP address in the subnet (for
example,
<literal>192.168.1.1</literal>)</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>cidr</literal></term>
<listitem>
<para>The CIDR that corresponds to the subnet
(for example,
<literal>/24</literal>)</para>
</listitem>
</varlistentry>
</variablelist>
<para>Using the <command>nova</command> command-line tool,
use the <literal>--hint</literal> flag. For example,
to specify the IP subnet
<literal>192.168.1.1/24</literal></para>
<screen><prompt>$</prompt> <userinput>nova boot --image cedef40a-ed67-4d10-800e-17455edce175 --flavor 1 \
--hint build_near_host_ip=192.168.1.1 --hint cidr=/24 server-1</userinput></screen>
<para>With the API, use the
<literal>os:scheduler_hints</literal> key:</para>
<programlisting language="json"><xi:include href="../../common/samples/server-scheduler-hints4.json" parse="text"/></programlisting>
</section>
</section>
<section xml:id="weights">
<title>Weights</title>
<?dbhtml stop-chunking?>
<para>The Filter Scheduler weighs hosts based on the config
option <literal>scheduler_weight_classes</literal>, this
defaults to
<literal>nova.scheduler.weights.all_weighers</literal>,
which selects the only weigher available -- the
RamWeigher. Hosts are then weighed and sorted with the
largest weight winning.</para>
<programlisting language="ini">scheduler_weight_classes=nova.scheduler.weights.all_weighers
ram_weight_multiplier=1.0</programlisting>
<para>The default is to spread instances across all hosts
evenly. Set the <literal>ram_weight_multiplier</literal>
option to a negative number if you prefer stacking instead
of spreading.</para>
</section>
<section xml:id="chance-scheduler">
<title>Chance scheduler</title>
<?dbhtml stop-chunking?>
<para>As an administrator, you work with the Filter Scheduler.
However, the Compute service also uses the Chance
Scheduler,
<literal>nova.scheduler.chance.ChanceScheduler</literal>,
which randomly selects from lists of filtered
hosts.</para>
</section>
<xi:include
href="../../common/section_cli_nova_host_aggregates.xml"/>
<section xml:id="compute-scheduler-config-ref">
<title>Configuration reference</title>
<xi:include href="../../common/tables/nova-scheduling.xml"/>
</section>
</section>