diff --git a/doc/source/admin/index.rst b/doc/source/admin/index.rst index f19e3b8f7da3..2ff1873f2003 100644 --- a/doc/source/admin/index.rst +++ b/doc/source/admin/index.rst @@ -53,3 +53,4 @@ operating system, and exposes functionality over a web-based API. support-compute.rst system-admin.rst secure-live-migration-with-qemu-native-tls.rst + mitigation-for-Intel-MDS-security-flaws.rst diff --git a/doc/source/admin/mitigation-for-Intel-MDS-security-flaws.rst b/doc/source/admin/mitigation-for-Intel-MDS-security-flaws.rst new file mode 100644 index 000000000000..316f2cb5761c --- /dev/null +++ b/doc/source/admin/mitigation-for-Intel-MDS-security-flaws.rst @@ -0,0 +1,127 @@ +====================================================================== +Mitigation for MDS ("Microarchitectural Data Sampling") Security Flaws +====================================================================== + +Issue +~~~~~ + +In May 2019, four new microprocessor flaws, known as `MDS +`_ , have been +discovered. These flaws affect unpatched Nova compute nodes and +instances running on Intel x86_64 CPUs. (The said MDS security flaws +are also referred to as `RIDL and Fallout `_ or +`ZombieLoad `_). + + +Resolution +~~~~~~~~~~ + +To get mitigation for the said MDS security flaws, a new CPU flag, +`md-clear`, needs to be exposed to the Nova instances. It can be done +as follows. + +(1) Update the following components to the versions from your Linux + distribution that have fixes for the MDS flaws, on all compute nodes + with Intel x86_64 CPUs: + + - microcode_ctl + - kernel + - qemu-system-x86 + - libvirt + +(2) When using the libvirt driver, ensure that the CPU flag ``md-clear`` + is exposed to the Nova instances. It can be done so in one of the + three following ways, given that Nova supports three distinct CPU + modes: + + a. ``[libvirt]/cpu_mode = host-model`` + + When using ``host-model`` CPU mode, the ``md-clear`` CPU flag + will be passed through to the Nova guests automatically. + + This mode is the default, when ``virt_type=kvm|qemu`` is + set in ``/etc/nova/nova-cpu.conf`` on compute nodes. + + b. ``[libvirt]/cpu_mode = host-passthrough`` + + When using ``host-passthrough`` CPU mode, the ``md-clear`` CPU + flag will be passed through to the Nova guests automatically. + + c. A specific custom CPU model — this can be enabled using the + Nova config attributes: ``[libvirt]/cpu_mode = custom`` plus a + particular named CPU model, e.g. ``[libvirt]/cpu_model = + IvyBridge`` + + (The list of all valid named CPU models that are supported by + your host, QEMU, and libvirt can be found by running the + command ``virsh domcapabilities``.) + + When using a custom CPU mode, you must *explicitly* enable the + CPU flag ``md-clear`` to the Nova instances, in addition to the + flags required for previous vulnerabilities, using the + ``cpu_model_extra_flags``. E.g.:: + + [libvirt] + cpu_mode = custom + cpu_model = IvyBridge + cpu_model_extra_flags = spec-ctrl,ssbd,md-clear + +(3) Reboot the compute node for the fixes to take effect. (To minimize + workload downtime, you may wish to live migrate all guests to + another compute node first.) + +Once the above steps have been taken on every vulnerable compute +node in the deployment, each running guest in the cluster must be +fully powered down, and cold-booted (i.e. an explicit stop followed +by a start), in order to activate the new CPU model. This can be done +by the guest administrators at a time of their choosing. + + +Validate that the fixes are in effect +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +After applying relevant updates, administrators can check the kernel's +"sysfs" interface to see what mitigation is in place, by running the +following command (on the host):: + + # cat /sys/devices/system/cpu/vulnerabilities/mds + Mitigation: Clear CPU buffers; SMT vulnerable + +To unpack the message "Mitigation: Clear CPU buffers; SMT vulnerable": + + - The ``Mitigation: Clear CPU buffers`` bit means, you have the "CPU + buffer clearing" mitigation enabled (which is mechanism to invoke a + flush of various exploitable CPU buffers by invoking a CPU + instruction called "VERW"). + + - The ``SMT vulnerable`` bit means, depending on your workload, you may + still be vulnerable to SMT-related problems. You need to evaluate + whether your workloads need SMT (also called "Hyper-Threading") to + be disabled or not. Refer to the guidance from your Linux + distribution and processor vendor. + +To see the other possible values for the sysfs file, +``/sys/devices/system/cpu/vulnerabilities/mds``, refer to the `MDS +system information +`_ +section in Linux kernel's documentation for MDS. + +On the host, validate that KVM is capable of exposing the ``md-clear`` +flag to guests:: + + # virsh domcapabilities kvm | grep md-clear + + +Also, refer to the 'Diagnosis' tab in this security notice document +`here `_ + + +Performance Impact +~~~~~~~~~~~~~~~~~~ + +Refer to this section titled "Performance Impact and Disabling MDS" from +the security notice document `here +`_, under the +'Resolve' tab. (Note that although the article referred to is from Red +Hat, the findings and recommendations about performance impact apply +for other distributions as well.) diff --git a/doc/source/admin/security.rst b/doc/source/admin/security.rst index 31c6b0a52fea..a7300f8eef86 100644 --- a/doc/source/admin/security.rst +++ b/doc/source/admin/security.rst @@ -48,3 +48,13 @@ streams of a nova instance—i.e. guest RAM, device state, and disks (via NBD) when using non-shared storage. For further details on how to set this up, refer to the :doc:`secure-live-migration-with-qemu-native-tls` document. + + +Mitigation for MDS (Microarchitectural Data Sampling) security flaws +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +It is strongly recommended to patch all compute nodes and nova instances +against the processor-related security flaws, such as MDS (and other +previous vulnerabilities). For details on applying mitigation for the +MDS flaws, refer to the :doc:`mitigation-for-Intel-MDS-security-flaws` +document.