Kitz ADSL Broadband Information
adsl spacer  
Support this site
Home Broadband ISPs Tech Routers Wiki Forum
   Compare ISP   Rate your ISP
   Glossary   Glossary
Please login or register.

Login with username, password and session length
Advanced search  


Author Topic: Interesting discovery about meltdown mitigation performance  (Read 410 times)


  • Content Team
  • Addicted Kitizen
  • *
  • Posts: 5719
Interesting discovery about meltdown mitigation performance
« on: October 05, 2018, 02:53:55 PM »

I been messing with proxmox virtualization lately and it has a specific hardware option related to PCID as well as another option for foreshadow/spectre.

I will paste below the outcome of a diagnostic script for checking vulnerabilities.

The hardware is leased from hetzner, which I ordered yesterday from their auction section (old gen bargain basement section).

The cpu is a xeon 1245v2  which the important bit is a ivy bridge generation cpu (v1 sandy, v2 ivy, v3 haswell and so on).  I think it is unlikely haswell have flashed a recent bios on the server, but I have not verified.

PCID I had always believed to be introduced in haswell and newer chips, simply because on Windows OS it is required to have a haswell or newer to have optimised meltdown mitigation performance, as stated in the inspectre app and various websites.  But I was then surprised when checking the cpuflags on the system that PCID was present and indeed proxmox passes it onto hypervisors.  The diagnostics script also confirms meltdown mitigation is accelerated via PCID instructions.

As it turns out PCID was actually introduced in sandy bridge, but there is a second instruction set similarly named called NVPCID introduced in haswell.  The linux implementation only requires PCID for enhanced performance so sandy bridge and newer chips benefit from it, whilst the windows implementation requires PCID and NVPCID and as such requires haswell or newer, so this will probably annoy ivy bridge and sandy bridge owners who run windows.

I checked the FreeBSD mailing list and they also seem to have an implementation that can take advantage of plain PCID without NVPCID as well.  But I have yet to bench on a BSD machine to confirm this.  But will soon.

Also of interest is that debian (and probably other linux distros) can load cpu microcode updates at boot, without needing an updated bios firmware.  Now this I already knew, and I believe windows 10 can do this as well (but not older versions of windows).  I think its likely this hardware I am playing with right now from hetzner has not had its bios flashed, so most of the stuff been mitigated below is the debian microcode updates.

If you virtualize via ESXi, on the same cpu and using their latest security patches I have no microcode update.  My existing ESXi hosted at soyoustart ironically is the exact same cpu, so I am doing performance testing on that to compare proxmox and ESXi hypervisor performance, but the affect of spectre and co may also be tested.  I can still test on proxmox without all the mitigations as well as they can be disabled during the boot process.

This script was run on the proxmox host (which runs on top of debian), there was no specific tuning done to disable or enable mitigation's so this is out of box status on proxmox 5.1 which has not been updated since install.

The summary is colour coded on the shell and not here, but OK means mitigated, KO means not mitigated.

Code: [Select]
root@vhde1:~# ./
Spectre and Meltdown mitigation detection tool v0.40

Checking for vulnerabilities on current system
Kernel is Linux 4.15.17-1-pve #1 SMP PVE 4.15.17-9 (Wed, 9 May 2018 13:31:43 +0200) x86_64
CPU is Intel(R) Xeon(R) CPU E3-1245 V2 @ 3.40GHz

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available:  YES
    * CPU indicates IBRS capability:  YES  (SPEC_CTRL feature bit)
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available:  YES
    * CPU indicates IBPB capability:  YES  (SPEC_CTRL feature bit)
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available:  YES
    * CPU indicates STIBP capability:  YES  (Intel STIBP feature bit)
  * Speculative Store Bypass Disable (SSBD)
    * CPU indicates SSBD capability:  YES  (Intel SSBD)
  * L1 data cache invalidation
    * FLUSH_CMD MSR is available:  YES
    * CPU indicates L1D flush capability:  YES  (L1D flush feature bit)
  * Enhanced IBRS (IBRS_ALL)
    * CPU indicates ARCH_CAPABILITIES MSR availability:  NO
    * ARCH_CAPABILITIES MSR advertises IBRS_ALL capability:  NO
  * CPU explicitly indicates not being vulnerable to Meltdown (RDCL_NO):  NO
  * CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO):  NO
  * CPU/Hypervisor indicates L1D flushing is not necessary on this system:  NO
  * Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA):  NO
  * CPU supports Software Guard Extensions (SGX):  NO
  * CPU microcode is known to cause stability problems:  NO  (model 0x3a family 0x6 stepping 0x9 ucode 0x20 cpuid 0x306a9)
  * CPU microcode is the latest known available version:  YES  (latest version is 0x20 dated 2018/04/10 according to builtin MCExtractor DB v84 - 2018/09/27)
* CPU vulnerability to the speculative execution attack variants
  * Vulnerable to CVE-2017-5753 (Spectre Variant 1, bounds check bypass):  YES
  * Vulnerable to CVE-2017-5715 (Spectre Variant 2, branch target injection):  YES
  * Vulnerable to CVE-2017-5754 (Variant 3, Meltdown, rogue data cache load):  YES
  * Vulnerable to CVE-2018-3640 (Variant 3a, rogue system register read):  YES
  * Vulnerable to CVE-2018-3639 (Variant 4, speculative store bypass):  YES
  * Vulnerable to CVE-2018-3615 (Foreshadow (SGX), L1 terminal fault):  NO
  * Vulnerable to CVE-2018-3620 (Foreshadow-NG (OS), L1 terminal fault):  YES
  * Vulnerable to CVE-2018-3646 (Foreshadow-NG (VMM), L1 terminal fault):  YES

CVE-2017-5753 aka 'Spectre Variant 1, bounds check bypass'
* Mitigated according to the /sys interface:  YES  (Mitigation: __user pointer sanitization)
* Kernel has array_index_mask_nospec:  YES  (1 occurrence(s) found of x86 64 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch:  NO
* Kernel has mask_nospec64 (arm64):  NO
> STATUS:  NOT VULNERABLE  (Mitigation: __user pointer sanitization)

CVE-2017-5715 aka 'Spectre Variant 2, branch target injection'
* Mitigated according to the /sys interface:  YES  (Mitigation: Full generic retpoline, IBPB, IBRS_FW)
* Mitigation 1
  * Kernel is compiled with IBRS support:  YES
    * IBRS enabled and active:  YES  (for kernel and firmware code)
  * Kernel is compiled with IBPB support:  YES
    * IBPB enabled and active:  YES
* Mitigation 2
  * Kernel has branch predictor hardening (arm):  NO
  * Kernel compiled with retpoline option:  YES
    * Kernel compiled with a retpoline-aware compiler:  YES  (kernel reports full retpoline compilation)
> STATUS:  NOT VULNERABLE  (Full retpoline + IBPB are mitigating the vulnerability)

CVE-2017-5754 aka 'Variant 3, Meltdown, rogue data cache load'
* Mitigated according to the /sys interface:  YES  (Mitigation: PTI)
* Kernel supports Page Table Isolation (PTI):  YES
  * PTI enabled and active:  YES
  * Reduced performance impact of PTI:  YES  (CPU supports PCID, performance impact of PTI will be reduced)
* Running as a Xen PV DomU:  NO

CVE-2018-3640 aka 'Variant 3a, rogue system register read'
* CPU microcode mitigates the vulnerability:  YES
> STATUS:  NOT VULNERABLE  (your CPU microcode mitigates the vulnerability)

CVE-2018-3639 aka 'Variant 4, speculative store bypass'
* Kernel supports speculation store bypass:  NO
> STATUS:  VULNERABLE  (your kernel needs to be updated)

CVE-2018-3615 aka 'Foreshadow (SGX), L1 terminal fault'
* CPU microcode mitigates the vulnerability:  N/A
> STATUS:  NOT VULNERABLE  (your CPU vendor reported your CPU model as not vulnerable)

CVE-2018-3620 aka 'Foreshadow-NG (OS), L1 terminal fault'
* Kernel supports PTE inversion:  NO
* PTE inversion enabled and active:  UNKNOWN  (sysfs interface not available)
> STATUS:  VULNERABLE  (Your kernel doesn't support PTE inversion, update it)

CVE-2018-3646 aka 'Foreshadow-NG (VMM), L1 terminal fault'
* This system is a host running an hypervisor:  YES
* Mitigation 1 (KVM)
  * EPT is disabled:  NO
* Mitigation 2
  * L1D flush is supported by kernel:  NO
  * L1D flush enabled:  UNKNOWN  (can't find or read /sys/devices/system/cpu/vulnerabilities/l1tf)
  * Hardware-backed L1D flush supported:  NO  (flush will be done in software, this is slower)
  * Hyper-Threading (SMT) is enabled:  YES
./ 3867: [: Illegal number:
> STATUS:  VULNERABLE  (disable EPT or enabled L1D flushing to mitigate the vulnerability)

> SUMMARY: CVE-2017-5753:OK CVE-2017-5715:OK CVE-2017-5754:OK CVE-2018-3640:OK CVE-2018-3639:KO CVE-2018-3615:OK CVE-2018-3620:KO CVE-2018-3646:KO

To mitigate the last CVE, cpu pinning would work, the issue is when HTT cores are shared between different VM's, disabling EPT I think possibly also mitigates it.
« Last Edit: October 05, 2018, 02:59:10 PM by Chrysalis »
Sky Fiber Pro - Billion 8800NL bridge & PFSense BOX running PFSense 2.4 - ECI Cab - LINE STATISTICS CLICK HERE


  • Global Moderator
  • Senior Kitizen
  • *
  • Posts: 26385
  • Over the Rainbow Bridge
    • The ELRepo Project
Re: Interesting discovery about meltdown mitigation performance
« Reply #1 on: October 05, 2018, 05:19:00 PM »

You might find it interesting to go to the /sys/devices/system/cpu/vulnerabilities/ directory and read each file contained therein.

Example --

$ cd /sys/devices/system/cpu/vulnerabilities
$ ll
total 0
-r--r--r--. 1 root root 4096 Oct  5 17:14 l1tf
-r--r--r--. 1 root root 4096 Oct  5 17:14 meltdown
-r--r--r--. 1 root root 4096 Oct  5 17:14 spec_store_bypass
-r--r--r--. 1 root root 4096 Oct  5 17:14 spectre_v1
-r--r--r--. 1 root root 4096 Oct  5 17:14 spectre_v2
$ more *
Mitigation: Page Table Inversion
Mitigation: PTI
Mitigation: __user pointer sanitization
Mitigation: Full generic retpoline
:cat:  100% Linux and, previously, Unix. Co-founder of the ELRepo Project.

Please consider making a donation to support the running of this site.