Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Weaver]

[This topic has been created by splitting off the following posts from leonroy's High packet loss and occasional disconnection under load thread.

Maybe it's time for a new pfSense install.

I run a virtual instance, it's been faultless.

Apologies for straying off topic - a random question about your VM and major software components - can your software setup make use of paravirtualization ?

[dee.jay]

I don't believe so no - paravirtualization was no longer supported by ESX 5 upwards because of Intel VT and AMD VT technologies, which apparently are a lot faster.

[Chrysalis]

I always thought of paravirtualisation been modified kernel/drivers that dont fully emulate but instead are cut down specifically designed for a virtual environment, and in that respect it is still around e.g. vmx network drivers.  As emulating full hardware is expensive.

But maybe I am wrong and it isnt that.

actually seems I am right.

https://docs.vmware.com/en/VMware-vCloud-NFV-OpenStack-Edition/3.0/vmwa-vcloud-nfv30-performance-tunning/GUID-E2271E36-12BB-47CE-A765-5ECB5BBE7CC7.html

I think if you have hardware cpu virtualisation, then its probably fair to call it partial paravirtualization, as the hardware is usually just limited to the cpu and memory subsystem but rest of kit still has to be emulated, like graphics, i/o and networking.

If you have vt-d support you can also directly passthru hardware like i/o cards and gpu's.

[dee.jay]

Ah yes, VT-d. I use that to passthrough the onboard SAS controller through to a FreeNAS VM.

You can get pretty crazy with VT-d if devices can support it. Probably the favourite thing I've done was to passthrough an nVidia GTX680 to a Windows VM with the machine itself running Arch Linux.

https://www.youtube.com/watch?v=YRFO942RLNE

^^ That's me. Old video, apparently that was 4 years ago I did that. Should try it on my more modern desktop.

[Chrysalis]

one of the reasons I am migrating away from esxi is you wouldnt be able to do that on consumer nvidia hardware as an artificial block in the drivers to make you buy quadro cards :p, but no problem with proxmox, qemu etc.

[Weaver]

Yes, to @chyrs, basically. Paravirtualisation is where you run a special o/s or driver or app code which contains well-known secret handshake codes which are a hint for the VM to tell it to do something. It’s a very very high performance way of cheating, by giving hints to the VM that you wish for some operation to be done but you don’t have to emulate the hardware needed to carry it out, and I say cheating because full emulation including all the hardware is not necessarilly being done by the VM at all. It’s like the equivalent of adding a new instruction to the CPU’s instruction set, not to imply that that is literally the method used, but can be implemented by any distinctive hint sequence that the VM can recognise and which will not happen accidentally in existing code. If you provide paravirtualisation-aware drivers then several different similar operating systems could be used.

[dee.jay]

one of the reasons I am migrating away from esxi is you wouldnt be able to do that on consumer nvidia hardware as an artificial block in the drivers to make you buy quadro cards :p, but no problem with proxmox, qemu etc.

Yeah that is very true. I believe you can flash BIOS'es to consumer cards, but I don't know if that is still a valid exercise these days.

For a desktop it works fine with QEMU etc as you say.

[niemand]

https://www.unixarena.com/2017/12/para-virtualization-full-virtualization-hardware-assisted-virtualization.html/ is a nice summary.

Guest OSes make calls to the hypervisor via APIs so it doesn't have to emulate everything - a microkernel sits on top of the hardware handling these calls.

There are also hybrids that not only use PV via the OS but also make calls directly to the device drivers the hypervisor would use. Hardware assistance as VMWare ESXi uses, where hypervisor spends much of its time sanitising then feeding directly to bare metal, plus PV calls that go entirely around the OS alleviating the need for the hypervisor to sanitise and handle edge case instructions and use the device drivers the hypervisor has installed as modules.

I'm most familiar with ESXi and, as a lot of what it does is close to bare metal, it can be pretty performant. Obviously certain instructions and sequences are trapped by it as they can't be handled by its normal operation and have to be emulated.

It's a fascinating technology overall and, like many other technologies, one that's becoming less of a concern as we increasingly run our applications on other people's computers.

[Weaver]

Off topic: I wonder if someone could start trapping all unrecognised instructions and then handling them in software in cases where the processor is too old to know more recent instructions. I remember a runtime for a dos C compiler (JPI C) that would detect the presence of absence of an 8087 or the equivalent internal h/w floating point in a later processor and it would then patch code in the code segment of the loaded image so that 80x87 FP instructions would be changed to calls or jumps or traps/software interrupt instructions I forget which, anyway something that was short enough to fit in the byte length of the instruction (although I suppose you could in extremis put in a short jump and overwrite following instructions too, copying affected bytes to a short jump-off area somewhere provided that the instructions were relocatable [!!]). The chosen patch was short enough so that it could fit with no trouble anyway.

This was such a well thought out idea because it gave 100% full speed to modern processors and old processors did what they could, with the solution being about as good as anything could be, provided that the method chosen for the call-return patch code, be it an int instruction or a call or whatever, was as fast as possible to keep the overhead down.

[dee.jay]

I am pretty sure that ESX will only expose to the VM the instruction set that is available from the physical CPU.

It is important to keep CPU families the same when dealing with a vCenter environment, where VM's move across blades and such. If a VM moves to a blade with a newer instruction set CPU - it cannot then move back. (Note, when dealing with a large deployment of multiple ESX servers on shared storage, VM's will move around all day long in order to balance out the demand on CPU and memory.)

So, if one were to run a VM and run an application that expected newer instructions, it would simply fail to run - as the VM would only receive the same instruction set as the underlying hardware beneath it.

[niemand]

Absolutely. VMs that are allowed to be vMotioned like that haven't a clue they have moved. They are snapshot and moved to a new LUN / volume. The VM is running with the same settings it was when originally powered on.

[Chrysalis]

When I updated my ESXi rig, the machine changed from intel i5 750 to AMD 2600X, different vendor and a much different era of hardware.  In terms of migrating VM's it was like as Carl said, as if there was no hardware swap at all, the reason been that the emulated kit is still the same, the only change was the cpu really.  Both AMD and Intel chips are x86_64 chips so share same cpu drivers.  It's only the cpu scheduling that differs between the two.

[niemand]

Should mention that in the case of ESXi at least an awful lot of what it does is just sending instructions straight to the physical CPUs. There are key instructions it will trap and handle in different ways but a full-on CPU emulator it is not. It takes advantage of  VMMs much as a bunch of other full virtualisation options do.

I can't say I've had much exposure to paravirtualisation and I'm fine with that. Ideally hypervisor and hardware should take care of everything.