Re: nvmm & pku

[Dave Voutila <dv%sisu.io@localhost>]

Taylor R Campbell <campbell+netbsd-tech-kern%mumble.net@localhost> writes:

>> Date: Thu, 26 Jan 2023 10:29:00 -0500
>> From: Dave Voutila <dv%sisu.io@localhost>
>>
>>  1) disable PKU support for guests by masking the cpuid extended feature
>>     bit advertising support (I believe this is already done) and *also*
>>     intercepting and prohibiting flipping the CR4 PKE bit to 1. The
>>     interception of the CR4 bit flip is key. Without it the bit set,
>>     wrpkru/rdpkru generate #UD, so you can prohibit usage.
>>
>>     I believe the CR4 intercept is done only in nvmm's VMX functionality
>>     and luckily is preventing that bit being set, but I'm not 100%
>>     positive about this. I do know it is *not* done for SVM guests.
>
> I've drafted a patch to filter the CR4 bits for SVM guests.  However,
> I don't have any SVM hardware, so I'll need to find someone to test
> it.
>
>> or
>>
>>   2) save and restore the PKRU value at guest entry/exit, similar to FPU
>>      state.
>>
>> I think (2) is the right approach and I have a diff for OpenBSD already
>> that should be landing shortly. For now, as we're testing the use of PKU
>> in OpenBSD, we have it disabled when we detect we're running under a
>> hypervisor.
>
> That sounds reasonable.  If you would like to share the diff I'd be
> happy to take a look.  Need to familiarize myself with the relevant
> parts of the architecture manual.
>

I committed our simplified approach yesterday:

https://freshbsd.org/openbsd/src/commit/PB5K056kGxKiZfxJ

>> We plan on enabling it prior to the release of OpenBSD 7.3 and this will
>> have ramifications for OpenBSD guests under nvmm potentially trampling
>> or breaking other guests. I'm happy to have a conversation with any
>> NetBSD devs looking to update nvmm to support these features.
>
> Thanks -- do you have an image we can run in a guest to test the
> potential trampling?
>

I don't have an image. What I do for testing is using two test programs
that use inline asm to issue rdpkru and wrpkru instructions. A guest
should be able to be isolated such that if I write a pkru value multiple
times, such as 0xbabaf000, it can read back the same value.

You can also use a more recent linux distro, such as Ubuntu 22.04, that
ships with PKU enabled. Linux should always have a program see the
default pkru value 0x55555554 iirc. If one guest is able to hammer the
pkru and change it such that another guest can read the value, something
is wrong in the hypervisor.

Similarly, a host program shouldn't be able read or set a value exposed
to a guest. So if you run a test program in the host writing something
distinguishable like my 0xbabaf0000 test value and a guest sees it just
by running rdpkru instructions, something is wrong in the hypervisor.

-dv