kern/57066: Concurrent O_DIRECT and fork() can flood the kernel log and reduce tmpfs capacity to 0

[hildenbr%in.tum.de@localhost]

>Number:         57066
>Category:       kern
>Synopsis:       Concurrent O_DIRECT and fork() can flood the kernel log and reduce tmpfs capacity to 0
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Oct 20 09:25:00 +0000 2022
>Originator:     David Hildenbrand
>Release:        9.3
>Organization:
>Environment:
NetBSD localhost 9.3 NetBSD 9.3 (GENERIC) #0: Thu Aug  4 15:30:37 UTC 2022  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64
>Description:
Hi,

as part of our research, we developed three test cases that stress the COW+page pinning logic
by mixing O_DIRECT with concurrent fork(), to be ran on different OSes that support O_DIRECT.

The main motivation for these tests is to more easily identify issues in the COW+page pinning
logic in OSes. Especially when page pinning is used inside the OS for other features beside
O_DIRECT, whereby page pins can be held for a very long time, such issues become more relevant
(e.g., as shown by the recent vmscplice() security issue under Linux).

According to our understanding, NetBSD so far only uses page pinning for O_DIRECT.

We also ran these test cases on NetBSD 9.2 and 9.3, and found that 2 out of 3 tests fail, indicating
correctness issues in these corner cases. While correctness is one thing, the side effects of the
incorrect handling are being able to flood the kernel log and suddenly having a tmpfs *capacity* of 0
Byte, triggerable by an unprivileged user. Due to these side-effects, I set the severity as critical.


I. test1

localhost$ ./test1 tmp
Using 'tmp' with O_DIRECT.
Wrong data read: Secret data

The test takes a long time to fail. It helps to run it in a VM and overload the CPU in
the hypervisor (e.g., "stress -c `nproc --all`" under Linux)

The kernel log is flooded with messages
like:
	pmap_unwire: wiring for pmap 0xffff8ea0e1bfedc8 va 0x6fc6de4fd000 did not change!

In addition, tmpfs suddenly has a capacity of 0B (was 2 GiB), making commands like "cc" that use /tmp fail:

localhost# df -ah
Filesystem         Size       Used      Avail %Cap Mounted on
/dev/wd0a           21G       2.4G        18G  11% /
tmpfs              4.0K       4.0K         0B 100% /tmp
kernfs             1.0K       1.0K         0B 100% /kern
ptyfs              1.0K       1.0K         0B 100% /dev/pts
procfs             4.0K       4.0K         0B 100% /proc
tmpfs              4.0K       4.0K         0B 100% /var/shm


II. test2

localhost$ ./test2 tmp
Using 'tmp' with O_DIRECT.

The test will run forever and not fail. The kernel log is flooded with pmap_unwire error messages only.


III. test3

localhost$ ./test3 tmp
Using 'tmp' with O_DIRECT.
mem[0] != 0xff, mem[0] == 0x0
Wrong data read

The test fails very fairly quickly. Otherwise, the symptoms match test1.
>How-To-Repeat:
The test cases are located at: https://gitlab.com/cop_paper/o_direct_fork_tests.git

To run them, simply build them with "make" and then execute them on a
dummy file. The tests will run until an error is detected.

Assuming git is installed and running as an unprivileged user:

localhost$ git clone -c http.sslVerify=false https://gitlab.com/cop_paper/o_direct_fork_tests.git
Cloning into 'o_direct_fork_tests'...
remote: Enumerating objects: 6, done.
remote: Counting objects: 100% (6/6), done.
remote: Compressing objects: 100% (6/6), done.
remote: Total 6 (delta 2), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (6/6), 4.70 KiB | 301.00 KiB/s, done.
Resolving deltas: 100% (2/2), done.
localhost$ cd o_direct_fork_tests/
localhost$ make
cc -O2 -MMD -MP -O2 -Wall   -c test1.c
cc -O2 -MMD -MP -O2 -Wall   -lpthread -o test1 test1.c 
cc -O2 -MMD -MP -O2 -Wall   -c test2.c
cc -O2 -MMD -MP -O2 -Wall   -lpthread -o test2 test2.c 
cc -O2 -MMD -MP -O2 -Wall   -c test3.c
cc -O2 -MMD -MP -O2 -Wall   -lpthread -o test3 test3.c 

Run the tests as documented above.
>Fix: