kern/54759: vm.ubc_direct deadlock when read()/write() into mapping of itself

[jaromir.dolecek%gmail.com@localhost]

>Number:         54759
>Category:       kern
>Synopsis:       vm.ubc_direct deadlock when read()/write() into mapping of itself
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Dec 12 22:10:00 +0000 2019
>Originator:     Jaromir Dolecek
>Release:        9.99.17
>Organization:
>Environment:
NetBSD pure-amd64 9.99.17 NetBSD 9.99.17 (QEMU-VIRT) #240: Sun Nov 10 01:40:13 CET 2019  dolecek@.../sys/arch/amd64/compile/QEMU-VIRT amd64
>Description:
ubc_direct code path locks buffer chage pages for the I/O when doing read()/write(). If the buffer passed to the I/O is actually a mmapped() buffer cache page, deadlock occurs. When this happens, the calling process is blocked, and also other processes fail to execute/finish.

This has been noted again in http://mail-index.netbsd.org/current-users/2019/12/03/msg037131.html and also in discussion when the code was introduced.

This problem needs to be fixed before we can turn the ubc_direct to on by default.
>How-To-Repeat:
1. sysctl -w vm.ubc_direct=1
2. compile and run attached program:

#include <sys/types.h>
#include <sys/uio.h>
#include <unistd.h>
#include <stdio.h>
#include <sys/mman.h>
#include <fcntl.h>
#include <string.h>

int
main(int argc, const char **argv)
{
        FILE *fp;
        int fd;
        char buffer[8192];
        char buffer2[8192];

        fp = tmpfile();
        fd = fileno(fp);

        int rndfd = open("/dev/urandom", O_RDONLY);
        read(rndfd, buffer, sizeof(buffer));
        close(rndfd);

        fwrite(buffer, 1, sizeof(buffer), fp);
        fflush(fp);

        void *content;

        if ((content = mmap(NULL, sizeof(buffer), PROT_READ|PROT_WRITE, MAP_SHARED,
                fd, 0)) == NULL)
                perror("mmap");

        lseek(fd, 0, SEEK_SET);

        read(fd, buffer2, sizeof(buffer2));
        if (memcmp(buffer, buffer2, sizeof(buffer2)) != 0)
                fprintf(stderr, "memcmp mismatch\n");

        lseek(fd, 0, SEEK_SET);
        if (write(fd, content, sizeof(buffer)) < 0)
                perror("Write");
}
>Fix:
N/A