kern/60993: pty master read race leads to EIO failures

["jgoerzen%complete.org@localhost via gnats" <gnats-admin%NetBSD.org@localhost>]

>Number:         60993
>Category:       kern
>Synopsis:       pty master read race leads to EIO failures
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Nov 11 14:00:00 +0000 2026
>Originator:     John Goerzen
>Release:        11.1
>Organization:
n/a
>Environment:
NetBSD [redacted] 11.1 NetBSD 11.1 (GENERIC) #0: Mon Nov  2 16:34:32 UTC 2026  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64
NetBSD [redacted] 11.1 NetBSD 11.1 (GENERIC) #0: Mon Nov  2 16:34:32 UTC 2026  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/alpha/compile/GENERIC alpha

>Description:
When reading from a pty master, the read will occasionally fail with EIO.  This seems to be correlated with multiple processes opening and closing ptys at the same time.

I first noticed this as a CI failure in the C-Kermit test suite, which runs multiple C-Kermit processes in parallel, using ptys to drive them (and they also often are using ptys for the tests).  One particular failure indicated that a C-Kermit process under testing had exited normally, but the test controller never received its output.  This PR is a result of working to isolate that cause.

The test program below starts 4 worker process, each of which runs 5000 trials.  Each trial calls openpty() and forks a writer process.  The writer process writes 5 bytes to the slave side of the pty, calls tcdrain() on it, and exits.  The worker tries to read the expected bytes, waits on the writer process, and closes the master.

I duplicated this on two platforms, both running NetBSD 11.1:

On amd64 running with 4 CPUs under qemu+KVM, I agerage around 80-90 failures per execution of the test program.  On my single-CPU bare-metal 600MHz 21164a Alpha, some runs produce no failures, and others less than 10.  Compiling with -O2 makes no difference.  The example program detects failures elsewhere (ie, in write()) but I have only ever seen failures in read().

I also tried writing to the master and reading from the slave.  I was never able to produce a failure in that direction.

With minor modifications to include lines, I ran the same program on OpenBSD, FreeBSD, and Linux.  None of them triggered any failures.

I had previously reported kern/60935, which was pty data loss related to kern.tty.qsize.  This example is intentionally far below qsize, so it is a different issue.  (Though I can't speak to whether it's the same underlying cause.)
>How-To-Repeat:
Compile the below program with:

cc -o ptyeio ptyeio.c -lutil

You will see lines like:

read() returned -1 after 0 bytes: Input/output error

#include <sys/wait.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>
#include <termios.h>
#include <unistd.h>
#include <util.h>

#define PROCS   4
#define TRIALS  5000

int
main(void)
{
        char buf[64];
        ssize_t n = 0;
        size_t got;
        int i, t, m, s, lost;

        setvbuf(stdout, NULL, _IOLBF, 0);
        for (i = 0; i < PROCS; i++) {
                if (fork() != 0)
                        continue;
                lost = 0;
                for (t = 0; t < TRIALS; t++) {
                        if (openpty(&m, &s, NULL, NULL, NULL) == -1) {
                                perror("openpty");
                                continue;
                        }
                        if (fork() == 0) {
                                close(m);
                                if (write(s, "hello", 5) != 5)
                                        perror("write");
                                if (tcdrain(s) == -1)
                                        perror("tcdrain");
                                _exit(0);
                        }
                        close(s);
                        got = 0;
                        while (got < 5 &&
                            (n = read(m, buf + got, sizeof(buf) - got)) > 0)
                                got += (size_t)n;
                        if (got != 5) {
                                printf("read() returned %zd after %zu bytes: "
                                    "%s\n", n, got,
                                    n == -1 ? strerror(errno) : "EOF");
                                lost++;
                        }
                        wait(NULL);
                        close(m);
                }
                printf("process %d: lost %d of %d\n", i, lost, TRIALS);
                _exit(0);
        }
        while (wait(NULL) > 0)
                ;
        return 0;
}

>Fix:
I don't know.