kern/60935: Data loss when pty queue exceeds kern.tty.qsize

["jgoerzen%complete.org@localhost via gnats" <gnats-admin%NetBSD.org@localhost>]

>Number:         60935
>Category:       kern
>Synopsis:       Data loss when pty queue exceeds kern.tty.qsize
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Oct 29 00:30:00 +0000 2026
>Originator:     John Goerzen
>Release:        NetBSD 11.0
>Organization:
>Environment:
Verified on:

NetBSD alpha.lan.complete.org 11.0 NetBSD 11.0 (GENERIC) #0: Thu Jul 30 15:23:12 UTC 2026  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/alpha/compile/GENERIC alpha

and

NetBSD anyvm.org 11.0_RC5 NetBSD 11.0_RC5 (GENERIC) #0: Tue Jun 16 15:48:07 UTC 2026  mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64

>Description:
When the pty reader is slower than the writer, and there is data in the pty queue that exceeds the number of bytes specified in the sysctl kern.tty.qsize, part of the data written can be silently lost.

This appears to be the case when the size of the write to the sending side is less than the qsize, or perhaps it just has to be smallish (I had difficulty reproducing with large writes).

This comment in /etc/sysctl.conf implies this has been an issue for a long while:

# Default tty/pty character queue sizes. Should be bumped to 32K or so if
# used in networking (ppp/pppoe)

I have an isolated reproduction below.

I initially ran into this while developing the test suite for C-Kermit 11, and documented it at https://www.openkermit.org/ckermit/platforms/ (see the "Weird problems on NetBSD" section).  Normally, C-Kermit requires ACKs to its packets and has sliding windows, and will detect and workaround this issue.  However, it has a streaming mode in which it requires the link to be completely error-free, and it just sends data as fast as possible, aborting the transfer if a checksum fails, rather than retry.  In this mode, I was observing failures as I added additional tests with larger data sizes.

None of the other platforms I test on (FreeBSD, OpenBSD, Linux, and MacOS) exhibit this problem.

I suspect that this could also be an issue in other circumstances.  For instance, if a user is on a serial console running at 2400bps, and uses ssh to connect to a remote system, then runs "ls -l /usr/bin".  Thanks to large TCP packets and ssh compression, it is likely to overwhelm the qsize at the default (1024) very quickly, resulting in a corrupted display.

I could also see this being the issue behind the sysctl warning about PPP.  PPP may be able to recover from this, but at an efficiency cost: lost packets and, if compression is enabled, a compression desynchronization requiring a compression restart.
>How-To-Repeat:
This C program reliably repeats the problem and reports how many bytes were sent before bytes were skipped.

#include <sys/wait.h>
#include <fcntl.h>
#include <poll.h>
#include <stdio.h>
#include <stdlib.h>
#include <termios.h>
#include <time.h>
#include <unistd.h>

#define TOTAL 1000000L          /* bytes to send */
#define WSIZE 90                /* bytes per write() */
#define RSIZE 100               /* bytes per read() */
#define STARTWAIT_MS 500        /* reader delay before the first read */
#define IDLE_MS 3000            /* reader stops after this much quiet */

static void writer(int fd) {
    unsigned char buf[WSIZE];
    long off = 0;
    size_t i, len, done;
    ssize_t n;

    while (off < TOTAL) {
        len = TOTAL - off < WSIZE ? (size_t)(TOTAL - off) : WSIZE;
        for (i = 0; i < len; i++)
            buf[i] = (off + (long)i) % 251;
        done = 0;
        while (done < len) {
            if ((n = write(fd, buf + done, len - done)) < 0) {
                    exit(10);
            }
            done += (size_t)n;
        }
        off += (long)len;
    }
}

int main(void) {
    struct termios t;
    struct timespec ts = { STARTWAIT_MS / 1000,
                           (STARTWAIT_MS % 1000) * 1000000L };
    struct pollfd pfd;
    unsigned char buf[RSIZE];
    long got = 0, intact = -1;
    ssize_t n, i;
    int mfd, sfd, status;
    pid_t pid;

    if ((mfd = posix_openpt(O_RDWR | O_NOCTTY)) < 0 ||
        grantpt(mfd) < 0 || unlockpt(mfd) < 0)
            exit(11);
    if ((sfd = open(ptsname(mfd), O_RDWR | O_NOCTTY)) < 0)
        exit(12);
    if (tcgetattr(sfd, &t) < 0)
        exit(13);
    cfmakeraw(&t);
    if (tcsetattr(sfd, TCSANOW, &t) < 0)
        exit(14);

    if ((pid = fork()) < 0)
        exit(15);
    if (pid == 0) {
        writer(mfd);
        exit(0);
    }

    /* The parent holds mfd open so the master stays open after the
     * writer exits. */
    nanosleep(&ts, NULL);
    pfd.fd = sfd;
    pfd.events = POLLIN;
    while (got < TOTAL && poll(&pfd, 1, IDLE_MS) > 0) {
        if ((n = read(sfd, buf, sizeof(buf))) <= 0)
            break;
        for (i = 0; i < n && intact < 0; i++)
            if (buf[i] != (got + i) % 251)
                intact = got + i;
        got += n;
    }

    if (waitpid(pid, &status, 0) < 0 || status != 0) {
        fprintf(stderr, "writer failed\n");
        return 2;
    }
    if (intact < 0)
        intact = got;
    printf("bytes written: %ld\n", TOTAL);
    printf("bytes read:    %ld\n", got);
    printf("intact before first loss: %ld\n", intact);
    return got == TOTAL && intact == TOTAL ? 0 : 1;
}

The exact loss seems to be a bit run-dependent.  For instance, I gathered these outputs:

amd64, qsize=1024 (default):
    bytes written: 1000000
    bytes read:    921978
    intact before first loss: 1024

amd64, qsize=65536:
    bytes written: 1000000
    bytes read:    945236
    intact before first loss: 65536

alpha, qsize=65536:
    bytes written: 1000000
    bytes read:    894806
    intact before first loss: 65536

The amd64 machine is running in a 4-CPU qemu+KVM VM.  The Alpha is running on bare metal, a 600MHz 21164a.  The amd64 one is significantly faster.  Perhaps that accounts for the fact that, despite encountering the first loss at the same spot, the Alpha dropped more bytes overall.

Different write sizes produce different loss patterns.  With a write size of 90, multiple runs of the test program produce loss at the same place.  With larger write sizes, they all produce loss but the location is less consistent.

I could not reproduce the problem with O_NONBLOCK and poll().  I suspect there is something in the blocking path that isn't working right.
>Fix:
I don't know