Re: kern/59175: posix_spawn hang, hanging other process too

["Taylor R Campbell via gnats" <gnats-admin%NetBSD.org@localhost>]

The following reply was made to PR kern/59175; it has been noted by GNATS.

From: Taylor R Campbell <riastradh%NetBSD.org@localhost>
To: Thomas Klausner <wiz%NetBSD.org@localhost>
Cc: gnats-bugs%NetBSD.org@localhost, netbsd-bugs%NetBSD.org@localhost,
	martin%NetBSD.org@localhost, chs%NetBSD.org@localhost
Subject: Re: kern/59175: posix_spawn hang, hanging other process too
Date: Fri, 14 Mar 2025 14:28:38 +0000

 So here's what I think is happening:
 
 1. do_posix_spawn in the parent p1 creates a process p2 with
    proc_alloc, which has:
    - p2->p_stat = SIDL (can't be looked up with proc_find)
    - p2->p_vmspace = &vmspace0 (initial vmspace, kernel's) or NULL
    - p2->p_psstrp = garbage from recycled struct proc or NULL
 
    In particular, if p2 is freshly allocated, it will have null
    p_vmspace and p_psstrp; if it was recycled, p_vmspace will have
    been set to proc0.p_vmspace=&vmspace0 shortly before recycling
    in uvm_proc_exit:
    https://nxr.netbsd.org/xref/src/sys/uvm/uvm_glue.c?r=1.182#414
    And p_psstrp will have been left untouched.
 
 2. do_posix_spawn in the parent creates a lwp in p2 to run
    spawn_return, sets p2->p_stat = SACTIVE (so it _can_ be looked up
    with proc_find), makes the lwp runnable, and waits for it to ack
    (sed_cv_child_ready).
 
 3. spawn_return in the child p2 acks immediately (sed_child_cv_ready)
    because neither POSIX_SPAWN_RETURNERROR nor POSIX_SPAWN_SETPGROUP
    is set.
 
 4. do_posix_spawn in the parent returns the child pid to the caller.
 
 5. parent tries sysctl kern.proc_args.<childpid>.argv or something
    which takes p2->p_reflock and does copyin_psstrings(p2, ...) which
    copies in p2->p_psstrp (garbage from recycled struct proc) from
    p2->p_vmspace (&vmspace0, kernel's).
 
    I'm not sure exactly what will happen here but it seems to send the
    kernel into a loop; in any case, whatever the symptom is, nothing
    good can come of attempting copyin_psstrings(p2, ...) before
    p2->p_vmspace and p2->p_psstrp are initialized.
 
 6. spawn_return in the child waits for p2->p_reflock before it can
    call execve_runproc to initialize p2->p_vmspace and p2->p_psstrp.
 
 In step (5), the parent gets stuck in a loop holding p2->p_reflock
 because p2->p_vmspace and p2->p_psstrp aren't initialized, while
 concurrently in step (6) the child is stuck waiting for p2->p_reflock
 before it can initialize p2->p_vmspace and p2->p_psstrp -- deadlock.
 
 (The window of time between (3) and (6) is pretty small, so the parent
 -- or any other process like a concurrent htop(1) -- has to be pretty
 quick about invoking sysctl during that window to win the race.)
 
 
 I think the general problem here is that the pid should not be usable
 by anything outside (like ps(1) or top(1), via sysctl kern.proc_args
 and stuff, in turn via proc_find) until execve_runproc has fully
 initialized the vmspace and psstrp.
 
 But I don't know if the child process can even run before p2->p_stat
 is set to SACTIVE, at which point it can be looked up with proc_find.
 Maybe we can safely defer the transition SIDL -> SACTIVE to
 spawn_return, if the lwp can run even if the process is not SACTIVE.
 And we should audit fork1 vs do_posix_spawn to make sure everything
 that fork1 initializes is also initialized by do_posix_spawn by the
 time of the SIDL -> SACTIVE transition.
 
 Separately, the parent can't return the child pid to the caller until
 that's done and the pid is usable (e.g., for waitpid(2) or kill(2)).
 So the early parent-wakeup path in spawn_return is probably just not
 workable.