toolchain/58748: atf should have a background process supervisor

[campbell+netbsd%mumble.net@localhost]

>Number:         58748
>Category:       toolchain
>Synopsis:       atf should have a background process supervisor
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    toolchain-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Oct 12 21:25:00 +0000 2024
>Originator:     Taylor R Campbell
>Release:        current
>Organization:
The NetBSD Foundatfzombies
>Environment:
>Description:
We have a lot of tests that:

1. run a daemon in the background, such as a rump server or an http server;
2. do various tests that interact with the daemon; and then
3. on cleanup, kill the daemon via pidfile or something.

And we have some long-standing PRs about tests that fail to clean up after themselves because this process is not actually very reliable:

https://gnats.netbsd.org/48892 some tests will not clean up rump server processes
https://gnats.netbsd.org/56770 rump threxec test randomly hangs the rump server process

We should 
>How-To-Repeat:
write and diagnose broken tests that use daemons
>Fix:
Teach atf-run to spawn subprocesses on request (in a process group or session or whatever), with an intermediate process whose only job is to send SIGKILL to the subprocesses on exit.

There should be a new ATF API call in C that works like posix_spawn, and a new ATF API call in sh that takes a command line and executes it as a daemon.  It should also have some way to coordinate with daemon processes so that it doesn't return until the daemon has finished setup, e.g. binding to sockets.