lib/55836: popen(3)'s type "r+" doesn't work correctly

[Michael Scholz <mike%fth-devel.net@localhost>]

>Number:         55836
>Category:       lib
>Synopsis:       popen(3)'s type "r+" doesn't work correctly
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Dec 01 23:25:00 +0000 2020
>Originator:     Michael Scholz
>Release:        NetBSD 9.99.76
>Organization:
>Environment:
System: NetBSD pumpkin.fth-devel.net 9.99.76 NetBSD 9.99.76 (PUMPKIN) #0: Mon Nov 30 10:24:54 CET 2020 root%pumpkin.fth-devel.net@localhost:/usr/obj/usr/src/sys/arch/amd64/compile/PUMPKIN amd64
Architecture: x86_64
Machine: amd64
>Description:

While I was playing with read/write pipes with popen(command, "r+") I
realized that this doesn't work as expected.  The plus triggers creating
the pipe object with socketpair(2) instead of pipe2(2) in
src/lib/libc/gen/popen.c, but socketpair doesn't seem to work properly.
The test program below, executed with ktrace, shows, after vfork, dup2
and some mmaps, that one read call is made but it never gets something
back and has to be stopped with Ctrl-C or so.

Mike

PS:  This may correspond to PR kern/55690 where problems with C-compiler
option -pipe appeared and the kernel was compiled with options
PIPE_SOCKETPAIR.

>How-To-Repeat:

% cat ./popen_t.c

#include <stdio.h>
#include <stdlib.h>
#include <err.h>

int
main(void)
{
	char		buf[BUFSIZ];
	FILE	       *fp;

	if ((fp = popen("cat", "r+")) == NULL)
		err(EXIT_FAILURE, "popen");

	while (fgets(buf, sizeof(buf), fp) != NULL)
		fputs(buf, stdout);

	if (pclose(fp) == EOF)
		err(EXIT_FAILURE, "pclose");

	return EXIT_SUCCESS;
}

/* end of popen_t.c */

Compile it to -o popen_t and

% ls /bin | ./popen_t

shows only the expected result if the popen type is "r"; set type to
"r+" and no output will appear.

>Fix: