>>> [...] when reading from a network pipe as in popen.
>> A "network pipe"?  Pipes and network connections are actually quite
>> different in recent NetBSD.
> I refered to it was a network connection based on the following
> statements in the manual for popen() in NetBSD-2.0 beta that says it
> uses sockets.

Sockets != network connections.  Sockets were created largely to
support network connections, it's true, but the designers were smart
enough that they're not restricted to that.  AF_LOCAL sockets are
always host-local, by definition, and there are more esoteric forms of
sockets, such as AF_ROUTE (and my own AF_TIMER), which also are never
network connections.

In historical NetBSD, pipes did not really exist as distinct entities
in their own right; pipe() was a libc wrapper around socketpair(2) with
AF_LOCAL and SOCK_STREAM.  In modern NetBSD, pipes are completely
distinct entities from sockets of any stripe; they are a third kind of
thing an open file descriptor can refer to, coeval with sockets and
vnodes in this regard.

>     Historically, popen was implemented with a unidirectional pipe;
>     ...
>     Since popen is now implemented using sockets, the type may
>     request a bidirectional data flow.

This statement is somewhat out of date.  Checking -current's source, I
see that popen() uses a socketpair when the mode incldues a +
(requesting bidirectional data flow) and a pipe (which as I explain
above has nothing to do with sockets now) otherwise.

>> (The part that would require complicated explanation is why the
>> first write writes only 1024 bytes - and why later writes don't
> Yes, that was the part that puzzled me the most, is why that behavior
> was consistent.  I would think it would have been more random as to
> which reads were short.

Actually, "random" things like pipe buffering are quite often
surprisingly deterministic.  While it's certainly true that code cannot
assume very much about buffering of data flow thorugh pipes (or
sockets, actually), in that there are reasonably foreseeable
circumstances in which practically any imaginable breaking-up and
re-collapsing of data will occur, it's also true that if you do
basically the same thing twice you'll usually get basically the same
behaviour each time.

> If the developers feel this is normal behavior, I will try not to use
> any more bandwidth in the list on the issue :-).

It's normal behaviour in that it's not something the interfaces promise
won't occur, and thus for your code to be correct it has to be prepared
to encounter it.  Whether it's normal behaviour in the sense of being
what you can expect to most frequently encounter under diverse
circumstances, I can't say.

> However, if you know the cause or probable cause and can find the
> time to tell me through direct email or point me to some
> documentation that would explain it, I would appreciate it very much.

I don't.  1024 is so small - only a quarter of a page; you said in your
first post that this was on i386 - that I have trouble accounting for
it.  The most plausible explanation I can think of offhand is that the
writing program is writing a large buffer that starts 3/4 of the way
through a page, with the second page not in core.  What I'm imagining
is that first 1024 bytes are available immediately and get put in the
pipe, but to get more, it has to page data in off disk.  This takes
time, and the writing process is put to sleep while waiting for it.
There not being anything else to do, your reader runs and finds the 1K
waiting.  After that, the writer writes whole pages - 4K - or more, at
once, so the pipe always contains more than you try to read.

I don't know how closely that description corresponds to reality, nor
even possibility under the current implementation; it's pure
speculation on my part, trying to come up with something plausible that
would produce the observed behaviour.

/~\ The ASCII				der Mouse
\ / Ribbon Campaign
 X  Against HTML	       mouse@rodents.montreal.qc.ca
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B