[ On Monday, May 10, 1999 at 12:19:58 (-0700), Erik E. Fair wrote: ]
> Subject: Re: uvm and/vs splice() (was: re: Ongoing projects)
>
> Dennis Ritchie's motivation when designing "streams" was to replace TTY
> line disciplines, because they are incompletely specified, hard to
> implement and get right. He never intended that "streams" be used for a
> general-purpose networking API as done in System V; that was a raised
> middle index finger to the BSD community by the idiots at USG/USL - they
> took "streams" and made "STREAMS" (occasionally known as "SEWERS") just to
> be different from the sockets API and thereby FUD the UNIX market where
> networking was concerned.

That doesn't seem to be entirely true....

I find tonight to my delight that dmr's "streams" paper ("A Stream
Input-Output System" by Dennis M. Ritchie) is directly available off his
home page:

	http://cm.bell-labs.com/cm/cs/who/dmr/st.html

This, the first sentence of the abstract of the above, should show
clearly that "streams" were considered for networks as well as line
disciplines, right from the start:

          In a new version of the Unix operating system, a flexible
          coroutine-based design replaces the traditional rigid
          connection between processes and terminals or networks.

There are of course lots of other references to implementing networking
protocols with streams in that paper too.

David Presotto & dmr's paper on 9th edition streams, describing the
second implementation of streams (the first was in 8th edition) is also
available online, and it shows how the idea of streams is extended and
complimented by some other low level tools to make IPC convenient
regardless of whether the processes are running on the same or different
(networked) machines:

	http://cm.bell-labs.com/cm/cs/who/dmr/ipcpaper.html

(originally published in the Proceedings of the 10th Usenix Conf.,1985)

This is where streams start to really show their strengths and by the
time streams were re-designed for Plan 9 (Bell Labs CSTR 158e, also
published as: D. Presotto, ``Multiprocessor Streams for Plan 9'', UKUUG
Proc. of the Summer 1990 Conf., London, England, 1990) they were firmly
embedded as part of the networking architecture.  Plan 9 of course takes
an entirely different and much more "unix"-like approach to networking
APIs and does everything through files with simple ascii commands (which
I think beats both sockets and TLI/XTI hands down!).

In the end "UNIX System V STREAMS" are still really not all that
different from dmr's original proposal.  The streams modules provided by
SysV for use with them are of course quite different.  I think it's the
TLI library which you're turning your nose up at.  TLI is just a user-
level API for communcating with the streams modules that implement
various transport protocols, such as UDP and TCP.  TLI (and it's
standardized descendant X/Open's XTI) has a very good reason to exist as
it does.  It's only real failing is that it wasn't the same as what
every existing Unix networking programmer and program was familiar with
and coded in.  It's not necessarily bad to be different, especially when
the differences allow for a better fit on a "new" paradigm.  If there
were a perfect system then one might hope that it would prevail and all
the world would adopt it universally, but until that happens some degree
of diversity is an exellent thing to have.

You'll note in the progression from the original streams, regardless of
which fork you take (i.e. the research group or the commercial folks),
streams has always inspired its developers and users to implement novel
new APIs for IPC and networking and you could say that as a result
they've *all* thumbed their noses to some degree at BSD sockets.
Interestingly though the commercial implementors were able to show how
streams could easily and transparently accomodate a sockets API
simultaneously.

BTW, there's also similar work mentioned in the Plan 9 Streams technical
report:

    Larry Peterson "RPC in the X-Kernel: Evaluating New Design
    Techniques" in Proc. Twelfth Symposium on Operating Systems
    Principles; Litchfield Park, AZ, Dec. 1989

I've yet to find a copy of this paper [though I've not yet searched the
ACM digital library], so if anyone has a link I'd much appreciate it.

The Plan 9 streams are also briefly discussed in context with the full
networking system in:

	http://plan9.bell-labs.com/plan9/doc/net.html

(and notably the above paper recommends Peterson's X-Kernel
implementation as possibly superior)

> Where I came into the "splice()" discussion was when Chris Demetriou
> pointed out that such a system call would not be able to do data transforms
> as required by various protocols (e.g. UNIX ASCII to NETASCII for FTP &
> SMTP, TELNET byte stuffing, etc), and I suggested "streams" as an answer to
> that objection. The model I had in my head was pushing a streams module on
> top of one of the file descriptors, and then calling "splice()" to move the
> data with the transform. I don't remember if dmr ever generalized the
> "streams" model to talk to anything other than devices.

Hmmm.... I kind of like that idea of having all file descriptors be
streams end points and thus potentially allow streams modules to be
pushed onto them!

-- 
							Greg A. Woods

+1 416 218-0098      VE3TCP      <gwoods@acm.org>      <robohack!woods>
Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>