Subject: Re: FreeBSD Bus DMA
To: Jason Thorpe <thorpej@nas.nasa.gov>
From: Justin T. Gibbs <gibbs@plutotech.com>
List: tech-kern
Date: 06/11/1998 21:33:06
>I'm only going to make the following comments on this thread:
>
>	(1) NetBSD invented the interface.  (It was primarily me, plus
>	    the input of some others who are Very familier with all
>	    of the issues.  See the manual page for the canonical list.)
>
>	(2) NetBSD did the first, second, third, etc. implementations.
>	    (Our underlying implementation continues to evolve, and
>	    there has been exactly one interface change, inspired by
>	    the fact that we run on a distributed parallel supercomputer.)
>
>	(3) NetBSD has widely deployed the bus_dma interface AS IT IS,
>	    before FreeBSD even considered adopting it.

Agreed.

>	(4) You are proposing changing an interface we have widely deployed.

I'm not proposing anything.  I'm telling you how FreeBSD has changed the
interface.  What happens from there is anyones guess.

>	(5) You have shown no concrete win for doing so.  All you can say
>	    is that you think that it's better.

I believe I have made a compelling case, along with real world numbers
comparing the NetBSD and FreeBSD implementations, for why allowing
deferred allocations is a win.  I have also given concreate numbers
showing the space savings possible for implementations that don't need
to keep dm_segs like data in the bus_dmamap_t object.  You can also infer
that a system that uses substantially less memory for bus dma glue (on the
order of MBs of data) will achieve better performance simply from having
additional free pages to apply to other applications.  Take these arguments
as you will.  I lose nothing if you disagree.

>I entertained your ideas during the year-long design phase of bus_dma,
>and when I raised technical based concernes (based on my experience working
>with the NetBSD kernel on a _WIDE_ variety of architectures), you not only
>refused to address them, but failed to demonstrate any real gain to be had.

I don't have any email archives to prove otherwise so lets assume that this
is the case.  The changes in the FreeBSD implementation occurred after this
review process and were communicated to you in the 12/97-1/98 time frame.
Whether you believe that there is a gain to be had is something only you
can decide.  I let my ideas stand on their own merit.

>(I.e. I don't dispute that you could save a small amount of memory by
>eliminating the array of segments within the DMA map, but you never
>once quantified HOW much, and ignored the fact that it can actually
>lead to waste and inefficiency on systems where you have no choice
>but to have a static copy of that information.)

In the recent discussions on this list, I did quantify exactly "how much"
of a difference in space can be saved.  As Matt Thomas has already pointed
out, if an implementation needs to store this data in the dma map, at most
this change is a wash.  Where are the performance penalties of making the
map opaque?

>Why should we change our interface based on your theories?

Did I ask you to?

>We have proof
>that our interface works, is adequate, easy to use, etc.  It was a clean
>win over the old one, because the old one _didn't work_ on many of our
>platforms.

I never stated that it "didn't work".  I merely contend that it "wastes 
space" - in fact the NetBSD interface would have wasted a considerable
amount of space over and above what the old bounce buffer code in FreeBSD
consumed.

> > I must say that it still amazes me how much this group lets its emotions
> > get in the way of progress.  Is it so inconceivable that a piece of code
>
>No, it's not emotions.  It has nothing to do with "refusing to cooperate".
>It's called common sense and good engineering.  Until you can demonstrate a
>CLEAR BENEFIT to changing a perfectly adequate interface that we have already
>widely deployed, why should we entertain the idea of changing?

I don't really care wether NetBSD adopts the change or not.  It may prove
that your deployment of the old interface is so entrenched as to make this
unfeasable.  What I do care about, are concreate examples of why this
modified interface doesn't work.

>FreeBSD _should_ have adopted the interface unchanged, gained some experience,
>and then consider changing the interface (or not!), and show a definite win by
>doing so.

Which is easier?  Fixing a deficiency in an interface as you deploy it?  or
deploying an interface with a deficiency and going in after deployment to
fix the problem?

>But that didn't happen.  Instead you're saying:
>
>	We changed the interface.  Here's a vague outline of how.  We
>	think it's better.  Now you're going to have to live with it.

I'm sorry that hurts your feelings so much.

>I'm even told you refused to write a document detailing all of the exact
>differences, and how to port a driver from one to another.

Ha.  Okay Jason, it's t-minus 6 days to your USENIX presentation on a large
body of non-busdma related code, you're working non-stop on polishing up
your slides and your on-line documentation between flame baits on tech-kern
and someone pops there head in and says, "did you folks at least document
clearly the changes you made from the NetBSD interfaces?"  I thought my
response was quite reasonable:

"Not yet.  I'm the only person to use them so far and I have been 
 concentrating on documenting the new SCSI code.  The bus dma
 interfaces will be documented at some point, its just a question of
 having the time, not the inclination."

Unfortunately I don't have the luxury of having my pay work sponsor bus
dma activities like some.

>I don't think asking for concrete justification is unreasonable.

I don't think it is either.  I have attempted to give it and you have
decided that it's not good enough.  I'm not going to press the issue.

>And,
>until you provide it, I'm not really willing to consider changing the
>interface in any way that you suggest.

How open minded of you.

>Jason R. Thorpe                                       thorpej@nas.nasa.gov
>NASA Ames Research Center                            Home: +1 408 866 1912
>NAS: M/S 258-5                                       Work: +1 650 604 0935
>Moffett Field, CA 94035                             Pager: +1 650 428 6939

--
Justin