This is something that I found odd, but I don't have time right now to
delve into...  I was playing around with progress(1) today to report a
display glitch to someone and found something odd...

I'm running on a 2-way 3.2GHz Xeon with HT enabled, so 4 virtual CPUs,
each with 2MB L2 cache, running NetBSD/amd64 -current as of this morning.
2GB of RAM installed.

(1) dd if=/dev/zero count=64k bs=64k | progress -l 4g dd of=/dev/null bs=64k

gets me about 95MB/s.  It looks like progress(1) uses a 1k buffer for
reading and writing, which is kind of lame for bulk data but not really
a huge deal.

(2) dd if=/dev/zero count=64k bs=64k of=/dev/null

on this platform reports about 5.2GB/s.  With the default setup,

(3) dd if=/dev/zero count=64k bs=64k | dd of=/dev/null bs=64k

reports about 385MB/s, which seems a bit low.

Matt Green, in private communication suggested trying:

# sysctl -w kern.pipe.maxloankvasz=0
kern.pipe.maxloankvasz: 16777216 -> 0

This improves (3) by nearly 5x (to 1.5GB/s).

So I added a -b buffersize option to progress(1) to play with the
amount of data read or written (I suppose I could have just used
the block size from dd, but this is also closer to my original
setup).  Before the sysctl, I see something like:

-b 4k	180MB/s
-b 7k	220MB/s
-b 8k	126MB/s		<- page loaning kicks in
-b 32k	181MB/s
-b 64k	195MB/s

After the sysctl, I see something like:

-b 4k	272MB/s
-b 7k	452MB/s
-b 8k	399MB/s
-b 32k	719MB/s
-b 64k	1.01GB/s

The numbers fluctuate a bit, but those are about the relative sizes.

Anyway, I think this points to something funny going on with the page
loan code for pipes.  I don't know if this is exacerbated by running
MP or what.  Obviously something is suboptimal with the default
configuration choices here.

Does anyone want to look into this further and try to figure out what's
going on?  I don't have time to look at it anytime soon and it would
be nice to track down the slowdown here or have more performant default
settings.

-allen

-- 
Allen Briggs  |  http://www.ninthwonder.com/~briggs/  |  briggs@ninthwonder.com