Re: bsdcpio and bsdtar installed by default

[Joerg Sonnenberger <joerg%britannica.bec.de@localhost>]

On Thu, Jun 05, 2008 at 02:10:16PM -0400, Greg A. Woods; Planix, Inc. wrote:
> On 5-Jun-08, at 1:56 PM, Joerg Sonnenberger wrote:
>>
>> For gzip/libz based compression, I think the overhead of pipe is clearly
>> as large as the gain from running on different cores (e.g. sharing L2
>> caches).
>
> That may be a problem with pipes then, perhaps.

You have to copy all the data and deflate is quite efficient otherwise,
so the overhead of system calls is mostly eating the parallelism.

>> For my Core2 laptop (hw.ncpu=2):
>>
>> bsdtar xf libX11-1.1.3.tar.bz2
>> 1.18s user 0.12s system 99% cpu 1.305 total
>>
>> bsdtar --use-compress-program /usr/bin/bunzip2 -xf libX11-1.1.3.tar.bz2
>> 1.25s user 0.16s system 109% cpu 1.285 total
>
> Those numbers don't quite make sense to me.  Something looks (a little bit) 
> wrong.  The results seem to indicate the combined CPU use exceeded that 
> possible by one CPU alone.  There also seems to be a tiny savings in wall 
> clock time too.  However the user time is higher than I would expect for 
> the overhead of reading and writing through a pipe (from user-land alone).

Yeah, might be misaccounting on the shell builtin. Running the same test
with /usr/bin/time:
% /usr/bin/time bsdtar --use-compress-program bunzip2 -xf libX11-1.1.3.tar.bz2 
     1.38 real         1.22 user         0.15 sys
% /usr/bin/time bsdtar xf libX11-1.1.3.tar.bz2
     1.31 real         1.22 user         0.08 sys

For reference:
% /usr/bin/time bsdtar xf libX11-1.1.3.tar   
     0.14 real         0.03 user         0.11 sys
% /usr/bin/time bunzip2 -c libX11-1.1.3.tar.bz2 > libX11-1.1.3.tar
     1.20 real         1.16 user         0.04 sys

Essentially, the time is dominated by bzip2 and writing to disk directly
is the cheaper part.

Joerg