tech-userlevel archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: bsdcpio and bsdtar installed by default

On 5-Jun-08, at 1:56 PM, Joerg Sonnenberger wrote:

For gzip/libz based compression, I think the overhead of pipe is clearly
as large as the gain from running on different cores (e.g. sharing L2

That may be a problem with pipes then, perhaps.

For bzip2 the ratio changes somewhat, but it seems like the
overhead due to pipe usage and the additional cache trashing are mostly
compensating each other.

For my Core2 laptop (hw.ncpu=2):

bsdtar xf libX11-1.1.3.tar.bz2
1.18s user 0.12s system 99% cpu 1.305 total

bsdtar --use-compress-program /usr/bin/bunzip2 -xf libX11-1.1.3.tar.bz2
1.25s user 0.16s system 109% cpu 1.285 total

Those numbers don't quite make sense to me. Something looks (a little bit) wrong. The results seem to indicate the combined CPU use exceeded that possible by one CPU alone. There also seems to be a tiny savings in wall clock time too. However the user time is higher than I would expect for the overhead of reading and writing through a pipe (from user-land alone).

The former uses internal lbibz2 handling, the second call uses the
external program. Both cases to tmpfs. So while it is slightly faster to
use the external program, it means 10% more over all resource usage.

I would expect a small amount of resource usage overhead, but 10% does seem high.

What most concerns me here is the time spent in user-land -- it would be interesting to profile both processes in the 2nd case to see where the overhead really is.

For extraction to a real file system, the situation will change somewhat
as the operations are generally IO bound in that case.


                                        Greg A. Woods; Planix, Inc.

Home | Main Index | Thread Index | Old Index