[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]
slow pipes chewing up system time (and real time)?
I have finally got around to modifying the Mercurial CVS conversion tool so
that it handles the NetBSD CVS repositories and I would like to run it on
NetBSD to provide daily incremental repository updates, but I can't because the
NetBSD performance is so bad.
I did all my development work on OS X because that's the biggest box I own and
converting the src repository requires about 13GB of RAM. When I moved to
running the tool on NetBSD, I started with the othersrc repository because that
makes testing somewhat more manageable. My problem is that the conversion is
five times slower on NetBSD than it is on OS X in real time even though CPU
usage appears to be comparable.
My Mac is 3.4GHz Intel Core i7 with 4 cores and one extra virtual hyper thread
core for each real core. The OS X 10.8.2 is run from an external thunderbolt
attached RAID 0 disk pair and the data I'm working with is on the internal WD
Caviar Black formatted with a case sensitive journaled HFS+ file system. The
cvs server uses a RAM disk for it's temporary storage.
My NetBSD box is a 3.3GHz AMD Phenom II with 6 cores. NetBSD 6.0 is run from
an AHCI attached SSD (an old SSD) and the data I'm working with is variously on
tmpfs or the SSD. The SSD file systems are formatted with FFSv2 without
logging, with noatime, and for the designated destination file system, with
async. The use of tmpfs or async doesn't make an obvious change to performance.
"time -l" on the Mac and NetBSD give me similar user CPU and system CPU numbers
for the conversion of othersrc (half a minute and five seconds respectively),
but real time on the Mac is 68s and on NetBSD it is 359s. Even though the
reported system time is only five seconds, when I run top on NetBSD, one of the
CPUs just sits there at 100% system.
I don't think the horrible wall clock time is caused by hardware constraints
(disk performance etc.) because I have tested this on tmpfs with no change and
after running ktrace I'm a bit suspicious of the pipe performance. I can't see
a way to find out how long each system call is taking, so I don't know for sure
though. The application talks to the cvs server with a stop and wait
implementation of the cvs protocol and the workload is a lot of small requests.
I see support in NetBSD for pipes handling lots of large requests and I'm
wondering is anyone knows of any problems with NetBSD pipes handling lots of
Main Index |
Thread Index |