NetBSD-Users archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

Re: vm.bufcache and git



  From: Thor Lancelot Simon <tls%panix.com@localhost>
  Subject: Re: vm.bufcache and git
  To: Greg Troxel <gdt%ir.bbn.com@localhost>
  Cc: netbsd-users%netbsd.org@localhost
  Date: Mon, 24 May 2010 14:18:30 -0400

  On Mon, May 24, 2010 at 01:24:06PM -0400, Greg Troxel wrote:
  > 
  >   Is each inode in the buffer cache really 512 bytes?

  Oh, it's larger than that.  It's at least one on-disk fragment.  If you
  use tmpfs, it's at least one *page*.

  Systems like nbanoncvs need 4k/512 filesystems and an added buffer cache
  pool for 512 byte items -- by default the smallest pool is 1K.


Thanks - that makes sense - I had forgotten that of course the
traditional cache is caching disk blocks.


I consider making vm.bufcache big enough most of the solution.  Then, we
get down to comparison of speed vs other systems.

"git config core.preloadindex true" enables some parallelism.  I don't
understand exactly, but it helps.

On NetBSD 5 amd64, git status in a repo with ~130K files takes about
1.8s (preloaded cache of course, with preloadindex true, and 2.5s
without preloadindex).  On similar hardware with Linux, it takes about
0.85s.  Most of the time on NetBSD seems to be in stat.

On a 2.5 year old mac laptop, status takes about 7 s, comparable to a
similar vintage netbsd desktop.

Given all that, NetBSD's performance seems reasonable, but Linux stat is
very fast, and any speedups we can make to stat would help git
performance with NetBSD-sized repos.


Can anyone point me to a stat microbenchmark?  "lat_syscall stat" from
lmbench works, but I think the key is rotating stat over large numbers
of files such that the buffer cache working set no longer fits in L1/L2
cpu cache.  What I'd like is a program that gets a variable-sized list
of files to stat and times N loops over that list, just doing stat.
Then I could plot stat time in us vs log of number of files stated or
something like that.





Attachment: pgpYA1lQVXKPa.pgp
Description: PGP signature



Home | Main Index | Thread Index | Old Index