Subject: bad filesystem performance
To: None <netbsd-help@netbsd.org>
From: Chris Jones <chris@cjones.org>
List: netbsd-help
Date: 06/10/2003 15:10:53
A few days ago, I noticed my NFS server getting really slow; I sent a 
message to this list at that time.  I'm starting to think it's a problem 
with the filesystem on the server, but I'm at a loss for how to really 
nail down the cause of it.

The machine is running raidframe across two IDE drives, wd1 and wd2. 
They're partitioned almost identically, the difference being that one 
has a boot partition, and the other has a swap partition that's online 
but never actually used -- it's there for dumps.

My complaint is that performance is really bad.  It's gotten 
dramatically worse in the past few days, though I can't really quantify 
it at this point.  Various CVS operations are taking many times longer 
than normal, and NFS performance is bad enough that the clients think 
the server's not responding at all.

The strange thing is the output of iostat.  The two drives, wd1 and wd2, 
are the only devices connected to an IDE controller.  The drives are 
identical, and they're being used in a RAID1 array.  But wd2 sees 2-3 
times as many accesses as wd1, and they're in smaller chunks.  If 
anybody has suggestions on where else to look, I'd appreciate it.

Chris


iostat output:

       tty            wd0             wd1             wd2 
fd0             cpu
  tin tout  KB/t t/s MB/s   KB/t t/s MB/s   KB/t t/s MB/s   KB/t t/s 
MB/s  us ni sy in id
    0   56  0.00   0 0.00  13.73  38 0.51   6.31  98 0.60   0.00   0 
0.00   3  0  6  1 90
    0   79  0.00   0 0.00  12.42  43 0.52   6.33  99 0.61   0.00   0 
0.00   1  0  5  1 94
    0   56  0.00   0 0.00  15.84  31 0.47   6.32  91 0.56   0.00   0 
0.00   0  0  4  1 95
    0   53  0.00   0 0.00  15.12  25 0.37   6.41  70 0.44   0.00   0 
0.00   0  0  2  1 97


uname -a:

NetBSD gamera.mt.sri.com 1.6 NetBSD 1.6 (GAMERA) #0: Mon Sep 23 16:47:30 
MDT 2002 
chris@legolas.mt.sri.com:/usr/src/sys/arch/i386/compile/GAMERA i386


mount -a:

/dev/raid0a on / type ffs (soft dependencies, local)
/dev/raid0f on /var type ffs (soft dependencies, local)
/dev/raid0e on /usr type ffs (soft dependencies, local)
/dev/wd1a on /altroot type ffs (soft dependencies, local)
mfs:135 on /tmp type mfs (local)
/dev/raid0h on /users type ffs (soft dependencies, NFS exported, local)
/dev/raid0g on /data type ffs (soft dependencies, NFS exported, local)
/dev/wd0e on /stage type ffs (soft dependencies, local)
kernfs on /kern type kernfs (local)


disklabel wd1 excerpt:

5 partitions:
#        size    offset     fstype  [fsize bsize cpg/sgs]
  a:    300384         0     4.2BSD   1024  8192    16   # (Cyl.    0 - 297)
  c: 156355584         0     unused      0     0         # (Cyl.    0 - 
155114*)
  d: 156355584         0     unused      0     0         # (Cyl.    0 - 
155114*)
  e: 156055200    300384       RAID                      # (Cyl.  298 - 
155114*)


swapctl -a:

Device      1K-blocks     Used    Avail Capacity  Priority
/dev/raid0b   1048576    10320  1038256     1%    0
/dev/wd2b      150192        0   150192     0%    1
Total         1198768    10320  1188448     1%


dmesg excerpt:

pciide1 at pci0 dev 14 function 0: Promise Ultra100/ATA Bus Master IDE 
Accelerator (rev. 0x02)
pciide1: bus-master DMA support present
pciide1: primary channel configured to native-PCI mode
pciide1: using irq 9 for native-PCI interrupt
wd1 at pciide1 channel 0 drive 0: <MAXTOR 6L080J4>
wd1: drive supports 16-sector PIO transfers, LBA addressing
wd1: 76345 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 156355584 
sectors
wd1: 32-bit data port
wd1: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6
wd1(pciide1:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using 
DMA data transfers)
pciide1: secondary channel configured to native-PCI mode
wd2 at pciide1 channel 1 drive 0: <MAXTOR 6L080J4>
wd2: drive supports 16-sector PIO transfers, LBA addressing
wd2: 76345 MB, 16383 cyl, 16 head, 63 sec, 512 bytes/sect x 156355584 
sectors
wd2: 32-bit data port
wd2: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 6
wd2(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using 
DMA data transfers)

-- 
Chris Jones               chris@cjones.org                www.cjones.org