One thing I noticed with recent kernels is that ata disk performance
has decreased markedly on one of my machines, supposedly the fastest
one of three fairly equivalent machines. Two drives (seagate ATA-IV)
are essentially identical while the other (wd) is roughly the same
performance. Disk perf on the machine in question is noticably
slower and bonnie results appear to agree.  All three machines are
running the same -current build (one is an MP) on roughly the same
speed cpus with sysctl settings as mentioned recently:

kern.maxvnodes=131072
vm.bufcache=5

Note that all three run slightly into swap with the new kernels,
even with the above changes, when they used to *never* run into
swap. Two have .5G and one has 1G ram...

#1)

old bonnie results:
    -------Sequential Output-------- ---Sequential Input-- --Random--
    -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
 MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
100 45140 59.8 55066 42.7 53620 50.8 73079 84.2 359877 92.3 12769.8 65.3

new:
    -------Sequential Output-------- ---Sequential Input-- --Random--
    -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
 MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
100 25066 22.6 23989  7.3 29172 10.4 95010 96.2 415142 85.9 11538.2 45.3

Any ideas as to why disk performance on this supposedly faster
machine has decreased with recent -current kernels? Note that %cpu
has drastically decreased on the Sequential Output sections, along
with performance.

-----------------
#1)
NetBSD 1.6ZG (APMLM) #0: Wed Jan  7 17:25:13 CST 2004
total memory = 511 MB
avail memory = 493 MB
...
cpu0 at mainbus0: (uniprocessor)
cpu0: AMD Athlon XP 2200+ (686-class), 1830.48 MHz, id 0x681
...
pchb0: VIA Technologies VT8377 Apollo KT400 CPU to PCI Bridge (rev. 0x00)
...
pcib0 at pci0 dev 17 function 0
pcib0: VIA Technologies VT8235 (Apollo KT400) PCI-ISA Bridge (rev. 0x00)
viaide0 at pci0 dev 17 function 1
viaide0: VIA Technologies VT8235 ATA133 controller
viaide0: bus-master DMA support present
viaide0: primary channel configured to compatibility mode
viaide0: primary channel interrupting at irq 14
atabus0 at viaide0 channel 0
viaide0: secondary channel configured to compatibility mode
viaide0: secondary channel interrupting at irq 15
atabus1 at viaide0 channel 1
...
wd0 at atabus1 drive 0: <ST340016A>
wd0: drive supports 16-sector PIO transfers, LBA addressing
wd0: 38166 MB, 77545 cyl, 16 head, 63 sec, 512 bytes/sect x 78165360 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd0(viaide0:1:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA data transfers)

    -------Sequential Output-------- ---Sequential Input-- --Random--
    -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
 MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
100 25066 22.6 23989  7.3 29172 10.4 95010 96.2 415142 85.9 11538.2 45.3

-----------------
#2)
NetBSD 1.6ZG (APMLM) #0: Wed Jan  7 17:25:13 CST 2004
total memory = 511 MB
avail memory = 493 MB
...
cpu0 at mainbus0: (uniprocessor)
cpu0: AMD Athlon XP 2100+ (686-class), 1764.17 MHz, id 0x662
...
pchb0: Advanced Micro Devices AMD761 System Controller (rev. 0x13)
pcib0 at pci0 dev 7 function 0
pcib0: VIA Technologies VT82C686A (Apollo KX133) PCI-ISA Bridge (rev. 0x40)
viaide0 at pci0 dev 7 function 1
viaide0: VIA Technologies VT82C686A (Apollo KX133) ATA100 controller
viaide0: bus-master DMA support present
viaide0: primary channel configured to compatibility mode
viaide0: primary channel interrupting at irq 14
atabus0 at viaide0 channel 0
viaide0: secondary channel configured to compatibility mode
viaide0: secondary channel interrupting at irq 15
atabus1 at viaide0 channel 1
...
wd0 at atabus0 drive 1: <ST380011A>
wd0: drive supports 16-sector PIO transfers, LBA48 addressing
wd0: 76319 MB, 155061 cyl, 16 head, 63 sec, 512 bytes/sect x 156301488 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd0(viaide0:0:1): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA data transfers)

    -------Sequential Output-------- ---Sequential Input-- --Random--
    -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
 MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
100 54090 74.5 54430 50.1 46070 49.8 81885 94.6 369656 96.1 12027.5 63.0

-----------------
#3)
NetBSD 1.6ZG (GENERIC.MP) #0: Wed Jan  7 14:55:09 CST 2004
total memory = 1023 MB
avail memory = 994 MB
...
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Athlon MP 2000+ (686-class), 1666.80 MHz, id 0x662
cpu1: starting
cpu1: AMD Athlon MP (686-class), 1666.73 MHz, id 0x662
...
pchb0: Advanced Micro Devices AMD762 NorthBridge (rev. 0x11)
pcib0 at pci0 dev 7 function 0
pcib0: Advanced Micro Devices AMD768 PCI to ISA/LPC Bridge (rev. 0x05)
viaide0 at pci0 dev 7 function 1
viaide0: Advanced Micro Devices AMD768 IDE Controller (rev. 0x04)
viaide0: bus-master DMA support present
viaide0: primary channel configured to compatibility mode
viaide0: primary channel interrupting at ioapic0 pin 14 (irq 14)
atabus0 at viaide0 channel 0
viaide0: secondary channel configured to compatibility mode
viaide0: secondary channel interrupting at ioapic0 pin 15 (irq 15)
...
wd0 at atabus0 drive 0: <WDC WD400BB-00DEA0>
wd0: drive supports 16-sector PIO transfers, LBA addressing
wd0: 38166 MB, 77545 cyl, 16 head, 63 sec, 512 bytes/sect x 78165360 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd0(viaide0:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA data transfers)

    -------Sequential Output-------- ---Sequential Input-- --Random--
    -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
 MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
100 38386 44.5 47806 24.1 47475 32.0 78156 99.8 248311 99.9 12062.0 193.9