Subject: Network to harddrive performance
To: None <current-users@netbsd.org>
From: Arto Huusko <arto.huusko@utu.fi>
List: current-users
Date: 02/25/2004 17:00:59
I am completely out of any ideas what is wrong with my setup here,
and I hope someone please could enlighten me.

The problem is this: receiving data from network and putting it
to a file on a harddisk is incredibly slow. Receiving to MFS
is very fast. But, receiving from a slower link: it doesn't
matter whether the data is discarder, goes to harddrive, or
MFS, or whatever; the speed is always about right. The situation
is the same when receiving from  external network (which is also
behind a slower link).

My setup is this: I've got an alphastation 500/266 with rtk NIC,
a 2GHz Athlon with 3c905c-tx, and a laptop running W2k with
some 10mbit NIC. The alpha is connected to the internet, and
both pcs are connected to the alpha, which does NAT for them.
The link to internet is 10mbit.

All the machines are connected to each other via a no-name
switch.

 1. Moving data from i386 to alpha is OK. The transfer rate
    with ftp is about 4mbytes per second. This is to a
    hard drive on the alpha. To MFS, the speed is under
    2mbytes per second(!? Why would this be slower?).

 2. Moving data from/to the 10mbit W2k machine to/from either
    the alpha or i386 is OK. The transfer rate is always
    near 1mbyte per second, which pretty much maxes out
    the 10mbit.

 3. Moving data from alpha to i386 is not OK.
    Using ftp to receive a file to MFS, and I get about 7mbytes
    per second.
    Using ftp/web browser/scp/whatever to receive a file to
    hard drive, I get under 100kbytes per second!

    (Downloading from alpha to i386 under windows, using
    web browser, I get about 500kbytes per sec; so Windows
    has the same problem)

 4. Moving data from external sources, e.g. ftp.funet.fi,
    to the i386, and the speed is what I would expect.
    It seems to be a bit slower than it should be:
    using ftp to receive from ftp.funet.fi to the alpha,
    and I usually get 700-800kbytes per second. The
    i386 behind alpha can make perhaps 500kbytes per second.

This whole situation seems completely crazy to me. I know
that I can get data from network to harddrive on the
i386 at least at the rate of 1mbytes per second, because
that's the speed I get when receiving from the slower
W2k machine. But when receiving from 100mbit link, all
it can do is under 100kbytes per second.

Also, since receiving to mem buffer (or MFS), I can get
7mbytes per sec, the NIC or switch or cabling is not to blame.

I have also tried different NICs: the alpha has also had
21140 NIC, and I've tried rtk in the i386 as well. The
results are the same.

The i386 runs 1.6ZJ, alpha 1.6ZG. I have also tried
1.6ZG and ZI GENERIC kernels on the i386.

What the hell is the problem here? Is this a hardware
issue?

My dmesg is:

NetBSD 1.6ZJ (LADY) #2: Thu Feb 12 15:37:15 EET 2004
	root@lady:/data/netbsd/current/i386/obj/sys/arch/i386/compile/LADY
total memory = 511 MB
avail memory = 498 MB
BIOS32 rev. 0 found at 0xfdad0
mainbus0 (root)
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: AMD Athlon XP 2400+ (686-class), 2000.19 MHz, id 0x681
cpu0: features c3c3fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR>
cpu0: features c3c3fbff<PGE,MCA,CMOV,PAT,PSE36,MMXX,MMX>
cpu0: features c3c3fbff<FXSR,SSE,3DNOW2,3DNOW>
cpu0: I-cache 64 KB 64b/line 2-way, D-cache 64 KB 64b/line 2-way
cpu0: L2 cache 256 KB 64b/line 16-way
cpu0: ITLB 16 4 KB entries fully associative, 8 4 MB entries fully associative
cpu0: DTLB 32 4 KB entries fully associative, 8 4 MB entries 4-way
cpu0: calibrating local timer
cpu0: apic clock running at 266 MHz
cpu0: 8 page colors
ioapic0 at mainbus0 apid 2 (I/O APIC)
ioapic0: pa 0xfec00000, version 3, 24 pins
acpi0 at mainbus0
acpi0: using Intel ACPI CA subsystem version 20031203
acpi0: X/RSDT: OemId <AMIINT,VIA_K7  ,00000010>, AslId <MSFT,00000097>
acpi0: SCI interrupting at int 9
acpi0: fixed-feature power button present
ACPI Object Type 'Processor' (0x0c) at acpi0 not configured
acpi: activated PNP0C0F
PNP0C0C at acpi0 not configured
PNP0A03 at acpi0 not configured
PNP0C02 at acpi0 not configured
PNP0200 at acpi0 not configured
PNP0100 at acpi0 not configured
PNP0B00 at acpi0 not configured
PNP0800 at acpi0 not configured
npx0 at acpi0 (PNP0C04)
npx0: io 0xf0-0xff irq 13
npx0: using exception 16
pckbc0 at acpi0 (PNP0303): kbd port
pckbc0: io 0x60,0x64 irq 1
fdc0 at acpi0 (PNP0700)
fdc0: io 0x3f2-0x3f5,0x3f7 irq 6 drq 2
com0 at acpi0 (PNP0501-1)
com0: io 0x3f8-0x3ff irq 4
com0: ns16550a, working fifo
com1 at acpi0 (PNP0501-2)
com1: io 0x2f8-0x2ff irq 3
com1: ns16550a, working fifo
lpt0 at acpi0 (PNP0401)
lpt0: io 0x378-0x37f,0x778-0x77b irq 7 drq 3
ACPI Object Type 'Power' (0x0b) at acpi0 not configured
ACPI Object Type 'Power' (0x0b) at acpi0 not configured
ACPI Object Type 'Power' (0x0b) at acpi0 not configured
ACPI Object Type 'Power' (0x0b) at acpi0 not configured
PNP0C0F at acpi0 not configured
PNP0C0F at acpi0 not configured
PNP0C0F at acpi0 not configured
PNP0C0F at acpi0 not configured
PNP0C0E at acpi0 not configured
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
pchb0 at pci0 dev 0 function 0
pchb0: vendor 0x1106 product 0x3189 (rev. 0x00)
agp0 at pchb0: aperture at 0xe0000000, size 0x10000000
ppb0 at pci0 dev 1 function 0: vendor 0x1106 product 0xb168 (rev. 0x00)
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled
vga0 at pci1 dev 0 function 0: vendor 0x10de product 0x0171 (rev. 0xa3)
wsdisplay0 at vga0 kbdmux 1: console (80x25, vt100 emulation), using wskbd0
wsmux1: connecting to wsdisplay0
ex0 at pci0 dev 6 function 0: 3Com 3c905C-TX 10/100 Ethernet with mngmt (rev. 0x78)
ex0: interrupting at ioapic0 pin 17 (irq 10)
ex0: MAC address 00:0a:5e:1d:63:5f
exphy0 at ex0 phy 24: 3Com internal media interface
exphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
pcib0 at pci0 dev 17 function 0
pcib0: vendor 0x1106 product 0x3177 (rev. 0x00)
viaide0 at pci0 dev 17 function 1
viaide0: VIA Technologies VT8235 ATA133 controller
viaide0: bus-master DMA support present
viaide0: primary channel configured to compatibility mode
viaide0: primary channel interrupting at ioapic0 pin 14 (irq 14)
atabus0 at viaide0 channel 0
viaide0: secondary channel configured to compatibility mode
viaide0: secondary channel interrupting at ioapic0 pin 15 (irq 15)
atabus1 at viaide0 channel 1
auvia0 at pci0 dev 17 function 5: VIA VT8235 AC'97 Audio (rev 0x50)
auvia0: interrupting at ioapic0 pin 22 (irq 12)
auvia0: ac97: Avance Logic ALC650 codec; 20 bit DAC, 18 bit ADC, Realtek 3D
auvia0: ac97: ext id 5c7<AC97_22,LDAC,SDAC,CDAC,SPDIF,DRA,VRA>
audio0 at auvia0: full duplex, mmap, independent
isa0 at pcib0
pcppi0 at isa0 port 0x61
sysbeep0 at pcppi0
ioapic0: enabling
wd0 at atabus0 drive 0: <WDC WD800BB-00CAA1>
wd0: drive supports 16-sector PIO transfers, LBA addressing
wd0: 76319 MB, 155061 cyl, 16 head, 63 sec, 512 bytes/sect x 156301488 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd0(viaide0:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA data transfers)
atapibus0 at atabus1: 2 targets
cd0 at atapibus0 drive 1: <_NEC DVD_RW ND-1300A, , 1.06> cdrom removable
cd0: 32-bit data port
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
wd1 at atabus1 drive 0: <Maxtor 52049H3>
wd1: drive supports 16-sector PIO transfers, LBA addressing
wd1: 19541 MB, 39704 cyl, 16 head, 63 sec, 512 bytes/sect x 40021632 sectors
wd1: 32-bit data port
wd1: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd1(viaide0:1:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using DMA data transfers)
cd0(viaide0:1:1): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33) (using DMA data transfers)
boot device: wd1
root on wd1a dumps on wd1b
root file system type: ffs
wsdisplay0: screen 1 added (80x25, vt100 emulation)
wsdisplay0: screen 2 added (80x25, vt100 emulation)
wsdisplay0: screen 3 added (80x25, vt100 emulation)
wsdisplay0: screen 4 added (80x25, vt100 emulation)

   -- Arto