Subject: Lousy filesystem performance, or is it my motherboard?
To: None <current-users@netbsd.org, port-i386@netbsd.org>
From: Thor Lancelot Simon <tls@cloud9.net>
List: current-users
Date: 11/30/1994 13:05:33
After months of piddling around with awful IDE disks on my machine at home,
I went and plugged in the BT445S I'd bought with the machine, and bought
myself a Micropolis 1-gig disk. (roughly equivalent to the ST312000N/
Fujitsu 2694/et al.; a 5400RPM, 8ms drive).
The first thing I noticed was that performance didn't seem much better than
it had been with the IDE disk, from a subjective point of view. So, I went
and did some benchmarking with iozone.
On a type 2 UFS, reads maxed out around 500K/s, and writes around 600K/s,
once I was writing files big enough for the cacheing not to matter. This
seemed Ungood. One of my clients has a machine running FreeBSD 1.1R, with
an Adaptec 1742, and the same Micropolis drive I have, so I went over there
and ran iozone again.
Performance was _much_ better. On his machine, I got about 2.5MB/s write, and
1MB/s read. (figures are from memory -- actual iozone data appended below.)
I wasn't sure if this was a consequence of the operating system, the
controller, or perhaps that he has a reasonably high-quality EISA motherboard,
and I have an el-cheapo UMC VESA. Both machines are 486/66 with 256k cache;
to make things even scarier, I did my tests on my machine in single-user
mode, but his is a busy news server, and I was running iozone on the disk the
news spool is on!
So I went home and realized I could likely localize the problem by running
iozone directly on the device. I did that with both the raw and block
devices; the block device gave somewhat better performance when everything
was just going into and out of cache, but at larger file sizes exhibited a
strange behaviour in which performance was reduced by an order of magnitude
when the record size was 4096. I assume this has something to do with the
page size and the buffer cache? In general, of course, the raw device
performed better.
My raw device scores were respectable -- 4MB/s write and 3MB/s read. To
me, this would seem to indicate that the problem is in the UFS code somewhere
and not with my motherboard/SCSI controller. Unfortunately, I wasn't able
to wipe out a partition on the FreeBSD machine to get raw scores for that
os/motherboard/scsi controller.
All my test data is appended below. If anyone has any idea what's going on
here, I'd love to know.
I'm running NetBSD1.0 (1.0A now, but the scores are no better :-( ), with
type 2 filesystems. Incidentally, while futzing around with this I think
I found an spl problem in the LFS code, if anyone cares. Oh -- and one other
thing -- I noticed that the disklabel thought my disk was 3600RPM (the install
script should really *ask*, no? Most new disks are 5400RPM at least by now),
but newfs-ing the filesystem in question with -r 5400 didn't help any.
IOZone was built with -DUSE_FSYNC.
-- IOZone data for NetBSD 1.0A, UFS
IOZONE: Performance Test of Sequential File I/O -- V2.01 (10/21/94)
By Bill Norcott
Operating System: POSIX 1003.1-1990
IOZONE: auto-test mode
MB reclen bytes/sec written bytes/sec read
1 512 1978445 294543
1 1024 2833989 284166
1 2048 3495253 281119
1 4096 4032984 278136
1 8192 5518821 504123
2 512 779610 499321
2 1024 896218 502914
2 2048 525602 496955
2 4096 533626 494611
2 8192 554802 498135
4 512 586615 494611
4 1024 607870 494611
4 2048 623224 494611
4 4096 632625 497544
4 8192 647269 494611
8 512 531934 494903
8 1024 539460 494903
8 2048 542600 492867
8 4096 545068 493447
8 8192 550433 494319
16 512 536527 495634
16 1024 536699 494611
16 2048 539807 494611
16 4096 542951 494028
16 8192 541375 495634
Completed series of tests
--IOZone data for FreeBSD 1.1R, UFS
IOZONE: Performance Test of Sequential File I/O -- V2.01 (10/21/94)
By Bill Norcott
Operating System: POSIX 1003.1-1990
IOZONE: auto-test mode
MB reclen bytes/sec written bytes/sec read
1 512 1066348 635500
1 1024 1429876 3932159
1 2048 2029501 1367707
1 4096 2419790 599186
1 8192 3311292 1187067
2 512 1113532 905245
2 1024 1850428 817072
2 2048 2374134 1066348
2 4096 2419790 526481
2 8192 3400787 344737
4 512 960527 401368
4 1024 1981560 809190
4 2048 2542002 731564
4 4096 1784810 1089429
4 8192 3447373 733697
8 512 1471685 1064094
8 1024 1973790 910156
8 2048 2443283 797648
8 4096 2594414 747869
8 8192 2197888 944308
16 512 1221641 1117239
16 1024 2071261 972592
16 2048 2062772 1072026
16 4096 1874549 1035630
16 8192 1857256 1122221
Completed series of tests
--IOZone data for NetBSD 1.0A, no filesystem.
IOZONE: Performance Test of Sequential File I/O -- V2.01 (10/21/94)
By Bill Norcott
Operating System: POSIX 1003.1-1990
IOZONE: auto-test mode raw device /dev/rsd0g
MB reclen bytes/sec written bytes/sec read
1 4096 1263344 1278751
1 8192 2184533 2139951
1 16384 3495253 3084047
1 32768 4993219 3276800
1 65536 6168094 3615779
2 4096 1255779 1327311
2 8192 2207528 2255002
2 16384 3554494 3328812
2 32768 4993219 3554494
2 65536 4993219 3744914
4 4096 1255779 1340033
4 8192 2184533 2279513
4 16384 3554494 3382503
4 32768 4415056 3524625
4 65536 4032984 3847985
8 4096 1253902 1316892
8 8192 2190237 2310911
8 16384 3524625 3466366
8 32768 3994575 3615779
8 65536 3938313 3956890
16 4096 1255779 1348650
16 8192 2184533 2314098
16 16384 3532045 3600260
16 32768 3910772 3663147
16 65536 3787181 3947580
Completed series of tests
IOZONE: Performance Test of Sequential File I/O -- V2.01 (10/21/94)
By Bill Norcott
Operating System: POSIX 1003.1-1990
IOZONE: auto-test mode raw device /dev/sd0g
MB reclen bytes/sec written bytes/sec read
1 4096 223101 2330168
1 8192 7489828 2330168
1 16384 7489828 2383127
1 32768 8065969 2383127
1 65536 7489828 2383127
2 4096 223576 2496609
2 8192 4877097 2438548
2 16384 4766254 2438548
2 32768 4766254 2467237
2 65536 4877097 2467237
4 4096 222627 2542002
4 8192 3554494 2496609
4 16384 3495253 2526689
4 32768 3495253 2542002
4 65536 3495253 2542002
8 4096 222568 2573192
8 8192 3072750 2534322
8 16384 3095427 2549728
8 32768 3072750 2557502
8 65536 3095427 2565323
16 4096 222627 2589076
16 8192 2922859 2557502
16 16384 2912711 2573192
16 32768 2907663 2577145
16 65536 2917776 2577145
Completed series of tests