Re: 5.x filesystem performance regression

[Lloyd Parkes <lloyd%must-have-coffee.gen.nz@localhost>]

??? Why is this in tech-net?

On 5/06/2011, at 2:40 AM, Edgar Fuß wrote:

> Having fixed my performace-critical RAID configuration, I think there's some
> serious filesystem performance regression from 4.x to 5.x.
> 
> I've tested every possible combination of 4.0.1 vs. 5.1, softdep vs. WAPBL,
> parity maps enabled vs. disabled, bare disc vs. RAID 1 vs. RAID 5.

Excellent.

> The test case was extracting the 5.1 src.tgz set onto the filesystem under 
> test.
> The extraction was done twice (having deleted the extracted tree in between);

I always reboot between such tests to ensure that the buffer cache has been 
cleared out. If I ever get around to running RAID benchmarks again, I'll script 
it all in /etc/rc.d with reboots between each run so that I can get a number of 
runs without having to run anything by hand.

> in some cases the time for the first run is missing because I forgot to time
> the tar command.

That's a problem because that is what is required to show the effect of the 
buffer cache.

> So, almost everywhere, 4.0.1 is three to fiveteen times as fast as 5.1.

I'm afraid that is isn't even close to almost everywhere because there are so 
many missing measurements. If we ignore all of the second runs because of the 
buffer cache issue, we only have two columns that contain enough data. The 
first is the plain disc column and it shows things looking pretty good for 5.1. 
The second is RAID 5 32k which doesn't look so good. For some reason, RAID 5 
appears to be very slow and it needs looking at.

If we want to look at the second runs in order to work out why 5.1 looks so 
much worse in the second runs, we still only have enough data in the plain disk 
and RAID 5 32k columns. For the plain disk, 5.1 does perform better in the 
second run than the first, just nowhere near as well as 4.0.1. My guess is that 
the VM parameters changed between 4.0.1. and 5.1 (they did change, I just can't 
remember when). Try comparing the output of "sysctl vm" on the two versions of 
NetBSD. My experience is that the VM settings need adjusting in order to get 
acceptable performance from any specialised workload and I suspect that under 
4.0.1 your file set fits in memory, but under 5.1 it doesn't fit in the allowed 
file memory. Once again RAID 5 appears to be very slow and it needs looking at. 

Cheers,
Lloyd