Re: zfs pool behavior - is it ever freed?

[Greg Troxel <gdt%lexort.com@localhost>]

Mr Roooster <mrrooster%gmail.com@localhost> writes:

> I'm not sure they did a lot more than expose the ARC limit as a sysctl.

I'm not either, but if there is a precise description/code of what they
did, that lowers the barrier to us stealing* it.  (* There is of course
a long tradition of improvements from various *BSD being applied to
others.)

> I moved to FreeBSD from Net a few years ago (mainly to get ZFS), and
> have had similar issues under heavy load with a large ARC. It wouldn't
> crash or hang, but it would always favour killing something over
> flushing the ARC under pressure. I did a little bit of digging and got
> the impression this was the way it was intended to work. (Although
> reading this thread it may be a little more complex than that. :) )

Somebody may intend that, but it seems obviously buggy to kill processes
than to drop data from a cache.

> Once I limited my ARC my problems went away. I limited mine to 16 gig
> on a 96 gig system, but I was running some processes with high memory
> usage. I've not had cause to increase it though, and the system runs
> reliably. It has a few zpools, and I'm running a VM of an iSCSI
> exposed ZVOL, so it get a decent amount of use.

Did I hear that right -- you had problems on a 96 GB system with the
default settings?  What was the default limit?

Did you -- or could you -- characterize the performance impact on ZFS of
having ARC limited to say 8/16/24G?  And is this with spinning disks or
SSD, with or without L2ARC?

> (This is my home system, not a production system, however it does have
> something like 10 HDDs in, so is often quite I/O loaded).

Wow, that's a lot of disks!