Re: ZFS tunable parameters

[Simon Burge <simonb%NetBSD.org@localhost>]

Greg Troxel wrote:

> Brad Spencer <brad%anduin.eldar.org@localhost> writes:
>
> > I left the ARC setting stock and performed my usual abuse build testing
> > and I can say with certainly that it performed as one would expect.  No
> > lock ups and while swapping was higher than with the adjustments
> > mentioned in the patch, that would have been expected.  The system
> > performed perfectly under pretty high memory pressure.
>
> That is great news.  I expect that 11 will behave the same way, as I
> think all the fixes have been pulled up (a while ago)
>
> > I think I can say, at this point, that the fixes worked and worked very
> > well.
> >
> > This is a very welcome change and brings the experience of using ZFS
> > with NetBSD very close to what you could expect with Solaris.  There are
> > likely more problems, in the form of missing support, still present with
> > ZFS, expecially around failed devices, but in -current it works very
> > well now with far fewer quirks.
> >
> > Thanks to everyone who made this a reality.
>
> I think what really made the difference was yamt's fixes.  Before, our
> code did not write back dirty pages that, I think, resulted from mmap,
> and that's fixed, if I follow the diffs (but the underlying code is
> complicated).
>
> chuq identified a deadlock, where FreeBSD has two separate cleaner
> threads in a way that matters for mmap-dirtied-pages.  I suspect that's
> actually a problem but that it rarely matters now that we don't vastly
> accumulated mmap-dirtied-pages.
>
> I have not had a crash on my 11 box, and am going to update my desktop
> from 10 to 11 soonish (async from whatever tags exist on netbsd-11 :-).
> I hope that makes the same difference you see.  Currently, I perceive an
> increasing over time since boot risk of crash, which fits with the bugs
> yamt found.

Brad's experience matches my experience.  I am running with 128GB of
memory though, in contrast to their 8-16GB.  I had been running an
experimental patch from chuq that attempted to deal with the pagedeamon
deadlocks and made things better but still had the occasional hang when
memory was really low.  One set of yamt's changes was related to the
deadlock case.  It's been 64 days since I rebooted with yamt's fixes and
haven't had any problems since then (touch wood).

It's also worth highlighting yamt's fix for the slow rm case.  That has
made a significant usability improvement with ZFS.

Michael's patch did limit the size of the ARC to approximately half the
memory instead of almost all of the memory once the memory size grew
over about 8GB.  I don't know if this is helpful or not, but having some
tuning knobs available definitely makes it easier to tweak these if
there are some usage patterns or configurations that run into problems.

Cheers,
Simon.