Subject: Re: Bad side effect of "newdirpref"
To: Thor Lancelot Simon <email@example.com>
From: Charles M. Hannum <mycroft@MIT.EDU>
Date: 08/21/2006 13:57:22
On Mon, Aug 21, 2006 at 12:19:02PM -0400, Thor Lancelot Simon wrote:
> On Mon, Aug 21, 2006 at 10:04:45AM -0400, Charles M. Hannum wrote:
> > This is no longer the case. Rather, "newdirpref" tends to fill up
> > some CGs first -- often a contiguous chunk, even. This has the
> > effect of reducing the pool of CGs that these large files can come
> > from. Worse, because the full or nearly-full CGs tend to be in a
> > contiguous region, as the disk fills, the "rehashing" mechanism in
> > blockpref ends up hitting an even smaller number of CGs frequently.
> > Worse, the policy itself tends to blindly put new directories (and
> > thus their files) in the same CG much of the time anyway. The result
> > of all this is that we have really bad contention, and rather often
> > the two processes I referred to above end up writing files to the
> > same CG, interleaving their blocks (because the block rotor is also
> > just that dumb).
> > And this culminates in me being unable to write the data faster than
> > 4x to my DVD drive, because reading it back from the hard disk is
> > just too damned slow. Even though I've tested my DVD drive at 16x
> > and it works just fine.
> > This sucks.
> I wonder if this effect might not be greatly reduced if the number of
> CGs were greatly reduced.
Did you even process what I wrote, or did you just rapid-fire your stock
Given the simplistic "block rotor" used for block allocation, reducing
the number of CGs can *only* make the problem worse.