On Mon, Aug 21, 2006 at 01:35:03PM -0700, Bill Studenmund wrote:
> 
> The current scheme just stops a process, so we basically pump the breaks 
> on a writer. If a process stops writing (say it goes back to processing 
> data to write to another file), we stop hitting the breaks. With tweaking 
> the scheduling, we would be applying a more-gradual breaking. I'm not 100% 
> sure how to do this as I _think_ I want whatever we do to decay; if a 
> program shifts away from writing, I'd like it to move back to being 
> scheduled as if it had never written. I know the scheduler does this, I'm 
> just not sure how to map the dynamics from disk usage to those for CPU 
> usage.

Here is what bothers me -- and it's clear to me now that I did not adequately
understand one key design decision early in this process.  I do not believe
that it is _ever_ appropriate to throttle a writer simply because the
number of writes it is issuing exceeds _X_, for any _X_, without some metric
of whether a congestion condition has actually occurred.

I cannot imagine how, in general, doing that could actually have any other
than a negative performance impact.

If we were actually measuring some value that could indicate congestion,
such as the change in request latency, then I do not think the current
approach would cause problems, and I believe that sleeping writers for
something like the measured minimum request latency would in fact be the
correct way to pace their writes -- it's almost exactly what network
protocols do.  But without that, simply putting processes to sleep
because we've decided they issued a "write flood" is just going to have
a negative effect on performance that will cascade through the system.  I
don't know how to explain the negative effect we're seeing right now,
but I certainly wouldn't expect a positive one.

Thor