Subject: Throttling IO Requests in NetBSD via Congestion Control
To: None <tech-kern@netbsd.org>
From: Sumantra Kundu <sumantra@gmail.com>
List: tech-kern
Date: 08/21/2006 14:16:36
Hi,

   As part of NetBSD SoC project, we have implemented a congestion
control algorithm (CCA) that slows down the rate of write requests by
individual processes when the algorithm perceives that the process is
initiating too many writes in a small interval of time.

The motivation for such an algorithm was to push back write requests
back to processes by making them sleep for a variable length of time
(using ltsleep()) so that corresponding read requests are not heavily
penalized.

In order to benchmark the performance of CCA, we modified Postmark
(v1.5) so that the program forks off a child process that executes "dd
bs=1m count=10000 if=/dev/zero of=<some name>" in the background while
Postmark continues to randomly read from a set of files it had created
during initialization phase.

Now some interesting results which we are trying to understand:
Link for results:
http://netbsd-soc.sourceforge.net/projects/congest/Benchmarking_Results.html

Note: (a) softdeps was --not-- turned off, and the bufq strategy was
the default one.
         (b) TPS= Transactions per Second defined over reads only. NO writes.

+ With **no** CCA, TPS (with dd and Postmark) drops by 42%.
This is encouraging indicating that writes are indeed affecting reads.

+ With CCA, writes are indeed slowing down, as evident from the
decrease in the rate of data being transferred by "dd". But, what is
intriguing is the fact that reads are slowing down too !!!!

We have tried around 50 scenarios and only in 3/4 cases, the rate of
read requests (measured in TPS) showed an improvement of only 16%. In
over 90% of the cases, the decrease in the rate of writes, is being
accompanied by a decrease in the read rate.

And now the **worst part**.... The read rate (with CCA) is less that
read rate (without CCA).......This is baffling since as we have
decreased the write requests rates, read requests should be serviced
more frequently....

+ Disabling softdep yields similar trends.. Remember that no other
heavy duty processes are running on the system that is eating away CPU
timeslice. (As shown by top).
Also, the results are for a system that is running in single user mode.

While we are trying to discern such behavior, any insights would be
greatly appreciated..

Thanks,
Sumantra
==PS:==
Initial information (along with results) is available at the NetBSD
SoC website:
http://netbsd-soc.sourceforge.net/projects/congest/index.html