Subject: Re: loaning for read() of regular files
To: Jonathan Stone <jonathan@dsg.stanford.edu>
From: Chuck Silvers <chuq@chuq.com>
List: tech-kern
Date: 02/23/2005 08:14:03
On Tue, Feb 22, 2005 at 09:43:30PM -0800, Jonathan Stone wrote:
> In message <20050223052206.GB21919@spathi.chuq.com>Chuck Silvers writes
> >On Tue, Feb 22, 2005 at 02:56:50PM -0500, Stephan Uphoff wrote:
> >> On Tue, 2005-02-15 at 12:36, Chuck Silvers wrote:
> 
> >> I think that you also need to check if the individual pages that are
> >> replaced (or shadowed - but I guess this is broken anyway) are wired to
> >> avoid areas wired by uvm_vslock() :-(
> >
> >yea, that's true, I'll add it.
> 
> Are there yet more corner cases?  Is there any way to use a cheap
> conservative test that says a page definitely *is* safe for read
> loanout, rather than enumerating all the cases when it isn't safe
> (and potentially missing some)?

enumerating the cases where it's illegal is all we can do.
in this case, the issue is with the pages being replaced
rather than the pages being loaned.


> Also: I assume it's benign, but have you to confirmed by testing that
> read()ing through files much larger than physical memory isn't
> penalized by the changes for loanout?

my initial mail gave results for reading a file that wasn't already
in the cache, which is basically the same as what you're talking about.

but I ran your case just now anyway, reading through 2GB of a file
on a machine with 512MB of RAM, using the same two buffer sizes as
before (4KB and 1MB):

w/ loaning
# time pt -r -c 2048 -s 1048576 /var/mysql/ibdata1 
2147483648 bytes transferred in 162.522 secs (13213415 bytes/sec)
0.028u 4.284s 2:42.53 2.6%      0+0k 130+1io 0pf+0w
# time pt -r -c 524288 -s 4096 /var/mysql/ibdata1
2147483648 bytes transferred in 162.283 secs (13232897 bytes/sec)
0.453u 10.856s 2:42.30 6.9%     0+0k 0+1io 1pf+0w

w/o loaning
# time pt -r -c 2048 -s 1048576 /var/mysql/ibdata1
2147483648 bytes transferred in 163.738 secs (13115344 bytes/sec)
0.020u 34.523s 2:43.76 21.0%    0+0k 0+1io 1pf+0w
# time pt -r -c 524288 -s 4096 /var/mysql/ibdata1
2147483648 bytes transferred in 163.480 secs (13136010 bytes/sec)
0.604u 26.182s 2:43.48 16.3%    0+0k 0+1io 1pf+0w

-Chuck