Subject: Re: dump is slow over network and/or raidframe
To: Aaron J. Grier <agrier@poofygoof.com>
From: Chuck Swiger <cswiger@mac.com>
List: netbsd-users
Date: 11/28/2006 09:34:51
On Nov 27, 2006, at 6:03 PM, Aaron J. Grier wrote:
> most helical scan drives (DAT and DLT) don't do well if you can't keep
> them streaming.  if they get underflows, they have to reposition the
> tape to figure out where they left off.

DAT uses a helical scan mechanism; DLT means "Digital Linear Tape",  
and it uses a multi-track linear format and cartridge which derives  
from the classic QIC-150/Travan design.  This said, it's correct that  
pretty much any tape drive performs well when they're kept streaming,  
and will slow down dramatically if they see a data underflow.

> rmt only has a single queue.  when transferring data, it reads from  
> the
> network, then writes to the tape device.  this maintains  
> synchronism and
> is the simplest safest model, but can perform poorly with  
> simplistic rmt
> clients like dump.
>
> dump has only a single output queue.  combine this with rmt's
> synchronism, and you get a half-duplex network operation.  there  
> can be
> no data in-flight or buffered on the network, since additional  
> blocks of
> data cannot be sent until the one in-flight has been written to the
> device and ack'd to dump by rmt.  as latency and buffering go up,
> performance goes down.
>
> the whole dump -> rmt -> tape write interaction ends up going in fits
> and starts.

Agreed.  Note that you can use dd to rebuffer the output of dump,  
which will speed things up considerably, assuming that your version  
of dump does not support configuring the blocksize it uses.  Most  
tape drives seem to like getting 64K of data in a block, although  
experimentation will show that some do better with 63K, or with 128K  
(or 127K)-- using slightly smaller than a power-of-2 size seems non- 
intuitive, but it seems to work better with the built-in compression  
that modern drives like to use.

-- 
-Chuck