Subject: Re: Questions regarding dump
To: Greywolf <greywolf@starwolf.com>
From: Matthew Jacob <mjacob@feral.com>
List: current-users
Date: 07/21/2000 01:18:19
> - NDMP does not preclude dump, that I can tell -- if anything, it might
>   augment it.

Right.

> 
> - Yes, there are proprietary multiplexers for tapes.  I don't see why we
>   can't write one opensource.  On the other hand, while it speeds backups
>   by keeping the tape drives streaming, it makes a NIGHTmare out of doing
>   a restoral.  Restores are very slow from multiplex-dumped tape sets.
>   And if one of those tapes across which how-many-machines are dumped
>   in parallel goes bad, guess what?  You're well and fscked.  I'd just
>   as soon pass on the multiplexing and go for raw multiple drives.

If you do this, you can also do this as a RAIDtape format. There are
also poliy differences you could set such that fulls are striped
but incrementals are single taped (so as to hit the most common
recovery case- recovering the file ate yesterday...).

> - I just finished a Veritas installation, and I don't want anything to do
>   with either Veritas or Legato.  While it's flexible, it's also amazingly
>   convolved.  We had to call in a consultant three times in order to get
>   it set up properly.  The documentation is something like eight volumes.
>   There has GOT to be a simpler approach to setting up a backup system.

Well, I have to agree with you. I'm fairly comfortable with Legato stuff-
but I've been doing it for years and know the limits, etc. But such
a system is fundamentally undesirable in the kind of broad environment
I want to see this stuff in (I want to support large *and* small
scale systems- and I don't like seeing the cost as high as it has been).

> - Personally, I *like* dump and restore; all the database manglement
>   and indexing for tape location, etc., is just front end goop, really,
>   and it could be glued on at any time.

If I understand you correctly, what you're saying is you like the
dump/restore tape format and that you feel index management can
be a simple addon- I'm not so sure about that.

> - Whatever solution we use, it's GOT to have an option to force a tape
>   rewaffle upon detection of end of tape, and it's got to have an option
>   to say "spew data until you reach end of tape".  Or, better yet, if you
>   give it the size/density parameters, that will do nothing more than to
>   tell the program to calculate how much of the tape/how many tapes it's
>   going to use.

But don't try and do size calculations- differing tape defect rates
make actual size hard to predict. You actually have two cases- one
easy, one hard, to cope with:

+ You hit Early Warning. This allows you enough space to write
trailer records (that can identify potential 'next volume' information,
etc.). Requires OS support in the driver so that Early Warning can
be propagated back to the application.

+ You have some kind of write error. No way to put trailer records- you
just have to ask for the next tape.


> 
> - Some form of offline (removable) storage will need to be present;
>   disaster recovery in the form of colocated redundancy may be common
>   among businesses, or growing so, but it is not common practice among
>   individuals.  I don't think tape storage for the individual market is
>   going to suddenly evaporate; nor should our support for said devices.

I don't believe I suggested we desupport tape devices. I believe what I'm
trying to get at is that even with low cost 'individual' systems (and I
count Feral Software as part of this- I grumbled like hell when I bought
a single DDS-3 drive last year) the amount of disk storage is vastly
outstripping the capacity of the tape drives people can afford to buy.

I *don't* have a silver bullet to solve this conundrum- but I *do* feel
that getting serious about an open source network backup utility that is
well supported in *BSD and Linux (yes, I'd put it there too) *AND* works
and plays well with some of the commercial packages can't be anything
but a good idea- even for small sites (2-10 systems). At the very least
it would be an improvement on rmt and sharing of tape drives across
multiple systems.

> 
> Just a few thoughts; I don't think I'm all that far off the mark.

Nope - good summary I'd say.

> # Amanda backs up 30+ GB of disk storage with a 4GB Tandberg SLR5 streamer
> # for me. Only requirement is that partition sizes have to be below 4GB
> # because level 0 dumps have to fit on one tape. But 50GB QIC drives are
> # readily available,. not to mention DLT or AIT drives.
> 
> Amanda is broken.  Period.  It can't handle multiple volumes.  Now if
> that support gets added...

I still think that there's more to all of this than just the despatcher-
and I think that indexing and media management is a bit more than a bolt-on
to such stuff, but this is the kind of thing where I don't have to
convince anyone- I can just *do* this. If you don't want it, don't buy it
(even at the 'free' sales price). This is different than the CAM negative
lovefast a year or so ago in that none of what I refer to is invasive
to the system.

-matt