Re: "Virtual" RAID1

[tlaronde%polynum.com@localhost]

On Wed, Jun 26, 2019 at 07:55:03AM +0930, Brett Lymn wrote:
> On Mon, Jun 24, 2019 at 10:38:17AM +0200, tlaronde%polynum.com@localhost wrote:
> > 
> > I have read the presentation of the features (I have a copy of a book on
> > AFS that I need to read to), so it has features that I'm after but it
> > lacks also one feature: the reality of disks and the need for
> > dissymmetry: RAID1 puts an equal burden on both disks meaning that to
> > serve one data, you stress two disks; whether you put two high end
> > server's disks---and it is a waste---or youp put two desktop/terminal
> > disks, and you will spend your time replacing disks praying that they
> > not both die at the same moment.
> > 
> 
> well you would be pretty unforutnate for that to happen and, besides, in
> backups we trust ;)
> 
> > I have in mind a simple block upon which one could imagine building
> > distributed data.
> > 
> 
> Have you looked at glusterfs?  It sounds like you may be trying to
> implement something like that.
> 
> > 
> > One can see that distributed file system and disconnection can be
> > handled with this element: since writes (depending on filters) is always
> > duplicated, in one member is a "local" storage, there is always (if not
> > discarded by filter) a local copy of written data on the node. So if the
> > other component is a remote file service, the disconnected client can
> > work.
> > 
> 
> OK, so you then have to make some hard decisions on how you handle
> performance.  Are you going to choke write performance to the slowest
> disk or have some sort of buffering to hide the write delay.  If you
> rely on syncing of the mirror components then you still can lose data if
> the up-to-date component fails before the mirror sync is done.

I don't say there is (for distributed filesystem) an easy solution.
These problems have to be faced, by the very nature of the distribution
itself, by whatever system implements it.

But the thing I have sketched could be used for distributed filesystems.

At first, what I search is not a distributed filesystem because it will
be put on the "single" node serving: duplicating on fly the files I want
to backup in a dissymmetric couple of two storages: the first one is the
one serving the data, the second one is only backing it and physically
the second one can be "cheaper" disks because there will be less
intensive usage; plus, I want to have a filter to discard some data that
I don't backup.

For the moment, I will keep what I have put in place, that is not
"synchronous" duplication but, from time to time, dump/restore (using
chflags to discard what I don't want to dump) or find/pax depending
on the connection of the backup disks.

And I will play a little, when time permits, with what I have sketched.
At the moment it is little less than a first approach of design and a
name: DWARF (Duplicate? Write : Always; Read: on Failure). 
-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                     http://www.kergis.com/
                       http://www.sbfa.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C