Re: Can NetBSD cgd be used for encrypted backup?

[Swift Griggs <swiftgriggs%gmail.com@localhost>]

On Mon, 19 Jun 2017, Mayuresh wrote:
> Just curious. How does iscsi compare with NFS? Guess even NFS has a 
> notion of block size, that would help optimize io.

Sorry for butting in, but I'd point out that NFS is file-based and layers 
on top of an existing filesystem. So, the block size of the underlying 
file system is going to determine the block size. There is also the 
consideration of the network parameters such as send and receive buffer 
sizes and several others that matter quite a bit (depending on the 
layer-4 protocol in use and the version of NFS).

iSCSI only provides block devices, it can't do file-based I/O natively 
without a filesystem on top of it. My experience with iSCSI has overall 
been quite poor. I once did a long whitepaper on iSCSI vs AoE. Being a big 
fan of SCSI (and not a huge fan of ATA) I was hoping & expecting iSCSI was 
going to be better than it turned out. However, the experience turned out 
completely opposite. Not only did AoE stomp it in every performance test I 
tried, it also scaled better, recovered from failures better, and so 
forth. iSCSI also has a million dials and settings for mostly useless crap 
few are going to fiddle with. It feels like some kind of top-heavy 
machination designed by some committee somewhere that never has to use 
network block storage in-real-life.

I've also seen large scale iSCSI deployments be fraught with pain and 
peril simply because network engineers can't be trusted to leave the VLANs 
it runs on alone and can't be bothered to put it on discrete switches.

Of course AoE runs on top of layer-2 and iSCSI is a layer-5 protocol. The 
extra layers underneath iSCSI make it routable, but destroy performance. 
With AoE you don't have to tune TCP/IP (but it's non-routable).

I also remember hearing about HyperSCSI which is supposed to be hybrid 
strategy that uses SCSI CDB's over Ethernet frames like AoE does. My guess 
is, based on AoE's good-showing, that approach would rock if they got it 
off the ground.

I guess I should also point out that iSCSI is widely supported across a 
larger number of operating systems than AoE and has much more vendor 
acceptance since AoE is seen as the domain of the CORAID (or whatever they 
are called now) folks.

Anyhow, based on my bad experience, I wouldn't recommend iSCSI for anyone 
unless they simply had no other choice. I have seen it be workable, 
especially with dedicated hardware (Equallogic gear seems to work okay, 
and it's got NetBSD bits in there too!), but overall, I'd run screaming 
away.

iSCSI does give a block device to use with CGD, though. I bet it would 
work fine with CGD, despite being kind of a poor idea in general (iSCSI 
not CGD).

-Swift

Just my opinions here. If you use iSCSI and love it, YMMV, and more power 
to you.