> > Perhaps we're getting off on a tangent here, but...
> Yeah, but I bet it'll be fun. So let's go. :-)

Woo-hoo! I'm in.....

> > Sure, you can eliminate the CPU usage with iSCSI by purchasing an
> > expensive iSCSI offload adapter, but now where is the cost savings?
>
> One thing that I think iSCSI will have going for it is that while you're
> right about the needs of the switches, Ethernet will have a lot more
> competitive drive to reduce cost than will FC. So I think things will get
> better in the future. :-) Yes, FC will get cheaper, but there
> will be more things driving Ethernet down so I think it'll win.

Two things here.

iSCSI with offload engines can perform
very well. It also means there is a storage
(peripheral) infrastructure accessible with
software-only initiators. Throw offload engines
and money at apps as necessary. No such thing
as a software-only FC initiator. Hence, its
a capability FC can't match.

There is this emerging tech called RDMA (Remote DMA).
It is in early stages. Much of the CPU costs of software-only
iSCSI is the copying between network buffers and app/fs
buffers. It's not just the CPU costs, but the consumption
of memory bus cycles. RDMA has good early results of
nearly eliminating those costs. NFS over RDMA and especially
iSCSI over RDMA are active areas of R&D.


> > performance of one FC port, you need *TWO* Gig-E ports.  And forget
> > about having a transaction latency as low as FC... iSCSI's protocol
> > overhead is just plain higher (Ethernet, IP, TCP, *and* the iSCSI
> > transport protocol).
>
> Yes, transaction latency will be an issue. iSCSI will really need lots of
> outstanding transactions.

There are bona-fide FC-over-IP gateway products. I've
not heard talk of network-over-FC for many years, and
even then it never got passed the talk stage. For all
IP/ethernet's warts, the FC folks seemed to have found
value. For all FC's virtues, networking folks haven't
found value.

> Folks are happy with local disks that get say 30 MB/sec, and iSCSI
> can do that easily. So there could be a lot of options where iSCSI is
> "good enough." :-)

Yup.

I've been told by a reliable source that only 20% of
shared storage (not including desktop storage) is connected
to SANs at enterprise sites. There is a lot of cost and
care-and-feeding. I don't have access to the raw data.
It seems cleared that SANs are deployed reluctantly, though.

For us folks old enough to remember SMD disks (and nd protocol :)
we'll also remember when SCSI was thought of as a "toy" disk.
ATA and SATA are toys? Seen this before, and the toys win.
Shared storage technologies, FC and iSCSI included, are in
competition with direct attached, including PCI RAID cards.

All the fancy aggregation technologies are less and less
meaningful as single disk, or at least enclosed disks,
provision applications sufficiently.

Yes, there are benefits to having lots and lots of disk
arms, and all the piping (FC, FC-attached RAID, etc, etc)
and cooling to have those disks.

Nevertheless, I've been at panel presentations where the
FC advocate lost ground very quickly to another panelists
account of succesfully provisioning a large application
(500tb) with ATA drives, admitedly aggregated through FC
attached RAID controllers.

iSCSI appears a toy. So does SATA. So did SCSI at one time.
The toys will win.

Regards,
	-gww