Thursday, October 13, 2011

Fibre Channel at Dead End, Better to Invest in Other Technologies

One of the most used solutions for storage connectivity is Fibre Chanel. Personally for high performance computing I prefer to use Lustre, but in my organization you can find enterprise class systems with enterprise storage networks, which were mainly Fibre Channel based.

As we evaluate every technology before we acquire new systems, I recently reviewed the options for an enterprise class storage solution. A quick history background: While Fibre Channel was created back than for general usage, it has become a storage networking solution. Fibre Channel is standardized in the T11 Technical Committee of the InterNational Committee for Information Technology Standards (INCITS).

In the early 2000s, Fibre Channel speed was at 2Gb/s, Ethernet was just getting into 1Gb/s and InfiniBand was not there. Having the storage capacity at 2x the communication capacity was a good reason for the Fibre Channel adoption. In the mid 2000s, Fibre Channel was at 4Gb/s, Ethernet at 1GigE and InfiniBand was moving into 20Gb/s. Nowadays, Fibre Channel is at 8Gb/s, Ethernet at 10Gb/s and InfiniBand at 56Gb/s. The next speed bump of Fibre Channel is 16Gb/s, Ethernet 40Gb/s and InfiniBand 100Gb/s. With this faster evolution of Ethernet and InfiniBand compared to Fibre Channel, there is clearly no reason to use Fibre Channel anymore and any investment in it for future deployments is clearly a mistake.

If we consider latency as another factor, the appearance of SSDs does not help Fibre Channel as well. The latency benefit of SSD is eliminated when Fibre Channel is being used, and the only options for SSD based storage are Ethernet or InfiniBand.

FCoE (Fibre Channel over Ethernet) is not the solution, and it seems to be just another try of the Fibre Channel vendors to extend the life time of the dying Fibre Channel storage. There is no reason to continue and use Fibre Channel. You better invest in getting Ethernet (iSCSI for example) or InfiniBand storage for your next system – higher throughput, lower latency, and better economics. I also believe that iSCSI deployments are on the increase while Fibre Channel on the decrease. Yet another proof point.


  1. Are you aware of any good SAN-like solutions that support Infiniband? Lustre installations are good in a lot of cases, but there are plenty of situations where I'd like to be able to manage SAN-like storage over IB... if I could find any good support for it.

  2. I am aware of several companies with InfiniBand solutions - Kove, DataDirect, Xyratex, Violin Memory, Oracle ZFS, IBM, AMAX etc. You can also self-build NFS system as well.