Pondering Fibre Channel over Ethernet

Currently the twittersphere and blogosphere is actively discussing Fibre Channel over Ethernet (FCoE).  The conversation was triggered by a post by Hu Yoshida from HDS, and I wanted to share my thoughts.

One of the most interesting responses was this one by Nigel Poulton where he explains the infrastructure required for FCoE.  He goes into great detail highlighting the lossless nature of FCoE and the required hardware and cabling.  The key takeaway is that FCoE is not iSCSI; it does not use generic 10 Gigabit Ethernet (10 GigE) hardware.  You will need different cabling and advanced switches to meet the more stringent demands of FCoE.

The requirement for specialized FCoE hardware will increase costs versus iSCSI.  iSCSI can leverage existing LAN  investments and benefits from the economies of scale of Ethernet hardware.  The specialized nature of FCoE means that it will ship in lower volumes than vanilla 10 GigE and thus it will take longer for FCoE hardware prices to decline.  This will impact the ROI of FCoE solutions since the higher cost will extend the payback period.  Thus, FCoE will be at a price disadvantage to iSCSI; however, I believe that the two technologies address different markets.  FCoE mimics the reliability and consistency of Fibre Channel, but will also mimic the high price.  iSCSI in contrast, runs on mainstream Ethernet infrastructures and will provide a more cost effective although less reliable and slower performing option.

A real world experience would be useful in clarifying my thoughts.  I sold iSCSI solutions (EqualLogic, if you must ask) to small businesses in a previous job.  The prospects were companies who wanted the benefit of a shared storage environment and had limited budgets and sophistication.  They were excited about iSCSI because it provided the benefit of a SAN without the cost and complexity of Fibre Channel.  In my opinion, these customers will continue to favor iSCSI over FCoE.  However, I believe that the target FCoE customer is a larger environment that already has an FC SAN in place.

It will be interesting to see where the market moves from a backup perspective.  iSCSI is less common in today’s Gigabit Ethernet data protection environments.  As 10 GigE prices decrease, high performance iSCSI will become widely available.  Will this drive penetration for iSCSI backup targets?  It definitely could for smaller companies like the one described above.  However, I believe that enterprises have a strong need for the performance and reliability of FC/FCoE and will favor these technologies over iSCSI in the datacenter.

Another question is whether companies will favor FCoE over FC.  Given the newness of FCoE, I believe that it will take a few years before companies start seriously implementing FCoE in the core especially since most enterprises already have large investments in FC.  FC speeds will also continue to increase and keep the technology competitive.  Long-term, FCoE will co-exist in corporate datacenter infrastructures along with FC, and it will be interesting to see how companies divide their I/O workload between the two.

3 replies on “Pondering Fibre Channel over Ethernet”

Good concise post and a very balanced view (in my opinion).

An interesting twist to the storey could lie in the fact that iSCSI can also benefit from many of the enhancements in Enhanced Ethernet. For example, Priority based Flow Control (802.3Qbb) and Enhanced Transmission Selection (802.1Qaz).

My understanding is that it is possible, for example, to give iSCSI frames, obviously encapsulated in IPv4 frames as normal, a high priority meaning that iSCSI can benefit from lossless behaviour………….. up to a point. I say “up to a point”, because if iSCSI were to co-exist with FCoE or any other traffic type that required higher priority, it would have to sit lower down the pecking order and be more likely to have PAUSE conditions inflicted. E.g. If FCoE and iSCSI were operating on the same unified fabric and the fabric started to experience congestion, FCoE would be given a higher priority than iSCSI, BUT iSCSI could still have a higher priority than other traffic types. Hope Im making sense.

Of course, if iSCSI were also running over Enhanced Ethernet, it would benefit from the better physical aspects too including higher rated cabling and hardware meaning less packets drops for other reasons etc.

Just my penny’s worth.


Thank you for your comment; you bring up a great point that iSCSI could benefit from the same advanced networking features required by FCoE.

I think that the challenge comes back to cost. If these features can be implemented cost effectively then they will be relevant to iSCSI. If a substantial premium is charged versus vanilla 10 GigE then I think that most iSCSI users with stick with standard 10 GigE as a “good enough” solution.

I have an example were this chlikecst would have been relevant.About 4/5 Years ago I had to setup a Postfix Mail server on some new hardware that was given to me for the setup. The company that I worked for at the time had a Fedora only policy.After a lot of trial and error I found out that the on-board fake RAID was not supported by Linux yet and RAID was a requirement of the setup.At the same time I was personally playing with and learning FreeBSD. So I gave FreeBSD 5.x a try and it worked without any problems, RAID and all. I left that company shortly there after and I know that this FreeBSD box is still running without a days problems or reboots, but unfortunately without any security updates or software update at all and the people currently work in my place don’t have any knowledge of FreeBSD or even want to learn FreeBSD.Its not often that I hear of, or experience BSD supporting some hardware before Linux but this experience is what got me started with FreeBSD and I have been experimenting with it ever since. Unfortunately I have not been given the opportunity to put FreeBSD in production again. If it was up to me and if support is not a factor (see chlikecst) it would be FreeBSD every time. Jails Rock!

Leave a Reply

Your email address will not be published.