InfiniBand for the Home in Less Than $150 (10Gb Networking on the Cheap)

I came across Dave Hunt's Blog - http://davidhunt.ie/wp/?p=232, talking on using older generation of InfiniBand (10Gb) for the home. Think about your home entertainment system, Blu-ray streaming from a storage box to your PC and other usage models…

When it comes to Ethernet and 10Gbs, the price is in the sky, but you can get the same speed with InfiniBand for pennies.  Dave built a system that passes over 700MB/sec throughput between his PCs at home for under $150! As Dave says, that’s like a full CD’s worth of data every second…

From Dave blog – “So, I now have an InfiniBand Fabric working at home, with over 7 gigabit throughput between PCs. The stuff of high-end datacenters in my back room. The main thing is that you don’t need a switch, so a PC to PC 10-gigabit link CAN be achieved for under $150! Here’s the breakdown: 2 x Mellanox MHEA28-XTC InfiniBand HCA’s @ $34.99 + shipping = $113 (from eBay), 1 x 3m Molex SFF-8470 InfiniBand cable include shipping = $29. Total: $142”.

Next will be for me to bring InfiniBand into my home…

PCI-Express 3.0 is Finally Here…

Long time since my last post… You know how it is – new academic year, lots of preparations, going to Supercomputing 2011 in the freezing Seattle…

One of the new technologies that I was really waiting for is PCI Express 3.0. PCI Express 2.0 was released in 2008 and is the bottleneck whenever you use faster than 20Gb/s network - such as InfiniBand. PCI Express 2.0 was released in 2008, and it is about time to get the new generation out.

While sources said that the official release of PCI Express 3.0 will be in the March 2012 time frame, the first systems based on PCI Express 3.0 (and InfiniBand FDR!!) are already out there. If you were at SC’11, or if you monitor the news from the TOP500 list, you could hear (or read) on the new systems. One of them is the Carter supercomputer in Purdue University, which is said to be the US fastest campus supercomputer. Carter systems was ranked 54th on the November TOP500.org list and was built using the latest technologies from Intel, HP and Mellanox, including not-yet-released Xenon E-5 "Sandy Bridge" Intel processors, HP Proliant servers and the already-released InfiniBand FDR.

The folks from Purdue claim that "Carter is running twice as fast as the supercomputer we were using and is using only half of the nodes. That will allow us to scale our models for better forecasts." So higher performance at a lower operational cost. Great deal…