[SLL] Rant: ATA-over-Ethernet 0x88a2

[Andrew Sweger]

This subject has me a little hot under the collar. I'm not angry, just
very frustrated and feeling a bit impatient. There is a very nice
introductory article by Ed L. Cashin in the June 2005 issue of Linux
Journal in the Kernel Korner about "ATA over Ethernet: Putting Hard Drives
on the LAN".

The basic concept is to minimally translate the IDE interface to a
Ethernet physical link (twisted pairs) and modprobe aoe to have your box
talk to the device. The AoE protocol is dead simple, doesn't involve
TCP/IP (just Ethernet), and runs over commodity network gear. From the
sysadmin perspective, you would never realize the you're working with
anything unusual except the device names have a new flavor
(/etc/etherd/e0.0). Just configure md, lvm, etc. and start building
file-systems. I've got my 2.6.11 kernel ready to go and aoe is loaded, I'm
just waiting for the hardware.

The article does a great job explaining how it works and points out that,
right now, there is only one hardware vendor putting together devices that
present the AoE interface: Coraid Inc.  Interestingly enough, as you turn
to the next page in the article, there is a full-page ad for Coraid right
there (and the next is another for Arkeia, a Coraid partner). Also, the
article is written by none other than a Coraid employee. Now, stop right
there. That doesn't bother me in the least. In fact, I have been ignoring
the Coraid ads for months until I read Ed's article. So, this is a good
thing.

Up until yesterday, this system was only available for PATA from Coraid.
It looks like they just added SATA gear today. I don't care much about
SATA. What is bothering me is the present cost of this equipment. Coraid
appears to be the only player in this field. Judging by the information on
their website (which is of terrible quality, way too many bloody PDFs), it
looks like they're cranking out copies of the prototype circuit board
still. It's a big PCB that acts as a tray under the hdd, which makes it
big and clunky. And they want USD$295.00 *per* drive to give you this
interface (266 per drive in the 15 drive SATA "shelf"). That's more than
the cost of the damn hard drive! They even want this much for the
bare-bones evaluation kit. What's worse is they insist on referring to
this circuit board as a blade. It's the buzz-word-band-wagon, folks.
*sigh*

The thing that's really bothering me is seeing this stuff take off and
head down the SATA path and leave a huge opportunity behind. Maybe some
clever drive maker will simply put an RJ-45 jack on the back of the drive,
cutting Coraid out. Probably not and it doesn't matter. What we really
need is a dirt cheap dongle that fits behind the 3.5" by 1" profile of all
the old 40GB and 80GB PATA drives lying around. Something that I can stuff
inside old portable drive clamshells (converting old USB or Firewire kits,
etc.). We need high-density drive chassis backplanes that take a minimal
drive carrier and let us jam in these old drives and have it present a
single GigE copper connection on the back (built-in Ethernet switch). I
think the DIP switch addressing scheme needs to go away too. There are
plenty of device mapping abstractions in modern kernels (plus drive
content fingerprints) that having random device addresses is not a problem
worth trying to solve in hardware.

I think this AoE could be a powerful enabling technology that circumvents
the baggage and complexity of things like iSCSI and the NAS hype. Not only
could people build modern 12 petabyte arrays with 40,000 harddrives if
they wanted to, but people could also make brilliant and efficient use of
a lot of old hardware. (Wow, imagine the failure rates in a 40,000 drive
array!)

Are there any hardware experts in the audience that have connections with
production fabrication facilities? Want to start a new business? Like I
said, I've got aoe loaded in the kernel. I'm still waiting for the
hardware.

-- 
Andrew B. Sweger -- The great thing about multitasking is that several
                                things can go wrong at once.