Think Outside the Stack

When I look at where messaging technology is today, I am amazed that most solutions still rely on an IP stack that dates back to the seventies, with only minor enhancements. In that same timeframe, networks have gone from 10 Megabit shared to 10 Gigabit switched and Intel processors have advanced from the 286 with about five MIPS of processing power, to today’s latest generation processors with over 100,000 MIPS. Bottom line, messaging software has not innovated anywhere near as fast as hardware and it has become the bottleneck to performance.

To address these failings, we launched Data Fabric over three years ago. From the start, Data Fabric was designed and engineered to take advantage of next generation hardware. By leveraging technology from companies such as Intel and Mellanox , Data Fabric delivers unprecedented performance with unmatched throughput, scalability and latency without the need for exotic hardware accelerators.

Data Fabric was one of the first commercial messaging products to implement RDMA – Remote Direct Memory Access. This allows the movement of huge amounts of messages with deterministic, low latency (single digit microseconds). While this is still an industry leading solution with a large number of production deployments, we continue to look for better ways to do things. With the latest version of Data Fabric, we are providing MultiVerb – an IB verb level implementation of multicast for 10 Gigabit Ethernet and InfiniBand networks. This release  presents massive scalability, while delivering extremely low and deterministic latency and true kernel bypass.

As part of our testing of Data Fabric, we were able to utilize Intel's Customer Response Team (CRT) datacenter in DuPont WA. There we deployed Data Fabric MultiVerb onto 360 Intel Westmere class servers with Mellanox C-X2 cards and a QDR Infiniband network. We published a million 200 byte messages per second to 1,000 clients with an average latency of 5 micro seconds and 99.99 percentile latency of 19 microseconds – that’s a billion messages per second with minimal latency. Scalability and latency are almost non issues now with Data Fabric – so go ahead and “Think Outside the Stack”.