Cloud Computing: Big data analytics computing requires a ‘maverick fabric’ network

Grazed from IDG. Author: Bob Fernander.

The high-performance computing (HPC) scientific/academic sector is accustomed to using commodity server and storage clusters to deliver massive processing power, but comparable large-scale cluster deployments are now found in the high-end enterprise as well. Large Internet businesses, cloud computing suppliers, media and entertainment organizations, and high-frequency trading environments, for example, now run clusters that are on par and in some cases considerably larger than the top 100 clusters used in HPC.

What differentiates the two environments is the type of networks allied to the application programming models and the problem sets used. In the scientific/academic sector, it is typical to use proprietary solutions to achieve the best performance in terms of latency and bandwidth, while sacrificing aspects of standardization that simplify support, manageability and closer integration with IT infrastructure. Within the enterprise the use of standards is paramount, and that means heavy reliance upon Ethernet. But plain old Ethernet won’t cut it. What we need is a new approach, a new "maverick fabric."…

Such a fabric should have a way to eliminate network congestion within a multi-switch Ethernet framework to free up available bandwidth in the network fabric. It also should significantly improve performance by negotiating load-balancing flows between switches with no performance hit and, use a "fairness" algorithm that prioritizes packets in the network and ensures that broadcast data or other large frame traffic, such as localized storage sub-systems, will not unfairly consume bandwidth…

Read more from the source @ http://news.idg.no/cw/art.cfm?id=F9E288B9-A508-EEC3-B6F4353CCD6DCC0C