Big Data and the case for IT relevancy
March 15, 2012
The case for IT’s long-term relevancy in the workplace is about to head for the jury, and big data and cloud computing are going to be a large part of the closing arguments.
Ever since the last hype-cycle of cloud computing, there have been a lot of gloom-and-doom predictions about the role of IT in the enterprise. The logic was straightforward: if server operations were to be heavily automated, then what would be the need for large cadres of system administrators?…
This sort of talk tends to make sysadmins understandably edgy, particularly given the standard corporate response whenever stock prices need to go up: trim the workforce. Other hotter-than-the-Sun IT concepts–such as DevOps–tend to put sysadmins on the defensive, too. After all, if developers can encode job allocations within cloud-based applications, that’s even less need for system and database administrators in the loop, right?
And then there’s big data, which some argue has the potential to shove IT out to the margins even farther.
The argument is not as straightforward, but it basically assumes that, as business intelligence and data analytic tools get smarter and easier to use for those employees with the MBAs, there will be less need for "pure" data managers and therefore further marginalization of IT as a whole.
Toss in the coming wave of big-data appliances and hosted big data services, and you’ve got yourself a recipe for more IT downsizing, right?
Perhaps not.
Based on what I saw at the Strata conference a couple of weeks ago, I would submit that there is not only plenty of room for IT in the big data sector, but that big data may actually make IT more relevant in the business place.
It is clear to almost everyone that businesses are under a deluge of data from a variety of sources, and they are trying to figure out (a) what to do with it and (b) how to apply what they learn back to their business practices.
These are tasks that, unlike system operation and automation, can’t really be scaled, because at the end of the day there’s going to need to be a data scientist (or a team of data experts) looking at the data and making decisions. And right now there is a huge shortage of data scientists.
Big data does not have to be a push for business workers to crowd out IT staff–instead, it offers IT an opportunity be more part of the business process than ever.
To be a data scientist, you need a lot of cross-disciplinary skills. Typically, most people assume that this is going to be businesspeople learning how to be techies, but it can go the other way, too: technologists can just as easily learn to be business savvy.
In fact, given that IT is more of a moving target than business practices (both general and within a company), I might risk the ire of businesspeople by putting forth the idea that learning about business is going to be a bit easier for IT staff than vice versa.
Even if all things are equal, big data offers a big chance for IT to become much more important to the success of a company, rather than being regarded as support staff who help get the "real" work done.
That’s a huge shift in IT’s role, and not every company is going to be ready for it. But those that want to be ready would do well right now to get the business and IT departments sitting down to have frank discussions on the role each side can play to make the shift to a data-driven organization happen.