By Clay Dillow
FORTUNE -- Big data has been favorably cast as "the new oil" and held up as the economic counterweight to America's sinking manufacturing sector. And while the "data is the new oil" analogy isn't perfect or even necessarily sound (data is both abundant and renewable, after all), there's some merit to the metaphor. As oil did at the beginning of the last century, big data is going to drive economies in the century ahead. But it may not do so in the way that many people think it will.
As with oil, companies know data is out there in large quantities and that it's not enough to simply know where it is -- it has to be extracted, refined, and delivered in a usable format to be valuable. And like the energy economy before it, the data economy needs dedicated people -- 4.4 million of them by 2015 in the IT field alone, according to an oft-cited Gartner Research analysis.
But here the similarities end. The oil patch has never had much trouble finding and training enough roughnecks to get oil out of the ground, but training up skilled big data professionals is a different enterprise entirely. In the U.S. alone, a McKinsey & Company report projects a shortfall of between 140,000 and 190,000 "deep analytical" big data professionals by 2018 -- that is, people with highly technical skills in machine learning, statistics, and/or computer science, the actual hands-on big data people that know how to crunch huge data sets into meaningful information.
But what's often overlooked in this dim projection of the big data labor market is that the impact of big data on employment goes far deeper than the deep analytics and IT fields. Companies need professionals at all levels that are not necessarily schooled in deep analytics but are nonetheless big data-savvy. These professionals don't need degrees in computer science or statistics.
A VP at management consulting and technology advisory outfit Booz Allen Hamilton recently told InformationWeek that the company has had great success bringing physicists and music majors onto data science teams -- creative thinkers who know less about computer science and more about how to look at big data problems in a different way. Though companies and economies will certainly need data scientists to manage their massive databases and information technology teams to support them, to a far greater degree they'll need professionals knowledgeable and creative enough to leverage big data to the greatest possible advantage.
"Advances in software, in interface design, and things like that will make it easier to analyze big data in the future," says Dr. Betsy Page Sigman, a professor at Georgetown University's McDonough School of Business and an expert on technology and information systems. "So it won't be as big of a technological hurdle. The more important thing for companies will be to have a lot of people that understand not just how to produce statistics and analytics, but understand how to make better decisions because they have this information."
Any employment bump tied to the proliferation of big data analytics won't be confined to IT departments or even to dedicated "data divisions" that emerge within companies. And it isn't just big data specialists like data scientists and statisticians that stand to benefit from this boom. Big data opportunities are already being exploited in data-centered pursuits like risk management, marketing, and research science, but the applications are virtually limitless.
Academically, big data is playing a role in decidedly non-data disciplines, like some portions of the social sciences and humanities, says Jim Spohrer, computer scientist and director of IBM's Global University Relations Programs. It will increasingly become integral in medical research, various kinds of product development and modeling, and all types of research science. To remain competitive, companies will require professionals at all levels that fundamentally grasp big data concepts and and know how to use them to their advantage.
"What we're seeing now is the tip of the iceberg," Spohrer says. "You think of all the different jobs that exist today. How is all the data associated with each person's job going to change the job and the skills they need?"
Part of this shift will simply require the retooling of existing jobs, but there is also a new class of positions and skills emerging as well, Spohrer says, positions like "chief data officer" that will become more common within existing companies large and small. These jobs won't necessarily be occupied by deep analytics-types either, but by non-data professionals educated and experienced in their chosen industry while also skilled in the use of big data tools (which, as Sigman noted above, will only become easier for non-data professionals to use).
Existing companies will expand as they deploy more big data resources -- both human and technological -- to leverage big data to their advantage. But the space to watch isn't necessarily existing companies reorganizing to embrace big data, but the emerging big data industry where deep analytical job growth is likely to make its biggest economic impacts.
As with so many other specialized endeavors that fall outside the purview of core business -- things like advertising and marketing, the latter of which is itself being completely transformed by big data analytics -- many big data applications will be farmed out to outside contractors who specialize in big data analysis and problem solving, Sigman says. Scores of big data startups are already emerging (largely with the help of venture capital) to meet this demand, including MapR, ParStream, ScaleArc, and Cloudant. The ones who are able to best meet their customers' data analytics needs are poised to become home to many of those many millions jobs big data will generate over the next few years.
The voice of Windows in the enterprise discovers that Mac users are more productive
That 41% of enterprises won't let Apple (AAPL) PCs anywhere near their computing services -- not even e-mail or the Internet -- should come as no surprise to the IT professionals who subscribe to Forrester Research's market research reports. After all, it reflects the advice that Forrester has been giving information technology departments for decades. Take, for MOREPhilip Elmer-DeWitt - Oct 27, 2011 8:00 AM ET
This is one in a series of articles leading up to Fortune Brainstorm Tech, which started today in Aspen, Colo. The articles look back at the progress of companies that presented at Brainstorm in 2009 as well as look forward to those that will present this year.
By Benjamin Snyder, contributor
HCL Technologies, an offshore IT and software development company, handles everything from setting up a company's cloud computing services and MOREJul 22, 2010 4:10 PM ET
But in the 19% of enterprises that do, Apple's 10% share is expected to double within a year
It's easier to get a camel through the eye of a needle than to get some tech departments that support Research in Motion's (RIMM) BlackBerry to change gears and support Apple's (AAPL) iPhone.
But that's starting to change, according to a report to clients issued Thursday by Merrill Lynch's (BAC) Scott Craig.
Drawing on a MOREPhilip Elmer-DeWitt - Jul 8, 2010 11:32 AM ET
Companies are dealing with much more data than they were twenty years ago, so why are they still storing it the same way?
By Charlie Silver
Based on some estimates, our current growth pace of data is setting mankind on a course to produce nearly 10 trillion gigabytes - or 10 "Zettabytes" - of data by 2015, enough to store over one quintillion pages of text or almost 70 trillion MOREJun 15, 2010 3:27 PM ET
Ever wondered why your company will support Research in Motion's (RIMM) BlackBerry but not the iPhone? Does it seem like the corporate deck is stacked against Apple (AAPL)?
A conference call with four chief information officers organized by Morgan Stanley's Kathryn Huberty last week might have confirmed your worst fears.
Of the four guests, only one -- the CIO of a multi-billion dollar company who runs a Mac shop for the "creatives" MOREPhilip Elmer-DeWitt - Jul 12, 2009 11:10 AM ET
The World Wide Developers Conference (WWDC) that opens Monday morning in San Francisco would be a relatively obscure technical gathering of programmers and IT administrators - with sessions on "Advances in OpenGL" and "What's New in Objective-C" - were it not for one thing.
The keynote address that Apple's CEO is scheduled to give starting at 10 am Pacific Time (1 pm ET) is perhaps the second most closely watched MOREPhilip Elmer-DeWitt - Jun 6, 2008 3:26 PM ET
In the weeks before the iPhone was released last June, there were few analysts more reviled by Apple (AAPL) enthusiasts than Gartner's Ken Delanay.
On June 19, ten days before the device went on sale, he gave it the information technology kiss of death. "We're telling IT executives to not support it because Apple has no intentions of supporting (iPhone use in) the enterprise," he told Network World's Jon Brodkin. "This MOREPhilip Elmer-DeWitt - Mar 20, 2008 7:59 AM ET
Wondering why your corporate Information Technology department won't buy you an Apple (AAPL) iPhone or support the one you bought yourself? Here's your answer.
Or, rather, 10 answers. Channeling the thought processes of IT managers who don't need many excuses not to support yet another platform, Forrester Research(FORR) has put together the definitive top 10 reasons not to support this one -- fully documented, complete with footnotes. Why now, just when MOREPhilip Elmer-DeWitt - Dec 13, 2007 9:12 AM ET
|The Deep Web you don't know about|
|Pizza chain Sbarro files for bankruptcy|
|Colorado gets $2 million from marijuana taxes|
|Invest $1 million, try for a U.S. green card|
|Shodan: The scariest search engine on the Internet|