Big data is large in a number of ways. It’s gaining a lot of attention for its ability to help executives make better decisions and uncover unexpected opportunities. But big data is also important for career opportunities. As companies need experts in the techniques and tools, people with the right skills can improve their incomes.
Technology publisher O’Reilly undertook an extensive survey in 2013 at a big data conference, looking at trends and what skills brought professionals better compensation. This is practical information that anyone with an information management degree should consider.
The most prevalent tool was SQL, with 71 percent of respondents using it. That shouldn’t be a surprise. Most companies will pull data out of their databases, and that means a good chance of a SQL-based relational system such as Oracle, Microsoft SQL Server, or even an open source product like MySQL. Excel was used by just over a third of respondents. Interestingly, more used the open source tools R and Python (43 percent and 40 percent, respectively).
The results showed a correlation between the number of tools that a professional used and salary. The average respondent used ten different tools and had a median income of $100,000. Focus on the people using 15 different tools and the income jumped to $130,000. What exactly that means is a little tricky, because the correlation was with the current use of tools and not necessarily the knowledge of them. It could be that people who made more happened to have a broader repertoire of tools than those who made less.
It could also be a matter of environment. Perhaps companies with a more demanding environment requiring a greater number of tools were ready to pay more for people who seemed to have the necessary experience, so the requirements, that ultimately had to include the tools, drove the salary. But knowing the additional tools would have put people in a better competitive position to get such jobs, so one clear lesson is to expand your knowledge.
There was also a split in tools, with open source on one side and commercial on the other. The respondents who focused on the commercial tools tended to “use them in isolation, without many other tools.” With the correlation between tool use and salary, it would sound as though embracing more open source tools could pay off.
In fact, the survey found exactly that. The median salary for people use used 5 of 13 commercial tools listed was $90,000. For those who used 6 of 19 open source tools, the median salary was $130,000. According to the analysis, because there seem to be fewer people who know the open source tools, it becomes a seller’s market and demand drives up salaries.
Higher salary also correlated with tools designed for big data, including Hadoop, MapR, Cassandra, Hive, and MongoDB. As the report says, though, the people who responded were on the cutting edge. Maybe it’s time to join them.