How to prepare for a big data world

It’s one of those apocryphal facts which turns out to be true: 90% of all the data in the entire world was created in the last two years. Each day we make and store around 2.5 quintillion bytes of data. Experts are now predicting a so-called ‘Data Apocalypse’ where we’ll simply run out of room to store all this new information. Cloud-based companies like Google, Dropbox and pCloud are building data centres at about the same rate that McDonalds builds restaurants, but it might not be enough; scientists are now looking at coding data into DNA as a possible solution.

The point is: companies are collecting more data than ever before. And it’s not purely to sell you stuff. We now use data analytics to help track rhino poachers in South Africa, measure economic health by the number of lorries on the road, and help Germany beat Brazil at soccer. The International Data Corporation predicts that global revenues from big data and business analytics will reach $210b in 2020.

“The trend we’re seeing in the market is not just about highly skilled data scientists. The growth we’re seeing is the need for knowledge. We need workers to understand and analyse data to extract insights,” says KJ Kim, from Tableau Software. “This trend is partly due to the sheer amount of data we’re handling day to day, as well as the availability of user-friendly software that makes analytics more accessible to people of varying skill levels.”

The future might be data-driven, but that’s not necessarily a bad thing. There are opportunities, if you know where to look.

Upskill

Data Analysts and Data Scientists are some of the fastest growing professions in the world. These days, every company is a data company, whether they like it or not, and they need skilled workers to help collect, organize and analyse data patterns. The McKinsey Global Institute estimates 200,000 data job openings in 2019, with salary benchmarks at around $147,000. Upskilling in data analytics, data science, machine learning and Python programming is probably one of the best ways to future-proof your résumé. And although demand is booming, there’s a significant data skills gap as tech leaders and universities pivot into data education. Which brings us to…

Learn, learn, learn

The brilliant thing about data is that you don’t need formal STEM qualifications to get started. In fact some people think the data skills gap is partly fuelled by senior executives misunderstanding what it means to be a data scientist. “Engineers or data scientists are going to need specialised skills that are relevant to their industry or area of analysis,” says KJ. “However, there’s also a need for business professionals that understand how to explore and gather insight from the data. Their knowledge of the data’s business relevance is critical to weave a coherent story, backed by good numbers.” For many data jobs, an online short course (or series of short courses) is sufficient to get your foot in the door. It’s why RMIT recently launched it’s Big Data programs, including Business Analytics, Data Science and Machine Learning.

Use data for good

Big data and privacy concerns go hand in hand. Experts have pointed out that the rise of machine learning and data analytics has, in some ways, outstripped our ability to regulate these fields. Here’s an example: in the 1970s, The Belmont Report laid out the core principles for ethical research with human participants. But nowhere in that document does it make allowances for data companies monitoring social media behavior, or Facebook’s notorious 2014 ‘emotional contagion study’, which manipulated the newsfeeds of 689,000 users to assess their emotional response. ‘Opting out’ and the notion of digital privacy is becoming increasingly difficult for users. Understanding ethical obligations and legal boundaries is going to become increasingly important for data scientists, especially as governments move to tighten global data restrictions.

Get creative

Anyone who’s read Freakonomics knows that data isn’t simply numbers: data tells a story. And it can be used in thousands of creative and interesting ways. The story of the Oakland A’s using data to turn the tables on league baseball is already the stuff of legend. Organisations like the Farmers Business Network collect seed data to help US farmers make better crop decisions and increase yield. Parisian hospitals are using machine learning to help predict the number of incoming patients. IBM is running similar systems to decrease the morbidity of heart attack victims. Scientists are even hoping that big data might hold the key to saving the Great Barrier Reef. The limits of data application are really the limits of the human imagination. What that means for the future is anyone’s guess – but if the last five years are any guide, change will be drastic, far-reaching and very, very swift.