Big Data: A Revolution That Will Change How We Live, Work and Think

We are witnessing the beginnings of a revolution. Big data, the explosion of information that digitisation has sparked, is changing our world in ways we are only just starting to appreciate. The book, ‘Big Data: A revolution that will change how we live, work and think’, is co-authored by Professor Viktor Mayer-Schonberger, Professor of Internet Governance and Regulation at the Oxford Internet Institute, University of Oxford, with Kenneth Cukier, Data Editor of The Economist .

The book aims to explain where we are, how we got here, and offers a guide to the benefits and dangers that lie ahead. In the 20th century, value shifted from physical infrastructure to intangibles, from land and factories to brands and intellectual property. Now another shift is taking place, in which a new source of value is not the power of one’s computer hardware but the data that is fed into it, and how it is analysed, they argue. Data is becoming a significant corporate asset, a vital economic input, and the foundation of new business models.

Big data, when allowed to speak, allows us to make connections that we never knew existed.  It is poised to shake up everything from businesses and the sciences, to healthcare, government, education, economics, the humanities and every other aspect of society, say the authors.  

The effect on individuals may the biggest shock of all: expertise matters less as probability and correlation become paramount, they say. Area specialists will have to contend with what the data says. Traditional ideas of management, decision-making, human resources and education will need to adjust.  Often it will not be humans making decisions but machines.The book concludes that this creates an ethical dilemma: should individual volition trump big data, even if statistics argue otherwise?  The authors also warn that the age of big data will require new rules to safeguard the sanctity of the individual.

Books asked Professor Mayer-Schonberger about his research

First, explain what big data actually means? Is there a threshold of online information that defines it as such or is the term quite loose?
The world is awash with data, and the amount we collect and store doubles every three years. So it is tempting to understand big data in terms of absolute size. But in our book we suggest that such a view is too limited. Rather we suggest three characteristic qualities of big data that we label “more”, “messy”, and “correlations”.
“More” implies that we have now more data available relative to a particular question or phenomenon we study than before. “Messy” means that with so much extra data, we can accept a bit of messiness in the data.
This leads to the third, “correlations”. With big data, we cannot prove causality. Big data does not shed light on the “why”, only the “what”. But in many instances this is good enough. In fact, as we suggest the quest to uncover underlying causes is hard, and often leads us humans to incorrect hunches. Looking for the “what” is a pragmatic alternative.

Which organisations or sectors are ahead of the game in the way they use and analyse big data?
To be a successful user of big data, one needs to have the data (or access to it), the skills to analyse it, but – at least for now – most importantly what we term a big data mindset: the ability to understand that data’s value lies in its secondary uses, and that through innovative uses, and combinations of data sources, this hidden value can be uncovered. It is little surprise that companies dealing with large amounts of data, such as Google, are generally excellent big data users. But other sectors are catching up fast.

What are the main risks in using the logic of machines to make decisions rather than relying on intuition and experience?
More often than we would think, intuition and experience are just plain wrong. So using empirical data rather than hunches should improve our economy and our society. But there are risks. One is that we humans use big data correlations to make decisions that imply causality. That is abusing data and would lead us down a very dangerous path. Another potential risk that is exacerbated in the big data age is what we call the “dictatorship of data” – that we rely too much on it.

What aspects of society are set to be shaken up most by this development?
There is no single aspect that stands out. But one of the most important aspects that are changing is how we investigate and understand the world around us. In a world of small data, we crafted a concrete hypothesis perhaps from a broader theory. We then collected the data we needed to “prove” or “disprove” the hypothesis. But that requires that we humans come up with the right hypothesis. With big data we can “let the data speak” – and have algorithms and data help us in coming up with and then testing hypotheses. As part of its project to predict the spread of the flu using Internet search terms, Google tested 500 million possible mathematical models to find the best fit.

How will it affect the way universities carry out research?
Big data has already changed many of the hard sciences. The medical sciences are up next for a flood of data, and big data insights. The social sciences will greatly benefit from big data, as it permits us to shed light on social dynamics at scale and much closer to real time. Significant parts of the humanities, such as history and literature, will benefit from big data insights – and at least to an extent be transformed into more empirical sciences. But the biggest change of all is perhaps how we communicate our knowledge. As we collect and analyse our teaching and teaching materials, we’ll find out what really works and what does not – and thus be able to improve learning, perhaps more than anything in the last century.

What are the main risks presented by big data?
The main risks are, in addition to relying on big data for causal decisions and fetishising data (mentioned above), of course the challenge that big data poses to individual privacy.

What safeguards are already in place to ensure our personal information is not shared without our knowledge? Does more need to be done?
The way we protect information privacy stems from the pre-Internet age. In the big data era, it stands to lose much of its effectiveness, and needs an update. What this update could include is detailed in an entire chapter in the book.

Is there anything we as individuals could be doing to protect ourselves?
Big data required a societal response, rather than something we humans can easily do individually.

On balance, is big data going to change our lives for the good or do you think the risks outweigh the benefits?  
Big data is like electricity or antibiotics – it is a very potent tool that can significantly improve the human condition if it is used wisely. But much like electricity or antibiotics, the benefits far outweigh the risks if we put in place the appropriate safeguards.