Sollicité à l'automne 2013 pour contribuer à un ouvrage sur les big data et l'évaluation des politiques publiques, j'avais adressé à l'éditeur une note d'intention, bien moins académique que le chapitre finalement publié. En voici le plan : 12 positions, 2 constats de révolutions, et 10 opportunités d'affaires.
- Big data and us: revolution was just around the last corner
- What big data is, and is not
- Big data is our daily experience: basically, the inputs and outputs of our phone, stored forever.
- Thus, big data is just a fact, just another logical consequence of the decrease in the cost of data storage.
- Thus, big data is pervasive, too. When every behavior turned digital, it turned tracked, at least potentially.
- Big data is all over the world. It bridged an information gap.
- Yet big data did not change, by itself, physical reality. You can ignore data as well — just switch off your phone.
- Big data is just the persistent digital layer, or digital skin, that covers almost every physical reality.
- So, big data crunched the world.
- “Big data” is not a new idea or paradigm.
- And even the collection, storage and use of big data are not that new.
- And big data doesn’t track your mind. It tracks behaviors. So far, it doesn’t track thoughts; that would be another revolution.
- Whether it is about behaviors or thoughts, big data can’t know anything; just as water dams can’t fly. Their electricity can power airplanes; data can feed algorithms that will generate useful information.
- Therefore, big data isn’t used so often to support decision making.
And that’s where the revolution starts. Much is on the way.
If data oceans are not new (their volume just doubles every year), building dedicated tools, methods, algorithms, science, in order to fish in such oceans, is rather new.
- What the big data revolution is about
The big data revolution is about becoming effective in handling data oceans, in “turning the data deluge into decisions”.
It involves a technological move, and a scientific one. Both moves started in the early 2000s.
Data Technology processing data of large volume with velocity.
Data Science turning statistical methodology upside down, from sowing in an information desert, to transforming data oceans into fried fish.
- What big data is, and is not
- Big data, public policies and evaluation: 10 opportunities or challenges
- The economics of public service change: big data brings a cheap deposit of information.
- A newly found deposit makes the fortune of shovels and pickaxes suppliers.
- Big data allows mass customization of service… (that's the less new side of big data)
- …and this customization gets in touch with people within their place and time
- Big data adds power to the monitoring of internal processes — as soon as they monitor things
- Big data can bring added intelligence to decision-makers, stakeholders and evaluators — even if it’s the perspective under which “big data delusion” is most often raised.
- Big data can bring added intelligence to users and citizens too.
- Crowd-sourced data transforms collective behaviors into information.
- Open data can make added value.
- Closed data can make added value, too.
As columnist Steve Lohr put it in the New York Times in 2012, "Big Data does seem to be facing a work-force bottleneck". Yes, big data skills may be missing sometimes; then, training and hiring are adequate solutions, while ignoring the issue is probably not. Big data is here to stay.
As the same Steve Lohr put it in the same column, "Big Data is great. But so is intuition." Sometimes, senior program managers or evaluators will feel that intuition skills are missing among junior staff, even if they were raised in the era of big data. Then, training and hiring are adequate solutions, while ignoring the issue is probably not. The need for intuition is here to stay, too.