Every minute the world generates 1.7 million billion bytes of data, equivalent to 360,000 standard DVDs. More digitised data was created in the last two years than in the rest of human history. This trend and the mountains of data it produces is what we call "Big data". The big data sector is growing at a rate of 40% a year.
Handling big data requires increased technological capacity, new tools and new skills.
Even in traditional sectors like farming, the use of big data can have a huge impact. The tractor of the future will be enhanced with sensors that collect data from the machine, the soil and the crops it processes. The data will be analysed and combined with other data about weather and crop features.
What makes big data so important?
Take the example of the farmer. The results will help farmers make better choices on what crops to grow, and exactly when & where to sow them. The novel elements here are the use of data gathered by sensors, the integration of data from different sources, the use of real-time data-processing and the provision of visualisation tools for desk-top computers as well as hand-held devices for the farmer out in the field. All this can be combined with data about agricultural markets, altering logistics and the next season’s investments.
Big data could also predict the outbreak of an epidemic by analysing information on social media, such as Twitter. Analysing geographic patterns for people tweeting something vague such as: "In bed with fever” and “weird spots on my skin" may allow health authorities to identify epidemics much faster than t notifications by doctors and hospitals. Comparing data from social networks with official reports, including patterns of past epidemics, can refine our predictive capacity and response.
In summary big data is already affecting all areas of the economy. Studies indicate that data-driven decision making leads to 5-6% efficiency gains in the different sectors observed.
Intelligent processing of data is also essential for addressing societal challenges. Data could be used to enhance the sustainability of national health care systems and to tackle environmental problems by, for example, processing energy consumption patterns to improve energy efficiency or of pollution data in traffic management.
Why does the EU care about big data?
The core of the European Union is a single market that helps all our families, and businesses and national economies to prosper. Anything that affects each of our daily lives and our economy is automatically something the EU needs to think about.
Even though the European Union is the largest economy in the world, and makes up arund 20% of global GDP – today only 2 out of the top 20 companies changing lives and making money out of big data are European. We should improve that situation.
What is the EU doing about big data?
The most recent European Council Summit (October 2013) concluded that: "EU action should provide the right framework conditions for a single market for big data". We must ensure that relevant legislation supports entrepreneurship in this area. One example is the recent European legislation to open up government information and turn it into a source for innovation. We would expect Member States to transpose these rules rapidly and in an ambitious way into their national legislation.
Spreading this effort beyond “open data” will also necessarily mean achieving a critical mass of research and innovation on data in the Horizon 2020 programme, with 90 million euros available over the next two years..
Useful links
Big data in the Digital Agenda