March 21, 2013; Source: Harvard Business Review

In the social sciences and industry, researchers have been mining the deluge of newly available and high frequency digital data. As researchers struggle with the meaning, methods, and limits of this data, the United Nations’ Global Pulse initiative is making inroads in discovering its potential for social good. Digital sources of this “big data,” according to a Global Pulse Initiative report, include:

  1. “Data Exhaust: passively collected transactional data from people’s use of digital services like mobile phones, purchases, web searches, etc., and/or operational metrics and other real-time data collected by UN agencies, NGOs and other aid organisations to monitor their projects and programmes (e.g. stock levels, school attendance); these digital services create networked sensors of human behaviour;
  2. Online Information: web content such as news media and social media interactions (e.g. blogs, Twitter), news articles obituaries, e-commerce, job postings; this approach considers web usage and content as a sensor of human intent, sentiments, perceptions, and want;
  3. Physical Sensors: satellite or infrared imagery of changing landscapes, traffic patterns, light emissions, urban development and topographic changes, etc.; this approach focuses on remote sensing of changes in human activity; and
  4. Citizen Reporting on Crowd-sourced Data: information actively produced or submitted by citizens through mobile phone-based surveys, hotlines, user-generated maps, etc…”

Robert Kirkpatrick, the director of the Global Pulse initiative, rightly points out that much of this data is owned by private companies. So, although big data can be used to deliver low-cost estimates of household income in developing countries, inflation rates, or natural disasters and disease outbreak impacts (just to name a few applications), its use for these purposes is at the discretion of the companies that own the data.

Kirkpatrick ultimately calls on private companies to form and contribute to a real-time data commons in such a way that protects privacy and doesn’t compromise market competitiveness. He concludes, “Big data is a raw public good, and we must work together to find ways to harness it for massive social impact, both safely and responsibly. For this to happen, data philanthropy has to become a private sector priority.” In order to take steps toward this ambitious goal, he argues that innovative private-public partnerships will be required. We would add that private-nonprofit partnerships will also be necessary in order to better put this data to use for public benefit. – Michelle Shumate