Can Big Data Better Serve the Public?

By Gidon Ben-Zvi

2.2.2016 twitter linkedin facebook

As the digital data universe grows bigger by the day local and national governments are embracing innovative methods to harness Big Data technology to better serve citizens.

Initially, the unparalleled performance of Big Data platforms were seized upon by the private sector as a revolutionary way to boost analytics performance and rapidly grow a company’s revenues.

As Big Data has matured and transformed into a mainstream business agenda, governments around the world are realizing its potential as a key decision maker.

For the People: Big Data can help deliver better services, improve efficiency of operations and enhance communications between the governing and the governed.

Big Data Nation

On the national level, interesting initiatives have been developed in Japan, China and the United States, to name three.

In 2013 the Japanese government began developing a program that would use Big Data technology to ultimately enable the creation of more responsive, less bureaucratically dense services to citizens.  The “My Number” law was enacted to break down the data-sharing barriers that existed between various government departments. Recently, Japanese citizens began receiving a unique 12-digit “my number” that will allow them to access tax, social security and disaster relief benefits from all applicable levels of government. This eliminates duplication, delay and frustration while promoting seamless data-sharing between the departments.

The Chinese government has gone a step further in its embrace of Big Data with the launch of the first of several planned big data zones in Guizhou. As part of this first data zone, the government will build a cloud-based Big Data platform to integrate city and county-level government departments around Guizhou. If the initiative is successful, the government hopes to expand the initiative to a single, unified and nation-wide platform for all government data by 2018.

Meanwhile in the United States many government agencies are increasingly leveraging analytics and Big Data to gain actionable insights about a range of topics.

In 2012 the Obama administration committed $200 million to Big Data to drive research and development. In 2014 the Executive Office of the President published “Big Data: Seizing Opportunities, Preserving Values,” which examined how Big Data will transform the relationships between government, citizens, businesses and consumers. The report notes that Big Data and advanced analytics can enable every part of government to “serve citizens better.”

President Data:  In 2015 the US federal government officially acknowledged the importance of Big Data.   


In February 2015 the White House announced the appointment of US Chief Data Scientist DJ Patel, the first person with that title.

Big Data Challenges

Despite the progress in implementing Big Data solutions, government agencies are grappling with many of the same issues as their counterparts in the private sector.

Specifically, lack of staff resources – business and data analysts, data scientists, engineers and data architects – and the costs of an infrastructure refresh are two leading concerns. Other issues include the challenge of creating an analytic system that provides real time analysis and implementing Big Data analytics.

Companies such as SQream Technologies are making it possible for city hall to implement hassle-free, cost-efficient and readily available Big Data solutions. In an era of increasingly tight local budgets SQream provides real-time, comprehensive, actionable insights from enormous data sets in an effective and cost-effective manner.

SQream is uniquely suited to provide public agencies with the power of a full-rack database machine – like Oracle Exadata, Teradata and IBM Netezza – without busting the budget, by condensing data into a single standard 2U server, delivering the most cost-effective performance for Big Data. The result: dramatic decreases in energy expenditures, datacenter floor space and a significant lowering of the TCO.

Big Data’s Transforming Public Services

Distilling the flood of data into meaningful information is helping bridge the gap between citizens’ needs and governments’ actions. Big Data is enabling public agencies to better target services, better design those services and better evaluate the economic benefits of government programs on valued constituents.


Data Deluge: With the amount of data being collected, analyzed and stored by government agencies growing exponentially, Big Data’s time is now.


Big Data companies such as SQream have the technology required to help facilitate real time analysis in new ways to improve a wide variety of public services.

Along with the requisite technical requirements, governments big and small can more effectively use Big Data by creating more positions for data scientists, analysts and statisticians.

By investing today in Big Data the government can become more efficient, save money and ultimately help public bodies better serve their citizens.