Introduction

By 2050 it is expected that 70% of the world’s population will live in cities. Management of infrastructure and utility assets is therefore a growing priority. After decades of poor resource management, cities are now turning to data analytics and computational models to get it right.

Some areas where Big Data is valuable include utilities resource management, fraud detection and policing. An example of a city seeking to leverage Big Data tools extensively is Chicago where Mayor Rahm Emanuel created the Chicago Open Data Initiative in 2011 to provide more transparency over how private data is stored and used.

Like with private organisations, Big Data is useful to cities since it is a low-cost option with little physical storage required. Also, machine learning techniques that have been around for decades, but rarely used, can now solve key problems in urban planning and management. And the business potential for Big Data in cities globally is enormous – consulting firm McKinsey estimates that $1 trillion can be saved globally by optimising public infrastructure.

Civic Services

With increasing urbanisation comes increasing pressures on civic services, which can be eased using Big Data.

Engineering & Development

As urban populations grow, the scale of engineering developments required in cities is growing. With projects that are larger and more complex than designers have ever experienced before, data analytics can no longer be ignored.

For projects spanning entire neighborhoods, using intuition is less reliable as a tool than it is when building a single structure or city block, which is why engineers are turning to computational models. These models incorporate factors such as the impact of utilities, environmental factors, social impact and how local infrastructure and traffic is impacted and can be used to plan responses to risk events such as power outages, terrorist attacks or weather disasters.

Such engineering projects present opportunities to test new technology – building smart cities on existing infrastructure is difficult, as original city designs won’t have anticipated or accommodated for technologies such as sensors. This is similar to challenges faced by cities in the 20th century when developing motorways and gas/electricity distribution networks. For this reason urban planners tend to focus on piloting Big Data solutions in new build development projects, where such solutions can have maximum impact.

Examples of large scale projects currently in construction include Chicago’s 600 acre Lakeside Development and China’s “Turn the Pearl Delta River into One” project in South China which aims to merge nine cities into one mega city 5,000 square kilometers in area, via 150 infrastructure projects.

Policing

Due to Big Data’s low-cost, local police forces now have resources that only federal or state/national services would have had previously.

Crime patterns can be analysed to find when and where crime may happen and policing resources can be allocated using LAPD’s PredPol^[1]. In its first year, areas that used it saw a 10% reduction in crime rates while areas that didn’t reported a rise of 0.4% in crime rates. Some critics argue that the crime rate was on the way down, so it is hard to show the crime rate reduction is mainly due to PredPol.

Regardless of what one might think of PredPol, it cannot be denied that many local police forces are benefiting from more data being available with less paperwork to physically store. Police have access to data on suspects, data from body-worn cameras and from automated plate readers. Police can now also use smartphone data to try to find corroborating evidence placing suspects at the scene of the crime when it occurred.

One of these measures, body worn cameras, is being used to try to improve levels of trust between the public and police. There have even been instances where police have switched the camera off^[2], resulting in the dismissal or resignation of the officers involved. These cameras have been introduced in some US and UK police forces, however concerns have been voiced by civil liberties unions over protecting the privacy of officers and civilians^[3] while police officers have expressed doubts over their effectiveness questioning whether domestic abuse victims would be hesitant to call police if they felt they would be recorded^[4].

Utilities

Utilities companies are now using smart meters to address many of the issues they face such as billing problems, fraud and ensuring optimal service availability. Monitoring availability, usage and key system parameters allows for more accurate billing and therefore less over or underpaying for customers. Also utilities companies can identify key times when their grid or reserves might be stressed and act accordingly.

With smart meters, many issues with infrastructure can be automatically detected. From a safety perspective this is a significant improvement upon more traditional systems, where many issues would have gone undetected until an accident happened such as faulty electrical/gas mains or burst pipes.

Analysing data from smart meters for usage anomalies or line tampering makes finding fraud easier. In the UK, £100-400m per annum is estimated to be lost in revenues to power firms due to energy theft^[5]. Given that most utility fraud in UK is used for illegal drugs production, requiring enormous amounts of electricity and water, this is a key issue for police authorities.

Tracking Human Movement

As already noted, researching human behaviour is the key to avoiding urban planning mistakes of the past. And looking specifically at human movement can give enormous insights.

Traffic Management

Managing traffic has been a goal for cities since motorways were first built. Now with modern sensors, data analytics and coordinated use of technology it is possible to identify issues with traffic, control traffic flow and glean insights about civic related issues.

For instance, a project was recently put in place in Los Angeles to synchronise all the city’s traffic lights^[6] so congestion can be removed manually. Meanwhile, using cars’ sensor data can reveal the city’s traffic hotspots and how traffic is affected by events, seasons and days of the week. In addition public parking prices can be modified to reflect supply and demand. This isn’t the first attempt at managing traffic in Los Angeles – previous experiments in the 1980s involved installing loop detectors below roads to monitor traffic flow which was a much more expensive project than the more modern Big Data approach.

The City of Los Angeles isn’t the only entity using traffic data in such a way – INRIX, a Seattle-based company, provides traffic management services and analysis to 40 state DOTs in the United States, the United Kingdom Highways Agency and the UK’s two main terrestrial TV broadcasters BBC and ITV^[7]. INRIX provide traffic services based on real-time reports and sensor data, mixed with historical traffic flow trends.

To add to such traffic services, municipalities are also seeking data from cab companies to identify where public transport needs are not met. The global rideshare firm Uber have agreed to share their trip data on a quarterly basis with the City of Boston^[8]. Uber plan to provide similar services to other municipalities, having previously been hesitant about sharing their data.

This would complement cities’ existing uses of social media where information on delays in public transport is now available in real-time in many cities. But there’s more to be gained than just public transport information – this could alert authorities to where street repairs are required. This is in addition to monitoring data from calls to emergency services, where issues such as potential rat infestation or public health hazards to children can be detected.

Mapping Disease and Shortages in Food & Water

Mobile phones can be used track human behaviour and diseases. Approximately 80% of the world’s cellphones are in developing countries and, although many handsets will have basic features, they reveal location data while using phones for making payments is quite common. This can reveal data on health patterns, employment trends, social tensions, economics, poverty and transport in countries where such data is generally unavailable through more traditional means.

Tracking population movements using cellphone signals results in not only a better understanding of disease outbreaks but an eradication strategy. This data can also highlight abnormalities in disease outbreaks, with a level of detail not previously possible. An example of a pioneering project to map diseases is being run by Caroline Buckee and Nathan Eagle between the Harvard School of Public Health and MIT’s Human Dynamics Lab^[9]. This project shows how human movement influences disease spread, using mobile phone signals to track movement, and how it can be contained.

A key strength of this project is that it combines data analytics with ground level intelligence. This is an improvement upon previous attempts at mapping disease such as Google Flu Trends, which estimates the number of visits patients will make to the doctor for flu related illness, and that has consistently overestimated these rates since 2011.

Data from mobile phones and satellites can reveal where food and water shortages may occur. Many countries face potential water shortages in the coming decades – reports of California’s groundwater reserves depleting originate from an analysis of data from NASA satellites^[10]. Similarly, many countries in North Africa, the Middle East and South Asia could face water shortages.

Data analytics can be used to identify arable land and plan food distribution, giving more scope to anticipate potential shortages. Furthermore links between food shortages and civil unrest, terrorism and war can be analysed more closely – indeed in 2010 RecordFuture, a cyber security and analytics firm, noted how shortages affected civil unrest and war in Yemen^[11].

Data Privacy Issues

Municipal data analytics projects involve using enormous amounts of public data, therefore data privacy is a key issue. Within data protection laws in most jurisdictions it is impossible to legally obtain private data without an individual’s consent, unless this data is required to enact a contract, for legal purposes or for the purposes of a police investigation.

Passing private information to 3rd parties and/or using it for the purpose of marketing is usually only allowed under the strictest circumstances. Furthermore data, such as sensor data for traffic monitoring or mobile app data, is usually required to be anonymised so that it is impossible to reverse the process and identify the person it relates to. In particular, Uber will anonymise any trip data they provide to the City of Boston so that users and drivers cannot be identified.

However as many firms and governmental organisations have found over the years, public trust remains an issue particularly when the public is unaware of how their information is used. Indeed much confusion exists over what data police and intelligence services can access without a warrant. Also, while the anonymisation of data is legal, the technique can often come across to the public as a legal loophole to use private data rather than as a protection measure for the public’s benefit. Finally, the risk of cyber attack exposing private data remains a very real possibility from which there would be no legal recourse in a worst case scenario.

Municipalities such as the City of Chicago under Mayor Rahm Emanuel have therefore put initiatives in place, such as the Chicago Open Data Initiative, to make public archives readily available and to raise citizens’ awareness of this and their rights, particularly to see what’s kept about them on digital files.

Conclusion

As urban populations increase, pressures on city services are expected to grow. This is why municipalities are turning to Big Data and computational modeling to manage their resources more efficiently. But Big Data should not replace intuition or more established processes, as Google learnt when tracking flu rates, rather it should complement and strengthen urban planners’ existing skills.

Many Big Data solutions are new but companies such as PredPol, INRIX and Uber already show positive results. As Big Data solutions get adopted globally the $57 trillion investment in public assets McKinsey say is required between 2013-2030 presents significant commercial opportunities for Big Data. There are also opportunities to gain new insights using private companies’ data – for example, retailers’ data could potentially warn of food and water shortages. However companies’ willingness to share data may be limited.

Finally, cities will face a big challenge to protect the public’s privacy as well as maintaining their trust. This is an especially controversial issue in policing – both in general use of data and with body worn cameras where many forces have refused to use them. In general, only a small number of privacy breaches are required to set public trust back significantly, so the onus will be on municipalities to go out of their way to be transparent about how they use the public’s data and to adopt data policies similar to Chicago’s Open Data Initiative.

References

1. ^Scientifically Proven Field Results.
2. ^Daytona Beach police officer resigns after body camera turned off during arrest, 14 May, 2014.
3. ^New era of policing: Will the benefits of body-worn cameras outweigh the privacy issues?, 21 November, 2014.
4. ^Jersey City cops urge caution on plan for police body cameras, December 5, 2014.
5. ^What is Energy Theft?
6. ^Less Traffic on Your Commute? Thank Big Data, October 1, 2014.
7. ^How big data can end gridlock on the roads, 9 September, 2013.
8. ^Uber to Hand Over Trip Data to Boston, January 13, 2015.
9. ^Big Data from Cheap Phones, April 23, 2013.
10. ^Why global water shortages pose threat of terror and war, 9 February, 2014.
11. ^Yemen Heading for Disaster in 2010, January 12, 2010.

Authored by:
Liam Murray

Liam Murray is a data driven individual with a passion for Mathematics, Machine Learning, Data Mining and Business Analytics. Most recently, Liam has focused on Big Data Analytics – leveraging Hadoop and statistically driven languages such as R and Python to solve complex business problems. Previously, Liam spent more than six years within the finance industry working on power, renewables & PFI infrastructure sector projects with a focus on the financing of projects as well as the ongoing monitoring of existing assets. As a result, Liam has an acute awareness of the needs & challenges associated with supporting the advanced analytics requirements of an organization.