Heading home at the end of the day? Chances are you will pull up the Waze or Google Maps app to check for peak hour traffic to either find the best route or the best time to leave. This seemingly trivial convenience today was impossible just 20 years ago. At the centre of this ability to zoom up to 30,000 feet to observe the state of the city is the availability of one key ingredient: Big data.
Sure, data had always been available prior to the 21st century, but what really changed at the turn of the millennium was the speed by which we collect and process these data points. Previously, hundreds of hours would have been spent on station counters and surveyors at key junctions, counting vehicles and conducting Level of Service surveys, for traffic studies. Today, your smartphone transmits these data points to a central core for processing and, in turn, shapes traffic in real time.
Big data has transformed the way we travel in our cities. Government and private sectors around the world are also beginning to use data mapping, modelling and analytics to gain more insights into human behaviour and preferences and make better decisions in planning and management. At the 25th MSC Malaysia Implementation Council Meeting (ICM) in 2013, the Malaysian government announced a Big Data Analytics (BDA) roadmap to unlock the value of its open big data with a pilot project targeting four segments—price monitoring, sentiment analysis, crime prevention and infectious disease forecasting.