Collaborating with the private sector and government partners, Smart Dubai is the government office charged with facilitating Dubai’s citywide smart transformation, to empower, deliver and promote an efficient, seamless, safe and personalised city experience for residents and visitors.
In order to deliver the goal of 100 smart initiatives and 1,000 smart services over the course of three years, Smart Dubai set out a data strategy to modernise the city’s infrastructure and deliver innovation through efficient, seamless, safe and personalised city experiences, ultimately leading to a stronger economy, better living, improved governance, and mobility.
As data sharing became mandatory under the Dubai Data Law, Smart Dubai was created to lead the world’s most ambitious and comprehensive data initiative in the world. As part of this unprecedented Digital Transformation journey spanning AI, IoT, Big Data and several other technologies, Smart Dubai’s initial vision was to create a data repository to host open and shared data for both public and private sector entities, allowing for an array of use cases ranging from easing traffic congestion to limiting queuing in hospitals.
With the view to become an impartial market broker for citywide data by creating an open and well-regulated data market to spark new opportunities – Smart Dubai needed to establish a suitable governance framework and a platform for organisations to use and access data in real-time, in order to build a smarter, more prosperous city.
Solution
As Smart Dubai’s data platform – Dubai Pulse – started to grow in size, hosting a larger number of datasets, the initiative was faced with a deluge of data to be stored, ingested and computed to gain valuable insights for the city. With the view of delivering real-time dashboards based on available information – either ingested or streamed – Smart Dubai needed a scalable data platform to run deep analyses and correlations.
“When looking for a suitable partner, open-source technology was a crucial factor as part of the tender. After evaluating an array of options and providers, we chose Cloudera due to the openness of the platform and the opportunity offered to our development team in addition to dedicated support,” said Matar Al Humairi, Assistant CEO, Smart Infrastructure Sector, Smart Dubai. “As part of our technology stack, we implemented Apache NiFi as part of Cloudera DataFlow (CDF) for real-time ingestion in order to sort, filter and parse data to help with the pace of real-time analysis. We also deployed Cloudera for ingestion and storage.”
Results
Having set the foundation to support a scalable data warehousing infrastructure, Smart Dubai was able to unlock unprecedented insights and deliver a variety of use cases across planning and development, social services and energy that would benefit the local population.
As part of its data journey, the organisation built in collaboration with Dubai Electricity and Water Authority (DEWA) an electricity and water consumption dashboard that allows comparisons at building level, or across entire communities by analysing DEWA consumption data, GIS data and other district cooling operators’ datasets. “Our big idea was to create a live dashboard that visualises how these resources are consumed, allowing for the comparison of use across buildings or entire communities, leading to a much better understanding of how water is used throughout the city and subsequently better planning,” explained Al Humairi.
Additionally, by building a 360-degree profile of citizens based on DEWA’s open data records, Smart Dubai has been able to illustrate flows of people moving across the city – whether to move to a new house, switch community or simply traveling – by building a real-time dashboard.
Al Humairi added that: “Our City Flow Dashboard has become invaluable for city planners, government entities, real estate developers and even new business start-ups. It has allowed us to turn raw utility provider data into a powerful visual tool that helps track community population trends and illustrates how Dubai’s residents are moving over time and space.”
Smart Dubai’s data platform has identified more than 2,000 datasets from participating Dubai government and semi-government entities, of which over 550 have already been ingested. It has also attracted more than two million hits from visitors and data seekers in 2019 alone, a figure that reflects the growing interest and importance of the city data.
In an effort to go digital as part of the most recent Dubai Paperless Strategy, the organisation is looking to expand the remit of its ‘DubaiNow’: a consumer application aimed to host any mobile transaction for a better and safe customer experience that is already offering over 115 city services to citizens.
Click below to share this article