Improving businesses in the ME by analysing big data

Over the last decade or so, the promise of big data has reignited new opportunities for innovation. In many organisations, the CIO is now truly considered to be part of the elite C-suite of executives, thanks in large part to the insights that IT can now deliver to the business in terms of identifying new business models, lucrative customer segments, and problematic products and processes. In other words, big data can make IT matter.

The use cases for big data technology, within and outside the enterprise, are growing in volume and variety. Within the enterprise, many use cases are focused on increasing profitability and operational efficiency, including identifying wasteful expenses and improving security. The most meaningful use cases may be found outside the enterprise in philanthropic organisations, non-profit entities, and even some government bodies.

Data may become a big differentiator for countries in the Middle East as they embark on their digital transformation journey. Analysing big data—including the often-overlooked “dark data” can yield valuable insights that enterprises can use to improve business.

 Data, data everywhere.

Every time a person swipes a credit or debit card, digital information is released and this information can be analysed by online retail stores to gauge customer interests and send out ‘push notifications’ with personalised offers that might be attractive to customers.

In healthcare, big data is being used to cure diseases, predict epidemics, and improve quality of life. Wearable devices, such as Fitbit, Jawbone and other health monitors, can transmit real-time data, including activity levels, nutrient intake, and other health indicators that doctors can use to diagnosis and treat ailments. On a macro level, information from devices can be fed into a general public healthcare database and analysed in order to find patterns and remedies for medical conditions. These are only some examples of the invaluable importance of big data and analytics and their potential benefits and impacts.

Irrespective of geography, industry or company size, in many organisations, the modern CIO’s role is closely tied to being able to extract actionable insights and quantifiable business value from enterprise data.

 The secret to extracting value from big data

Deriving value from analytics requires organisations to prepare data for use, however, today’s explosion of data volumes, types and sources makes this challenging.

Before big data is used, the organisation needs the right foundation to have confidence that the data is comprehensive, reliable, and timely. This foundation should enable the enterprise to easily integrate traditional data management technologies, like data warehouses and databases, with new ones like Hadoop and Spark; adapt to changes in the competitive landscape, emerging technologies, and changes in business operations; and avoid becoming locked in to any one approach or vendor’s stack. If the foundation is weak, no amount of investment in analytics software may be able to compensate for it.

Choosing the right big data solution only solves one part of the problem. Another challenge can be finding the right skills in the IT group to manage new big data workloads and applications. Since business intelligence and data warehousing were older models and analysis was done on structured and historical data, a data analyst typically would look at data from a single source. Today, big data analysis involves real-time analysis of both structured and unstructured data, data-at-rest and data-in-motion. This gives rise to the role of the data scientist with the ability and acumen to examine data from several sources, spot trends and provide unique insights that can offer businesses a competitive advantage.

Organisations should keep in mind the five key traits of an effective big data deployment:

1. Open – big data workloads and technologies are quickly evolving. Architectures should be built with interoperable, modular blocks so big data products can work with other solutions in the datacentre.
2. Agile – A nimble infrastructure lets enterprise IT respond to competitive threats, changes in industry trends, and consumer behaviour. Solutions should be chosen with easy scalability and deployment flexibility to help build apps and integrate data sources with extreme flexibility.
3. Secure – Securing big data workloads that support the IT governance standards of the enterprise. Solutions should help normalise the security models used across the datacentre, giving a simple yet holistic view of role-based security to data and apps.
4. Cloud-ready – Big data deployments can span physical, virtual, and private-, public-, and hybrid-cloud environments. Your organisation should be able to manage data and building apps across all environments should be done seamlessly.
5. Cost-effective – Investing in open source solutions can help free up valuable resources to help enterprises focus on more valuable tasks and support the business by discovering better ways to engage customers.

In running big data applications, organisations in the Middle East need to invest in aggregating data from social media in order to understand public sentiment towards their company, among other use cases. In addition, regional governments and enterprises should understand that big data holds tremendous potential in the development of smart cities and revolutionising industries such as healthcare, retail, oil and gas, banking, and telecommunications.