Poor public sector data quality hampers investigation

Poor public sector data quality hampers investigation

An attempt to investigate the connection between GP prescriptions and the contributing factors affecting them, has been hampered by the poor quality of public sector data. Next-generation business intelligence and data science company, Polymatica, announced the findings of a comprehensive analysis of government GP data, which discovered the amount of asthma medication prescribed has increased by 17% in six years, whilst the amount of antibiotics prescribed dropped by 12% during the same period.

However, as Polymatica CEO, Mark Hinds, explained, any conclusions were difficult to come by, with poor data quality limiting the ability to assess possible root causes. “We wanted to investigate the factors behind the findings, but we’ve been hindered by the quality of the data. What we’re able to see is that the message about reducing the number of antibiotics being prescribed is largely getting through. However, we can’t dig into why this has decreased, or why the level of asthma medication being prescribed has risen.”

The amount of antibiotics prescribed stood at 28.7m items in 2017, down from 32.8m in 2011, with Penicillin still the predominant antibiotic issued by GPs. The amount of asthma medication prescribed in 2017 hit 54.6m items for the year, up from 46.5m in 2011. Polymatica sought to use NHS Digital’s GP practice data for prescriptions written in England – consisting of over 700m rows of data – to study potential links between the rise and fall in prescriptions and other factors.

Hinds continued: “We wanted to see if external factors such as socioeconomic status or pollution would affect the level of prescriptions. But the data left us questioning whether the infrastructure and processes in place for data entry and management are up to standard. The government is clearly willing to make changes to public health policy – but what are they basing these decisions on? You need clean data to understand the root cause of problems like rising asthma medication. This is something that as a society we need to monitor and understand, but poor data quality is restricting these efforts. The consequences for this could be sizeable – impacting policy decisions based on data analysis and limiting the effectiveness of new technologies such as Artificial Intelligence (AI).”

The main data quality issue stemmed from incorrect and varying inputs on addresses. Polymatica found that cities and counties were often entered incorrectly or inconsistently; suggesting a manual input process instead of drop-down lists, resulting in spelling mistakes, use of various abbreviations and other inconsistencies. This made aggregating the data difficult, which in turn made it challenging to connect it with pollution and socioeconomic data – making analysis unreliable.

One focus of the analysis was understanding variations in the rate of prescriptions between regions. For instance, Yorkshire was the top prescriber for asthma medication in 2017 – with 4.2m items of asthma medication prescribed in 2017, which was far higher than London’s 2.9m, despite having significantly fewer GP surgeries and a smaller population. Yorkshire also had 2m items of antibiotics prescribed in 2017, compared to London’s 1.8m and bucked the national trend by increasing prescriptions since 2011.

“Ultimately, poor data quality harms results and creates inconsistent insights,” continued Hinds. “We can see Yorkshire is an anomaly in the UK – but we can’t begin to understand why that might be and certainly couldn’t create a national strategy around what might be bad insights from bad data. The government is taking the right steps on data quality but must do better. A good example is the British National Formulary for drug classification, which has allowed the NHS to accurately collect information on prescriptions. This is a great example of the gold standard in data quality that the government should be aiming to apply across the board to ensure they get accurate insights. Without quality data, we cannot unlock data’s potential to do good – helping spot the patterns that humans alone simply can’t. We cannot proceed as a society without learning, drawing conclusions and changing the way we operate, whether this is for a business or public authority. This is why it’s vital to improve data quality, or risk falling behind and failing to adopt revolutionary technology.”

Click below to share this article

Browse our latest issue

Intelligent CIO Europe

View Magazine Archive