{"id":29008,"date":"2019-09-18T09:28:50","date_gmt":"2019-09-18T08:28:50","guid":{"rendered":"https:\/\/www.intelligentcio.com\/africa\/?p=29008"},"modified":"2019-09-20T11:11:13","modified_gmt":"2019-09-20T10:11:13","slug":"decision-inc-experts-on-working-towards-a-better-data-management","status":"publish","type":"post","link":"https:\/\/www.intelligentcio.com\/africa\/2019\/09\/18\/decision-inc-experts-on-working-towards-a-better-data-management\/","title":{"rendered":"Decision Inc experts on working towards a better data management"},"content":{"rendered":"\n

Big data is seeing organisations becoming more mindful of aligning their data management practices with newer paradigms such as data lakes, elastic and cluster computing, and real-time data. Stefan Kruger and Peter Gaertner, senior data architects at Decision Inc, say that the cloud provides an environment capable of managing the volume and scale required to do so.<\/strong><\/p>\n\n\n\n

There has been a continued drive to improve the mechanisms for dealing with the processing and analysis of big data volumes, such as Machine Learning (ML). <\/p>\n\n\n\n

However, despite these innovations,\nchallenges such as bad or biased ML models are not easily solved and require\ncontinuous attention and improvement moving forward.<\/p>\n\n\n\n

This is where an effective data\nmanagement plan (DMP) is essential. It empowers the business stakeholders to\nconsume and model the necessary data and generate business insights from it. A\ndata management plan does not compromise on security and serves to establish a\nflexible, agile, and forward-thinking environment needed to cater for big data\nin a digital world.<\/p>\n\n\n\n

Regulatory drive<\/strong><\/p>\n\n\n\n

An integral part of the DMP is to\nensure the company remains compliant with current regulations. This requires an\nunderstanding of how the collected data is stored, used, and analysed within a\ncomplex regulatory environment.<\/p>\n\n\n\n

Companies must ensure the affected\ndata is anonymised and encrypted as necessary while applying transparent\nprinciples on how it is collected, and the use cases it provides for. All this\nshould be applied in a legal framework suited to all relevant legislative\nguidelines as set out by governments (think South Africa\u2019s POPI, the GDPR of\nthe EU, and Australia\u2019s CDR).<\/p>\n\n\n\n

And while this regulatory context\nis key, companies who are adapting their DMPs must also be aware of the most\nimpactful trends in data management. Things such as automation and DataOps, the\ncontinued hybrid and full cloud migration of organisations, data streaming from\nInternet of Things devices, and of course the compliance focus.<\/p>\n\n\n\n

Subtle\ndifferences<\/strong><\/p>\n\n\n\n

Part of this \u2018new\u2019 data environment\nis DataOps. Contrary to popular belief, this is not just DevOps for data. It\nrequires data management solutions to take into consideration key aspects that\nhave traditionally been less prevalent in the data management space.<\/p>\n\n\n\n

For example, ubiquitous version\ncontrol and collaboration. This is necessary to carry the full solution in\ncentralised source control and facilitate collaboration between stakeholders.\nIt often has the nett effect of reducing the reliance on traditionally\nUI-driven tasks and changing these to automated scripts.<\/p>\n\n\n\n

Secondly, DataOps sees\ninfrastructure as code also playing a role. This defines the infrastructure\nneeded by a solution and packaging it as a defined, dynamic, and versionable\npart of the data management process.<\/p>\n\n\n\n

Underpinning these elements is the\nneed to enable automated testing routines that run when a new version update is\ndone as part of the DMP process. Automated feedback and flagging of testing\noutcomes to the required stakeholders form a core part of this.<\/p>\n\n\n\n

Good data<\/strong><\/p>\n\n\n\n

Part of ensuring an effective DMP\nis put in place comes down to process engineering. Companies must embrace good\nprocess and change existing ones where necessary. Decision-makers must avoid\nthe risk of simply digitising or regurgitating bad processes.<\/p>\n\n\n\n

Also, it must be remembered that\non-premise infrastructure does not necessarily always translate directly into\ncloud architecture concepts. Companies must ensure that any cloud-bound\nmigration (whether hybrid or full) is designed to leverage the targeted cloud\nbenefits where it makes the most sense and delivers the most impact from a functionality\nand a cost savings perspective.<\/p>\n\n\n\n

Data management will continue to\ngrow in impact as more businesses embark on exercises in digital\ntransformation. The cloud-oriented focus will continue, but the legacy of\non-premise solutions will still be around for some time, especially in\nenterprise environments.<\/p>\n\n\n\n

The convergence of solutions\ncatering for data management (such as master data, data quality, integration,\nstorage and intelligence) will remain an interesting area to watch as\ntechnologies evolve. However, a DMP requires a considered approach that factors\nin both existing and future data requirements. Companies must be willing to\nembrace change in this new digital world.<\/p>\n","protected":false},"excerpt":{"rendered":"

Big data is seeing organisations becoming more mindful of aligning their data management practices with newer paradigms such as data lakes, elastic and cluster computing, and real-time data. Stefan Kruger and Peter Gaertner, senior data architects at Decision Inc, say that the cloud provides an environment capable of managing the volume and scale required to […]<\/p>\n","protected":false},"author":25,"featured_media":29009,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[4950,134,182,46,275,13,262],"tags":[9727,116,11071,2239,842,11072,1059,291,11070,11065,478,445,392,7257,11068,876,4569,453,11067],"class_list":["post-29008","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-analysis","category-blog","category-cloud","category-data-centres","category-intelligent-technology-newsletter","category-top-stories","category-used","tag-australia","tag-big-data","tag-cdr","tag-data-lakes","tag-data-management","tag-dataops","tag-decision-inc","tag-devops","tag-dmp","tag-elastic-and-cluster-computing","tag-eu","tag-gdpr","tag-machine-learning","tag-ml","tag-peter-gaetner","tag-popi","tag-real-time-data","tag-south-africa","tag-stefan-kruger"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/posts\/29008","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/users\/25"}],"replies":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/comments?post=29008"}],"version-history":[{"count":3,"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/posts\/29008\/revisions"}],"predecessor-version":[{"id":29017,"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/posts\/29008\/revisions\/29017"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/media\/29009"}],"wp:attachment":[{"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/media?parent=29008"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/categories?post=29008"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/africa\/wp-json\/wp\/v2\/tags?post=29008"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}