{"id":34840,"date":"2024-04-12T10:56:02","date_gmt":"2024-04-12T09:56:02","guid":{"rendered":"https:\/\/www.intelligentcio.com\/apac\/?p=34840"},"modified":"2024-04-12T10:56:04","modified_gmt":"2024-04-12T09:56:04","slug":"why-organisations-are-spending-time-at-the-lakehouse","status":"publish","type":"post","link":"https:\/\/www.intelligentcio.com\/apac\/2024\/04\/12\/why-organisations-are-spending-time-at-the-lakehouse\/","title":{"rendered":"Why organisations are spending time at the lakehouse"},"content":{"rendered":"\n

James Wright, Senior Director, Asia Pacific and Japan, Cloudian, warns that the lack of broad understanding of \u2018spending time at the lakehouse\u2019 won\u2019t be a viable excuse for risking data security or breaking compliance rules and regulations around sovereignty.<\/em><\/strong><\/p>\n\n\n

\n
\"\"
James Wright, Senior Director, Asia Pacific and Japan, Cloudian<\/figcaption><\/figure><\/div>\n\n\n

First there were data lakes, then data warehouses, now \u2013 the inevitable data lakehouse.<\/p>\n\n\n\n

Most people don\u2019t really understand what any of these systems mean and most definitions do little to change that.<\/p>\n\n\n\n

But to put it in terms of value that people can understand, it\u2019s about easier, cheaper storage and management of, as well as creating value out of (you guessed it) data.<\/p>\n\n\n\n

Organisations have been storing unfathomable amounts of data on their business, customers, partners, and other areas for years now. The latest estimates indicate 0.33 zettabytes or 328.77 billion gigabytes are created every single day.<\/p>\n\n\n\n

That\u2019s the equivalent of more than 164 billion hours of HD video or nearly half a trillion physical filing cabinets worth of paper, every day.<\/p>\n\n\n\n

Cloud and data centre directory Cloudscene says Australia now has more than 300 data centres crunching our share of that pie, and that doesn\u2019t account for the many thousands of on-premises and edge data centres across the country.<\/p>\n\n\n\n

Still, we\u2019ve done very little to extract value out of all that data, and the more we add to the pile the harder it can become to do so.<\/p>\n\n\n\n

Essentially, the data lakehouse promises to change that with companies such as HPE, Snowflake and Vertica are leading the revolution. There are a whole bunch of technologies, applications, databases, storage components and other data \u2018things\u2019 involved that ingest data from diverse sources, but it\u2019s easier to understand it better through examples.<\/p>\n\n\n\n

Imagine an Australian telco wants to quickly pull up every single transcript of customer service calls and web chats that have occurred in the last year to assess key issues, customer sentiment, and the effectiveness of the company\u2019s response.<\/p>\n\n\n\n

A human would take years to do that, whereas an older IT system might take days, weeks, months or be unable to do it at all. If it could, it would also come at enormous cost in both real terms and to the IT capacity that telco needs for a range of business functions.<\/p>\n\n\n\n

The right data lakehouse with a touch of AI is pulling up that data and querying the issues, sentiment, and response in minutes, at little financial or resource cost. Business leaders can look at it, think \u2018huh, interesting\u2019 and make some real decisions based on it. The IT team can move onto the next query.<\/p>\n\n\n\n

But as with all things technology, and particularly our data, issues like security and sovereignty must be top of mind.<\/p>\n\n\n\n

The advent of AI, data lakehouses and everything that comes with them means that more and more data is going to be pulled up, analysed, and shared within and between organisations.<\/p>\n\n\n\n

Data is the crown jewel cybercriminals are after and organisations need to be careful about where that data is stored, who has access, and where and how it\u2019s backed up.<\/p>\n\n\n\n

These technologies are only going to increase the value of our data and it\u2019s already any organisation\u2019s most valuable resource. That means ransomware gangs will demand even more in ransom to release data they can hack.<\/p>\n\n\n\n

There are compliance standards to be mindful of too, such as the OAIC\u2019s Consumer Data Right system which requires businesses to follow strict information security requirements around governance, minimum system controls, testing, monitoring, evaluation and reporting.<\/p>\n\n\n\n

And as the Government moves to reshape the cyber legislative landscape to support the Cyber Security Strategy, new requirements may come into effect.<\/p>\n\n\n\n

It\u2019s also well established in the cyber security industry that there\u2019s no way to fully shut out cybercriminals, which means organisations using more data for more uses need to think creatively.<\/p>\n\n\n\n

They have to create an immutable, unencryptable version of their key data, siphoned away from cybercriminals even after they infiltrate.<\/p>\n\n\n\n

Data sovereignty is important in this context as data lakehouses regularly use public cloud infrastructure to host data, meaning that data can be removed from Australian shores and be accessible from abroad.<\/p>\n\n\n\n

Immutable copies of sensitive and critical datasets should be kept on-premises, regardless of the rounds they\u2019re doing at the lakehouse.<\/p>\n\n\n\n

With this in place, companies can reinject their crown jewel data without needing to pay the ransom and keep operating, selling and using tools like AI and data lakehouses to help them do that.<\/p>\n\n\n\n

Whether they understand the lakehouse world or not \u2013 Australian organisations need to understand that security and sovereignty must be prioritised when spending time there.<\/p>\n","protected":false},"excerpt":{"rendered":"

James Wright, Senior Director, Asia Pacific and Japan, Cloudian, warns that the lack of broad understanding of \u2018spending time at the lakehouse\u2019 won\u2019t be a viable excuse for risking data security or breaking compliance rules and regulations around sovereignty. First there were data lakes, then data warehouses, now \u2013 the inevitable data lakehouse. Most people […]<\/p>\n","protected":false},"author":58,"featured_media":34842,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[4664,1272,41,44],"tags":[2156,2808,2957,3180,6121,699],"class_list":["post-34840","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-intelligent-technologies-data","category-expert-voice","category-storage","category-top-stories","tag-cloudian","tag-compliance","tag-data-security","tag-james-wright","tag-sovereignty","tag-storage"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/posts\/34840","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/users\/58"}],"replies":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/comments?post=34840"}],"version-history":[{"count":1,"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/posts\/34840\/revisions"}],"predecessor-version":[{"id":34843,"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/posts\/34840\/revisions\/34843"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/media\/34842"}],"wp:attachment":[{"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/media?parent=34840"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/categories?post=34840"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.intelligentcio.com\/apac\/wp-json\/wp\/v2\/tags?post=34840"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}