From Data to Deployment: Creating a Strong Foundation for AI
Francesco Tisiot
|RSS FeedHead of Developer Experience at Aiven
Subscribe for the latest news and insights on open source, Aiven offerings, and more.
The third article in our Data Economy series examines what it takes to unlock the opportunities with AI. Based on some of the key findings of our own research ‘Predicting the Data and AI Revolution’ and exclusive insights provided by leaders in the data economy, this piece explores how organizations are currently using AI. This includes the major challenges they are facing and, more perhaps importantly, what can be done to address them in order to create businesses that are built to be data-ready with AI at their core.
It’s no secret that businesses around the world are in an arms race to adopt AI. But, in keeping their eyes on the shiny prize at the end, many are skipping a crucial first step for successful deployments: putting their data house in order.
It’s the modern day equivalent of putting the cart before the horse and, much like with the simile, a crash is inevitable unless businesses approach AI adoption in the right way.
It’s why, despite so many organizations investing in AI, that the value derived from those solutions has been limited. Indeed, research conducted by Aiven among executives found only 17% of organizations have successfully moved AI projects into full production despite widespread experimentation. The research also found that more than half of these C-Suite executives cite data-related challenges as the main barrier to success with AI deployments.
What is happening is that AI is the epitome of the ‘garbage-in, garbage-out’ concept. Because most businesses were, quite understandably, not primed for the emergence of AI, they have taken to throwing data at the problem and hoping some of it sticks. Spoiler alert: it won’t.
As part of our data economy campaign, we have spoken to individuals all playing differing roles in its development. One of those individuals is Nilesh Bansal, Co-Founder, WorkOrb who told me, “companies pursuing AI purely for fear of missing out are skipping the important step of building a solid data foundation layer first. My advice would be to start with the data and then build on top of it.”
To tackle the foundational challenges businesses are facing when it comes to getting their data AI-ready, we first need to know what they are.
As data is moved between environments, fed into ML models or leveraged in advanced analytics, considerations around topics like security and compliance are top of mind for many businesses. Our research found that well over half (58%) of executives see data security and privacy as the number one obstacle unlocking the potential of AI. There is a sense of ‘if we don’t move it, it won’t break’.
There is, of course, the other side of the issue. That businesses are reluctant to put too much data into an AI because of a mistrust about where the data goes and/or how it is used. They keep confidential and business-sensitive information away from AI which creates the sense of security businesses are looking for but compromises the AI output — and therefore the wider perception of AI — in the process.
This is why we’re seeing a groundswell of popularity for AI sovereignty. The majority (80%) of executives in our research emphasized that data sovereignty and control are equally critical to AI-driven strategic success.
While on the one hand we have a burgeoning economy built on data, on the other hand there is a large proportion of businesses that simply don’t know how to handle it. And, if we are to unlock the proposed USD 15.7 trillion that AI is expected to contribute to the global economy by the end of the decade, this has to change.
The question is, how?
The data economy is something to behold. The change it has enacted and the speed at which it has done so has not been seen in human history. But progress is not linear and it is clear that ambition is being hindered by inadequate data readiness with significant gaps between AI aspirations and data capabilities.
Organizations need to be data-ready to truly be AI-ready. The accessibility of GenAI has created a false sense of ease around AI as a technology. However, unlocking the full value of proprietary data is complex and has revealed stress fractures within organizations. Businesses are eager to differentiate through AI-powered innovations but require efficient ways to manage and leverage their data.
As Bansal points out, “The data practices that the companies need in the new era of AI don't change. The fundamentals remain the same. Companies need to start with making sure they're collecting the data they need for their businesses. Having the right infrastructure in place that allows teams to access this data in an easy manner, while maintaining governance and privacy controls.”
Luca Eisenstecken, Partner, Atomico also told us, “Data is the foundation for AI and production. It is a necessary building block to create and run AI applications that deliver real business value at scale, which in itself drives the data economy much akin to a perpetual flywheel.”
The more data an organization has, the more it can use and the bigger role it plays in the data economy. It’s a self-fulfilling prophecy and one of the reasons why there is a disparity between companies at either and of the data scale. A point explained by Jérémy Barneron, Senior Software Engineer at fintech start-up, Dojo who said, “The larger tech companies have already mastered the data economy and how to make sense of their data in real-time to grow their revenue and deliver value to their customers. We need to start leveling the playing field and find a way to make data accessible to start-ups so they can compete with the larger companies.”
The whole premise of the data economy falls down if the data — and therefore power and influence — is held by a minority of big, influential companies. Organizations must be part of a data economy based on ecosystems of parties cooperating on shared goals whilst not losing sight of their respective bottom lines.
Looking ahead to 2025, businesses must prioritize the basics of data management and storage to make sure the right data is in the right place at the right time. Businesses need to stop looking at the AI endgame but instead focus on the basics of ‘doing data well’.
That is what will unlock the impact of AI and allow those businesses, currently on the outside of the data economy looking in, a prime seat at the table of change.
