Data design is the backbone of decision-making, strategy and innovation. However, harnessing the power of data isn’t easy. It requires deliberate planning, strategic thinking and the right technology. Do it right and it will save you time, money and lead to better business outcomes. Do it wrong and it can create security risks, increase costs and lead to misguided decision-making. Here’s how to design for data the right way.
What you should design for
When we talk about designing for data, what we’re really talking about is designing for outcomes. Data design isn’t merely about collecting and storing information. It’s about shaping systems and processes to generate actionable insights, drive operational efficiency and facilitate informed decision-making.
Every step in the data design process should work towards achieving these overarching outcomes. The more you can plan your data process to address how you want to use that data, the more successful your outcomes will be.
Where to start when designing for data
The data design and planning process has to start somewhere. Many enterprises like to start with technology or tools. While those elements are important, they are a middle or later stage of the process.
While there’s no “one-size-fits-all” approach, designing for data usually starts with understanding the questions you want your data to answer. Whether it’s identifying market trends, optimizing operational processes or enhancing customer experiences, clarity on desired outcomes is paramount. By starting with the end in mind, organizations can align their data strategies with business objectives from the beginning.
Only when you know the questions can you understand if you already have the data you need to answer them, or how you should design to capture, normalize, store, analyze and visualize new data sources to drive business outcomes.
Who should be involved
Who asks the question, or even who is thinking of what questions to ask, is an important step in the data design process. This should be a collaborative effort that requires multidisciplinary expertise.
While line-of-business owners provide valuable insights into specific pain points and challenges, data specialists bring technical proficiency and analytical skills to the table. Engaging stakeholders from various departments ensures a holistic approach to data design, fostering alignment between business needs and technological capabilities. This cross-functional approach also helps to break down silos. It’s important not to design systems where data can’t be universally accessed, or where certain departments or business units don’t even know that certain data streams or caches even exist.
The role of AI in designing for data
As with many other aspects of technology, the rapid rise in artificial intelligence (AI) technologies has the potential to revolutionize the data design process. But, it’s not a magic bullet that will do all of the work for you.
Much of the current AI hype is around generative AI, built on large language models (LLM). These models are great at taking inputs, unpacking them into a broader context, finding relevant information and then formulating responses for humans.
But, generative AI isn’t really designed to analyze data in transactional systems or structured data. You still need to have a design that includes a middle layer to convert and translate, actually doing the analytics. This layer can format data so your generative AI systems can predictably and consistently retrieve the right information and then present it in an easy-to-understand manner.
You can’t discount the effort and expertise of key resources in this process. As the adage goes, “Garbage in, garbage out.” If you don’t have the data science part of the design process in place, then that fancy AI platform you’ve spent so much on can quickly lead you in the wrong direction.
What happens if you design for data well
So, you’ve asked the right questions. Your team has thought through data collection and use across the organization. And you’ve structured your data so that AI can speed up the analysis process. What does success look like then?
Successful data design results in increased cost optimization, operational efficiency, actionable insights and organization flexibility. You start to understand your customers better. It becomes easier to predict where the market is heading and get out ahead of trends or potential pitfalls. You can identify and solve problems before they become catastrophes. And you can do it all faster and cheaper than before.
What happens if you don’t design for data
On the other hand, if you rushed the discovery phase or relied too much on AI to magically solve all your schema issues, you might be worse off than if you did nothing.
You’ll already have wasted time and money designing a system that doesn’t work now, and will only get harder to fix in the future. You’ll be stuck with increased costs as employees have to maintain faulty systems and process data manually to try and get any insights. And worst of all, you might make the news for all the wrong reasons. If you’re dealing with sensitive data – HIPAA, PII, children, etc. – not designing properly can lead to regulatory, compliance and other legal issues. Not to mention the overall security concerns of proper data collection and storage. It’s just a matter of time before something goes wrong.
Designing for data isn’t just a technical endeavor; it’s a strategic imperative to ensure business success. By prioritizing outcomes, involving diverse stakeholders, leveraging AI responsibly and investing in robust data infrastructure, organizations can unlock the full potential of their data.
This post is adapted from a Sourcing for Innovation podcast. You can watch a preview of and listen to the whole conversation below.
