Why do businesses struggle to access information easily despite modern data advances? What is data mesh and how can it solve this common business challenge? How quickly can you get a data mesh project live?
We answered these questions and more in our recent webinar, Data mesh & virtualisation: Unlock the true value of your business information. Our own data expert, Richard Fraccaro, together with Jack Steele and Ben Tzannes from Zetaris, one of our technology partners, discussed the principles of a well-built data mesh, what data mesh in practice looks like and how it can quickly add value for organisations and build business agility.
But there was still more we could have covered, so we’ve provided answers to some other common questions we get about data mesh and virtualisation below. We hope you find them helpful as you work towards building your data capabilities to make more informed business decisions.
1. What kind of skills are required to deliver data mesh and data products?
Since data mesh is meant to empower business users, the chosen technology platform needs to be designed around common business data skills. Nowadays, that could mean SQL (Structured Query Language): understanding its concepts of data and how to manipulate data through SQL coding. Even better though – and what most users expect from modern platforms nowadays – is a graphically driven interface so that SQL is created for the user, only requiring coding for highly bespoke data requirements. For these reasons, we like working with the Zetaris Networked Data Platform and recommend it to our clients.
2. How important is the choice of technology in implementing data mesh?
A virtualised data environment is essential for you to realise the benefits of the data mesh approach. And the platform in which virtualised data is catalogued needs to be optimised for the intensive, analytical workloads that data consumers will be needing. Although virtualised data platforms have been around for many years, it is the new virtualised data platform technologies just now starting to appear in the market that are designed to provide all the capabilities required for data mesh.
If you tried to avoid adopting these platforms altogether and instead built your data products in a traditional data warehouse or data lake, you would face the same data accessibility challenges that you were trying to resolve in the first place.
3. What kind of scale and complexity is required within an organisation for a typical data mesh project? Is the process very involved?
The push to start a data mesh project usually comes from one of two places – either you have struggled with data lakes and data warehouses not delivering against your business requirements and you are looking for a better way to access your information. Or your organisation is anticipating growth in data volume and complexity, and wants to avoid the problems and bottlenecks that come with data centralisation.
Whatever the impetus, the data mesh implementation is conducted at the pace of your business. This also means you can design your implementation approach so that it’s tailored specifically to your organisation and only includes data products that are valuable to your business. Depending on your existing data environment, your data mesh should be relatively simple to build. The idea is to start small, with the highest-value objectives first. From there it can be iteratively managed and scaled, which is one of its key benefits compared to traditional data warehouses.
4. How different are data models and data products? Isn’t it just doubling?
Whilst data products access information through the data models of centralised repositories, they aren’t duplicating those data models. Instead, data products allow business users to create their own associations between different parts of the data model (or even between data models of disparate repositories). This lets users create a view of the data they need, which wouldn’t otherwise be available without data mesh. All the while, the data models continue to look after data integrity and security, and the data mesh inherits those properties from the source system.
5. How do you manage data product changes (where changes in a traditional data warehouse model need to be carefully orchestrated)?
Data products are designed to be useful associations of disparate data that aligns with a business user’s perspective of the data. For each use case, a ‘view’ is created across one or more data products to serve the specific requirements of that use case. If that use case’s requirements change, then the view can be changed without impacting other views created for other use cases. The data products themselves only change when they are expanded to include additional data and so avoids data product changes that could compromise existing use cases.
6. Where does a data governance framework fit in with a data mesh approach?
To create data products that are usable, manageable, high-quality and secure, you need a good handle on your source data, as well as its surrounding services and frameworks. This makes governance a big part of data mesh.
The business team that owns a data product should have a very clear understanding of the data, as well as the processes that have generated the information. A governance framework ensures that each business user or team applies the same levels of governance across all the products and teams. In a data mesh, this is known as federated governance and is designed to work with the business teams, rather than being a centralised capability that limits access to insights.
Want to know more about how your organisation can improve reliability, agility and efficient access to meaningful business data with data mesh and virtualisation? Watch our on-demand webinar here.
Optivia’s data quality services empower reliable, informed decision-making. We help you strengthen your business with robust data governance principles designed to improve data accuracy and ensure you adhere to data privacy regulations. If you need a data quality framework that aligns with your strategic goals and helps you make rapid, meaningful decisions, contact us today.