Join us
@pmartinoli ・ Feb 21,2023 ・ 5 min read ・ 1181 views ・ Originally posted on blog.mia-platform.eu
Data is one of the most critical components of any business, as it allows us to personalize and customize our products for potential consumers. As important as data is, studies have shown that about 50‑70% of data collected by organizations goes unused and becomes what Gartner calls Dark Data. We can attribute this large amount of unused data to the inefficiencies in the systems that manage them.
This post discusses how methods like Data Meshes and Data Fabrics, which have emerged in the past decade, can help mitigate the problems associated with data management.
At the end of this post, you should understand what Data Meshes and Data Fabrics are, their differences, and why one may overtake the other.
According to IBM, a Data Mesh is a decentralized data architecture that organizes data by a specific business domain, providing more ownership to the producers of a given dataset. By decentralizing data, a Data Mesh offers an alternative to the central data lake and team culture that has been present in companies for decades. It is important to note that Data Meshes are language‑agnostic and technology‑agnostic as it is an approach that focuses more on organizational changes.
Data Meshes are built on four fundamental principles, which are discussed in the paragraphs below:
As defined by IBM, a Data Fabric is an architecture that facilitates the end‑to‑end integration of various data pipelines and cloud environments through intelligent and automated systems. It is adaptive, flexible, secure, and ensures a consistent user experience across all integrated environments.
With Data Fabric, we can monitor and manage our data applications regardless of where they live.
At the center of the Data Fabric is rich metadata that enables automation, which is designed to automate data integration, engineering, and governance between data providers and consumers.
Alongside automation, the Data Fabric is tasked with the following responsibilities.
As both data paradigms are created to aid data gathering, governance, and distribution, it is easy to notice similarities between them. However, the differences are also apparent and should be considered before an organization chooses a paradigm.
This section discusses the differences between the Data Mesh and Data Fabric paradigms.
It is conceivable to see Data Fabrics take the lead in the coming years regarding efficient data management. Data Fabrics connect the entire organization's data and facilitate frictionless data sharing.
Because Data Fabrics center on automation, we can optimize data management and send real‑time insights and analytics to data users. Moreover, Data Fabrics offer increased security: the virtualization layer ensures that the data is not unnecessarily moved. Data Fabrics are also cost‑efficient.
However, Data Meshes and Fabrics are not mutually exclusive. Data Fabrics can enable Data Mesh implementation by automating repetitive tasks using Data Fabrics’ metadata insights. With a Data Fabric, data owners in the Data Mesh paradigm can achieve the capabilities to create data products.
This article discusses the Data Mesh and Fabric paradigms, their differences, and, more importantly, what data management method is expected to take the lead in the coming years.
Mia‑Platform Fast Data is a perfect example of the cohabitation between the paradigms, and it can help you shift from one to the other if needed. To understand more about Mia‑Platform Fast Data, check out this article.
Join other developers and claim your FAUN account now!
Technical Writer, Mia-Platfom
@pmartinoliInfluence
Total Hits
Posts
Only registered users can post comments. Please, login or signup.