What is the implication for data quality?

The choice of which type of data storage to use can have a significant impact on data quality.

Data lakes are typically used for storing large amounts of unstructured data. Unstructured data is more difficult to govern and manage than structured data. As a result, data lakes are more likely to have lower data quality than data warehouses, and can lead to duplicate or inconsistent data. In contrast, data warehouses are more likely to impose strict rules that can exclude important data. 

The ability to manage and improve data quality is doubtless improved when data is governed by a schema, as is the case with data warehouses. When data is stored in its native format, as is the case with data lakes, the quality of the data can be more difficult to control. 

The choice of data storage architecture should be made based on the needs of the business and the nature of the data being stored.

Emerging concepts such as data mesh and data fabric attempt to exploit the benefits of data lakes, data warehouses and data lakehouses through a combination of approaches such as local governance, self-service solutions, and interoperable data standards. For more on this subject read this article on data fabric and data mesh.

What about the difference in cost?

The choice of data storage solution also affects the cost of storing and accessing data. Data warehouses are typically more expensive than data lakes because they require more hardware and software resources. Data lakehouses are usually more expensive than data lakes or data warehouses because they combine the features of both.

How about speed?

The choice of data storage solution also affects the speed at which data can be accessed. Data lakes can be faster than data warehouses because they can be queried in parallel. Data warehouses can be faster than data lakes if the right indexes are used. Data lakehouses can be faster than both if they are designed properly.

What is the impact on data pipelines, and data governance?

The impact of differing methods of data storage on how data is governed, managed and curated for healthy pipelines into businesses varies depending on the needs of the organization.

• Organizations that need to store large amounts of unstructured data may find that a data lake is the best solution for their needs.
• Organizations that need to store large amounts of structured data may find that a data warehouse is the best solution for their needs.
• Organizations that need to store large amounts of both structured and unstructured data may find that a data lakehouse is the best solution for their needs.

The decision of which method to use should be based on the specific needs of the organization rather than on generalities about each method.