DevOps Articles

The Future of Data Integration

Data integration has significantly evolved since data became centralized in data warehouses and lakes. ELT (Extract-Load-Transform) has replaced ETL (Extract-Transform-Load) by making analysts autonomous at getting access to the data they need in those warehouses.

And that leads to the question: how will data integration evolve given the indications we already have?

The reason is simple: the hard part about data integration is not building the connectors, but maintaining them.

With a platform that can do the following: ELT and reverse-ETL with open source connectors that you can customize at will; An active data engineering community incentivized to maintain the long tail of connectors; Data lineage; Observability across all data pipelines; Integration with other tools of the company data stack to ensure interoperability