Atlan Launches Integration with Databricks Unity Catalog to Provide Column-Level Lineage

The integration will help data teams collaborate on their data and AI processes with greater context and trust.

Atlan, the active metadata platform for modern data teams, today announced that it has partnered with Databricks, the data and AI company, to release an integration with Databricks’ Unity Catalog. This will help joint customers drive business value by unifying data and AI with increased visibility and context across the modern data stack.

Atlan powers modern data discovery, cataloging, and governance by sharing context across the modern data stack through active metadata, automated bots, and granular personalization. As a result of this integration, enterprise data teams can bring together context from their Databricks assets and other data assets into one single source of truth.

“We’re excited to partner with Databricks to make data and AI more accessible than ever”

The Databricks Lakehouse Platform helps organizations accelerate innovation by unifying data teams with an open, scalable platform for all of their data-driven use cases. From streaming analytics and AI to business intelligence (BI), Databricks provides a modern lakehouse architecture that unifies data engineering, data science, machine learning and analytics within a single collaborative platform.

Marketing Technology News: ESET Leader in G2 summer report for ESET PROTECT Advanced

Atlan connects to Databricks Unity Catalog’s REST API to extract all relevant metadata from Databricks clusters and workspaces, powering discovery, governance, and insights inside Atlan. This integration allows Atlan to generate column-level lineage for tables, views, and columns for all the jobs and languages that run on a Databricks cluster. By pairing this with metadata extracted from other tools in the data stack (e.g. BI, transformation, ELT), Atlan can create true end-to-end lineage.

Thanks to Unity Catalog’s simplified delivery system, which sends complete lineage through its API, this entire experience is near instantaneous with drastically reduced compute and cost for customers. This allows Databricks customers to holistically understand the flow of their data, gain deeper insight into the data populating their models, run RCA exercises, and even power programmatic governance at scale with Atlan’s metadata activation engine.

“We’re excited to partner with Databricks to make data and AI more accessible than ever,” said Prukalpa Sankar, Co-founder at Atlan. “Today, data is increasingly complex and the humans of data are increasingly diverse, so it’s critical for everyone to understand context across their data. This integration makes it easier for customers to understand and work with their lakehouse assets. We are making significant investments into our joint roadmap with Databricks, and we can’t wait to power the future of data governance together.”

Within just three years of its launch, Atlan is the tool of choice for a growing list of modern data teams around the world, including WeWork, Plaid, Postman, Scripps Health, TechStyle, SnapCommerce, and Delhivery. It was also recently named a Leader in The Forrester Wave™: Enterprise Data Catalogs for DataOps, Q2 2022.

Marketing Technology News: MarTech Interview with Jerry Luk, Co-Founder & President at Firework