Registry / data / collate-sqllineage

collate-sqllineage

JSON →
library2.1.7pypypi✓ verified 24d ago

Collate SQL Lineage is a Python tool designed to analyze SQL statements and extract their data lineage, mapping relationships between tables and columns. It builds upon `sqllineage` for core lineage parsing and `sqlfluff` for robust SQL parsing and linting. The library provides a programmatic interface for integrating lineage analysis into data governance, compliance, and impact analysis workflows. It is currently at version 2.1.0 and maintains an active development and release cadence.

pip install collate-sqllineage
INSTALL
IMPORT
SIG · COLLATE-SQLLINEAGE
C
collate-sqllineage
datapythonv2.1.7
Install
6.8s avg
Import
Disk
76MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v2.1.7 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.000s · 71.2MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 6.8s · import 0.000s · 75MB
76MB installed
● package 76MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

CollateSQLLineageRunner
from collate_sqllineage import CollateSQLLineageRunner
from collate_sqllineage import CollateSQLLineageRunner

This quickstart demonstrates how to initialize `CollateSQLLineageRunner` and use its `run()` method to extract data lineage from a sample SQL `INSERT` statement. The output includes source and target tables, with the full result containing more granular details like column-level lineage.

from collate_sqllineage.runner import CollateSQLLineageRunner # Example SQL statement to analyze sql_statement = """ INSERT INTO target_schema.target_table (col_a, col_b) SELECT source_schema.source_table_1.col_x, source_schema.source_table_2.col_y FROM source_schema.source_table_1 JOIN source_schema.source_table_2 ON source_schema.source_table_1.id = source_schema.source_table_2.id WHERE source_schema.source_table_1.status = 'active'; """ # Initialize the runner. Configuration can be passed here (e.g., for verbose logging). # runner = CollateSQLLineageRunner(config={'verbose': True}) runner = CollateSQLLineageRunner() # Run the lineage analysis lineage_result = runner.run(sql=sql_statement) # Print the extracted lineage information print("--- Extracted SQL Lineage ---") print(f"Source Tables: {lineage_result.get('tables', {}).get('source', [])}") print(f"Target Tables: {lineage_result.get('tables', {}).get('target', [])}") # The 'lineage_result' dictionary contains more detailed information including columns, statements, etc. # print(lineage_result)
sqllineage --version
Debug
Known issues
breakingVersion 2.0.0 introduced a significant API refactor, deprecating direct function calls for lineage analysis in favor of the `CollateSQLLineageRunner` class. Code written for versions prior to 2.0.0 will no longer work.
fix
Migrate existing code to use `CollateSQLLineageRunner` by instantiating the class and calling its `run()` method with the SQL string. For example, `runner = CollateSQLLineageRunner(); result = runner.run(sql=my_sql)`. Old direct function calls will raise an `AttributeError` or `ImportError`.
affects: <2.0.0 to >=2.0.0
gotchaLineage extraction relies on `sqllineage` and `sqlfluff`, which may have limitations in fully parsing highly complex, non-standard, or niche SQL dialects. Edge cases or unsupported syntax might lead to incomplete or incorrect lineage results.
fix
Test with representative SQL samples. For problematic queries, simplify them where possible or report issues to `collate-sqllineage` or its underlying dependencies. Enable verbose logging in `CollateSQLLineageRunner(config={'verbose': True})` to get more diagnostic information on parsing failures.
affects: All versions
gotchaWhile `collate-sqllineage` specifies dependency versions (`sqllineage`, `sqlfluff`), future updates to these underlying libraries might introduce subtle breaking changes in parsing behavior or output format that could impact lineage results, even if `collate-sqllineage`'s direct API remains stable.
fix
For production environments, pin `collate-sqllineage` and its direct dependencies to specific versions. Regularly test against new dependency versions in development before deploying to ensure consistent lineage results. Review `collate-sqllineage`'s release notes for compatibility updates.
affects: All versions (future dependency updates)
Upgrade
Version history
2.1.7latest on PyPI · released Aug 21, 2026
Audit
Dependencies

No dependency data recorded yet.

Agent activity
23 hits · last 30 days
node
16
OpenAI (training)
2
Resources
collate-sqllineage — pip install collate-sqllineage · libregistry