Registry / observability / openlineage-sql

openlineage-sql

library1.52.0pypypi✓ verified 24d ago

The `openlineage-sql` library provides a Python interface to a high-performance Rust library for SQL lineage extraction. It enables parsing SQL queries to identify input tables, output tables, and query types, crucial for data governance and ETL pipeline observability. Currently at version 1.46.0, it is actively maintained with frequent updates reflecting improvements in its core Rust library and broader OpenLineage specification, requiring Python 3.10 or newer.

observability data workflow

pip install openlineage-sql

package ↗homepage ↗

INSTALL

IMPORT

SIG · OPENLINEAGE-SQL

openlineage-sql

observabilitypythonv1.52.0

Install

1.7s avg

Import

—

Disk

23MB

Pass rate

5/ 10

Env Coverage5 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.52.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

build_error

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 1.7s · import 0.000s · 25MB

23MB installed

● package 23MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

parse

✓ from openlineage_sql import parse

✗ from openlineage.sql import parse

SqlMeta

✓ from openlineage_sql import SqlMeta

ColumnLineage

✓ from openlineage_sql import ColumnLineage

This quickstart demonstrates how to use the `parse` function to extract lineage information (inputs, outputs, and query type) from a given SQL query, including DDL statements.

from openlineage.sql import parse sql_query = "SELECT a, b FROM input_table JOIN other_table ON input_table.id = other_table.id WHERE a > 10" result = parse(sql_query) print(f"SQL Query Type: {result.query_type}") print(f"Input Tables: {result.inputs}") print(f"Output Tables: {result.outputs}") # Example with DDL sql_ddl = "CREATE TABLE new_table (id INT, name VARCHAR(255))" ddl_result = parse(sql_ddl) print(f"\nDDL Query Type: {ddl_result.query_type}") print(f"DDL Output Tables: {ddl_result.outputs}")

Debug

Known issues

gotchaUsers on less common operating systems or architectures might encounter compilation errors during installation if pre-built wheels are not available. This requires a Rust toolchain to be installed on the system.

fix

Ensure a Rust toolchain (e.g., rustup) is installed and correctly configured if `pip install` fails with compilation errors.

affects: All versions

gotchaWhile supporting many SQL dialects, complex, non-standard, or highly dynamic SQL queries (e.g., involving complex macros, stored procedures, or unusual syntax) might lead to incomplete or inaccurate lineage extraction.

fix

Simplify complex queries where possible or pre-process dynamic SQL to its final form before passing it to `openlineage-sql`. Validate results for critical lineage paths.

affects: All versions

gotchaThe `parse` function returns a `SqlMeta` object, not directly a list of tables. Users must access specific attributes like `.inputs`, `.outputs`, and `.query_type` to retrieve the desired lineage details.

fix

Always treat the return value of `parse` as an object and access its attributes (e.g., `result.inputs`) rather than assuming it's a direct list or dictionary.

affects: All versions

Upgrade

Version history

1.52.0latest on PyPI · released Jul 23, 2026

Audit

Dependencies

No dependency data recorded yet.

Agent activity

31 hits · last 30 days

node

OpenAI (training)

Resources

packagehttps://pypi.org/project/openlineage-sql/ ↗

homepagehttps://openlineage.io ↗