Install & Compatibility
Where this runs
tested against v1.10.15 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.920 runs
installs and imports cleanly · install 0.0s · import 0.000s · 321.3MB
glibcpy 3.10–3.920 runs
installs and imports cleanly · install 14.4s · import 0.000s · 297MB
311MB installed
● package 311MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
dbt.config
✓ def model(dbt, spark):
dbt.config(materialized='python_model', file_format='iceberg')
Used within Python models for model configuration.
dbt.ref
✓ source_df = dbt.ref("my_sql_model")
Used in dbt SQL and Python models to reference other dbt models.
dbt.source
✓ raw_data = dbt.source('my_source', 'my_table')
Used in dbt SQL and Python models to reference declared data sources.
To get started with `dbt-glue`, you'll need to configure your `profiles.yml` to specify connection details for AWS Glue interactive sessions, including the IAM role, region, worker types, and S3 location. SQL models define transformations, and experimental Python models allow for more complex logic using PySpark DataFrames. Ensure `DBT_ROLE_ARN` and `DBT_S3_LOCATION` environment variables are set for authentication and storage paths respectively.
import os
# profiles.yml example for dbt-glue
profiles_yml_content = """
dbt_glue_project:
target: dev
outputs:
dev:
type: glue
query-comment: dbt-glue-example
role_arn: "{{ env_var('DBT_ROLE_ARN', 'arn:aws:iam::123456789012:role/GlueInteractiveSessionRole') }}"
region: "{{ env_var('AWS_REGION', 'us-east-1') }}"
workers: 5
worker_type: G.1X
schema: dbt_glue_demo_schema
database: dbt_glue_demo_db
session_provisioning_timeout_in_seconds: 120
location: "{{ env_var('DBT_S3_LOCATION', 's3://your-s3-bucket/dbt-glue/') }}"
glue_version: "4.0"
conf: "--conf spark.sql.catalog.glue_catalog=org.apache.iceberg.spark.SparkCatalog --conf spark.sql.catalog.glue_catalog.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog --conf spark.sql.catalog.glue_catalog.io-impl=org.apache.iceberg.aws.s3.S3FileIO --conf spark.sql.catalog.glue_catalog.lock-impl=org.apache.iceberg.aws.glue.DynamoLockManager --conf spark.sql.catalog.glue_catalog.lock.table=DbtGlueLockTable --conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions"
"""
# Example dbt_project.yml (assuming project name 'dbt_glue_project')
dbt_project_yml_content = """
name: 'dbt_glue_project'
version: '1.0.0'
config-version: 2
profile: 'dbt_glue_project'
model-paths: ["models"]
analysis-paths: ["analyses"]
test-paths: ["tests"]
seed-paths: ["seeds"]
macro-paths: ["macros"]
snapshot-paths: ["snapshots"]
target-path: "target" # directory which will store compiled SQL files
clean-targets:
- "target"
- "dbt_packages"
models:
dbt_glue_project:
+materialized: table
"""
# Example SQL model (models/my_first_model.sql)
sql_model_content = """
{{ config(materialized='table', file_format='parquet') }}
SELECT
1 as id,
'dbt-glue' as name
"""
# Example Python model (models/my_python_model.py) - requires AWS Glue 4.0+ and Iceberg
python_model_content = """
from pyspark.sql.functions import lit
def model(dbt, spark):
dbt.config(
materialized='incremental',
file_format='iceberg',
unique_key=['id'],
incremental_strategy='merge'
)
if dbt.is_incremental():
max_id_query = f"SELECT coalesce(max(id), 0) FROM {dbt.this}"
max_id = spark.sql(max_id_query).collect()[0][0]
return spark.createDataFrame([(max_id + 1, 'new_incremental_record')]) \
.toDF("id", "name")
else:
return spark.createDataFrame([(1, 'initial_record'), (2, 'another_initial')]) \
.toDF("id", "name")
"""
# To run:
# 1. Ensure AWS credentials and DBT_ROLE_ARN, DBT_S3_LOCATION environment variables are set.
# 2. Create the project structure: ~/.dbt/profiles.yml, dbt_project.yml, models/my_first_model.sql, models/my_python_model.py
# 3. dbt debug
# 4. dbt run
dbt --version
Debug
Known issues
breakingBeginning with dbt Core v1.8, installing a dbt adapter no longer automatically installs `dbt-core`. You must explicitly install both `dbt-core` and `dbt-glue` to avoid missing dependencies or version conflicts.fixAlways install `dbt-core` and `dbt-glue` together: `pip install dbt-core dbt-glue`.
affects: dbt-core >=1.8.0, dbt-glue >=1.8.0
gotchaPython model support and Amazon S3 Tables support are currently experimental. They may have limitations or breaking changes in future versions and require AWS Glue 4.0+ for optimal support, and Iceberg file format for Python models.fixReview the official dbt-glue documentation and GitHub README for the latest status and specific requirements before relying on these experimental features in production. Ensure your AWS Glue environment is version 4.0 or higher.
affects: All versions with Python/S3 Tables support (from 1.10.9 onwards)
gotchaWhen working with Iceberg tables on AWS Glue, especially in dbt tests or certain queries, you might need to explicitly prefix table names with `glue_catalog.` (e.g., `glue_catalog.your_database.your_table`) in custom SQL or specific configurations if not handled automatically by the adapter's macros.fixFor issues with table discovery, particularly in tests, ensure Iceberg-specific Spark configurations are correctly set in `profiles.yml` and consider explicitly using the `glue_catalog.` prefix where direct table references are made.
affects: All versions
deprecateddbt Core v1.10 introduces deprecation warnings for several patterns, including duplicate keys in the same YAML file, unexpected Jinja blocks, and the `--models` / `--model` / `-m` CLI flags (which were renamed to `--select` / `--s` in v0.21).fixUpdate your YAML files to remove duplicate keys and ensure proper Jinja syntax. Use `--select` or `-s` instead of `--models` for CLI commands. If using `--warn-error`, configure `warn-error-options` to handle deprecations appropriately.
affects: dbt-core >=1.10.0, dbt-glue >=1.10.0
gotchaIncorrect IAM permissions for the Glue interactive session role can lead to `AccessDeniedException` errors, preventing dbt-glue from accessing S3 buckets or the Glue Data Catalog.fixEnsure the IAM role associated with your Glue jobs has the necessary permissions for S3 access, Glue Data Catalog operations, and Lake Formation if applicable. A least-privileged policy example is often available in the dbt-glue documentation.
affects: All versions
Upgrade
Version history
1.10.19latest on PyPI · released Feb 18, 2026
Audit
Dependencies
dbt-corerequireddbt-glue is an adapter for dbt Core, which provides the main CLI and framework.
dbt-sparkoptionaldbt-glue leverages dbt-spark for its underlying Spark compatibility, especially in earlier versions.