Registry / aws / pyathena

pyathena

JSON →
library3.35.4pypypi✓ verified 27d ago

PyAthena is a Python DB API 2.0 (PEP 249) client for Amazon Athena, enabling SQL queries on data stored in Amazon S3. It provides a familiar interface for database interactions, supports various cursor types (e.g., standard, Pandas, Arrow), SQLAlchemy integration, and asynchronous query execution. The library is actively maintained with frequent updates.

pip install pyathena
INSTALL
IMPORT
SIG · PYATHENA
P
pyathena
awspythonv3.35.4
Install
9.9s avg
Import
39ms
Disk
505MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v3.35.4 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.910 runs
installs and imports cleanly · install 0.0s · import 0.040s · 631.1MB
glibc
py 3.103.910 runs
installs and imports cleanly · install 9.9s · import 0.038s · 590MB
505MB installed
● package 505MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

connect
from pyathena import connect

This quickstart demonstrates how to establish a connection to Amazon Athena, execute a simple SQL query, and fetch results using `pyathena`. It expects AWS credentials to be configured via environment variables, IAM roles, or `~/.aws/credentials` (handled by `boto3`). The `s3_staging_dir` and `region_name` are mandatory connection parameters.

import os from pyathena import connect # Configure these environment variables or replace with actual values # AWS_S3_STAGING_DIR: S3 path for Athena query results (e.g., "s3://my-athena-results-bucket/") # AWS_REGION_NAME: AWS region (e.g., "us-east-1") # AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN will be picked up by boto3 if not explicitly passed s3_staging_dir = os.environ.get('AWS_S3_STAGING_DIR', 's3://your-athena-query-results-bucket/') region_name = os.environ.get('AWS_REGION_NAME', 'us-east-1') aws_access_key_id = os.environ.get('AWS_ACCESS_KEY_ID') aws_secret_access_key = os.environ.get('AWS_SECRET_ACCESS_KEY') aws_session_token = os.environ.get('AWS_SESSION_TOKEN') # Ensure mandatory parameters are set if not s3_staging_dir.startswith('s3://') or not region_name: print("Error: AWS_S3_STAGING_DIR and AWS_REGION_NAME must be set correctly.") else: try: # Connect to Athena conn = connect( s3_staging_dir=s3_staging_dir, region_name=region_name, aws_access_key_id=aws_access_key_id, # Optional: boto3 usually handles this aws_secret_access_key=aws_secret_access_key, # Optional aws_session_token=aws_session_token # Optional ) cursor = conn.cursor() # Execute a sample query cursor.execute("SELECT 1 as one, 'hello' as greeting") # Fetch results print("Query Results:") for row in cursor.fetchall(): print(row) # Close cursor and connection cursor.close() conn.close() except Exception as e: print(f"An error occurred: {e}") print("Please ensure AWS credentials are configured (e.g., via environment variables, ~/.aws/credentials, or IAM role) and AWS_S3_STAGING_DIR and AWS_REGION_NAME are set correctly.")
Debug
Known issues
breakingStarting with PyAthena v3.30.0, the library no longer infers Python types for scalar values inside complex Athena types (e.g., '123' to 123 in structs/arrays). Values are kept as strings unless `result_set_type_hints` is provided.
fix
If your code relies on the previous heuristic type inference for complex types, explicitly provide `result_set_type_hints` in your `connect` or `cursor.execute()` calls to specify the expected Athena DDL type signatures for affected columns. Otherwise, adapt your code to handle string values for complex type elements.
affects: >=3.30.0
gotchaThe `s3_staging_dir` and `region_name` parameters are mandatory when establishing a connection to Athena. Failure to provide them will result in a connection error.
fix
Always pass `s3_staging_dir` (e.g., `s3://your-bucket/path/to/results/`) and `region_name` (e.g., `us-east-1`) to the `pyathena.connect()` function.
affects: All
gotchaFor very large query results, the default cursor might be slow as it fetches results in smaller chunks. This can lead to performance bottlenecks for extensive data analysis.
fix
Consider using `PandasCursor` with the `chunksize` option (e.g., `cursor_class=PandasCursor, cursor_kwargs={'chunksize': 100000}`) for better memory management, or configure Athena to write results to S3 directly and then download/process the CSV file for optimal performance with massive datasets.
affects: All
gotchaEnsure your AWS environment is correctly configured for authentication (e.g., IAM role, `~/.aws/credentials`, or environment variables `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_SESSION_TOKEN`). PyAthena relies on `boto3` for credential resolution.
fix
Verify `boto3`'s credential chain can find valid AWS credentials. For explicit control, you can pass `aws_access_key_id`, `aws_secret_access_key`, and `aws_session_token` directly to `pyathena.connect()`.
affects: All
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'pyathena'
The `pyathena` library is not installed in the Python environment, or the Python interpreter cannot locate it in its search path.
fix
Install the library using pip: `pip install PyAthena` or ensure the correct virtual environment is activated.
pyathena.error.DatabaseError: An error occurred (InvalidRequestException) when calling the StartQueryExecution operation: This functionality is not enabled in the selected engine version.
This error occurs when attempting to use a feature, such as result reuse, that requires Athena engine version 3, but the configured workgroup is using an older engine version.
fix
Configure your Athena workgroup to use Athena engine version 3, or disable the feature causing the error (e.g., `result_reuse_enable=False`) if backward compatibility is required.
botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the StartQueryExecution operation: User: arn:aws:iam::... is not authorized to perform: athena:StartQueryExecution on resource: ...
The AWS IAM user or role configured for PyAthena lacks the necessary permissions to execute Athena queries or access the specified S3 staging directory.
fix
Grant the required AWS IAM permissions (`athena:StartQueryExecution`, `athena:GetQueryExecution`, `athena:GetQueryResults`, `s3:GetObject`, `s3:ListBucket`, `s3:PutObject`, `s3:DeleteObject` for the staging S3 bucket) to the IAM entity PyAthena is using.
botocore.exceptions.ParamValidationError: Parameter validation failed:
An invalid or incorrectly formatted parameter was passed to an underlying `boto3` call made by PyAthena, often due to a mismatch with expected types or values by the AWS API.
fix
Review the PyAthena connection parameters and query arguments, ensuring they conform to the expected types and formats as per PyAthena and `boto3` documentation for the Athena service.
AttributeError: 'NoneType' object has no attribute 'get'
This generic Python error occurs in PyAthena when an operation attempts to access an attribute (like `get`) on an object that is `None`, usually because a preceding step (e.g., connection, query execution, or result fetching) failed to return a valid object and instead returned `None`.
fix
Implement robust error handling and `None` checks around PyAthena API calls, especially for `connect()`, `cursor.execute()`, and result fetching methods, to identify and handle cases where these operations might fail and return `None`.
Upgrade
Version history
3.35.4latest on PyPI · released Jul 31, 2026
Audit
Dependencies
boto3requiredRequired for AWS API interactions.
botocorerequiredRequired for AWS API interactions (a dependency of boto3).
SQLAlchemyoptionalOptional, for SQLAlchemy dialect support.
pandasoptionalOptional, for PandasCursor to fetch results as DataFrames.
pyarrowoptionalOptional, for ArrowCursor to fetch results as Apache Arrow tables.
polarsoptionalOptional, for PolarsCursor to fetch results as Polars DataFrames.
Agent activity
19 hits · last 30 days
node
18
Resources
pyathena — pip install pyathena · libregistry