Registry / data / fastparquet

fastparquet

JSON →
library2026.5.0pypypi✓ verified 26d ago

fastparquet is a Python library providing performant read/write support for the Parquet file format, without needing a Python-Java bridge. It integrates well with Python-based big data workflows, particularly Dask and Pandas (versions < 3.0). As of March 2026, with Pandas 3.0 explicitly depending on PyArrow, `fastparquet` is being retired, and no further development is anticipated, though it remains usable for Pandas 2.x users.

pip install fastparquet
INSTALL
IMPORT
SIG · FASTPARQUET
F
fastparquet
datapythonv2026.5.0
Install
8.6s avg
Import
1157ms
Disk
180MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v2026.5.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 1.210s · 178.8MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 8.6s · import 1.104s · 172MB
180MB installed
● package 180MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

ParquetFile
from fastparquet import ParquetFile
Used for reading existing Parquet files.
write
from fastparquet import write
Used for writing Pandas DataFrames to Parquet files.

This quickstart demonstrates how to create a Pandas DataFrame, write it to a Parquet file using `fastparquet.write`, and then read the data back into a new DataFrame using `fastparquet.ParquetFile.to_pandas()`. It also includes basic file cleanup.

import pandas as pd from fastparquet import write, ParquetFile import os # Create a sample DataFrame df = pd.DataFrame({ 'col1':, 'col2': ['A', 'B', 'C', 'D'], 'col3': [True, False, True, False] }) filename = "example.parquet" # Write the DataFrame to a Parquet file with Snappy compression write(filename, df, compression='SNAPPY') print(f"DataFrame successfully written to '{filename}'.") # Read the Parquet file back into a DataFrame pf = ParquetFile(filename) df_read = pf.to_pandas() print(f"DataFrame successfully read from '{filename}':") print(df_read) # Clean up the created file os.remove(filename)
Debug
Known issues
breakingThe `fastparquet` project is being retired and is incompatible with `pandas` 3.0 and newer versions. Pandas 3.0 now explicitly depends on `pyarrow`, superseding `fastparquet` for many common workflows. Users should target `pandas<3.0` for continued use or migrate to `pyarrow`.
fix
Downgrade Pandas to a version less than 3.0, or migrate your Parquet I/O operations to use PyArrow or other alternatives like Dask with the PyArrow engine.
affects: pandas >= 3.0
gotchaPerformance can be significantly impacted by the presence of NULL values and variable-length string encoding in your data. For optimal performance, consider using sentinel values (e.g., NaN) for data types that support them, or fixed-length strings where compatible with your ecosystem.
fix
Pre-process data to minimize NULLs or use appropriate sentinel values. Consider fixed-length string types if supported by your data and downstream systems.
affects: All versions
gotchaWhen installing `fastparquet` via `pip`, it's advisable to install `numpy` first to aid the dependency resolver. If pre-compiled wheels are not available for your system/Python version, or when installing directly from the GitHub repository, a C compiler toolchain and `cython` are required for compilation.
fix
`pip install numpy` before `pip install fastparquet`. Ensure you have a C compiler (e.g., GCC on Linux, Xcode command-line tools on macOS, MSVC on Windows) and `pip install cython` if encountering compilation errors.
affects: All versions (especially when wheels are not available)
gotchaThe test script itself contains a `SyntaxError`, preventing the library from being properly evaluated. This issue is with the test script's Python syntax rather than a direct problem with the installed library or its dependencies.
fix
Review and correct the `SyntaxError: invalid syntax` in the test script at `/script.py` line 7. Ensure the Python code adheres to valid syntax rules.
affects: All versions
Errors
Common errors & fixes
AttributeError: module 'fastparquet.parquet_thrift' has no attribute 'SchemaElement'
This error typically occurs due to version incompatibility between `fastparquet`, `pyarrow`, and `pandas`, especially when `fastparquet` version 0.8.0 or newer is used with `pyarrow` versions older than 5.0.0 and specific Python/Pandas versions (e.g., Python 3.6.9, Pandas 1.1.5).
fix
Downgrade `fastparquet` to an older, compatible version like 0.7.2. Ensure `pyarrow` is also at a compatible version (e.g., `pyarrow==5.0.0` with `fastparquet==0.7.2`).

`pip install fastparquet==0.7.2 pyarrow==5.0.0`
ModuleNotFoundError: No module named 'fastparquet'
This error means the `fastparquet` library is not installed in the Python environment where the code is being run, or the environment is not correctly activated.
fix
Install `fastparquet` using pip or conda, making sure the installation targets the correct Python environment.

`pip install fastparquet` (or `conda install -c conda-forge fastparquet` if using Anaconda)
RuntimeError: Compression 'snappy' not available. Options: ['GZIP', 'UNCOMPRESSED']
This error indicates that a specified compression library (e.g., `snappy`, `lz4`, `zstandard`, `brotli`) is not installed or properly configured in the environment, even though `fastparquet` supports it.
fix
Install the missing compression library. For 'snappy', install `python-snappy`. For others, install the corresponding Python package (e.g., `lz4`, `zstandard`, `brotli`).

`pip install python-snappy` (or `conda install -c conda-forge python-snappy`)
ValueError: Can't infer object conversion type: 0 (6.0, 1.0, 1.0, 1.0, 1.0)
This error arises when `fastparquet` encounters a column in a Pandas DataFrame that contains complex or mixed data types (like lists, tuples, or objects that it cannot automatically convert to a Parquet-compatible type).
fix
Explicitly convert the problematic column(s) to a string type (e.g., `str`) or another simple, consistent type before writing to Parquet, or preprocess the data to ensure uniform, compatible types.

`df['problematic_column'] = df['problematic_column'].astype(str)`
ImportError: Unable to find a usable engine; tried using: 'pyarrow', 'fastparquet'. A suitable version of pyarrow or fastparquet is required for parquet support.
This error from `pandas.read_parquet` indicates that neither `pyarrow` nor `fastparquet` is found or correctly installed in the environment for Pandas to use as a Parquet engine.
fix
Ensure both `pyarrow` and `fastparquet` are installed and accessible in your Python environment. While `pandas >= 3.0` explicitly depends on `pyarrow`, for `pandas < 3.0` either engine is sufficient. If `fastparquet` is intended, ensure it's installed.

`pip install pyarrow fastparquet`
Upgrade
Version history
2026.5.0latest on PyPI · released May 15, 2026
Audit
Dependencies
numpyrequiredRequired for numerical operations, recommend installing first for pip.
pandasoptionalCore dependency for DataFrame integration, specifically versions <3.0.
cythonrequiredRequired if building from Pyx files or installing from source.
cramjamrequiredRequired for compression algorithms.
fsspecrequiredRequired for filesystem abstraction.
Agent activity
8 hits · last 30 days
node
6
Resources
fastparquet — pip install fastparquet · libregistry