Fastavro is a high-performance Python library for reading and writing Avro files. It provides a significantly faster alternative to the official Apache Avro Python library, leveraging C extensions (Cython) for optimal speed. The library supports various compression codecs and is actively maintained, making it a popular choice for high-throughput Avro serialization and deserialization in Python applications.
pip install fastavroVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to define an Avro schema, write a list of Python dictionaries (records) into an in-memory Avro binary format using `fastavro.writer`, and then read those records back using `fastavro.reader`. It also highlights the use of `parse_schema` for efficiency and `codec` for compression.
Do not rely on a global cache for schemas. Pass parsed schemas explicitly or re-parse them as needed. `parse_schema` returns a parsed schema object that should be passed to `writer` or `reader` functions.
Install `cramjam` (`pip install cramjam`) and ensure it's available in your environment for Snappy compression.
Always ensure your reader schema is compatible with the writer schema, especially when dealing with schema evolution. Use `parse_schema` and provide both `writer_schema` (from the file) and `reader_schema` (your application's expected schema) to `fastavro.reader` for schema resolution.
Ensure all required fields are present in your Python dictionary records before passing them to `fastavro.writer`. For optional fields, explicitly use `null` if the field is omitted, or define a default value in the schema.
Open the file with `mode='a+b'` and call `writer(file_object, None, more_records)`.
Avoid using `expand=True` if you intend to reuse the parsed schema for reading or writing. Instead, manage referenced schemas via the `named_schemas` argument in `parse_schema` if you have complex, inter-dependent schemas.
Be aware of the return type when `return_record_name` is set. Adjust your code to unpack the `(name, value)` tuple or use `return_record_name_override=True` if you prefer a simpler return for single-type unions.
Run `pip install fastavro` in your terminal to install the library.
Ensure that your data strictly adheres to the provided Avro schema. When reading, verify that the reader's schema is compatible with the writer's schema, paying close attention to field names, types, and the use of aliases or union types. Validate your schema structure, especially for enums or complex types.
Ensure that the records you are passing to `fastavro.writer` are lists of dictionaries, and that each dictionary's values precisely match the types defined in your Avro schema for the corresponding fields. For example, if your schema defines a record, you must pass a Python dictionary for that record, not a string or other type.
Install the required compression library. For Snappy, run `pip install python-snappy` (or `pip install cramjam` for `fastavro` versions that use it for Snappy). Similarly, for Zstandard, install `pip install python-zstandard` (or `pip install backports.zstd` for older Python versions).