Pandavro provides a convenient interface to read and write Avro files using pandas DataFrames. It simplifies the serialization and deserialization of tabular data between Python's pandas library and the Avro data format. The current version is 1.9.0, and it maintains an active release schedule with updates for Python, pandas, and NumPy compatibility.
pip install pandavroVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to create a pandas DataFrame, write it to an Avro in-memory stream using `pandavro.to_avro()`, and then read it back into a new DataFrame with `pandavro.read_avro()`.
Review your application's reliance on pandas/NumPy internals and behavior when upgrading to pandas 2.0+ or NumPy 2.0+. Test thoroughly after upgrading underlying dependencies.
Inspect the inferred schema if schema compatibility is critical (e.g., by writing to a file and examining it with an Avro tool). Pre-process your DataFrame to ensure consistent dtypes for columns before writing, or explicitly cast types to match your desired Avro schema.
Standardize `NaN` and `None` handling in your DataFrame. Use `df.fillna()` or explicit type conversions (e.g., `pd.Int64Dtype()`) to ensure consistent nullable types. For `object` columns, ensure all non-null values are of a consistent type that maps well to Avro (e.g., all strings).