Install & Compatibility
Where this runs
tested against v0.4.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
muslpy 3.10–3.910 runs
installs and imports cleanly · install 0.0s · import 0.000s · 31.1MB
glibcpy 3.10–3.910 runs
installs and imports cleanly · install 36.3s · import 0.009s · 30MB
30MB installed
● package 30MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
array_record_module
✓ from array_record.python import array_record_module
Provides file-level read/write APIs, with a one-to-one mapping to the underlying C++ API.
array_record_data_source
✓ from array_record.python import array_record_data_source
Wraps `array_record_module` for convenient access to multiple ArrayRecord files.
This quickstart demonstrates how to write records to an ArrayRecord file and then read them back using both the file-level API (`array_record_module`) and the multi-file API (`array_record_data_source`). It highlights the importance of `group_size` during writing for optimal reading patterns.
from array_record.python import array_record_module
import os
# Define output path
output_file = 'output.array_record'
# --- Writing Records ---
# Use `group_size:1` for optimized random access; larger sizes improve sequential/batch access and compression.
writer = array_record_module.ArrayRecordWriter(output_file, 'group_size:1')
for i in range(10):
data = f"Record {i} data".encode('utf-8')
writer.write(data)
writer.close()
print(f"Wrote 10 records to {output_file}")
# --- Reading Records (File-level API) ---
reader = array_record_module.ArrayRecordReader(output_file)
print(f"Reading records from {output_file}:")
for i in range(reader.num_records):
record = reader.read(i)
print(f" Record {i}: {record.decode('utf-8')}")
reader.close()
# --- Reading Records (Multi-file API with DataSource) ---
# Note: For DataSource, the writer MUST specify group_size='group_size:1'
from array_record.python import array_record_data_source
# In a real scenario, you'd have multiple files, e.g., ['file1.array_record', 'file2.array_record']
data_source = array_record_data_source.ArrayRecordDataSource([output_file])
print(f"Reading records using DataSource from {output_file}:")
for i in range(len(data_source)):
record = data_source[i]
print(f" DataSource Record {i}: {record.decode('utf-8')}")
# Clean up the created file
os.remove(output_file)
print(f"Cleaned up {output_file}")
arrayrecord --version
Debug
Known issues
breakingThe `__getitem__` method signature changed in `v0.4.0`. It now strictly accepts a single integer index and returns a single record, aligning with Python's standard `__getitem__` behavior. Batching multiple indexes is no longer supported directly via `__getitem__`.fixFor accessing multiple records simultaneously, use the newly introduced `__getitems__()` method. For single record access, ensure you pass a single integer index.
affects: >=0.4.0
breakingPrior to `v0.4.0`, there was a transition in how `__getitem__` handled batching (e.g., in `v0.3.0` batching was no longer *required* for good performance but was still implicitly handled, paving the way for the `v0.4.0` strict single-item access). Code relying on `__getitem__` to implicitly handle lists of indices will break.fixUpdate `__getitem__` calls to pass single integer indices. For batch reads, adapt to use iteration over single accesses or upgrade to `v0.4.0+` and use `__getitems__()`.
affects: >=0.3.0, <0.4.0
gotchaWhen using `array_record_data_source` for multi-file access and random access, it is crucial that the `ArrayRecordWriter` specified `group_size='group_size:1'` when creating the ArrayRecord files. If not, the `DataSource` may not function as expected or might be inefficient.fixAlways initialize `ArrayRecordWriter` with `group_size='group_size:1'` if you intend to use `ArrayRecordDataSource` with random access patterns.
affects: All versions
gotchaArrayRecord requires Python 3.11 or newer for current versions. Older versions (e.g., 0.2.0) supported Python 3.8+.fixEnsure your Python environment is version 3.11 or higher. Check the `Requires-Python` metadata on PyPI for the specific version you are installing.
affects: >=0.6.0 (approximate, current versions)
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'array_record'
The 'array_record' module is not installed or the import statement is incorrect.
fixEnsure the module is installed using 'pip install array-record' and import it correctly with 'import array_record'.
AttributeError: module 'array_record' has no attribute 'write'
Attempting to use a 'write' method that does not exist in the 'array_record' module.
fixRefer to the official documentation to use the correct method for writing records, such as 'array_record.write_records()'.
TypeError: 'NoneType' object is not iterable
A function in 'array_record' returned 'None' when an iterable was expected.
fixCheck the function's return value and ensure it returns an iterable object as expected.
ValueError: Invalid record index
Attempting to access a record index that is out of bounds or does not exist.
fixVerify that the record index is within the valid range before accessing it.
ImportError: cannot import name 'ArrayRecordWriter' from 'array_record'
The 'ArrayRecordWriter' class does not exist in the 'array_record' module or is not available in the installed version.
fixCheck the module's documentation for the correct class name or update to the latest version of 'array_record'.
Upgrade
Version history
0.8.3latest on PyPI · released Nov 13, 2025
Audit
Dependencies
absl-pyrequiredRequired runtime dependency for core functionality.
etilsrequiredRequired runtime dependency for core functionality.
apache-beamoptionalOptional dependency for Apache Beam integration.
grainoptionalOptional dependency for integration with the Grain data loading library.