The `ndjson` library for Python, currently at version `0.3.1`, provides a `JsonDecoder` and `JsonEncoder` for newline-delimited JSON (NDJSON), also known as JSON Lines. It offers a familiar interface similar to Python's built-in `json` module, enabling efficient reading and writing of NDJSON data to and from file-like objects and strings. This lightweight library has no external dependencies and is particularly useful for processing large datasets or streaming applications where each line represents a complete, independent JSON object. Although its last release was in 2020 and its PyPI status is 'Pre-Alpha', it is considered stable and functional for its stated purpose.
pip install ndjsonVerified import paths — ran on the pinned version, not inferred.
This quickstart demonstrates how to write and read NDJSON data using the `ndjson` library's `dump`/`load` functions for bulk operations and `writer`/`reader` classes for streaming line-by-line processing, similar to Python's `csv` module. This is particularly efficient for large files, avoiding the need to load the entire dataset into memory.
Be aware of the PyPI status, but understand that the library is stable for its intended use cases. The version 0.3.1 has been consistent since 2020.
Use `ndjson.load(file_object)` or iterate with `ndjson.reader(file_object)` for stream-based parsing, which correctly handles newline-delimited JSON objects.
The `ndjson` library handles this correctly with `ndjson.dump()` and `ndjson.writer()`. When manually constructing NDJSON, always ensure one valid JSON object per line.
Evaluate if the current feature set meets your needs. For very large-scale or high-performance NDJSON processing, consider alternatives like `ijson` for incremental parsing or `orjson` for faster JSON operations, or libraries integrated with dataframes like `polars.read_ndjson`.
Always specify `encoding='utf-8'` when opening NDJSON files. If a BOM is present and causing issues, use `encoding='utf-8-sig'` to automatically strip it.
pip install ndjson
Use `ndjson.load()` for file-like objects, or iterate through the file line by line, parsing each line as a separate JSON object using `ndjson.loads()`:
```python
import ndjson
# Using ndjson.load() for file-like objects
with open('data.ndjson', 'r', encoding='utf-8') as f:
data = ndjson.load(f)
# Or for very large files, process line by line
def read_ndjson_lines(filepath):
with open(filepath, 'r', encoding='utf-8') as f:
for line in f:
line = line.strip()
if line: # Skip empty lines
yield ndjson.loads(line)
for record in read_ndjson_lines('data.ndjson'):
print(record)
```Ensure each line in your NDJSON file represents a complete and syntactically correct JSON object. When parsing, you can include error handling to skip or log malformed lines:
```python
import ndjson
import json
def parse_ndjson_with_error_handling(filepath):
valid_records = []
with open(filepath, 'r', encoding='utf-8') as f:
for line_num, line in enumerate(f, 1):
line = line.strip()
if not line: # Skip empty lines
continue
try:
record = ndjson.loads(line)
valid_records.append(record)
except json.JSONDecodeError as e:
print(f"Error on line {line_num}: {e} - Content: {line[:100]}")
return valid_records
records = parse_ndjson_with_error_handling('malformed.ndjson')
print(f"Successfully parsed {len(records)} records.")
```Always specify `encoding='utf-8'` when opening files for writing and `ensure_ascii=False` when using `ndjson.dump()` or `ndjson.dumps()` if your data contains non-ASCII characters:
```python
import ndjson
data_with_unicode = [{'name': 'Cáceres', 'city': 'Spain'}, {'name': '京都', 'city': 'Japan'}]
# When dumping to a string
unicode_text = ndjson.dumps(data_with_unicode, ensure_ascii=False)
print(unicode_text)
# When dumping to a file
with open('output_unicode.ndjson', 'w', encoding='utf-8') as f:
ndjson.dump(data_with_unicode, f, ensure_ascii=False)
```No dependency data recorded yet.