Registry / data / singer-python

singer-python

library6.8.0pypypi✓ verified 25d ago

The `singer-python` library provides essential utilities for implementing Singer protocol taps (data extractors) and targets (data loaders). It simplifies tasks like parsing configuration, managing state, and emitting/consuming standard Singer messages (schema, record, state, activate_version, log, batch). It is currently at version 6.8.0 and is actively maintained, though more opinionated frameworks like Meltano SDK are also available for higher-level abstractions built on the Singer protocol.

data workflow devops

pip install singer-python

package ↗homepage ↗

INSTALL

IMPORT

SIG · SINGER-PYTHON

singer-python

datapythonv6.8.0

Install

3.3s avg

Import

402ms

Disk

22MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v6.8.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.426s · 24.6MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 3.3s · import 0.378s · 25MB

22MB installed

● package 22MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

singer

✓ import singer

get_logger

✓ singer.get_logger()

write_schema

✓ singer.write_schema(...)

write_record

✓ singer.write_record(...)

write_state

✓ singer.write_state(...)

cli

✓ import singer.cli

Used for command-line interface utilities like `singer.cli.main` or `singer.cli.with_cli` decorator for taps and targets.

This quickstart demonstrates the core functionality of `singer-python`: emitting schema, record, and state messages to standard output, which is the standard mechanism for Singer taps.

import singer import json import sys # Get a Singer logger (logs to stderr by default) LOGGER = singer.get_logger() # 1. Define a schema for a stream stream_name = "users" schema = { "type": "object", "properties": { "id": {"type": "integer", "key_properties": ["id"]}, "name": {"type": "string"}, "email": {"type": "string", "format": "email"} } } key_properties = ["id"] # 2. Write the schema message to stdout # This is how a Tap declares the structure of data it will send singer.write_schema( stream_name=stream_name, schema=schema, key_properties=key_properties ) LOGGER.info(f"Schema written for stream '{stream_name}'") # 3. Write record messages to stdout # These are the actual data rows records = [ {"id": 1, "name": "Alice", "email": "alice@example.com"}, {"id": 2, "name": "Bob", "email": "bob@example.com"} ] for record in records: singer.write_record( stream_name=stream_name, record=record ) LOGGER.debug(f"Record written for '{stream_name}': {record['id']}") # 4. Write a state message to stdout # This allows a Tap to checkpoint its progress for incremental syncs state = {"bookmarks": {stream_name: {"last_id": records[-1]["id"]}}} singer.write_state(state) LOGGER.info(f"State written: {state}") # The actual Singer messages are written to stdout. # To see them, run this script and redirect stdout: # python your_script.py > output.jsonl

Debug

Known issues

breakingAny non-Singer output (e.g., plain text via `print()`) to `stdout` will break downstream Singer targets. Singer targets expect `stdout` to contain only line-delimited JSON messages conforming to the Singer protocol.

fix

Use `singer.get_logger()` for all informational and debug output. By default, `singer.get_logger()` writes to `stderr`, ensuring `stdout` remains clean for Singer messages. Avoid all other `print()` statements.

affects: All versions

gotchaIncorrect or untimely state management can lead to unreliable incremental syncs, data loss, or reprocessing. State messages should accurately reflect processed data and be emitted only *after* all corresponding records for a given checkpoint have been successfully written.

fix

Thoroughly understand the Singer spec for state management. Ensure state is written atomically and reliably, typically after a batch of records has been confirmed processed, to correctly bookmark progress.

affects: All versions

gotcha`singer-python` provides utilities for writing schemas but does not automatically handle complex schema evolution logic (e.g., column renames, type changes requiring migration). Taps must adhere to the Singer spec for schema changes.

fix

Consult the Singer spec regarding schema evolution. Only introduce additive changes (new fields) to a stream's schema or utilize the `activate_version` message for breaking changes, ensuring the target supports it. Plan schema changes carefully.

affects: All versions

gotchaFor new projects requiring a more opinionated framework with robust CLI parsing, test helpers, and higher-level abstractions for Singer, consider using `meltano-sdk` which builds upon the Singer protocol, rather than `singer-python` directly.

fix

Evaluate your project's needs: `singer-python` for barebones Singer protocol adherence and maximum control; `meltano-sdk` for a more complete, feature-rich development experience for taps and targets.

affects: N/A

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'singer'

The `singer-python` library is not installed in the current Python environment.

fix

pip install singer-python

json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The input provided to `json.loads()` (e.g., from `sys.stdin` or a config/state file) is not valid JSON, often due to an empty file or malformed string.

fix

Validate the JSON input string for correctness before passing it to `json.loads()`. Ensure config/state files are properly formatted or handle empty input gracefully.

KeyError: 'type'

An incoming line from `sys.stdin` (or other source) was parsed as JSON, but the resulting dictionary does not contain the mandatory 'type' key, indicating it's not a valid Singer message.

fix

Ensure all messages processed are valid Singer protocol messages, each containing a 'type' field (e.g., 'SCHEMA', 'RECORD', 'STATE'). Add error handling for malformed input.

AttributeError: 'bytes' object has no attribute 'strip'

Input from `sys.stdin` is being read as raw bytes in Python 3, but subsequent string operations (like `strip()`, `decode()`, or `split()`) are attempted on the bytes object without first decoding it into a string.

fix

Explicitly decode each line read from `sys.stdin` using `line.decode('utf-8')` (or another appropriate encoding) before performing string operations.

TypeError: Object of type datetime is not JSON serializable

An attempt was made to serialize a Python object (e.g., `datetime.datetime` instance, custom class instance) into JSON using `json.dumps()` or `singer.write_message()` without a custom encoder or prior conversion to a serializable type.

fix

Convert non-serializable objects (like `datetime` objects) into a JSON-serializable format (e.g., ISO 8601 string) before passing them to the JSON serializer or `singer.write_message()`.

Upgrade

Version history

6.8.0latest on PyPI · released Feb 27, 2026

Audit

Dependencies

No dependency data recorded yet.

Agent activity

29 hits · last 30 days

node

OpenAI (training)

Resources

packagehttps://pypi.org/project/singer-python/ ↗

homepagehttp://singer.io ↗