Registry / serialization / python-schema-registry-client

python-schema-registry-client

JSON →
library2.6.1pypypi✓ verified 24d ago

A Python client to interact with the Confluent Schema Registry's REST API. It allows managing Avro and JSON schemas, including registering new schemas, retrieving existing ones by ID or subject, and performing compatibility checks. The library is currently at version 2.6.1 and is actively maintained, with releases typically tied to feature enhancements and bug fixes.

pip install python-schema-registry-client
INSTALL
IMPORT
SIG · PYTHON-SCHEMA-REGI
P
python-schema-registry-client
serializationpythonv2.6.1
Install
6.9s avg
Import
466ms
Disk
65MB
Pass rate
5/ 10
Env Coverage5 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v2.6.1 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
glibc
py 3.10
1/2 runs
✓ 6.6s
py 3.11
1/2 runs
✓ 5.55s
py 3.12
1/2 runs
✓ 7.6s
py 3.13
1/2 runs
✓ 7.3s
py 3.9
1/2 runs
✓ 7.55s
65MB installed
● package 65MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

SchemaRegistryClient
from schema_registry.client import SchemaRegistryClient
AsyncSchemaRegistryClient
from schema_registry.client import AsyncSchemaRegistryClient
AvroSchema
from schema_registry.client.schema import AvroSchema
JsonSchema
from schema_registry.client.schema import JsonSchema
MessageSerializer
from schema_registry.serializers import MessageSerializer

This quickstart demonstrates how to initialize the `SchemaRegistryClient`, define an Avro schema, register it with the Schema Registry, and then retrieve it by its ID and subject name. Ensure a Confluent Schema Registry instance is running and accessible at the specified `SCHEMA_REGISTRY_URL` (default: http://localhost:8081).

import os import asyncio from schema_registry.client import SchemaRegistryClient from schema_registry.client.schema import AvroSchema SCHEMA_REGISTRY_URL = os.environ.get('SCHEMA_REGISTRY_URL', 'http://localhost:8081') async def main(): client = SchemaRegistryClient(url=SCHEMA_REGISTRY_URL) avro_schema_definition = { "type": "record", "namespace": "com.example", "name": "SensorReading", "fields": [ {"name": "id", "type": "string"}, {"name": "value", "type": "int"} ] } # Create an AvroSchema object avro_schema = AvroSchema(avro_schema_definition) # Define a subject name subject = "sensor-readings-value" try: # Register the schema registered_schema = client.register(subject, avro_schema) print(f"Schema registered with ID: {registered_schema.schema_id}") # Get the schema by ID retrieved_schema = client.get_by_id(registered_schema.schema_id) print(f"Retrieved schema (ID {registered_schema.schema_id}): {retrieved_schema.schema.to_dict()}") # Get the latest schema for a subject latest_schema_info = client.get_latest_version(subject) print(f"Latest schema for '{subject}' (version {latest_schema_info.version}, ID {latest_schema_info.schema.schema_id}): {latest_schema_info.schema.to_dict()}") except Exception as e: print(f"An error occurred: {e}") if __name__ == "__main__": # For synchronous use, just call client methods directly. # The example uses async to show client.register which is an async method by default # if the client is AsyncSchemaRegistryClient. For SchemaRegistryClient, it's synchronous. # However, the example above will work for both if using the sync client # due to how the `register` method is implemented (it's not truly async with await here). # Let's adjust for clarity to ensure it runs correctly with the default SchemaRegistryClient. # For a truly async example, one would use AsyncSchemaRegistryClient and 'await'. # For this quickstart, we'll keep it simple with the synchronous client. # Rerunning the quickstart for synchronous client clarification sync_client = SchemaRegistryClient(url=SCHEMA_REGISTRY_URL) avro_schema_definition = { "type": "record", "namespace": "com.example", "name": "SensorReading", "fields": [ {"name": "id", "type": "string"}, {"name": "value", "type": "int"} ] } avro_schema = AvroSchema(avro_schema_definition) subject = "sensor-readings-value-sync" try: registered_schema = sync_client.register(subject, avro_schema) print(f"Sync: Schema registered with ID: {registered_schema.schema_id}") retrieved_schema = sync_client.get_by_id(registered_schema.schema_id) print(f"Sync: Retrieved schema (ID {registered_schema.schema_id}): {retrieved_schema.schema.to_dict()}") latest_schema_info = sync_client.get_latest_version(subject) print(f"Sync: Latest schema for '{subject}' (version {latest_schema_info.version}, ID {latest_schema_info.schema.schema_id}): {latest_schema_info.schema.to_dict()}") except Exception as e: print(f"Sync: An error occurred: {e}")
Debug
Known issues
gotchaThe `SchemaRegistryClient` may not be picklable, which can cause issues in distributed computing environments like Apache Spark or when using multiprocessing that relies on pickling objects. This is often due to its internal use of `requests.Session` (or `httpx.Client`).
fix
Avoid passing `SchemaRegistryClient` instances directly between processes. Instead, initialize a new client instance within each process or task where it's needed. For PySpark, consider using broadcast variables for configuration and re-initializing the client in UDFs.
affects: All versions
gotchaWhen using the `faust` extra (`pip install python-schema-registry-client[faust]`), the library pulls in `faust-streaming`, which is a fork of the original `faust` library. This might lead to version conflicts or unexpected behavior if your project already depends on a specific version of `faust`.
fix
If you require a specific `faust` version or wish to avoid the `faust-streaming` fork, install `faust` manually first, then install `python-schema-registry-client` without the `[faust]` extra: `pip install faust` followed by `pip install python-schema-registry-client`.
affects: All versions with faust extra
gotchaSSL certificate verification can fail, especially when connecting to Schema Registry instances with self-signed certificates or improperly configured CA chains. This manifests as `requests.exceptions.SSLError` (or similar `httpx` errors).
fix
Ensure `ca_location`, `cert_location`, and `key_location` are correctly configured with paths to valid certificates. For development or controlled environments, `verify=False` can be passed to the underlying HTTP client (if exposed, or by subclassing/monkey-patching carefully) but is NOT recommended for production. Refer to `httpx` or `requests` SSL documentation for advanced configuration.
affects: All versions
gotchaSchema compatibility violations (e.g., error code 409) occur when registering a new schema that is incompatible with previous versions based on the subject's compatibility level.
fix
Always test compatibility before registering a new schema using `client.test_compatibility(subject_name, new_schema)`. Ensure your schema changes adhere to the configured compatibility rules (e.g., `BACKWARD`, `FORWARD`, `FULL`). Consider adding default values to new fields in Avro schemas to maintain backward compatibility.
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'schema_registry'
The 'python-schema-registry-client' package is not installed in the current Python environment, or the import statement uses an incorrect module name.
fix
Install the library using `pip install python-schema-registry-client` and ensure the import is `from schema_registry.client import SchemaRegistryClient`.
requests.exceptions.ConnectionError: Failed to establish a new connection: [Errno 111] Connection refused
The Schema Registry service is not reachable from the client, likely because it's not running, the URL is incorrect, or network connectivity issues prevent the connection.
fix
Verify the Schema Registry URL (e.g., `http://localhost:8081` or remote address) is correct and that the Schema Registry service is running and accessible from the client's network.
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http://localhost:8081/subjects
The client failed to authenticate with the Schema Registry. This typically means incorrect or missing username/password for basic authentication when the Schema Registry requires it.
fix
Provide correct authentication credentials when initializing `SchemaRegistryClient`, e.g., `client = SchemaRegistryClient(url='...', auth=('username', 'password'))`.
schema_registry.client.errors.SchemaRegistryError: 42201: Invalid Avro schema: Provided schema is not a valid Avro schema
The Avro schema string provided to a method like `register` is not syntactically valid or does not conform to Avro specification according to the Schema Registry's validation rules.
fix
Review and correct the Avro schema definition to ensure it adheres to the Avro specification, checking for proper JSON structure, correct field types, and valid keywords.
Upgrade
Version history
2.6.1latest on PyPI · released Apr 4, 2025
Audit
Dependencies
python>=3.8,<4.0requiredRequired Python version as specified by the project.
httpxrequiredUsed internally for HTTP requests, although not explicitly listed as a direct dependency for users, it's a core component. May be pulled in by direct dependencies.
faust-streamingoptionalRequired for the optional 'faust' integration extra. This is a fork of faust.
Agent activity
13 hits · last 30 days
node
10
Resources