Registry / observability / python-logstash-async

python-logstash-async

JSON →
library4.1.0pypypi✓ verified 24d ago

Python-logstash-async is an asynchronous Python logging handler designed to send log events to a remote Logstash instance. It processes log events in a separate worker thread to avoid blocking the main application, crucial for performance-sensitive applications like web services. It supports TCP, UDP, and Beats protocols, with optional SSL for TCP, and can persist unsent logs to a SQLite database. The library is actively maintained with regular releases addressing bug fixes and introducing new features.

pip install python-logstash-async
INSTALL
IMPORT
SIG · PYTHON-LOGSTASH-AS
P
python-logstash-async
observabilitypythonv4.1.0
Install
3.0s avg
Import
628ms
Disk
22MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v3.0.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.660s · 24.2MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 3.0s · import 0.596s · 25MB
22MB installed
● package 22MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

AsynchronousLogstashHandler
from logstash_async.handler import AsynchronousLogstashHandler
LogstashFormatter
from logstash_async.formatter import LogstashFormatter
HttpTransport
from logstash_async.transport import HttpTransport
BeatsTransport
from logstash_async.transport import BeatsTransport

This quickstart demonstrates how to configure `python-logstash-async` with `AsynchronousLogstashHandler` and `LogstashFormatter`. It sets up a logger to send INFO, WARNING, and ERROR messages to a Logstash instance, including extra fields. A `database_path` is specified for persistent storage of events in case Logstash is unreachable or the application restarts. The example also includes a console handler for immediate feedback during development.

import logging import os from logstash_async.handler import AsynchronousLogstashHandler from logstash_async.formatter import LogstashFormatter # Configure Logstash host and port (replace with your Logstash details) LOGSTASH_HOST = os.environ.get('LOGSTASH_HOST', 'localhost') LOGSTASH_PORT = int(os.environ.get('LOGSTASH_PORT', '5959')) # Or 5000 for standard TCP/UDP LOGSTASH_DATABASE_PATH = os.environ.get('LOGSTASH_DB_PATH', 'logstash_events.db') # Get a logger instance logger = logging.getLogger('my_app_logger') logger.setLevel(logging.INFO) # Create a Logstash formatter formatter = LogstashFormatter(message_type='python-logstash', extra_prefix='dev', extra={'application': 'my-python-app'}) # Create an asynchronous Logstash handler # It's recommended to specify a database_path for persistence across restarts handler = AsynchronousLogstashHandler( host=LOGSTASH_HOST, port=LOGSTASH_PORT, database_path=LOGSTASH_DATABASE_PATH, # For TCP/Beats with SSL, set ssl_enable=True and configure certs # ssl_enable=True, # ssl_verify=True, # ca_certs='/path/to/ca.crt' ) handler.setFormatter(formatter) logger.addHandler(handler) # Add a console handler for local debugging console_handler = logging.StreamHandler() console_handler.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')) logger.addHandler(console_handler) # Log some messages logger.info('This is an info message from the quickstart.', extra={'user_id': 123}) logger.warning('A warning occurred with some extra data.', extra={'component': 'auth', 'status': 'failed'}) logger.error('An error message with detailed context.') # Ensure all queued logs are sent before exiting # In a real application, this might be handled by an atexit hook or proper shutdown logic logger.info('Flushing pending log events...') handler.close() # This will attempt to flush remaining events
Debug
Known issues
breakingVersion 4.0.0 and newer drop support for Python versions 3.8, 3.9, and 3.10.
fix
Upgrade to Python 3.11 or newer, or stick to `python-logstash-async` versions < 4.0.0.
affects: 4.0.0+
gotchaUsing `AsynchronousLogstashHandler` without specifying a `database_path` will result in the loss of unsent log messages if the application process restarts or crashes.
fix
Always provide a `database_path` argument to `AsynchronousLogstashHandler` to enable event persistence to a SQLite database. For example: `database_path='logstash_events.db'`.
affects: All versions
gotchaUsing multiple instances of `AsynchronousLogstashHandler` with *different* `database_path` settings in the same process will not work as expected. Only the `database_path` from the first handler that emits a log event will be used by the single underlying `LogProcessingWorker`. Using `python-logstash-async` with standard Python `multiprocessing` can also be problematic due to how the worker thread and database are managed.
fix
For multiple handlers in a single process, configure them to use the *same* `database_path`. Avoid using `AsynchronousLogstashHandler` directly in multi-process scenarios where logs need to be flushed from each child process independently; consider using `logging.QueueHandler` with a `QueueListener` in the parent process or writing to files and using a tool like Filebeat for robust multi-process logging.
affects: All versions
gotchaOlder versions (prior to 4.0.2) might experience hangs on socket errors or improper shutdown of UDP sockets, potentially leading to resource leaks or unresponsiveness.
fix
Upgrade to version 4.0.2 or newer to benefit from fixes addressing socket handling and preventing hangs during network errors.
affects: < 4.0.2
gotchaWhen adding `extra` fields to log messages, ensure the keys do not clash with reserved names used by Python's logging system (e.g., `levelname`, `asctime`, `filename`).
fix
Refer to Python's `logging.Formatter` documentation for a list of reserved attributes. Use a `extra_prefix` in `LogstashFormatter` to avoid conflicts (e.g., `extra_prefix='my_app_data'`).
affects: All versions
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'logstash_async'
The `python-logstash-async` library is either not installed, or the import statement uses an incorrect module name.
fix
Ensure the library is installed with `pip install python-logstash-async` and use the correct import: `from logstash_async.handler import AsynchronousLogstashHandler`.
ConnectionRefusedError: [Errno 111] Connection refused
The Python application failed to establish a connection to the Logstash server, usually because Logstash is not running, is inaccessible, or the configured host/port is incorrect.
fix
Verify that the Logstash instance is running and listening on the specified host and port, and check network connectivity between the client and server.
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED]
The client application could not verify the Logstash server's SSL certificate, typically due to a missing, invalid, or untrusted Certificate Authority (CA) certificate.
fix
Ensure `ssl_enable=True` and provide the correct `ssl_ca_certs` parameter pointing to the trusted CA certificate bundle when initializing `AsynchronousLogstashHandler`.
TypeError: an integer is required (got type NoneType)
The `port` parameter for `AsynchronousLogstashHandler` was provided with a `None` value instead of an integer.
fix
Ensure the `port` parameter is set to an integer representing the Logstash input port, e.g., `port=5000`.
Upgrade
Version history
4.1.0latest on PyPI · released Nov 23, 2025
Audit
Dependencies
pythonrequiredRequires Python 3.11 or newer for version 4.0.0+.
Agent activity
17 hits · last 30 days
node
14
OpenAI (training)
2
Resources
python-logstash-async — pip install python-logstash-async · libregistry