Registry / observability / log-rate-limit

log-rate-limit

JSON →
library1.4.2pypypi✓ verified 85d ago

Log Rate Limit is a Python library that provides a logging filter for the standard `logging` framework, designed to suppress excessive log output. It works by rate-limiting logs based on configurable streams, preventing log floods. The current version is 1.4.2, released on January 17, 2025, and it appears to be actively maintained. It requires Python versions 3.8.1 or newer, but not Python 4.0.0 or later.

pip install log-rate-limit
INSTALL
IMPORT
SIG · LOG-RATE-LIMIT
L
log-rate-limit
observabilitypythonv1.4.2
Install
1.6s avg
Import
58ms
Disk
16MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v1.4.2 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.920 runs
installs and imports cleanly · install 0.0s · import 0.061s · 17.9MB
glibc
py 3.103.920 runs
installs and imports cleanly · install 1.6s · import 0.056s · 18MB
16MB installed
● package 16MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

RateLimit
from log_rate_limit import RateLimit
Used to configure rate-limiting per log record via the 'extra' dictionary.
StreamRateLimitFilter
from log_rate_limit import StreamRateLimitFilter
The main logging filter class to apply to a logger.

This quickstart demonstrates how to apply `StreamRateLimitFilter` to a logger, allowing 3 messages per 10 seconds for each unique stream. It shows how the default behavior handles identical messages, how to use a custom `stream_id` to group dynamic messages, and how to explicitly disable rate-limiting for critical logs using `extra=RateLimit(stream_id=None)`.

import logging import time from log_rate_limit import RateLimit, StreamRateLimitFilter # Configure basic logging logging.basicConfig(level=logging.INFO, format='%(levelname)s:%(name)s:%(message)s') logger = logging.getLogger(__name__) # Apply the rate limit filter to the logger # This example allows 3 logs per 10 seconds for each unique stream_id logger.addFilter(StreamRateLimitFilter(period_sec=10, max_logs=3)) print("--- Demonstrating default rate limiting (same messages) ---") for i in range(5): logger.info("Repeating message!") time.sleep(0.5) print("\n--- Demonstrating custom stream_id for dynamic messages ---") # Messages with dynamic content can be grouped by a custom stream_id for i in range(5): device_id = "device_A" logger.warning("Error on %s: process %d failed!", device_id, i, extra=RateLimit(stream_id=f"error_on_{device_id}")) time.sleep(0.5) print("\n--- Demonstrating disabling rate limiting for a specific log ---") logger.error("CRITICAL: Something went terribly wrong, do NOT rate-limit this!", extra=RateLimit(stream_id=None)) logger.info("This info log will be rate-limited by default if repeated.") logger.info("This info log will be rate-limited by default if repeated.")
Debug
Known issues
gotchaThe `StreamRateLimitFilter` operates on the raw log message before `logging.Formatter` applies any formatting (like adding timestamps). If you want to rate-limit messages that include dynamic content but are logically 'the same' (e.g., 'Error on device X at Y time'), you must manually define a `stream_id` based on the static parts of the message.
fix
For dynamic log messages that should be rate-limited together, pass a custom `stream_id` via the `extra` dictionary: `logger.warning('Error: %s', var, extra=RateLimit(stream_id='my_static_error_type'))`.
affects: All versions
gotchaBy default, the filter assigns a unique stream ID to every distinct log message string. This means that if your log messages differ even slightly (e.g., 'Attempt 1 failed' vs. 'Attempt 2 failed'), they will be treated as separate streams and not rate-limited together. Only identical messages will be suppressed.
fix
If similar but not identical messages should share a rate limit, provide a consistent `stream_id` for them in the `extra` dictionary: `logger.info('Status update: %s', status, extra=RateLimit(stream_id='status_updates'))`.
affects: All versions
gotchaThe library explicitly does not support Python 4.0.0 or later. While Python 4.x is not yet released, this indicates a potential future breaking change if the library is not updated for compatibility.
fix
Ensure your project's Python interpreter is within the `<4.0.0,>=3.8.1` range. Monitor future releases of `log-rate-limit` for Python 4.x compatibility updates if planning to upgrade Python versions.
affects: All versions
Errors
Common errors & fixes
Logs are not being rate-limited as expected, even though the same log message appears repeatedly in the code.
The default behavior assigns a unique stream to each *exact* log message. If there are slight variations in the message string due to variables, timestamps, or other dynamic content, the filter treats them as different log streams, and thus does not rate-limit them together.
fix
Identify the static part of your log messages and manually assign a consistent `stream_id` via the `extra=RateLimit()` dictionary for all related log calls. For example, `logger.error("Device %s failed!", device_id, extra=RateLimit(stream_id="device_failure"))`.
My application logs are being suppressed too aggressively, and important messages are being dropped.
The `StreamRateLimitFilter` parameters (`period_sec`, `max_logs`) might be too restrictive for your logging volume, or the filter is applied to a logger that handles critical messages that should never be rate-limited.
fix
Adjust the `period_sec` (time window) or `max_logs` (number of allowed logs) parameters when initializing `StreamRateLimitFilter`. Alternatively, for individual critical log messages, explicitly disable rate-limiting by setting `stream_id=None` in the `extra` dictionary: `logger.critical("Emergency!", extra=RateLimit(stream_id=None))`.
Upgrade
Version history
1.4.2latest on PyPI · released Jan 18, 2025
Audit
Dependencies

No dependency data recorded yet.

Agent activity
10 hits · last 30 days
node
8
OpenAI (training)
2
Resources
log-rate-limit — pip install log-rate-limit · libregistry