drain3

library0.9.11pypypi✓ verified 84d ago

Drain3 is a Python library for mining log templates from raw log messages, designed for stream processing. It's based on the Drain algorithm and is suitable for real-time log analysis. The library is actively maintained with frequent patch releases, currently at version 0.9.11.

data ai-ml

pip install drain3

github ↗package ↗

INSTALL

IMPORT

SIG · DRAIN3

drain3

datapythonv0.9.11

Install

2.6s avg

Import

—

Disk

17MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v0.9.11 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.920 runs

installs and imports cleanly · install 0.0s · import 0.000s · 19.8MB

glibc

py 3.10–3.920 runs

installs and imports cleanly · install 2.6s · import 0.000s · 20MB

17MB installed

● package 17MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

TemplateMiner

✓ from drain3 import TemplateMiner

✗ from drain3 import Drain3

This quickstart demonstrates how to initialize Drain3 with a basic configuration, add log messages, and retrieve the identified log clusters. For production use, consider configuring persistence (file, Redis, or HTTP) via `TemplateMinerConfig` to save and load the model state.

from drain3 import Drain3 from drain3.template_miner_config import TemplateMinerConfig import os # Configure Drain3. For production, consider loading from a file or Redis. # Example for file persistence: # config = TemplateMinerConfig.load('drain3.ini') # Ensure 'persist_state_to_file' and 'state_file_path' are set in config. config = TemplateMinerConfig() config.load_default_config() config.drain_sim_th = 0.4 config.depth = 4 # If you want to use file persistence (ensure directory exists and is writable): # config.persistence_type = 'FILE' # config.file_persistence_path = os.path.join(os.getcwd(), 'drain3_state.bin') drain = Drain3(config) log_messages = [ "081109 203619 143 INFO dfs.DataNode$PacketResponder: PacketResponder " "0 for block blk_3886504917409280145 terminating", "081109 203619 369 INFO dfs.DataNode$PacketResponder: PacketResponder " "0 for block blk_-6755409170280820986 terminating", "081109 203620 357 INFO dfs.DataNode$PacketResponder: PacketResponder " "2 for block blk_814013142207908518 terminating", "081109 203620 543 INFO dfs.DataNode$DataXceiver: Receiving block blk_-6755409170280820986 " "src: /10.250.9.141:50106 dest: /10.250.9.141:50010", ] for log_message in log_messages: cluster_id = drain.add_log_message(log_message) print(f"Log: '{log_message}' -> Cluster ID: {cluster_id}") # After processing, it's good practice to save the state if using persistence # if config.persistence_type == 'FILE': # drain.save_state() print("\n--- Current Clusters ---") for cluster in drain.drain.clusters: print(cluster)

Debug

Known issues

breakingVersion 0.9.0 introduced significant breaking changes, including a major refactor of the API. The `LogViewer` module was removed, and `TemplateMinerConfig` was refactored from a static method `load()` to a class that can be instantiated and configured. Imports for `Drain3` and `TemplateMinerConfig` have also changed.

fix

Review the official documentation and update imports and configuration instantiation. For example, `config = TemplateMinerConfig()` instead of `config = TemplateMinerConfig.load('config.ini')`, and `from drain3 import Drain3`.

affects: >=0.9.0

gotchaDrain3 does not automatically persist its state (the learned log templates) by default. If your application restarts without explicit state saving and loading, it will lose all learned templates and start learning from scratch, leading to reprocessing and inconsistent cluster IDs.

fix

Configure a persistence backend (FILE, REDIS, or HTTP) using `TemplateMinerConfig`. Call `drain.save_state()` periodically or upon application shutdown, and `drain.load_state()` upon application startup to ensure state is maintained. Example: `config.persistence_type = 'FILE'; config.file_persistence_path = 'path/to/state.bin'`.

affects: All

gotchaThe quality of log templates heavily depends on the `TemplateMinerConfig` parameters, especially `drain_sim_th` (similarity threshold) and `depth`. Incorrect settings can lead to overly generic or too specific templates, reducing the effectiveness of log parsing.

fix

Experiment with `drain_sim_th` and `depth` based on your log data characteristics. A lower `drain_sim_th` (e.g., 0.3-0.4) creates more general templates, while a higher value (e.g., 0.6-0.8) creates more specific ones. `depth` controls how many tokens are considered for tree traversal before a new node is created.

affects: All

Upgrade

Version history

0.9.11latest on PyPI · released Jul 17, 2022

Audit

Dependencies

PyYAMLoptionalRequired for loading configuration from YAML/INI files.

redisoptionalRequired for Redis-based persistence backend.

requestsoptionalRequired for HTTP-based persistence backend.

Agent activity

36 hits · last 30 days

node

Amazon

OpenAI (training)

Resources

githubhttps://github.com/IBM/Drain3 ↗

packagehttps://pypi.org/project/drain3/ ↗