Registry / observability / pygtail

pygtail

JSON →
library0.14.0pypypi✓ verified 87d ago

Pygtail is a Python library and command-line tool that reads log file lines that have not been previously read. It is a 'port' of logcheck's logtail2 and is capable of handling log files that have been rotated. The current stable version is 0.14.0. Releases are infrequent, often tied to bug fixes or minor enhancements.

pip install pygtail
INSTALL
IMPORT
SIG · PYGTAIL
P
pygtail
observabilitypythonv0.14.0
Install
1.6s avg
Import
17ms
Disk
16MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v0.14.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.920 runs
installs and imports cleanly · install 0.0s · import 0.018s · 17.8MB
glibc
py 3.103.920 runs
installs and imports cleanly · install 1.6s · import 0.017s · 18MB
16MB installed
● package 16MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

Pygtail
from pygtail import Pygtail
import pygtail
The primary class 'Pygtail' needs to be imported directly, not accessed via the top-level package name.

This quickstart demonstrates how to use Pygtail to read new lines from a log file. It first creates a dummy log file, reads its initial content, then appends new lines, and finally uses Pygtail again to read only the newly added lines. Pygtail automatically manages an offset file (by default `logfile.offset`) to keep track of already read content.

import sys import os from pygtail import Pygtail # Create a dummy log file for demonstration log_file_path = 'example.log' with open(log_file_path, 'w') as f: f.write('Line 1\n') f.write('Line 2\n') print(f"Reading new lines from {log_file_path}:") for line in Pygtail(log_file_path): sys.stdout.write(line) # Simulate new log entries with open(log_file_path, 'a') as f: f.write('Line 3 (new)\n') f.write('Line 4 (new)\n') print("\nReading newly added lines:") for line in Pygtail(log_file_path): sys.stdout.write(line) # Clean up dummy log and offset files if os.path.exists(log_file_path): os.remove(log_file_path) if os.path.exists(log_file_path + '.offset'): os.remove(log_file_path + '.offset')
pygtail --version
Debug
Known issues
gotchaWhen log files are rotated using a 'copytruncate' method (where the file is emptied and rewritten), Pygtail might not detect this by default, leading to missing new entries. This is because the inode remains the same, but the content is reset.
fix
Initialize `Pygtail` with `copytruncate=True` (e.g., `Pygtail('logfile.log', copytruncate=True)`). This enables support for `copytruncate`-style rotation.
affects: <=0.14.0
gotchaOn Windows, Pygtail may encounter `UnicodeDecodeError` when reading log files containing non-ASCII characters, especially if the file was generated on a Linux system or uses UTF-8 encoding.
fix
Specify the correct encoding when initializing `Pygtail` using the `encoding` parameter (e.g., `Pygtail('logfile.log', encoding='utf8')`).
affects: <=0.14.0
gotchaPygtail automatically creates an offset file (e.g., `logfile.log.offset`) to track read progress. If this offset file is deleted or becomes corrupted, Pygtail will re-read the entire log file from the beginning, which can lead to duplicate processing of log entries.
fix
Manage the lifecycle of the offset file carefully. If you need to re-read a file from the end if the offset file is missing, use the `read_from_end=True` parameter. For temporary reading without persistent offsets, you can specify `offset_file='/dev/null'` (on Unix-like systems) or use `save_on_end=False` and manually manage offsets with `tail.with_offsets()` and `tail.write_offset_to_file()`.
affects: <=0.14.0
Errors
Common errors & fixes
UnicodeDecodeError: 'charmap' codec can't decode byte 0x__ in position __: character maps to <undefined>
Attempting to read a log file with an encoding (e.g., UTF-8) that differs from the system's default encoding, particularly common on Windows systems expecting CP1252.
fix
Pass the `encoding` parameter to the `Pygtail` constructor: `tail = Pygtail('your.log', encoding='utf8')`. Adjust 'utf8' to the actual encoding of your log file if different.
Pygtail stops updating offset file and always returns 0 for stats.
This can occur when log rotation is performed by moving the original log file and creating a new empty one, rather than truncating it (e.g., `copytruncate`). Pygtail, by default, might not recognize this as a rotation if the inode remains the same but the file content is reset.
fix
Enable `copytruncate` support by initializing `Pygtail` with `copytruncate=True`: `tail = Pygtail('your.log', copytruncate=True)`. If log rotation involves compression, `pygtail` might also need custom log patterns using the `--log-pattern` command-line option, though this is not directly supported in the Python API for dynamic patterns.
FileNotFoundError: [Errno 2] No such file or directory: 'some.log.offset'
The default offset file name is derived from the log file. If Pygtail cannot create or access this file due to permissions or an invalid path, it will raise this error.
fix
Ensure the directory where the log file resides (or where you specify the offset file) has write permissions for the user running the Pygtail script. Alternatively, specify an explicit and writable `offset_file` path: `tail = Pygtail('some.log', offset_file='/var/log/my_app/some.log.offset')`.
Upgrade
Version history
0.14.0latest on PyPI · released Nov 6, 2022
Audit
Dependencies

No dependency data recorded yet.

Agent activity
19 hits · last 30 days
node
16
OpenAI (training)
2
Amazon
1
Resources
pygtail — pip install pygtail · libregistry