sqlitedict

library2.1.0pypypi✓ verified 23d ago

SqliteDict is a Python library that provides a persistent dictionary interface, backed by sqlite3 and using pickle for serialization. It is designed to be multithread-safe as a workaround for Python's `sqlite3` thread limitations and supports multiple tables within a single database file. It offers a simple, Pythonic dict-like interface to an SQLite database, currently at version 2.1.0, and is actively maintained.

database serialization

pip install sqlitedict

github ↗package ↗

INSTALL

IMPORT

SIG · SQLITEDICT

sqlitedict

databasepythonv2.1.0

Install

2.4s avg

Import

42ms

Disk

17MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v2.1.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.046s · 19.5MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 2.4s · import 0.038s · 20MB

17MB installed

● package 17MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

SqliteDict

✓ from sqlitedict import SqliteDict

This quickstart demonstrates how to create and interact with an SqliteDict, covering both `autocommit=True` and manual `commit()` usage, as well as the recommended `with` statement for proper database closing. It stores dictionary objects that are automatically pickled.

from sqlitedict import SqliteDict import os db_path = 'my_data.sqlite' # Clean up previous run if exists if os.path.exists(db_path): os.remove(db_path) # Open a new SqliteDict with autocommit enabled (recommended for many small writes) with SqliteDict(db_path, autocommit=True) as db: db['user:1'] = {'name': 'Alice', 'age': 30} db['user:2'] = {'name': 'Bob', 'age': 25} db['settings:theme'] = 'dark' print(f"Added 3 items. Current length: {len(db)}") # Re-open the database (autocommit defaults to False if not specified) with SqliteDict(db_path) as db_read_only: print(f"Retrieved user:1: {db_read_only['user:1']}") print(f"All keys: {list(db_read_only.keys())}") # Manual commit is needed if autocommit is False db_read_only['new_item'] = 'This will NOT be saved without commit()' # db_read_only.commit() # Uncomment to save print("Demonstrating explicit commit") # Open without autocommit, requiring explicit commit with SqliteDict(db_path, autocommit=False) as db_manual: db_manual['product:101'] = {'name': 'Widget', 'price': 19.99} db_manual.commit() # Explicitly commit changes print(f"Added product:101. Current length: {len(db_manual)}") # Verify the manually committed item with SqliteDict(db_path) as db_verify: print(f"Retrieved product:101: {db_verify['product:101']}") # Clean up the database file if os.path.exists(db_path): os.remove(db_path)

Debug

Known issues

breakingVersion 2.0.0 and above dropped support for Python 2.x and requires Python 3.7 or newer. If you need support for older Python versions, you must use `sqlitedict` version 1.7.0 or earlier.

fix

Upgrade your Python environment to 3.7+ or pin `sqlitedict` to version `1.7.0` for older Python versions.

affects: 2.0.0+

gotchaBy default, `autocommit` is `False` for performance reasons. Forgetting to call `db.commit()` after modifications will result in unsaved data. Always ensure you commit changes or initialize with `autocommit=True`.

fix

Explicitly call `db.commit()` after a series of modifications, or initialize `SqliteDict` with `autocommit=True`. Using a `with` statement also ensures the database is closed, but doesn't implicitly commit if `autocommit` is `False`.

affects: All versions

gotchaWhile `sqlitedict` is marketed as 'multithread-safe', this primarily refers to a workaround for Python's `sqlite3` module's threading limitations, serializing requests internally. SQLite itself operates on a single-writer model. True concurrent *writes* from multiple processes or threads will still be serialized and can lead to performance bottlenecks or locking issues if not carefully managed (e.g., with WAL mode and proper error handling).

fix

For high write concurrency, consider alternative databases or ensure proper transaction management, retry mechanisms, and potentially enable SQLite's Write-Ahead Logging (WAL) mode. `sqlitedict` is not designed for concurrent high-volume writes across many distinct processes.

affects: All versions

gotchaImproperly closing an `SqliteDict` instance (e.g., not calling `db.close()` or not using a `with` statement) can lead to data not being saved (if `autocommit` is `False`) or resource leaks. Earlier versions also had reported `AttributeError` or `TypeError` upon closing.

fix

Always use the `with` statement when opening an `SqliteDict` instance: `with SqliteDict('path/to/db') as db: ...`. This ensures the database connection is properly closed.

affects: All versions

gotchaBy default, keys are expected to be strings. If using non-string keys (e.g., integers, tuples) or complex objects as keys, `sqlitedict` will serialize them. If deterministic serialization is critical for unordered containers or custom key types, you might need to provide custom `encode_key` and `decode_key` functions to `SqliteDict`.

fix

For non-string or complex keys, test thoroughly. If unexpected behavior occurs or custom serialization is desired, provide `encode_key` and `decode_key` functions as parameters to the `SqliteDict` constructor to handle key serialization/deserialization explicitly.

affects: All versions

Errors

Common errors & fixes

ModuleNotFoundError: No module named 'sqlitedict'

The 'sqlitedict' library has not been installed in the current Python environment.

fix

pip install sqlitedict

sqlite3.OperationalError: database is locked

Another process or thread is currently holding a lock on the SQLite database file, preventing the current operation from proceeding.

fix

Ensure that only one process or thread attempts to write to the database simultaneously, increase the `timeout` parameter when creating `SqliteDict`, or use separate database files for highly contended data.

sqlite3.OperationalError: unable to open database file

The specified path to the database file is invalid, the directory does not exist, or the process lacks the necessary file system permissions to create or access the file.

fix

Verify that the database file path is correct, ensure the directory exists, and check that the Python process has appropriate read and write permissions for the specified location.

TypeError: Object of type MyCustomClass is not JSON serializable

This error occurs when `encode=json.dumps` and `decode=json.loads` are explicitly used, and you attempt to store objects that JSON cannot natively serialize (e.g., custom classes, `datetime` objects) without providing a custom JSON encoder.

fix

Either remove the `encode` and `decode` parameters to use `sqlitedict`'s default (pickle-based) serialization, or provide custom `default` and `object_hook` functions to `json.dumps` and `json.loads` respectively to handle the serialization of your specific object types.

sqlite3.ProgrammingError: Cannot operate on a closed database.

An attempt was made to perform an operation (like setting or getting an item, or committing changes) on a `SqliteDict` instance after its `close()` method has been called, which severs the underlying SQLite connection.

fix

Ensure that `SqliteDict` instances are only closed when no further operations are needed, or if an instance must be used again after closing, create a new `SqliteDict` instance pointing to the same database file.

Upgrade

Version history

2.1.0latest on PyPI · released Dec 3, 2022

Audit

Dependencies

No dependency data recorded yet.

Agent activity

27 hits · last 30 days

node