Registry / database / sqlite-fts4

sqlite-fts4

JSON →
library1.0.3pypypi✓ verified 22d ago

sqlite-fts4 is a Python library providing custom SQLite functions designed for efficient full-text search (FTS4) ranking and decoding. It offers functions like `rank_score`, `rank_bm25`, `decode_matchinfo`, and `annotate_matchinfo` to enhance the utility of SQLite's built-in FTS4 extension. The library, currently at version 1.0.3, is actively maintained with a focus on stability and compatibility, addressing specific issues such as big-endian system support.

pip install sqlite-fts4
INSTALL
IMPORT
SIG · SQLITE-FTS4
S
sqlite-fts4
databasepythonv1.0.3
Install
1.5s avg
Import
12ms
Disk
16MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v1.0.3 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.012s · 17.8MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 1.5s · import 0.010s · 18MB
16MB installed
● package 16MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

register_functions
from sqlite_fts4 import register_functions
Registers all custom FTS4 functions with an SQLite connection.
rank_score
from sqlite_fts4 import rank_score
Imports only the rank_score function for selective registration.

This quickstart demonstrates how to establish an SQLite connection, register the `sqlite-fts4` custom functions, create an FTS4 virtual table, insert data, and then perform a full-text search with relevance ranking using `rank_score`.

import sqlite3 from sqlite_fts4 import register_functions # Connect to an in-memory SQLite database conn = sqlite3.connect(':memory:') # Register all custom FTS4 functions register_functions(conn) # Create an FTS4 virtual table conn.execute("CREATE VIRTUAL TABLE docs USING fts4(title, body);") # Insert some data conn.execute("INSERT INTO docs (title, body) VALUES (?, ?)", ("Hello World", "This is a test document with some words.")) conn.execute("INSERT INTO docs (title, body) VALUES (?, ?)", ("Python SQLite FTS4", "Exploring full-text search capabilities in Python with SQLite FTS4.")) # Perform a search and rank results using rank_score # 'pcx' is required for rank_score() cursor = conn.execute( "SELECT title, body, rank_score(matchinfo(docs, 'pcx')) as score FROM docs WHERE docs MATCH ? ORDER BY score DESC", ("python search",) ) for row in cursor.fetchall(): print(f"Title: {row[0]}, Score: {row[2]:.2f}") conn.close()
Debug
Known issues
gotchaThe `rank_score()` and `rank_bm25()` functions require specific `matchinfo` format strings ('pcx' for `rank_score`, 'pcnalx' for `rank_bm25`) to return correct results. Using an incorrect format string can lead to inaccurate scores or math domain errors.
fix
Always pass the exact required `matchinfo` format string (e.g., `matchinfo(table, 'pcx')`) to `rank_score()` or `rank_bm25()`.
affects: All versions
gotchaSQLite FTS query syntax can be complex and easily lead to errors if user-provided search strings are not carefully handled. Exposing raw FTS operators to users without validation or a custom query language can result in unexpected behavior or exceptions.
fix
Sanitize user input for FTS queries, consider disabling advanced FTS operators for untrusted input, or implement a custom parser to build safe FTS queries from user-friendly input.
affects: All versions
gotchaPrior to version 0.5.2, calling `matchinfo()` without a `MATCH` clause in the query would raise an error. From version 0.5.2 onwards, this scenario now fails silently, which may alter behavior for applications relying on the previous error.
fix
Ensure `MATCH` clauses are always present when `matchinfo()` is intended to be used with meaningful data, or explicitly handle the silently failing case for empty `matchinfo` buffers.
affects: <0.5.2
gotchaThe Python `sqlite3` module must be compiled with FTS4 support in the underlying SQLite library for `sqlite-fts4` to function. If your Python environment's SQLite is not compiled with FTS4, attempts to create FTS4 tables will result in a 'no such module: fts4' error.
fix
If encountering this error, ensure your Python installation uses an SQLite library compiled with FTS4. This may involve installing Python/SQLite from source or using a distribution that provides FTS4-enabled `sqlite3`.
affects: All versions
Errors
Common errors & fixes
sqlite3.OperationalError: no such function: rank_bm25
The sqlite-fts4 extension was not successfully loaded into the SQLite database connection before attempting to use its custom FTS4 functions.
fix
Ensure the extension is loaded using `conn.load_extension(sqlite_fts4.loadable_path())` on the `sqlite3.Connection` object after importing `sqlite_fts4`.
ModuleNotFoundError: No module named 'sqlite_fts4'
The `sqlite-fts4` Python package has not been installed in the current Python environment.
fix
Install the library using pip: `pip install sqlite-fts4`.
ImportError: cannot import name 'rank_bm25' from 'sqlite_fts4'
Users mistakenly attempt to import custom FTS4 functions (like `rank_bm25`) directly from the `sqlite_fts4` Python package, instead of loading them as an SQLite extension for use in SQL queries.
fix
The functions are exposed directly within SQL queries after loading the extension. Simply import `sqlite_fts4` to get the loadable path, then load it into your SQLite connection: `import sqlite_fts4; conn.load_extension(sqlite_fts4.loadable_path())`.
sqlite3.OperationalError: unable to open shared library "/path/to/sqlite_fts4.so"
SQLite cannot find or load the extension file at the specified path, possibly due to an incorrect path, insufficient file permissions, or the file being missing/corrupted.
fix
Verify that `sqlite_fts4.loadable_path()` returns a valid and accessible path. Check file permissions for the extension file and its containing directory, and ensure the `sqlite-fts4` package was installed correctly.
Upgrade
Version history
1.0.3latest on PyPI · released Jul 30, 2022
Audit
Dependencies

No dependency data recorded yet.

Agent activity
15 hits · last 30 days
node
12
OpenAI (training)
1
Resources