Registry / serialization / urlextract

urlextract

JSON →
library1.9.0pypypi✓ verified 23d ago

URLExtract is a Python library for collecting and extracting URLs from a given text based on locating Top-Level Domains (TLDs). It is currently at version 1.9.0 and is actively maintained, with regular updates to its TLD list and ongoing Python version compatibility.

pip install urlextract
INSTALL
IMPORT
SIG · URLEXTRACT
U
urlextract
serializationpythonv1.9.0
Install
1.9s avg
Import
367ms
Disk
18MB
Pass rate
10/ 10
Env Coverage10 / 10
glibc
3.93.13
musl
3.93.13
Install & Compatibility
Where this runs
tested against v1.9.0 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
musl
py 3.103.95 runs
installs and imports cleanly · install 0.0s · import 0.390s · 19.6MB
glibc
py 3.103.95 runs
installs and imports cleanly · install 1.9s · import 0.344s · 20MB
18MB installed
● package 18MB
Code
Verified usage

Verified import paths — ran on the pinned version, not inferred.

URLExtract
from urlextract import URLExtract

Initializes the URLExtract class and uses the `find_urls` method to extract all URLs from a given text string.

from urlextract import URLExtract extractor = URLExtract() text = "Check out our website: example.com or find us at https://www.another-example.org/path?query=1" urls = extractor.find_urls(text) print(urls) # Expected output: ['example.com', 'https://www.another-example.org/path?query=1']
Debug
Known issues
gotchaURLExtract's TLD-based detection can lead to 'false matches' in certain contexts, such as CSS class names (e.g., `p.bold.name` might be extracted if `.name` is a valid TLD). The library correctly identifies these as valid patterns, but they might not be the intended URLs.
fix
Review extracted URLs in contexts where non-URL patterns might coincidentally contain TLDs. Consider using the `with_schema_only=True` parameter in `find_urls` if you only need URLs with explicit schemes (e.g., 'http://', 'https://').
affects: <=1.9.0
gotchaUsers have reported `urlextract.cachefile.CacheFileError` or issues with custom cache directories not saving TLDs, especially in bundled applications (like PyInstaller) or read-only file systems.
fix
Ensure the application has write permissions to the default cache directory or a custom directory specified during `URLExtract` initialization. Manually update the TLD list using `extractor.update()` if necessary. Consider reporting the issue on the GitHub repository for specific edge cases.
affects: All versions
breakingSupport for Python 3.6 has been dropped in recent versions due to underlying dependency changes (e.g., `filelock`). Users on Python 3.6 will encounter errors.
fix
Upgrade to Python 3.7 or newer. Python 3.12 support was added in v1.9.0.
affects: >=1.5.0 (approx), explicitly in recent versions
gotchaOlder versions (prior to 1.9.0) might incorrectly parse URLs within Markdown links or have issues with filtering mixed-case hostnames, leading to incomplete or incorrect extractions.
fix
Upgrade to version 1.9.0 or later to benefit from fixes for Markdown link parsing and mixed-case hostname filtering.
affects: <1.9.0
Errors
Common errors & fixes
ModuleNotFoundError: No module named 'urlextract'
The urlextract library is not installed in the Python environment, or the import statement is incorrect.
fix
Install the library using pip, then ensure it's imported correctly.
```python
pip install urlextract

# In your Python code
from urlextract import URLExtract
```
AttributeError: 'URLExtract' object has no attribute 'extract_urls'
You are trying to use an outdated method name `extract_urls`, which was renamed to `find_urls` in urlextract version 0.10.0.
fix
Replace all occurrences of `extract_urls` with `find_urls`.
```python
extractor = URLExtract()
urls = extractor.find_urls("text with a url.com and another.net")
```
TypeError: expected string or bytes-like object
The `find_urls` method (or other text processing methods) expects a string or bytes-like object as input, but received a different data type.
fix
Convert the input data to a string before passing it to the `urlextract` method.
```python
extractor = URLExtract()
# Correcting integer input
urls = extractor.find_urls(str(1234567890))

# Correcting list input (example)
text_parts = ["visit example.com", "or check out test.org"]
urls_from_list = extractor.find_urls(" ".join(text_parts))
```
requests.exceptions.ConnectionError
The `URLExtract().update()` method failed to download the latest TLD list due to network connectivity issues, DNS problems, or firewall restrictions preventing access to the TLD source.
fix
Verify your internet connection, check firewall settings, or try running the `update()` method again later when network conditions are stable.
```python
import requests
from urlextract import URLExtract

extractor = URLExtract()
try:
    extractor.update()
    print("TLD list updated successfully.")
except requests.exceptions.ConnectionError as e:
    print(f"Failed to update TLDs due to connection error: {e}")
    print("Please check your internet connection and firewall settings.")
except Exception as e:
    print(f"An unexpected error occurred during TLD update: {e}")
```
Upgrade
Version history
1.9.0latest on PyPI · released Feb 29, 2024
Audit
Dependencies
idnarequiredRequired for converting links to IDNA format.
uritoolsrequiredRequired for domain name validation.
platformdirsrequiredRequired for determining the user's cache directory.
dnspythonrequiredRequired for caching DNS results when DNS checks are enabled.
Agent activity
10 hits · last 30 days
node
8
OpenAI (training)
1
Resources
urlextract — pip install urlextract · libregistry