extruct

library0.18.0pypypi✓ verified 35d ago

Extruct is a Python library for extracting embedded metadata from HTML markup. It currently supports W3C's HTML Microdata, embedded JSON-LD, Microformat (via mf2py), Facebook's Open Graph, experimental RDFa (via rdflib), and Dublin Core Metadata (DC-HTML-2003). The library is actively maintained with its current stable version being 0.18.0.

http-networking data serialization

github ↗package ↗

Install & Compatibility

Where this runs

tested against v0.18.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.940 runs

installs and imports cleanly · install 0.0s · import 1.564s · 44.1MB

glibc

py 3.10–3.940 runs

installs and imports cleanly · install 4.8s · import 1.435s · 45MB

42MB installed

● package 42MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

The primary function for all-in-one metadata extraction.

from extruct import extract

Commonly imported for resolving relative URLs in extracted metadata.

from w3lib.html import get_base_url

Example of importing a specific extractor if only certain formats are needed.

from extruct.opengraph import OpenGraphExtractor

This quickstart fetches HTML content from a URL, determines the base URL for resolving relative paths, and then uses `extruct.extract` to retrieve structured metadata in common formats (JSON-LD, Microdata, Open Graph). The `uniform=True` parameter ensures a consistent output structure for easier processing.

import extruct import requests from w3lib.html import get_base_url import pprint pp = pprint.PrettyPrinter(indent=2) # Replace with a real URL to test url = 'http://quotes.toscrape.com/scroll' r = requests.get(url) base_url = get_base_url(r.text, r.url) data = extruct.extract(r.text, base_url=base_url, uniform=True, syntaxes=['json-ld', 'microdata', 'opengraph']) pp.pprint(data)

extruct --version

Debug

Known footguns

breakingVersions of `extruct` prior to 0.18.0 might encounter `ImportError: cannot import name '_ElementStringResult' from 'lxml.etree'` when used with `lxml` versions 5.1.0 or higher due to internal API changes in `lxml`.

gotchaThe output structure of `extruct` can be inconsistent for certain metadata types, sometimes returning a list of dictionaries and other times a single dictionary, which can lead to `TypeError` or `IndexError` if not handled carefully in post-processing.

gotchaExtracting all supported syntaxes from very large or complex HTML documents can be memory-intensive and slow. By default, `extruct.extract()` attempts all formats.

gotchaThe command-line tool `extruct` (e.g., `extruct 'http://example.com'`) requires the `requests` library, which is an optional dependency and not installed by default with a basic `pip install extruct`.

Upgrade

Version history

Breaking-change detection hasn't run for this library yet.

Audit

Security & dependencies

CVE tracking and dependency tree are planned for a later release.

Agent activity

19 hits · last 30 days

bytedance

gptbot

ahrefsbot

script

Resources

githubhttps://github.com/scrapinghub/extruct ↗

packagehttps://pypi.org/project/extruct/ ↗