Registry / http-networking / MechanicalSoup

MechanicalSoup

library1.4.0pypypi✓ verified 25d ago

Python library for automating website interaction — form submission, cookie handling, link following. Built on requests + BeautifulSoup. Current version is 1.4.0 (2023). Install name is MechanicalSoup (capital M and S), import is mechanicalsoup (all lowercase). Does NOT execute JavaScript — use Playwright or Selenium for JS-rendered pages. Maintenance status: low activity since 2023.

http-networking web-framework testing

pip install MechanicalSoup

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · MECHANICALSOUP

MechanicalSoup

http-networkingpythonv1.4.0

Install

3.3s avg

Import

551ms

Disk

33MB

Pass rate

10/ 10

Env Coverage10 / 10

glibc

3.9–3.13

musl

3.9–3.13

Install & Compatibility

Where this runs

tested against v1.4.0 · pip install

no network on importno background threads

Install × environment matrix

Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux

musl

py 3.10–3.95 runs

installs and imports cleanly · install 0.0s · import 0.566s · 34.5MB

glibc

py 3.10–3.95 runs

installs and imports cleanly · install 3.3s · import 0.536s · 35MB

33MB installed

● package 33MB

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

StatefulBrowser

✓ from mechanicalsoup import StatefulBrowser

✗ import mechanicalsoup

StatefulBrowser for form-based workflows. Requires lxml or html.parser.

import mechanicalsoup # StatefulBrowser is the high-level interface browser = mechanicalsoup.StatefulBrowser( soup_config={'features': 'lxml'}, raise_on_404=True ) # Navigate to a page browser.open('https://httpbin.org/forms/post') # Select and fill a form browser.select_form('form') # or CSS selector like '#login-form' browser['custname'] = 'Alice' browser['custtel'] = '555-1234' browser['comments'] = 'Hello!' # Submit the form response = browser.submit_selected() print(response.status_code) # Access the resulting page print(browser.page.title.string)

Debug

Known issues

breakingMechanicalSoup does NOT execute JavaScript. Pages that render content via JS (React, Vue, Angular SPAs) will appear empty or broken. This is the most common misuse.

fix

For JavaScript-rendered pages use Playwright (pip install playwright + playwright install) or Selenium instead.

affects: all

gotchaInstall name and import name differ in capitalization. pip install MechanicalSoup (capital M and S), but import mechanicalsoup (all lowercase). import MechanicalSoup raises ModuleNotFoundError.

fix

pip install MechanicalSoup then import mechanicalsoup

affects: all

gotcharaise_on_404=False by default — 404 responses are silently treated as successful. Code that navigates to a missing page gets no error and browser.page will contain the 404 HTML.

fix

Always pass raise_on_404=True: StatefulBrowser(raise_on_404=True). This raises LinkNotFoundError on 404 responses.

affects: all

gotchaBrowser (stateless) and StatefulBrowser (stateful) have different APIs. Browser.get() returns a response. StatefulBrowser.open() returns a response AND updates internal state. Mixing them causes confusion.

fix

Use StatefulBrowser for multi-page workflows (login flows, form sequences). Use Browser only for simple one-shot requests.

affects: all

gotchaselect_form() with no argument selects the first form on the page. On pages with multiple forms (login + newsletter), this may select the wrong form.

fix

Always pass a CSS selector: browser.select_form('#login-form'). Or use the nr= argument to select by index: browser.select_form('form', nr=1).

affects: all

Upgrade

Version history

1.4.0latest on PyPI · released May 30, 2025

Audit

Dependencies

requests>=2.22.0requiredRequired. Installed automatically.

beautifulsoup4>=4.7requiredRequired. Installed automatically.

lxmloptionalOptional but recommended parser for BeautifulSoup.

Agent activity

67 hits · last 30 days

node

OpenAI (training)

Resources

githubhttps://github.com/MechanicalSoup/MechanicalSoup ↗

packagehttps://pypi.org/project/MechanicalSoup/ ↗

homepagehttps://mechanicalsoup.readthedocs.io/ ↗