pi-web-fetch

library1.1.0jsnpmunverified

pi-web-fetch is a pi coding agent extension that gives AI agents a proper web browser via headless Chrome, clean content extraction with trafilatura, and optional LLM processing. Version 1.1.0 brings concurrent batch fetching (up to 10 URLs with 6 parallel tabs), a 15-minute cache, smart large-page summarization, cross-host redirect detection, and an extensible hook system for site-specific handling. Unlike static fetchers like Claude Code's WebFetch, it handles JavaScript-rendered SPAs, strips boilerplate using ML-based extraction, and supports optional LLM distillation via a pi sub-agent. Requires Node.js 18+, a Python tool runner (uvx recommended), and puppeteer's bundled Chromium.

devops

npm install pi-web-fetch

github ↗package ↗homepage ↗

INSTALL

IMPORT

SIG · PI-WEB-FETCH

pi-web-fetch

devopsjavascriptv1.1.0

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

default

✓ import piWebFetch from 'pi-web-fetch'

✗ const piWebFetch = require('pi-web-fetch')

Package is ESM-only; CJS require() will not work.

WebFetchExtension

✓ import { WebFetchExtension } from 'pi-web-fetch'

✗ import WebFetchExtension from 'pi-web-fetch'

Named export, not default. Use named import syntax.

type WebFetchToolOptions

✓ import type { WebFetchToolOptions } from 'pi-web-fetch'

✗ import { WebFetchToolOptions } from 'pi-web-fetch'

TypeScript type; import with 'import type' to avoid runtime errors.

Demonstrates importing and initializing the pi-web-fetch extension with a pi agent, including example tool calls for single and batch URL fetching.

import { WebFetchExtension } from 'pi-web-fetch'; import { PiAgent } from '@mariozechner/pi-coding-agent'; const agent = new PiAgent({ extensions: [new WebFetchExtension()] }); // The agent can now call web_fetch tool // Example: web_fetch({ urls: ['https://example.com'], prompt: 'What is the main topic?' }); // Single URL fetch without LLM: // web_fetch({ urls: ['https://example.com'] }); // Batch fetch (up to 10 URLs): // web_fetch({ urls: ['https://example.com', 'https://example.org'] });

Debug

Known issues

gotchaFirst run triggers puppeteer Chromium download (~300MB) and trafilatura via uvx (~10MB).

fix

Set PUPPETEER_EXECUTABLE_PATH to existing Chrome binary to skip Chromium download.

affects: >=1.0.0

deprecateduvx alias may not exist on all systems; fallback to uv run may be slower.

fix

Ensure uv is installed and uvx alias is available, or rely on pipx fallback.

affects: >=1.0.0

gotchaNode.js 18+ is required; older versions may fail due to puppeteer compatibility.

fix

Update Node.js to version 18 or later.

affects: >=1.0.0

breakingESM-only package: CJS require() will throw an error.

fix

Use import syntax or convert project to ESM.

affects: >=1.0.0

gotchaMaximum 10 URLs per batch fetch; exceeding limit will cause tool failure.

fix

Split fetches into batches of 10 or fewer URLs.

affects: >=1.0.0

Errors

Common errors & fixes

Error: Cannot find module 'pi-web-fetch'

Package not installed or not in node_modules.

fix

Run 'npm install pi-web-fetch' in your project directory.

TypeError: Class constructor WebFetchExtension cannot be invoked without 'new'

Attempted to call WebFetchExtension as a function instead of using 'new'.

fix

Correct: 'new WebFetchExtension()'

Error: The 'web_fetch' tool is not registered. Available tools: ...

The extension was not loaded correctly or agent does not have the extension registered.

fix

Ensure the extension object is passed to PiAgent constructor in 'extensions' array.

Error: connect ECONNREFUSED ::1:9222

Chrome remote debugging port is not accessible. Puppeteer cannot start Chrome.

fix

Check that Chrome is installed and puppeteer can launch it. Set PUPPETEER_EXECUTABLE_PATH if needed.

Upgrade

Version history

1.1.0latest on npm

Audit

Dependencies

@mariozechner/pi-coding-agentrequiredProvides the pi coding agent runtime and tool registration system

@mariozechner/pi-tuirequiredProvides terminal UI components for progress display

@sinclair/typeboxrequiredUsed for tool schema validation

puppeteerrequiredDownloads and manages headless Chrome for JavaScript rendering

Agent activity

7 hits · last 30 days

node

Resources

githubhttps://github.com/georgebashi/pi-web-fetch ↗

packagepi-web-fetch ↗

homepagehttps://github.com/georgebashi/pi-web-fetch#readme ↗