Install & Compatibility
Where this runs
tested against v1.6.16 · pip install
no network on importno background threads
Install × environment matrix
Each cell = how many times install + import succeeded across repeated harness runs. Partial = flaky.
glibc = Debian/Ubuntu slim · musl = Alpine Linux
py 3.10
✕ timeout
✕ timeout
py 3.13
✕ build_error
✕ build_error
266MB installed
● package 266MB
Code
Verified usage
Verified import paths — ran on the pinned version, not inferred.
NewsPlease
✓ from newsplease import NewsPlease
This quickstart demonstrates how to extract structured information from a single news article URL using the `NewsPlease.from_url()` method. It prints the article's title, authors, publication date, and an excerpt of the main text.
from newsplease import NewsPlease
url = 'https://www.theguardian.com/world/2023/jan/01/ukraine-war-russia-new-year-attacks'
article = NewsPlease.from_url(url)
if article:
print(f"Title: {article.title}")
print(f"Authors: {', '.join(article.authors)}")
print(f"Publish Date: {article.date_publish}")
print(f"Main Text (excerpt): {article.maintext[:200]}...")
else:
print(f"Failed to extract article from {url}")
news-please --version
Debug
Known issues
gotchaWindows users may encounter issues installing direct dependencies like 'lxml' and 'pywin32' via pip, requiring manual installation of pre-compiled wheels.fixFor 'lxml', download a compatible wheel from Christoph Gohlke's Python page (unofficial but common source) and `pip install` it. For 'pywin32', download and run its installer.
affects: All versions on Windows
gotchaUsing the default User-Agent string can lead to aggressive crawling being blocked by many news websites.fixConfigure a custom and less generic `USER_AGENT` in the `config.cfg` file (default location `~/news-please/config`) to improve crawl success rates. For example, `USER_AGENT = 'news-please (+http://www.example.com)'`.
affects: All versions
gotchaThere's a distinction between CLI mode (for full website crawling or continuous RSS feeds) and library mode (for extracting individual URLs). Attempting full crawls directly through the library API might not yield expected results without proper setup.fixFor comprehensive website crawling or RSS-based continuous crawling, use the command-line interface. For individual article extraction from known URLs, use the library functions like `NewsPlease.from_url()` or `NewsPlease.from_html()`.
affects: All versions
gotchaWhen running news-please in CLI mode, pressing `CTRL+C` multiple times to terminate the process is not recommended and can lead to data inconsistencies. It's best to allow for a graceful shutdown.fixPress `CTRL+C` once and wait for news-please to shut down gracefully (typically 5-60 seconds). Only press `CTRL+C` twice for an immediate, forceful kill if absolutely necessary.
affects: All versions
Errors
Common errors & fixes
ImportError: cannot import name 'NewsPlease' from 'newsplease'
This typically occurs if Python cannot find the `NewsPlease` class within the `newsplease` package. Common reasons include a typo in the import statement, an improperly installed package, or a local file named `newsplease.py` shadowing the installed library.
fixEnsure `news-please` is correctly installed (`pip install news-please`). Verify your import statement is `from newsplease import NewsPlease`. Check that there isn't a Python file named `newsplease.py` or a folder named `newsplease` in your current working directory that might conflict with the installed package.
Failed to extract article from URL / Article object is empty or missing expected fields.
Web scraping is inherently fragile. Website layouts change frequently, and anti-scraping measures can prevent successful extraction. This means `news-please` might not always successfully parse an article or might return an incomplete object.
fixInspect the `article` object for `None` values or missing attributes. Consider trying a different URL or checking if the target website has implemented new anti-bot measures. For very dynamic sites, additional pre-processing or custom extraction logic might be required outside of `news-please`.
requests.exceptions.ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) or similar network errors.
This error often indicates that the target server actively rejected or closed the connection. This can be due to aggressive request rates, a blocked User-Agent, or IP-based blocking by the website's security systems.
fixImplement a custom, less identifiable `USER_AGENT` in your configuration (`config.cfg`). Consider adding delays between requests (rate limiting) to avoid overwhelming the server. If persistent, you might be rate-limited or IP-blocked, requiring a proxy or waiting period.
Upgrade
Version history
1.6.16latest on PyPI · released Sep 21, 2025
Audit
Dependencies
ScrapyrequiredCore web crawling framework.
Newspaper3krequiredArticle extraction and content analysis.
lxmlrequiredFor parsing HTML and XML documents, with specific installation considerations on Windows.
requestsrequiredFor downloading web pages.
tldextractrequiredFor extracting top-level domain information.
PyMySQLoptionalOptional database backend for results.
psycopg2-binaryoptionalOptional database backend for results.
elasticsearchoptionalOptional backend for storing and versioning extracted data.