Registry / utility / spatie-robots-txt

spatie-robots-txt

library2.5.4phppackagistunverified

Determine if a page may be crawled from robots.txt and robots meta tags.

composer require spatie/robots-txt

INSTALL

IMPORT

SIG · SPATIE-ROBOTS-TXT

spatie-robots-txt

utilityphpv2.5.4

harness data pending

Install & Compatibility

Where this runs

No compatibility data collected yet for this library.

Code

Verified usage

Verified import paths — ran on the pinned version, not inferred.

Robots

✓ use Spatie\Robots\Robots;

Check if a URL is allowed to be crawled.

use Spatie\Robots\Robots; $robots = Robots::create(); if ($robots->mayIndex('https://example.com/page')) { echo 'Page may be indexed'; }

Debug

Known issues

No known issues recorded.

Upgrade

Version history

2.5.4latest on Packagist

Audit

Dependencies

No dependency data recorded yet.

Agent activity

6 hits · last 30 days

node

Amazon

Resources