معيار استبعاد الروبوتات

Key facts

Technology standard.caption

Example of a simple robots.txt file, indicating that a user-agent called "Mallorybot" is not allowed to crawl any of the website's pages, and that other user-agents cannot crawl more than one page every 20 seconds, and are not allowed to crawl the "secret" folder

Technology standard.class

skin-invert-image

Technology standard.status

Proposed Standard

Technology standard.first_published

1994 published, formally standardized in 2022

Technology standard.website

,

Technology standard.title

robots.txt

Technology standard.long_name

Robots Exclusion Protocol

Technology standard.image

Robots txt.svg

via Wikipedia infobox

Wikidata facts

Official website

www.robotstxt.org