Build a robots.txt file with allow and disallow rules, presets for Googlebot, Bingbot and AI crawlers, and your sitemap URLs. Everything runs in your browser and nothing is uploaded.
Google ignores crawl-delay; Bing and a few others honour it.
No extra rules yet. Add a crawler above.
robots.txt
User-agent: * Disallow: /admin/ Disallow: /cart/ Sitemap: https://www.example.com/sitemap.xml
Upload this file to the root of your domain (example.com/robots.txt). Robots.txt controls crawling, not indexing; use noindex to keep a page out of results.
Pages fixed and ready? Send Googlebot to them in minutes — credit back if it doesn't visit.
Submit URLs free