If you target audiences in Russia, Kazakhstan, Belarus, Turkey or other markets where Yandex has a large share, being indexed there isn't optional. Yandex works differently from Google in several ways, and many sites that do well on Google are only partly indexed on Yandex simply because nobody set things up properly.
Here's how to get it right.
Step 1: Add your site to Yandex Webmaster
Yandex Webmaster is Yandex's equivalent of Google Search Console. Add your site and verify ownership with an HTML file, a meta tag or a DNS record. Once verified, you get:
- Indexing reports. Which pages are in the search index, which were excluded, and why.
- Reindex pages. A tool for submitting specific URLs for re-crawling, up to a daily limit shown in your account.
- Sitemap files. Submit and monitor your sitemaps.
- Crawl statistics. What YandexBot fetched and the response codes it got.
- Site diagnostics. Alerts about critical and potential problems.
- Regional settings. Tell Yandex which region your site is relevant to.
Step 2: Submit your sitemap
Add your XML sitemap in Yandex Webmaster under the indexing section, and reference it in robots.txt:
Sitemap: https://www.example.com/sitemap.xmlYandex follows the standard sitemap protocol (up to 50,000 URLs or 50 MB uncompressed per file, with sitemap index files for larger sites). Keep it to canonical, indexable URLs. Run it through the sitemap validator before you submit, because Yandex reports errors in the sitemap but won't fix them for you.
Step 3: Use IndexNow
Yandex co-launched IndexNow with Microsoft Bing in 2021 and supports it fully. Pinging IndexNow when you publish or update a page is the quickest way to let Yandex know, and the same submission is shared with Bing and the other participating engines (Seznam, Naver and Yep).
Yandex Webmaster shows IndexNow submissions, so you can confirm your pings are arriving. If you're comparing push options, read Google Indexing API vs IndexNow. The short answer is that IndexNow covers Yandex and Google's API doesn't.
Step 4: Get robots.txt right for Yandex
Yandex reads robots.txt with a few differences from Google worth knowing about:
- Clean-param. A Yandex-specific directive that tells YandexBot to ignore certain URL parameters, such as tracking or session IDs, so it doesn't crawl endless duplicates. For example:
User-agent: Yandex
Clean-param: utm_source&utm_medium&utm_campaign- Crawl-delay. Yandex no longer follows
Crawl-delay. Use the crawl speed setting in Yandex Webmaster instead. - Host. The old
Hostdirective for choosing the main mirror is deprecated. Use 301 redirects to your preferred domain version. - User-agent groups. Yandex obeys groups for
Yandex(all its robots) and specific bots such asYandexBot. Make sure a group meant for one crawler doesn't block another by mistake.
Use the robots.txt analysis tool in Yandex Webmaster, or our robots.txt tester, to check specific paths.
Step 5: Set your region
Yandex puts a lot of weight on regional relevance. In Yandex Webmaster you can set the region the site is relevant to. For local businesses, this, along with a consistent address and contact details on the site, helps Yandex show your pages to the right audience. Sites aimed at several countries can use hreflang, which Yandex supports.
Step 6: Use the Reindex tool for priority pages
For pages you've just published or substantially changed, submit them through Reindex pages in Yandex Webmaster. The daily limit is modest, so save it for the URLs that matter most. Use IndexNow for everything else.
Why Yandex excludes pages
Yandex Webmaster's indexing report lists excluded pages with a reason. The common ones:
- Duplicate. Yandex found another page with the same content and chose that one. Fix it with canonicals, 301 redirects or Clean-param for parameter duplicates.
- Low-value or low-demand page. Yandex decided the page isn't likely to be useful to searchers. This is the Yandex equivalent of Google's "Crawled – currently not indexed", and the fix is the same: make the page more substantial and distinct. Our guide to why Google is not indexing your page covers this kind of quality work.
- Prohibited by robots.txt or noindex. Check that it's intentional.
- Redirect or non-canonical. The page redirects or canonicalises elsewhere.
- HTTP error. The page returned a 4xx or 5xx code when YandexBot visited.
How long Yandex indexing takes
Yandex doesn't publish a fixed timeline, and it doesn't promise that any page will be indexed. On a verified site with a clean sitemap and IndexNow set up, new pages are often crawled soon after they're published. Whether they then appear in search depends on the page itself. Yandex updates its search index in batches, so a page can be crawled and still take a little while to show up. Check the indexing report in Yandex Webmaster rather than assuming the worst after a day or two.
Speeding up discovery for pages you don't own
The Reindex tool and IndexNow only cover verified sites. For backlinks and mentions on other domains, use the same discovery tactics as for any engine: links from crawled pages, genuine sharing, and indexing services that create crawl paths and record YandexBot visits.
Yandex indexing checklist
- Verify your site in Yandex Webmaster
- Submit your sitemap and reference it in robots.txt
- Set up IndexNow
- Add Clean-param rules for tracking and session parameters
- Set crawl speed in Webmaster rather than with Crawl-delay
- Set your site's region
- Use Reindex pages for priority URLs
- Review excluded pages regularly and fix the causes
Index on Yandex with QuickIndexx
QuickIndexx sends URLs to Yandex through IndexNow and its discovery channels, and logs each YandexBot visit. If a URL gets no crawl within 24 hours, the credit goes back to your balance. As always, Yandex decides what gets indexed. We make sure it gets the chance to look. Start on the Yandex indexer page or see pricing.