YandexVideo bot

What is YandexVideo?

YandexVideo is a specialized web crawler operated by Yandex, Russia's largest search engine company. This bot is specifically designed to discover, analyze, and index video content across the internet. As part of Yandex's suite of specialized crawlers, YandexVideo focuses exclusively on video files, embedded players, and related multimedia elements.

The bot identifies itself in server logs with the user agent string: Mozilla/5.0 (compatible; YandexVideo/3.0; +http://yandex.com/bots). This follows standard user agent syntax, including the product name, version number, and a link to Yandex's bot documentation page where webmasters can find more information.

Unlike Yandex's general-purpose crawler (YandexBot), YandexVideo is programmed to prioritize video metadata, including duration, resolution, and caption information. It exhibits specific behavioral patterns, including dynamic crawling schedules that favor sites with frequently updated video content and higher engagement metrics. The bot typically modifies HTTP headers to simulate browser environments when interacting with video content.

Why is YandexVideo crawling my site?

If YandexVideo is visiting your website, it's primarily looking for video content to include in Yandex's video search results. The crawler is particularly interested in sites that:

YandexVideo is looking for properly marked-up video content using standard HTML5 video tags, Schema.org VideoObject markup, or Open Graph video tags. It pays special attention to video attributes like thumbnails, durations, and descriptions.

The frequency of visits depends on how often you update your video content and its popularity with Yandex users. Sites with regularly refreshed video libraries or high traffic from Russian-speaking regions may experience more frequent crawling. The bot operates on a dynamic schedule, prioritizing content that receives engagement from Yandex users.

This is authorized crawling behavior, as Yandex is a legitimate search engine attempting to improve its video search capabilities for users.

What is the purpose of YandexVideo?

YandexVideo exists to power Yandex's video search functionality, enabling users to discover relevant video content across the web. By specializing in video content rather than using a general-purpose crawler, Yandex can more efficiently extract and process video-specific data.

The data collected by YandexVideo helps Yandex build a comprehensive index of videos available online, particularly those relevant to Russian-speaking audiences. This specialized approach allows Yandex to offer more accurate video search results, including proper thumbnails, durations, and other metadata that enhances the user experience.

For website owners, having your videos properly indexed by YandexVideo can drive additional traffic from Yandex's video search platform. This is especially valuable for content creators targeting audiences in Russia and other CIS countries where Yandex maintains significant market share.

How do I block YandexVideo?

YandexVideo respects the standard robots.txt protocol, making it straightforward to control its access to your site. If you wish to block this crawler from accessing all video content on your site, add the following directives to your robots.txt file:

User-agent: YandexVideo
Disallow: /

If you only want to block access to specific directories containing videos:

User-agent: YandexVideo
Disallow: /videos/
Disallow: /media/restricted/

You can also allow YandexVideo to crawl most of your site while restricting access to specific files or directories:

User-agent: YandexVideo
Allow: /
Disallow: /private-videos/

Keep in mind that blocking YandexVideo will prevent your video content from appearing in Yandex's video search results. This could reduce visibility and traffic, particularly from Russian-speaking audiences. If you're concerned about bandwidth usage rather than indexing, consider implementing proper cache control headers for your video assets instead of blocking the crawler entirely.

For more granular control, you can also use the noindex meta tag or HTTP header on specific pages containing videos you don't want indexed, while still allowing the crawler to access your site.