Yeti bot

What is Yeti?

Yeti is a web crawler operated by Naver Corporation (formerly NHN Corporation), South Korea's leading search engine company. It functions as the primary indexing bot for Naver, systematically visiting and analyzing web pages to include them in Naver's search results. Yeti was developed to help Naver maintain a comprehensive and up-to-date index of the web, with particular emphasis on Korean-language content.

Technically classified as a search engine crawler, Yeti works by following links between web pages, downloading content, and analyzing it for inclusion in Naver's search index. It identifies itself in server logs with user agent strings such as Yeti/1.0 (NHN Corp.; http://help.naver.com/robots/) or Yeti/1.0 (+http://help.naver.com/robots/) for older versions. More recent iterations may appear as Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Yeti/1.1; +https://naver.me/spd) Chrome/106.0.5249.0 Safari/537.36, showing its evolution to better handle modern web standards.

Yeti exhibits distinct behavioral patterns, including prioritization of text-heavy pages, special attention to Korean-language content, and varying crawl frequencies based on a site's relevance to Naver's primarily Korean user base. Its crawling operations are managed through a network of IP addresses primarily associated with Naver's infrastructure.

Why is Yeti crawling my site?

Yeti crawls websites to discover, analyze, and index content for inclusion in Naver's search results. If you're seeing Yeti in your logs, it means Naver has discovered your site and is evaluating its content for potential inclusion in their search index.

The crawler is particularly interested in text content, especially pages with Korean-language material or content relevant to Korean users. It also pays attention to structured data, meta information, and links between pages to understand your site's content and structure.

Crawl frequency varies significantly based on your site's perceived relevance to Naver's audience. High-authority Korean domains may receive daily visits, while international sites with little Korean-language content might see Yeti much less frequently. Changes to your site, especially the addition of new content or updates to existing pages, may trigger increased crawling activity.

Yeti's crawling is authorized and legitimate as part of normal search engine operations, similar to how Google's or Bing's crawlers work to maintain their search indexes.

What is the purpose of Yeti?

Yeti's primary purpose is to support Naver's search engine by discovering and indexing web content. As South Korea's dominant search platform, Naver relies on Yeti to build and maintain its search index, ensuring users receive relevant and up-to-date results.

The data collected by Yeti powers various Naver services beyond basic search, including Naver Blog, Naver Cafe (community forums), and Naver Maps. These interconnected services create a comprehensive ecosystem of information particularly valuable to Korean-speaking users.

For website owners, being indexed by Yeti can provide significant value through visibility to Naver's large user base, especially important for businesses targeting the Korean market. While Naver's market share is highest in South Korea, its influence extends to Korean-speaking communities worldwide.

The bot operates similarly to other legitimate search engine crawlers, focusing on content discovery rather than data scraping for competitive purposes.

How do I block Yeti?

Yeti respects the standard robots.txt protocol, allowing website administrators to control its access. To completely block Yeti from crawling your site, add the following directives to your robots.txt file:

User-agent: Yeti
Disallow: /

This instructs Yeti not to crawl any part of your website. If you want to block access to specific sections while allowing Yeti to crawl other areas, you can use more targeted directives:

User-agent: Yeti
Disallow: /private/
Disallow: /members-only/
Allow: /

Naver also offers a verification tool through their Naver Search Advisor platform, where webmasters can register their sites and gain additional control over how Yeti interacts with their content.

Blocking Yeti will remove your site from Naver's search results, which could significantly reduce your visibility to Korean audiences. Consider this impact before implementing complete blocking, especially if South Korea represents an important market for your website. For most sites, allowing Yeti to crawl public content while using robots.txt to protect private areas offers the best balance between search visibility and resource management.