Qwantbot

What is Qwantbot?

Qwantbot is the web crawler operated by Qwant, a French technology company that runs the Qwant search engine. The bot functions as a search engine crawler that systematically browses the web to discover, analyze, and index web content for inclusion in Qwant's search results. Qwantbot has been active since at least 2014, with various versions and improvements released over time.

In server logs, Qwantbot typically identifies itself with user-agent strings following patterns like Mozilla/5.0 (compatible; Qwantbot/1.0; +https://help.qwant.com/bot/) or variations such as Mozilla/5.0 (compatible; Qwantbot-prod/1.0; +Qwantbot@qwant.com). Earlier versions used the name "Qwantify" in their user-agent strings.

The crawler operates from a range of IP addresses primarily located in France, with most originating from the 91.242.162.x range. Qwantbot follows standard crawler behavior by requesting and processing web pages, following links to discover new content, and sending the information back to Qwant's indexing systems. The bot is designed to respect standard web protocols including robots.txt directives.

Why is Qwantbot crawling my site?

Qwantbot crawls websites to discover and index content that will appear in Qwant search results. The primary goal is to make your site's content findable by Qwant users. The crawler is particularly interested in text content, links, metadata, and other elements that help determine relevance and quality for search results.

The frequency of Qwantbot visits depends on several factors, including your site's popularity, how often your content changes, and its relevance to Qwant users. More established sites with frequently updated content may receive more regular visits. New content or changes to existing pages often trigger crawling activity as the bot works to keep Qwant's index current.

Qwantbot's crawling is considered authorized standard behavior for search engines. As a legitimate search engine crawler, it helps make your content discoverable to potential visitors using the Qwant search service.

What is the purpose of Qwantbot?

Qwantbot supports Qwant's search engine by gathering the information needed to build and maintain its search index. Qwant positions itself as a privacy-focused alternative to other search engines, emphasizing user privacy and neutrality in search results.

The data collected by Qwantbot is used to:

• Build and maintain Qwant's search index
• Determine page ranking and relevance for search queries
• Understand the structure and content of websites
• Keep search results current with fresh content

For website owners, having content crawled by Qwantbot provides the benefit of visibility in Qwant's search results, potentially driving traffic from users who prefer this privacy-focused search engine. This is particularly relevant for audiences in France and Europe where Qwant has a stronger presence.

How do I block Qwantbot?

Qwantbot respects the robots.txt protocol, making it straightforward to control its access to your site. To completely block Qwantbot from crawling your site, add the following directives to your robots.txt file:

User-agent: Qwantbot
Disallow: /

If you want to block Qwantbot from specific sections of your site while allowing it to crawl others, you can use more targeted directives:

User-agent: Qwantbot
Disallow: /private/
Disallow: /members/
Allow: /

Since Qwantbot may use different user-agent names in its various versions, you might want to include entries for both common identifiers:

User-agent: Qwantbot
Disallow: /

User-agent: Qwantify
Disallow: /

Blocking Qwantbot means your content won't appear in Qwant search results, potentially reducing visibility to users of this search engine. Consider whether selective blocking of sensitive areas while allowing crawling of public content might be a better approach than complete blocking.

If you need more granular control beyond robots.txt, you can implement HTTP response headers or server-side blocking based on the user-agent string or IP addresses, though these approaches are typically more complex to maintain.