Feedly bot

What is Feedly?

Feedly is a freemium news aggregator application and service developed by DevHD. Available at feedly.com, it compiles news feeds from a variety of online sources, allowing users to customize their reading experience and share content with others. First released in 2008, Feedly gained significant popularity in 2013 when Google Reader announced its shutdown, with the service quickly attracting millions of new users seeking an alternative RSS reader.

Technically, Feedly functions as a content aggregator and RSS reader that collects and organizes web content from sources that users choose to follow. When crawling websites, Feedly typically identifies itself with a user agent string such as Feedly/1.0 (+http://www.feedly.com/fetcher.html; like FeedFetcher-Google). This indicates that it's accessing content to update feeds for users who have subscribed to content from your site.

Feedly works by scanning RSS feeds, Atom feeds, and other structured content sources that websites publish. Unlike general web crawlers that index entire sites, Feedly specifically targets syndicated content feeds to deliver updates to its users. The service is available as both a web application and mobile apps for Android and iOS devices.

Why is Feedly crawling my site?

Feedly is crawling your site because one or more Feedly users have subscribed to your content feeds. When you publish new content, Feedly's systems detect these updates and fetch the new content to display it to subscribers within their Feedly interface. This is a normal and authorized form of content syndication, assuming you've published RSS or Atom feeds for your content.

The frequency of Feedly's visits depends on several factors: how many Feedly users follow your content, how frequently you update your site, and the freshness settings within Feedly's system. Popular sites with frequent updates may see Feedly crawlers more often, sometimes multiple times per day. Sites with fewer followers or less frequent updates might see Feedly visits only periodically.

Feedly's crawling is triggered primarily when it checks for updates to feeds that its users have subscribed to. It may also crawl your site when a new user adds your feed to their Feedly account.

What is the purpose of Feedly?

The primary purpose of Feedly is to provide a centralized reading experience where users can consume content from multiple sources in a single, organized interface. Rather than visiting dozens of individual websites, users can see all their followed content in one place, organized by categories they define.

For users, Feedly serves as a personalized news and content hub. The service helps people manage information overload by organizing content into collections and allowing for focused reading experiences. Users can save articles for later, share content with others, and customize how content appears in their feed.

For website owners, Feedly can drive engaged traffic to your site as users discover your content through the platform. It effectively extends your content's reach to audiences who prefer consuming information through feed readers rather than visiting individual sites directly. Many content creators appreciate the expanded distribution that feed readers like Feedly provide.

How do I block Feedly?

If you wish to control how Feedly accesses your site, you can use the standard robots.txt protocol, which Feedly respects. To block Feedly completely, add the following directives to your robots.txt file:

User-agent: Feedly
Disallow: /

This will instruct Feedly not to crawl any part of your site. If you only want to block specific sections, you can modify the Disallow directive accordingly:

User-agent: Feedly
Disallow: /private-section/

Keep in mind that blocking Feedly means users who rely on the service won't receive updates from your site in their feeds. This could potentially reduce your content's visibility and reach among users who prefer consuming content through feed aggregators. Many publishers consider feed readers like Feedly to be valuable distribution channels that help extend their audience.

If you publish RSS feeds but don't want them to be accessed by feed aggregators like Feedly, you might need to implement additional access controls on your feed URLs, as simply blocking the crawler won't prevent users from manually adding your feed URL to their Feedly account. Some sites implement authentication on their feeds or use other technical measures to control feed access more granularly.