Miniflux bot

What is Miniflux?

Miniflux is a minimalist, open-source feed reader application created by Frédéric Guillot. It allows users to subscribe to and read content from RSS, Atom, and JSON feeds from websites. Available at https://miniflux.app, Miniflux functions as a feed fetcher that regularly polls websites for new content, making it appear as a crawler in server logs.

Miniflux identifies itself in logs with a user-agent string following the pattern Mozilla/5.0 (compatible; Miniflux/2.x.x; +https://miniflux.app), where the version number reflects the current release. This self-identification helps website administrators recognize legitimate Miniflux traffic.

As a feed fetcher, Miniflux operates by periodically checking websites for updated content based on user subscriptions. It downloads feed files, parses them, and stores new entries in its database for users to read. Unlike general web crawlers that index entire sites, Miniflux only accesses specific feed URLs that users have explicitly subscribed to.

Why is Miniflux crawling my site?

If Miniflux is accessing your website, it means one or more Miniflux users have subscribed to your RSS, Atom, or JSON feed. The application is fetching your content to display it to these subscribers in their personal feed reader interface.

Miniflux typically visits feed URLs at regular intervals determined by its configuration settings. By default, it checks for updates every 60 minutes, though this can vary based on individual installations. Some Miniflux instances may use an "entry frequency" scheduler that adjusts polling frequency based on how often your content typically updates.

The crawling is legitimate and authorized as long as you're publicly offering feeds. Miniflux only accesses the specific feed URLs that users have subscribed to, not your entire website. Each visit represents real user interest in your content.

What is the purpose of Miniflux?

Miniflux serves as a personal content aggregator for its users, allowing them to follow updates from multiple websites in one centralized interface. Unlike commercial feed readers that might analyze reading habits or serve advertisements, Miniflux is privacy-focused and doesn't collect data for purposes beyond delivering content to subscribers.

For content publishers, Miniflux and similar feed readers provide a distribution channel that helps maintain audience connection. When users subscribe to your feeds through Miniflux, they're expressing interest in regularly consuming your content, potentially increasing your readership and engagement.

The service benefits website owners by extending content reach to users who prefer feed readers over visiting individual websites or receiving email newsletters. It also respects your server resources by making controlled, periodic requests rather than constant polling.

How do I block Miniflux?

Miniflux respects the robots.txt protocol, though it's important to understand that blocking it will prevent legitimate subscribers from receiving your content updates. If you still wish to restrict Miniflux's access, you can add the following to your robots.txt file:

User-agent: Miniflux
Disallow: /

This directive specifically blocks Miniflux while allowing other crawlers. Alternatively, you could block specific paths instead of your entire site:

User-agent: Miniflux
Disallow: /premium-content/
Disallow: /members-only/

Keep in mind that blocking Miniflux means users who rely on it as their feed reader won't receive updates from your site, potentially reducing your audience reach and engagement. Since Miniflux only accesses publicly available feeds that you've created specifically for distribution, blocking it generally works against the purpose of having feeds in the first place.

If you're concerned about server load, consider implementing conditional HTTP responses that use the If-Modified-Since header, which Miniflux supports. This allows you to return a 304 Not Modified status when content hasn't changed, reducing bandwidth usage while still allowing subscribers to receive updates when available.