What is PocketParser?

PocketParser is a specialized web crawler operated by Read It Later, the company behind Pocket, a popular "read later" service that allows users to save articles from across the web. First deployed around 2016, PocketParser functions as a dedicated fetcher bot designed specifically for metadata extraction. When users save content to their Pocket account, PocketParser visits the webpage to collect essential information needed to create preview cards within the Pocket application.

The bot identifies itself in server logs with the user agent string PocketParser/2.0 (+https://getpocket.com/pocketparser_ua), which includes a link to its documentation. Unlike more aggressive crawlers, PocketParser exhibits a relatively low request frequency, only visiting pages when triggered by user actions rather than systematically crawling entire websites.

PocketParser operates by extracting specific metadata elements from webpages, including titles, featured images, content summaries, and Open Graph protocol tags. It doesn't execute JavaScript, process CSS, or handle cookies, focusing solely on gathering the information needed to generate accurate preview cards for saved content.

Why is PocketParser crawling my site?

If you're seeing PocketParser in your logs, it means one or more Pocket users have saved content from your website to read later. The bot isn't systematically crawling your entire site – it's making targeted visits to specific URLs that users have explicitly saved to their Pocket accounts.

PocketParser typically looks for content-rich pages like articles, blog posts, and news stories. Its visits are intermittent and directly tied to user save actions, meaning popular content may receive more frequent visits than less-shared pages. The crawling is authorized and legitimate, representing actual user interest in your content.

The bot may revisit pages occasionally to refresh its metadata, especially if the content has been saved by multiple users or remains popular within the Pocket ecosystem.

What is the purpose of PocketParser?

PocketParser supports Pocket's core service by enabling rich, visual previews of saved content. When a user saves an article, PocketParser collects the metadata needed to create an engaging preview card – complete with title, featured image, and summary – without having to store the entire webpage content.

This metadata extraction process benefits both Pocket users and content creators. For users, it creates a visually appealing and organized reading list that helps them remember why they saved specific articles. For content creators and website owners, it ensures their content is accurately represented within the Pocket interface, potentially driving return visits when users decide to read the saved content.

By focusing on metadata rather than full content scraping, PocketParser maintains a balance between providing useful previews and respecting the original source of the content.

How do I block PocketParser?

While PocketParser doesn't automatically honor robots.txt directives by default, you can still implement controls if needed. To manage PocketParser's access to your site, consider adding specific directives to your robots.txt file:

User-agent: PocketParser
Crawl-delay: 10
Allow: /public-content/
Disallow: /private/

This approach lets you control which sections of your site PocketParser can access while setting reasonable crawl rate limits. The Crawl-delay directive helps manage server load by requesting that the bot wait at least 10 seconds between requests.

Keep in mind that blocking PocketParser may affect how your content appears when saved to Pocket. Without proper metadata, preview cards may display incorrectly or lack visual elements, potentially reducing engagement from Pocket users. Since PocketParser only visits pages explicitly saved by users, its traffic is usually minimal and represents genuine interest in your content.

If you're experiencing unusually high traffic from PocketParser or suspect abuse, you can implement user-agent blocking at the server level or contact Pocket directly for assistance.

PocketParser