Slackbot

What is Slackbot?

Slackbot is a collection of automated robots operated by Slack, the business communication platform. Slack employs these robots to enhance their messaging platform by fetching additional information from around the web when users share links or media. The main Slackbot identifies itself in server logs as Slackbot 1.0 (+https://api.slack.com/robots) and serves as their default, "kitchen sink" robot that handles HTTP requests not covered by their more specialized bots. Slack operates two other specialized robots: the Link Expanding robot which identifies as Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots) and the ImgProxy robot which appears as Slack-ImgProxy 0.19 (+https://api.slack.com/robots). While not traditional web crawlers, these robots do make HTTP requests to external websites to retrieve content for display within Slack channels.

Why is Slackbot crawling my site?

If you're seeing Slackbot in your site logs, it's likely because a Slack user shared a link to your website in a Slack channel. The most common visitor is the Slackbot-LinkExpanding robot, which fetches content to create rich previews of shared links. This robot retrieves only the minimum necessary data (using HTTP Range headers) to extract meta tags from your content, particularly looking for oEmbed, Twitter Card, and Open Graph tags. If your page references images, videos, or audio files in these tags, Slackbot will fetch those files to verify them and extract additional metadata.

Responses to these requests are cached globally across Slack for approximately 30 minutes, so you shouldn't see multiple requests for the same URL more frequently than that. The ImgProxy robot might visit to fetch and cache images posted in Slack channels, while the general Slackbot might appear when making API requests for integrated services or handling outgoing webhooks configured by Slack teams.

What is the purpose of Slackbot?

Slackbot's primary purpose is to enhance the user experience within Slack by providing rich, visual previews of content shared in conversations. When a user shares a link, the Link Expanding robot retrieves metadata to create informative previews that include titles, descriptions, and thumbnail images, making conversations more engaging and informative.

The ImgProxy robot serves a security and performance function by proxying images shared in Slack. This approach protects user privacy by hiding detailed referrer information that might include team or project names, ensures images are served over HTTPS for security, and improves overall performance.

The general Slackbot handles miscellaneous HTTP requests needed for Slack's functionality, including API requests for integrated services and outgoing webhooks. All these robots work together to create a seamless, information-rich communication experience for Slack users while maintaining security and performance standards.

How do I block Slackbot?

Slack has made a deliberate decision not to honor robots.txt files. According to their documentation, they initially implemented and experimented with respecting robots.txt, but received numerous complaints from users because a significant portion of the internet would become inaccessible to their service. Since Slackbot is not a traditional crawler (it doesn't follow links and acts on behalf of human users), Slack determined it made more sense not to process robots.txt files as if they were a crawler.

If you wish to block Slack's robots from accessing your site or want to change how your content appears in Slack embeds, you'll need to contact Slack directly. They state they are "more than happy to add you to a blocklist/allowlist" upon request. You can reach out to Slack through their contact page or via Twitter at @slackapi.

Alternatively, you could implement server-side user-agent detection to identify Slack's robots by their user-agent strings and control their access that way. However, keep in mind that blocking Slack's robots may impact how your content appears when shared in Slack channels, potentially reducing visibility and engagement with users of the platform.