Mediatoolkitbot

What is Mediatoolkitbot?

Mediatoolkitbot is an internet crawler operated by Determ, a company specializing in online media monitoring and analytics. It functions as an intelligence gathering bot that systematically browses websites across the internet, collecting publicly available content for Determ's media monitoring platform. The bot identifies itself in server logs with the simple user agent string Mediatoolkitbot, without additional version information or technical details.

As a web crawler, Mediatoolkitbot searches through more than 100 million web sources to identify and collect content that matches specific phrases or topics that Determ's users are monitoring. It adheres to standard web crawler protocols, including respecting robots.txt directives, which allows website administrators to control how it accesses their content. This makes it a well-behaved crawler that balances its data collection needs with website performance considerations.

Why is Mediatoolkitbot crawling my site?

Mediatoolkitbot visits your website to search for content that might be relevant to Determ's users, who are typically marketing professionals, PR specialists, and brand managers looking to monitor online mentions. The bot is specifically looking for content that matches phrases or keywords that Determ users have set up for tracking.

The frequency of visits depends on several factors, including how often your site publishes new content and whether your content regularly matches the monitoring criteria of Determ's users. Sites that frequently publish content relevant to brands or topics being monitored may see more regular visits.

The crawling is a normal part of how media monitoring services work, similar to how search engines crawl sites to index content. The bot only accesses publicly available content – it cannot access password-protected areas or private information.

What is the purpose of Mediatoolkitbot?

Mediatoolkitbot serves Determ's media monitoring platform, which helps businesses track brand mentions, analyze market trends, monitor competitors, and measure campaign performance across the web. The data collected by the bot enables Determ to alert its users when their monitored keywords or phrases appear in new content.

For marketers and PR professionals, this service provides valuable insights into brand perception, industry conversations, and potential collaboration opportunities. When the bot finds content matching a user's monitoring criteria, Determ notifies them, allowing for timely responses to mentions or emerging trends.

While website owners don't directly benefit from being crawled, the visibility provided by being included in media monitoring results can lead to increased exposure when your content is relevant to brands or topics being tracked. This could potentially result in new business opportunities or partnerships if your content is discovered by Determ users looking for industry voices or collaboration partners.

How do I block Mediatoolkitbot?

Mediatoolkitbot respects the robots.txt protocol, making it straightforward to control its access to your website. If you wish to completely block the bot from crawling your site, add the following directives to your robots.txt file:

User-agent: Mediatoolkitbot
Disallow: /

This configuration prevents Mediatoolkitbot from accessing any part of your website. If you'd prefer to allow access but want to reduce the frequency of visits to minimize server load, you can use the Crawl-Delay directive:

User-agent: Mediatoolkitbot
Crawl-Delay: 10

The value after Crawl-Delay represents the minimum number of seconds the bot should wait between consecutive requests. Adjusting this number lets you balance the bot's access with your server performance needs.

If you experience any issues with Mediatoolkitbot's behavior on your site, Determ provides a direct contact channel at complaints@determ.com where you can report problems. Before blocking the bot entirely, consider whether the potential visibility from being included in media monitoring results might benefit your site. Completely blocking the bot means your content won't be discovered by brands and marketers using Determ's platform, which could reduce potential exposure opportunities.