We are excited to share that we have expanded our predefined bot list to include 10 new AI-powered bots.
AI is becoming a go-to source of information for millions. From ChatGPT to Perplexity, users are increasingly relying on these platforms to answer questions, discover content, and make decisions. Now you can easily see which bot visits your website and when.
Our system can now identify the following newly added bots:
- Mistral AI Bot
- Allen Institute
- Cohere Bot
- Facebook Bot
- Apple Bot
- Perplexity Bot
- Claude Bot
- Anthropic Bot
- OpenAI Search Bot
- ChatGPT Bot
These join our existing list of recognized bots, including:
- SEOZoom bot
- Linkpad.ru bot
- CognitiveSEO bot
- Blekko bot
- IBM Watson bot
- Linkdex bot
- OpenLinkProfiler bot
- PagesInventory bot
- BacklinkTest crawler
- Lipperhey spider
- Sistrix bot
- SEOkicks bot
- SearchMetrics crawler
- Archive.org crawler
- BLEXbot (LinkAssistant)
- MOZ bot (DotBot, Rogerbot, Ezooms)
- Baidu bot
- Yandex bot
- Majestic bot
- SemRush bot
- Ahrefs bot
- Bing bot
- Google bot
In addition to identifying these bots by name, we also monitor every request. If we detect that a request might be coming from a bot, we will send this information via HTTP request headers as soon as the bot sends the request.
In PHP, you can access these headers as follows:
$_SERVER['HTTP_PP_BOT'] - 0 means it is not a bot, and 1 means it is a bot.
$_SERVER['HTTP_PP_BOT_TYPE'] - includes the name of the bot, for example: CHATGPTBOT
These headers are sent even if you're using your own web server!
Additionally, you can utilize these predefined bots within our firewall rules, allowing you to block, allow, or redirect these requests to another page.
We hope you find this update helpful.
If you have questions or suggestions, our support team is always ready to assist.