Articles on: Settings

Bots blocking (New Version)

This article applies to the New Banner (v5+). If you are on the Legacy Banner, please refer to Bots Blocking (Legacy Version).


Overview

When search engines, SEO tools, social media platforms, and other automated services visit your store, they do not interact with the cookie consent banner the way a real visitor would. Showing the banner to these bots serves no compliance purpose and can interfere with how they index and render your store.


The Disable for known bots & crawlers option automatically detects these visitors and hides the banner for them, allowing the page to load fully without the banner running at all for that session.


Where to find the setting

  1. Go to Settings → Banner.
  2. Select the Advanced tab.
  3. Scroll down to the Visibility section on the right side of the page.
  4. Enable the Disable for known bots & crawlers checkbox.


How detection works

The app identifies bots by matching the visitor's user agent against a predefined list of known bots and crawlers. This list is regularly updated to keep up with the evolving landscape of automated traffic.

Detected categories include:

  • Search engine bots — Google (Googlebot, Googlebot-Mobile, Googlebot-Image, Mediapartners-Google), Bing (bingbot, msnbot), Yahoo (Yahoo! Slurp), Yandex (yandexbot), Baidu (Baiduspider), DuckDuckBot, Exabot, SeznamBot, Sogou, Qwantify
  • SEO and analytics crawlers — AhrefsBot, SemrushBot, MJ12bot, BLEXBot, Lipperhey SEO Service, linkdex, Rogerbot, Ltx71, Sistrix Crawler, MegaIndex, GrapeshotCrawler, Seokicks-robot, xovibot, Ezooms, dotbot
  • Social media bots — Facebookexternalhit, Twitterbot, Facebot, Pinterest, Tumblr, vkshare
  • Archival and monitoring bots — archive.org_bot, ia_archiver (Wayback Machine), memorybot, changedetection, Pingability, Monitoring
  • Common crawlers and libraries — wget, curl, Python-urllib, Java, Commons-HttpClient, httpunit, nutch, phpcrawl
  • Other notable bots — Applebot, Coccoc, OrangeBot, Proximic, CrystalSemanticsBot, IntegroMedb, Toplistbot, WinHttpRequest, BnLAR, Adidxbot
  • GDPR-related crawlers — Pandectes
  • General patterns — spider, bot, crawler


Why enable this

  • Cleaner analytics — bot visits will not appear as unresolved consent sessions in your reports.
  • Better crawlability — search engines and SEO tools see your full page content without banner interference.
  • Accurate consent data — your consent statistics reflect real visitor behaviour only.

Updated on: 17/04/2026

Was this article helpful?

Share your feedback

Cancel

Thank you!