> ## Knowledge Base Index
> Fetch the complete knowledge base index at: https://help.pandectes.io/sitemap.xml
> Use this file to discover available pages before exploring further.
> Pure-Markdown content can be obtained by appending a '.md' suffix to the content URLs listed in the sitemap (without the trailing slash).

# Bots blocking (New Version)

### Overview

Search engines, SEO tools, social media platforms, monitoring services, and other automated systems may visit your store without interacting with the consent banner like a real visitor.

The **Disable for known bots & crawlers** option detects known automated visitors and prevents the banner from running for that session.

When this option is enabled, known bots can access the storefront without the consent banner being displayed or initialized for them.

### Where to find the setting

1. Open the **Pandectes GDPR Compliance** app.
2. Go to **Settings > Banner**.
3. Open the **Advanced** tab.
4. Find the **Visibility** section.
5. Enable **Disable for known bots & crawlers**.

### How it works

When a visitor loads your store, Pandectes checks the browser user agent.

If the user agent matches a known bot, crawler, or automated tool from the detection list, the app does not run the banner for that session.

This means:

* The banner is not displayed.
* The consent widget is not shown.
* The bot visit is not treated like a normal visitor banner session.
* The page can be rendered without banner interference.

### Detected bot categories

Pandectes checks for known bots and crawler patterns, including the following categories.

| Category | Examples |
| ---- |
| Search engine bots | Googlebot, Googlebot-Mobile, Googlebot-Image, Google favicon, Mediapartners-Google, bingbot, msnbot, Yahoo! Slurp, YandexBot, Baiduspider, DuckDuckBot, Exabot, SeznamBot, Sogou, Qwantify |
| SEO and analytics crawlers | AhrefsBot, SemrushBot, MJ12bot, BLEXBot, Lipperhey SEO Service, linkdex, Rogerbot, Ltx71, Sistrix Crawler, MegaIndex, GrapeshotCrawler, Seokicks-robot, xovibot, Ezooms, dotbot |
| Social media bots | Facebookexternalhit, Twitterbot, Facebot, Pinterest, Tumblr, vkshare |
| Archival and monitoring bots | archive.org_bot, ia_archiver, memorybot, changedetection, Pingability, Monitoring |
| Common crawlers and libraries | wget, curl, Python-urllib, Java, Commons-HttpClient, httpunit, nutch, phpcrawl |
| Other known bots and automated tools | Applebot, Coccoc, OrangeBot, Proximic, CrystalSemanticsBot, IntegroMedb, Toplistbot, WinHttpRequest, BnLAR, Adidxbot |
| GDPR-related crawlers | Pandectes |
| General bot patterns | spider, bot, crawler |

### Why enable this option

Enable **Disable for known bots & crawlers** if you want the banner to be suppressed for automated traffic.

This can help with:

* **Cleaner consent data**
  Bot visits are less likely to appear as unresolved or non-interactive banner sessions.

* **Better crawl rendering**
  Search engines and SEO tools can render page content without the banner covering or modifying the page view.

* **More accurate reporting**
  Consent reports can stay focused on real visitor interactions instead of automated visits.

* **Reduced unnecessary banner loading**
  The banner does not need to initialize for visitors that cannot provide meaningful consent.

### Important notes

* This setting applies only to bots and crawlers that can be detected from their user agent.
* Some automated tools may use uncommon, custom, or disguised user agents and may not be detected.
* This option does not replace your consent model for real visitors.
* Real visitors still see the banner based on your banner visibility, region, and consent model settings.
* The detection list may be updated over time as known bot traffic changes.

### Video tutorial

${youtube}[How to disable the New Banner for known bots and crawlers](nnYwhgvhj1w)

### Summary

Use **Disable for known bots & crawlers** to prevent the New Banner from running for known automated visitors. This helps keep crawler sessions separate from real visitor consent behavior and reduces banner interference for search engines, SEO tools, and monitoring services.