> ## Knowledge Base Index
> Fetch the complete knowledge base index at: https://help.pandectes.io/sitemap.xml
> Use this file to discover available pages before exploring further.
> Pure-Markdown content can be obtained by appending a '.md' suffix to the content URLs listed in the sitemap (without the trailing slash).

# Bots blocking (New Version)

### Overview

Search engines, SEO tools, social media platforms, monitoring services, and other automated systems may visit your store without interacting with the consent banner like a real visitor.

The **Known bots & crawlers** option in the **Bots & crawlers** card hides the banner for automated visitors that Pandectes recognizes, and treats them as having given consent. Scripts run for them as if they had accepted, and nothing is recorded in your consent records.

By default, bots follow the regular flow, exactly like a human visitor.

### Where to find the setting

1. Open the **Pandectes GDPR Compliance** app.
2. Go to **Cookie banner**.
3. Open the **Advanced** tab.
4. Find the **Bots & crawlers** card.
5. Under **Known bots & crawlers**, enable **Hide the banner and treat as consent given**.
6. Click **See the list** to review the user agent fragments that are treated as bots.
7. Save.

![Bots & crawlers card on the Advanced tab](https://images.pandect.es/knowledge-base/bots-blocking-new-version-1.png =900xauto)

The same card has a **Specific bots** section for your own rules. Type a user agent fragment, press Enter, and add it to one of three lists: **Show the banner**, **Hide and treat as consent given**, or **Hide and treat as consent denied**. Matching ignores case and looks anywhere in the user agent. The most specific match wins, so "googlebot" overrides "bot". For more detail, see [Bots & crawlers: control banner behavior and consent records](/en/article/bots-and-crawlers-control-banner-behavior-and-consent-records-9upcs6/).

### How it works

When a visitor loads your store, Pandectes checks the browser user agent.

If the user agent matches a known bot, crawler, or automated tool, the banner is hidden for that session and the visit is treated as consent given.

This means:

* The banner is not displayed.
* The bot visit is not recorded in your consent records.
* The page can be rendered without banner interference.

### Detected bot categories

Pandectes checks for known bots and crawler patterns, including the following categories.

| Category | Examples |
| ---- |
| Search engine bots | Googlebot, Googlebot-Mobile, Googlebot-Image, Google favicon, Mediapartners-Google, bingbot, msnbot, Yahoo! Slurp, YandexBot, Baiduspider, DuckDuckBot, Exabot, SeznamBot, Sogou, Qwantify |
| SEO and analytics crawlers | AhrefsBot, SemrushBot, MJ12bot, BLEXBot, Lipperhey SEO Service, linkdex, Rogerbot, Ltx71, Sistrix Crawler, MegaIndex, GrapeshotCrawler, Seokicks-robot, xovibot, Ezooms, dotbot |
| Social media bots | Facebookexternalhit, Twitterbot, Facebot, Pinterest, Tumblr, vkshare |
| Archival and monitoring bots | archive.org_bot, ia_archiver, memorybot, changedetection, Pingability, Monitoring |
| Common crawlers and libraries | wget, curl, Python-urllib, Java, Commons-HttpClient, httpunit, nutch, phpcrawl |
| Other known bots and automated tools | Applebot, Coccoc, OrangeBot, Proximic, CrystalSemanticsBot, IntegroMedb, Toplistbot, WinHttpRequest, BnLAR, Adidxbot |
| GDPR-related crawlers | Pandectes |
| General bot patterns | spider, bot, crawler |

### Why enable this option

Enable **Hide the banner and treat as consent given** under **Known bots & crawlers** if you want the banner to be suppressed for automated traffic.

This can help with:

* **Cleaner consent data**
  Bot visits are less likely to appear as unresolved or non-interactive banner sessions.

* **Better crawl rendering**
  Search engines and SEO tools can render page content without the banner covering or modifying the page view.

* **More accurate reporting**
  Consent reports can stay focused on real visitor interactions instead of automated visits.

* **Reduced unnecessary banner loading**
  The banner does not need to initialize for visitors that cannot provide meaningful consent.

### Important notes

* This setting applies only to bots and crawlers that can be detected from their user agent.
* Some automated tools may use uncommon, custom, or disguised user agents and may not be detected.
* This option does not replace your consent model for real visitors.
* Real visitors still see the banner based on your banner visibility, region, and consent model settings.
* The detection list may be updated over time as known bot traffic changes.

### Video tutorial

${youtube}[How to disable the New Banner for known bots and crawlers](nnYwhgvhj1w)

### Summary

Use **Known bots & crawlers** to prevent the New Banner from running for known automated visitors. This helps keep crawler sessions separate from real visitor consent behavior and reduces banner interference for search engines, SEO tools, and monitoring services.