Policy management
Publisher policies
Policies describe how crawler traffic should be evaluated for historical evidence and future imports. Changing saved policies does not mutate existing ClickHouse events; use the simulator to replay policy decisions against historical traffic.
Demo tenant: Demo Publisher
Create policy
Saved policies
These rules are evaluated during import and can also be replayed against historical evidence in the simulator.
| Priority | Enabled | Name | Scope | Crawler vendor | Crawler purpose | Verification status | Action | Redirect URL | Throttle RPM | Updated | Edit |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 10 | Yes | Allow verified Googlebot on public content | content_class: public_news | - | verified | allow | - | - | 2026-06-19 15:24:11 | Edit | |
| 20 | Yes | Allow OAI-SearchBot on public content | content_class: public_news | OpenAI | search | - | allow | - | - | 2026-06-19 15:24:11 | Edit |
| 30 | Yes | Block GPTBot on premium content | content_class: premium_analysis | OpenAI | training | - | block | - | - | 2026-06-19 15:24:11 | Edit |
| 40 | Yes | Block training crawlers on premium content | content_class: premium_analysis | - | training | - | block | - | - | 2026-06-19 15:24:11 | Edit |
| 50 | Yes | Redirect unknown bots hitting archive content | content_class: archive | - | - | unknown | redirect | /ai-licensing | - | 2026-06-19 15:24:11 | Edit |
| 1000 | Yes | Monitor unknown bots by default | global: * | - | - | unknown | monitor | - | - | 2026-06-19 15:24:11 | Edit |