Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions src/content/docs/bots/concepts/bot/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,15 +19,12 @@ Bots can be used for good (chatbots, search engine crawlers) or for evil (invent
For more background, refer to [What is a bot?](https://www.cloudflare.com/learning/bots/what-is-a-bot/).
:::

## Verified bots
## Verified bots and signed agents

<Render file="verified-bots" product="bots" />

For more information, refer to [Verified bots](/bots/concepts/bot/verified-bots/).

:::note

The method for allowing or blocking verified bots depends on [your plan](/bots/get-started/).
The method for allowing or blocking verified bots depends on [your plan](/bots/concepts/bot/verified-bots/#availability).
:::

## AI bots
Expand Down
14 changes: 11 additions & 3 deletions src/content/docs/bots/concepts/bot/verified-bots/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -55,10 +55,14 @@ The bot must be verified using one of the following validation methods:

You can segment your verified bot traffic by its type and purpose by adding the Verified Bot Categories field `cf.verified_bot_category` as a filter criteria in [WAF Custom rules](/waf/custom-rules/), [Advanced Rate Limiting](/waf/rate-limiting-rules/), and Late Transform rules.

:::note
:::caution
The Verified Bot Categories field is not compatible with legacy Firewall rules.
:::

:::note
Verified Bot Categories is available on all plans.
:::

| Name | String value | Example |
| -------------------------- | ---------------------------- | ----------------------------------------------------------------------- |
| Academic research | `Academic Research` | Library of Congress, TurnItInBot, Bibliothèque nationale de France |
Expand All @@ -79,6 +83,10 @@ The Verified Bot Categories field is not compatible with legacy Firewall rules.
| Webhooks | `Webhooks` | Payment processors, WordPress Integration tools |
| Other | `Other` | |

### Availability
---

Verified Bot Categories is available on all plans.
## Availability

Verified bots are excluded by default when [Bot Fight Mode](/bots/get-started/bot-fight-mode/) is enabled to block definite bots.

[Super Bot Fight Mode](/bots/get-started/super-bot-fight-mode/) and [Enterprise Bot Management](/bots/get-started/bot-management/) customers have the option to block or allow verified bots.
10 changes: 10 additions & 0 deletions src/content/docs/bots/get-started/bot-fight-mode.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ Bot Fight Mode is a simple, free product that helps detect and mitigate bot traf

<Render file="about-plan-considerations" product="bots" />

---

## Enable Bot Fight Mode

To start using Bot Fight Mode:
Expand All @@ -46,6 +48,8 @@ To start using Bot Fight Mode:

<Render file="sbfm-upgrade" product="bots" />

---

## Disable Bot Fight Mode

If you find that **Bot Fight Mode** is causing problems with your application traffic, you may want to disable it.
Expand All @@ -71,6 +75,8 @@ To disable Bot Fight Mode:
</TabItem>
</Tabs>

---

## Block AI bots

Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/).
Expand All @@ -80,10 +86,14 @@ Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/).
You can view blocked AI bot traffic via [Security Analytics](/waf/analytics/security-analytics/).
:::

---

## Visibility

You can see bot-related actions by going to **Security** > **Events**. Any requests challenged by this product will be labeled **Bot Fight Mode** in the **Service** field. This allows you to observe, analyze, and follow trends in your bot traffic over time.

---

## Limitations

You cannot bypass or skip Bot Fight Mode using the _Skip_ action in WAF custom rules or using Page Rules. _Skip_, _Bypass_, and _Allow_ actions apply to rules or rulesets running on the [Ruleset Engine](/ruleset-engine/). While Super Bot Fight Mode rules are implemented in the Ruleset Engine, Bot Fight Mode checks are not. This is why you can skip Super Bot Fight Mode, but not Bot Fight Mode. If you need to skip Bot Fight Mode, consider using [Super Bot Fight Mode](/bots/get-started/super-bot-fight-mode/).
Expand Down
10 changes: 4 additions & 6 deletions src/content/docs/bots/get-started/bot-management.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ This Enterprise product provides the most flexibility to customers by:
- Allowing customers to take action on this score with [WAF custom rules](/waf/custom-rules/) or [`Workers`](/workers/runtime-apis/request/#incomingrequestcfproperties).
- Allowing customers to view this score in Bot Analytics or Logs.

---

## Enable Bot Management for Enterprise

Bot Management is automatically enabled for Enterprise zones entitled with the add-on.
Expand Down Expand Up @@ -53,6 +55,8 @@ Bot Management is automatically enabled for Enterprise zones entitled with the a
If you are not seeing Bot Management enabled on your zone or if you still see **Add Bot Management** on the Cloudflare dashboard, contact your account team for the proper entitlements.
:::

---

## Setup

Cloudflare recommends that you deploy the following basic settings and customize them according to the traffic in your zone.
Expand All @@ -69,12 +73,6 @@ Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/).
You can view blocked AI bot traffic via [Security Analytics](/waf/analytics/security-analytics/).
:::

### Enable JavaScript detections

Enabling [JavaScript detections](/bots/additional-configurations/javascript-detections/) validates that the browser can run JavaScript, and is stored in the `cf.bot_management.js_detection.passed` variable.

<Render file="javascript-detections-enable" params={{ one: "Bot Management" }} product="cloudflare-challenges" />

### Deploy default templates

Cloudflare has [default templates](https://dash.cloudflare.com/?to=/:account/:zone/security/security-rules?template=bot_traffic) for definite bots, which have a [bot score](/bots/concepts/bot-score/) of 1, and likely bots which have a bot score of 2 to 29. In our templates, Cloudflare recommends to allow verified bots such as Google SEO Crawler and access to cached static resources.
Expand Down
10 changes: 8 additions & 2 deletions src/content/docs/bots/get-started/super-bot-fight-mode.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,10 @@ To start using Super Bot Fight Mode:
If your organization also uses [Cloudflare Tunnel](/cloudflare-one/connections/connect-networks/), keep **Definitely Automated** set to **Allow**. Otherwise, tunnels might fail with a `websocket: bad handshake` error.
:::

### Conditions

<Render file="flexible-sbfm" product="bots" />

---

## Disable Super Bot Fight Mode

If you find that **Super Bot Fight Mode** is causing problems with your application traffic, you may want to disable it.
Expand Down Expand Up @@ -99,6 +99,8 @@ To disable Super Bot Fight Mode:

<Render file="flexible-sbfm" product="bots" />

---

## Block AI bots

Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/).
Expand All @@ -108,6 +110,8 @@ Refer to [Block AI bots](/bots/additional-configurations/block-ai-bots/).
You can view blocked AI bot traffic via [Security Analytics](/waf/analytics/security-analytics/).
:::

---

## Analytics

### Bot Report
Expand All @@ -122,6 +126,8 @@ To access the **Bot Report**, go to **Security** > **Bots**. If you see a double

You can see bot-related actions by going to **Security** > **Events**. Any requests challenged by this product will be labeled **Super Bot Fight Mode** in the **Service** field. This allows you to observe, analyze, and follow trends in your bot traffic over time.

---

## Ruleset Engine

Super Bot Fight Mode runs during the `http_request_sbfm` phase of the [Ruleset Engine](/ruleset-engine/about/phases/).
Expand Down
8 changes: 3 additions & 5 deletions src/content/partials/bots/verified-bots.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,8 @@

---

import { GlossaryTooltip } from "~/components"
Cloudflare maintains an internal directory of [verified bot](/bots/concepts/bot/verified-bots/) and [signed agents](/bots/concepts/bot/signed-agents/) that are associated with search engine optimization (SEO), website monitoring, and more.

Cloudflare maintains an internal list of <GlossaryTooltip term="verified bot">verified bots</GlossaryTooltip> that are associated with search engine optimization (SEO), website monitoring, and more.
You can use this directory to prevent any bot protection measures from impacting otherwise helpful bots and agents, such as search crawlers.

You can use this list to prevent any bot protection measures from impacting otherwise helpful bots, such as search crawlers.

For a partial list of verified bots, refer to [Cloudflare Radar](https://radar.cloudflare.com/verified-bots).
For a partial list of verified bots and signed agents, refer to [Cloudflare Radar](https://radar.cloudflare.com/verified-bots).