Volumetric Abuse Detection

Cloudflare Volumetric Abuse Detection helps you set up a system of adaptive rate limiting.

About

After API Discovery, Cloudflare looks for endpoint abuse based on common user traffic.

For example, your API might see different levels of traffic to a /reset-password endpoint than a /login endpoint. Additionally, your /login endpoint might see higher than average traffic after a successful marketing campaign.

These two scenarios speak to the limitations of traditional rate limiting. Not only does traffic vary between endpoints, but it also can vary over time for the same endpoint. Volumetric Abuse Detection solves these problems with unsupervised learning to develop separate baselines for each API and better adjust to changes in user behavior.

Volumetric Abuse Detection rate limits are generated on a per-session basis. Unlike traditional rate limits, which are based on IP addresses, Volumetric Abuse Detection rate limits are not as susceptible to false positives when traffic to your API increases.

Volumetric Abuse Detection rate limits are a way to prevent blatant volumetric abuse while minimizing false positives. If you are trying to prevent abusive bot traffic altogether, refer to Cloudflare’s Bot solutions.

Process

Volumetric Abuse Detection analyzes your API’s individual session traffic statistics to recommend per-endpoint, per-session rate limits.

Volumetric Abuse Detection currently requires a session identifier, like an authorization token available as a request header or cookie.

After adding a session identifier, allow 24 hours for rate limit recommendations to appear on endpoints in the Cloudflare dashboard.

Old dashboard: Security > API Shield > Endpoint Management

New dashboard: Security > Web Assets > Endpoints

Recommendations will continue to update if your traffic pattern changes.

Observe rate limits

Once rate limit recommendations appear in Endpoints, select the endpoint row to view more detail about the recommendation. You will see the overall recommended rate limit value, as well as p99, p90, and p50 rate limit values.

Cloudflare recommends choosing the overall rate limit recommendation, as our analysis includes the variance of the request rate distribution across your API sessions. Choosing a single p-value may cause false positives due to a high number of outliers.

p-values

p-values describe what percentile of your traffic fits below the value. For example, if your p90 value is 83, then 90% of your sessions had maximum request rates less than 83 requests per 10 minutes.

In Endpoints, you can review our confidence in the recommendation and how many unique sessions we have seen over the last seven (7) days. In general, endpoints with fewer unique sessions and high variability of user behavior will have lower confidence scores.

Implementing low confidence rate limits can still be helpful to prevent API abuse. If you are hesitant due to the recommendation’s confidence, we suggest starting your rate limit rule in log mode and observing violations of the rule for false positives.

Create rate limits

Refer to the Rules documentation ↗ for more information on how to create an Advanced Rate Limiting rule.

API

Rate limit recommendations are available via the API if you would like to dynamically update rate limits over time.

Special cases

Rate limit by user (JWT claim)

You can rate limit requests based on any claim inside of a JSON Web Token (JWT), such as:

• Registered claims like aud or sub
• Custom claims like userEmail, including nested custom claims like user.email

Rate limiting based on JWT claim values will only work on valid JSON Web Tokens. If you do not block invalid JSON Web Tokens on your path, the JWT claims will all be counted and possibly blocked if high traffic is detected in the Point of Presence (PoP).

You must also count the JWT claim that uniquely identifies the user. If you select a claim that is the same for many of your users, their rate limits will all be counted together.

Rate limit by user tier

If you offer multiple tiers on your website or application and you want to enforce rate limiting based on the tiers, such as:

• If "aud": "free-tier", rate limit to five requests per minute.
• If "aud": "premium-tier", rate limit to 50 requests per minute.

You can follow the rate limiting rule example below:

Example rule expression

(http.request.method eq "GET" and
http.host eq "<YOUR_DOMAIN>" and
http.request.uri.path matches "</EXAMPLE_PATH>" and
lookup_json_string(http.request.jwt.claims["<JWT_TOKEN_CONFIGURATION_ID>"][0], "aud") eq "free-tier"

Limitations

API Shield will always calculate recommendations when session identifiers are configured. To enable session-based rate limits, subscribe to Advanced Rate Limiting.

Availability

Volumetric Abuse Detection is only available for Enterprise customers. If you are an Enterprise customer and interested in this product, contact your account team.