You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: modules/manage/pages/cluster-maintenance/manage-throughput.adoc
+23-12Lines changed: 23 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,15 +1,18 @@
1
1
= Manage Throughput
2
-
:description: Manage the throughput of Kafka traffic with configurable properties.
2
+
:description: Learn how to manage the throughput of Kafka traffic.
3
3
:page-categories: Management, Networking
4
4
// tag::single-source[]
5
5
6
-
Redpanda supports throughput throttling on both ingress and egress independently, and allows configuration at the broker and client levels. This helps prevent clients from causing unbounded network and disk usage on brokers.
7
-
8
-
* Broker-wide limits apply to all clients connected to the broker and restrict total traffic on the broker.
9
-
* Client limits apply to a set of clients defined by their `client_id` and help prevent a set of clients from starving other clients using the same broker. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`] or with the Kafka API. When no quotas apply, the client has unlimited throughput.
6
+
Redpanda supports throughput throttling on both ingress and egress independently, and allows configuration at the broker and client levels. This helps prevent clients from causing unbounded network and disk usage on brokers. You can configure limits at two levels:
10
7
8
+
* *Broker limits*: These apply to all clients connected to the broker and restrict total traffic on the broker. See <<broker-wide-throughput-limits, Broker-wide throughput limits>>.
9
+
ifndef::env-cloud[]
10
+
* *Client limits*: These apply to a set of clients defined by their `client_id` and help prevent a set of clients from starving other clients using the same broker. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with {ui}, or with the Kafka API. When no quotas apply, the client has unlimited throughput.
11
+
endif::[]
11
12
ifdef::env-cloud[]
12
-
NOTE: Throughput throttling is supported for BYOC and Dedicated clusters.
13
+
* *Client limits*: These apply to a set of clients defined by their `client_id` and help prevent a set of clients from starving other clients using the same broker. You can manage client quotas with xref:reference:rpk/rpk-cluster/rpk-cluster-quotas.adoc[`rpk cluster quotas`], with the {ui} UI, with the link:https://docs.redpanda.com/api/doc/cloud-dataplane/operation/operation-quotaservice_listquotas[Redpanda Cloud Data Plane API], or with the Kafka API. When no quotas apply, the client has unlimited throughput.
14
+
15
+
NOTE: Throughput throttling is supported for BYOC and Dedicated clusters only.
13
16
endif::[]
14
17
15
18
== Throughput throttling enforcement
@@ -18,6 +21,10 @@ NOTE: As of v24.2, Redpanda enforces all throughput limits per broker, including
18
21
19
22
Throughput limits are enforced by applying backpressure to clients. When a connection is in breach of the throughput limit, the throttler advises the client about the delay (throttle time) that would bring the rate back to the allowed level. Redpanda starts by adding a `throttle_time_ms` field to responses. If that isn't honored, delays are inserted on the connection's next read operation.
20
23
24
+
ifdef::env-cloud[]
25
+
In Redpanda Cloud, the throttling delay is set to 30 seconds.
26
+
endif::[]
27
+
21
28
ifndef::env-cloud[]
22
29
The throttling delay may not exceed the limit set by xref:reference:tunable-properties.adoc#max_kafka_throttle_delay_ms[`max_kafka_throttle_delay_ms`].
23
30
endif::[]
@@ -41,13 +48,13 @@ The properties for broker-wide throughput quota balancing are configured at the
41
48
| A broker's total throughput limit for egress Kafka traffic.
| The maximum delay inserted in the data path of Kafka API requests to throttle them down. Configuring this to be less than the Kafka client timeout can ensure that the delay that's inserted won't be long enough to cause a client timeout by itself.
57
+
| Maximum delay inserted in the data path of Kafka API requests to throttle them down. Setting this lower than the Kafka client timeout helps ensure throttling alone does not cause client timeouts.
51
58
52
59
|===
53
60
@@ -97,6 +104,10 @@ endif::[]
97
104
98
105
=== Individual client throughput limit
99
106
107
+
ifdef::env-cloud[]
108
+
NOTE: The following sections show how to manage throughput with `rpk`. You can also manage throughput with the link:https://docs.redpanda.com/api/doc/cloud-dataplane/operation/operation-quotaservice_listquotas[Redpanda Cloud Data Plane API].
109
+
endif::[]
110
+
100
111
To view current throughput quotas set through the Kafka API, run xref:reference:rpk/rpk-cluster/rpk-cluster-quotas-describe.adoc[`rpk cluster quotas describe`].
101
112
102
113
For example, to see the quotas for client ID `consumer-1`:
@@ -142,7 +153,7 @@ NOTE: A client group specified with `client-id-prefix` is not the equivalent of
142
153
143
154
You can apply default throughput limits to clients. Redpanda applies the default limits if no quotas are configured for a specific `client_id` or prefix.
144
155
145
-
To specify a produce quota of 1 GBps through the Kafka API (which means a 1 GBps limit across all produce requests sent to a single Redpanda broker), run:
156
+
To specify a produce quota of 1 GB/s through the Kafka API (applies across all produce requests to a single broker), run:
146
157
147
158
[,bash]
148
159
----
@@ -237,9 +248,9 @@ Replace the placeholder values with the new quota values, accounting for the con
237
248
rpk cluster config set kafka_client_group_byte_rate_quota=
238
249
----
239
250
240
-
=== View throughput limits in Redpanda Console
251
+
=== View throughput limits in {ui}
241
252
242
-
You can also use Redpanda Console to view enforced limits. In the menu, go to **Quotas**.
253
+
You can also use {ui} to view enforced limits. In the side menu, go to **Quotas**.
0 commit comments