Skip to content

Commit 99fe2fc

Browse files
committed
Adjust FAQ for Kafka and Kinesis ClickPipes
1 parent c7f8231 commit 99fe2fc

File tree

4 files changed

+22
-14
lines changed

4 files changed

+22
-14
lines changed

docs/cloud/manage/jan2025_faq/_snippets/_clickpipes_faq.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ This corresponds to **0.25** ClickHouse compute units (1 unit = 8 GiB RAM, 2 vCP
5454

5555
Yes, ClickPipes for streaming can be scaled both horizontally and vertically.
5656
Horizontal scaling adds more replicas to increase throughput, while vertical scaling increases the resources (CPU and RAM) allocated to each replica to handle more intensive workloads.
57-
Both can be done via the UI during the creation of the ClickPipe or later in the editing mode under "Settings" -> "Advanced Settings" -> "Scaling"
57+
This can be configured during ClickPipe creation, or at any other point under **Settings** -> **Advanced Settings** -> **Scaling**.
5858

5959
</details>
6060

docs/integrations/data-ingestion/clickpipes/kafka/04_best_practices.md

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -133,17 +133,17 @@ the ClickPipe will automatically restart the consumer and continue processing me
133133

134134
### Benchmarks {#benchmarks}
135135

136-
Below are some informal benchmarks for ClickPipes for Kafka that can be used get a general idea of ClickPipes performance. It is important to know that many factors can impact performance including message size, data types, and data format. Your mileage may vary, and what we show here is not a guarantee of performance.
136+
Below are some informal benchmarks for ClickPipes for Kafka that can be used get a general idea of the baseline performance. It's important to know that many factors can impact performance, including message size, data types, and data format. Your mileage may vary, and what we show here is not a guarantee of actual performance.
137137

138138
Benchmark details:
139+
139140
- We used production ClickHouse Cloud services with enough resources to ensure that throughput was not bottlenecked by the insert processing on the ClickHouse side.
140141
- The ClickHouse Cloud service, the Kafka cluster (Confluent Cloud), and the ClickPipe were all running in the same region (`us-east-2`).
141-
- The ClickPipe was configured with a single replica.
142-
- The sample data included nested data with a mix of UUID, String, and Int datatypes.
143-
- Other datatypes, such as Floats, Decimals, and DateTime types may be less performant.
144-
- There was no appreciable difference in performance between compressed and uncompressed data.
145-
146-
| Replica Size | Message Size | Data Format | Throughput |
147-
|---------------|-----------|-------------|-----------|
148-
| 1vcpu/4gb | 1.6kb | JSON | 63mb/s |
149-
| 1vcpu/4gb | 1.6kb | AVRO | 99mb/s |
142+
- The ClickPipe was configured with a single L-sized replica (4 GiB of RAM and 1 vCPU).
143+
- The sample data included nested data with a mix of `UUID`, `String`, and `Int` datatypes. Other datatypes, such as `Float`, `Decimal`, and `DateTime`, may be less performant.
144+
- There was no appreciable difference in performance using compressed and uncompressed data.
145+
146+
| Replica Size | Message Size | Data Format | Throughput |
147+
|---------------|--------------|-------------|------------|
148+
| Large (L) | 1.6kb | JSON | 63mb/s |
149+
| Large (L) | 1.6kb | Avro | 99mb/s |

docs/integrations/data-ingestion/clickpipes/kafka/05_faq.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,15 @@ No, the ClickPipes for Kafka is designed for reading data from Kafka topics, not
5555
Yes, if the brokers are part of the same quorum they can be configured together delimited with `,`.
5656
</details>
5757

58+
<details>
59+
60+
<summary>Can ClickPipes replicas be scaled?</summary>
61+
62+
Yes, ClickPipes for streaming can be scaled both horizontally and vertically.
63+
Horizontal scaling adds more replicas to increase throughput, while vertical scaling increases the resources (CPU and RAM) allocated to each replica to handle more intensive workloads.
64+
This can be configured during ClickPipe creation, or at any other point under **Settings** -> **Advanced Settings** -> **Scaling**.
65+
</details>
66+
5867
### Upstash {#upstash}
5968

6069
<details>

docs/integrations/data-ingestion/clickpipes/kinesis.md

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -161,10 +161,9 @@ If you have specific low-latency requirements, please [contact us](https://click
161161

162162
### Scaling {#scaling}
163163

164-
ClickPipes for Kinesis is designed to scale horizontally. By default, we create a consumer group with one consumer.
165-
This can be changed with the scaling controls in the ClickPipe details view.
164+
ClickPipes for Kinesis is designed to scale both horizontally and vertically. By default, we create a consumer group with one consumer. This can be configured during ClickPipe creation, or at any other point under **Settings** -> **Advanced Settings** -> **Scaling**.
166165

167-
ClickPipes provides a high-availability with an availability zone distributed architecture.
166+
ClickPipes provides high-availability with an availability zone distributed architecture.
168167
This requires scaling to at least two consumers.
169168

170169
Regardless number of running consumers, fault tolerance is available by design.

0 commit comments

Comments
 (0)