Skip to content

MINOR: Add 4.0.1 docs #719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: asf-site
Choose a base branch
from
Open

MINOR: Add 4.0.1 docs #719

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
82584e1
Add 4.0.1 docs
clolov Sep 11, 2025
f5d4e4a
Address review #1 comments
clolov Sep 30, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion 40/generated/connect_rest.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ info:
name: Apache 2.0
url: https://www.apache.org/licenses/LICENSE-2.0.html
title: Kafka Connect REST API
version: 4.0.0
version: 4.0.1
paths:
/:
get:
Expand Down
3,372 changes: 3,263 additions & 109 deletions 40/generated/protocol_messages.html
View file
Open in desktop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't expected changes here but that looks correct. Not sure what happened with protocol_messages.html for 4.0 but it looks like it's currently wrong.

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion 40/generated/topic_config.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,7 @@ <h4><a id="remote.log.delete.on.disable"></a><a id="topicconfigs_remote.log.dele
</li>
<li>
<h4><a id="remote.storage.enable"></a><a id="topicconfigs_remote.storage.enable" href="#topicconfigs_remote.storage.enable">remote.storage.enable</a></h4>
<p>To enable tiered storage for a topic, set this configuration as true. You can not disable this config once it is enabled. It will be provided in future versions.</p>
<p>To enable tiered storage for a topic, set this configuration to true. To disable tiered storage for a topic that has it enabled, set this configuration to false. When disabling, you must also set <code>remote.log.delete.on.disable</code> to true.</p>
<table><tbody>
<tr><th>Type:</th><td>boolean</td></tr>
<tr><th>Default:</th><td>false</td></tr>
Expand Down
2 changes: 1 addition & 1 deletion 40/js/templateData.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@ limitations under the License.
var context={
"version": "40",
"dotVersion": "4.0",
"fullDotVersion": "4.0.0",
"fullDotVersion": "4.0.1",
"scalaVersion": "2.13"
};
34 changes: 19 additions & 15 deletions 40/streams/developer-guide/config-streams.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -296,12 +296,12 @@ <h4><a class="toc-backref" href="#id45">num.standby.replicas</a><a class="header
<tr class="row-even"><td>commit.interval.ms</td>
<td>Low</td>
<td colspan="2">The frequency in milliseconds with which to save the position (offsets in source topics) of tasks.</td>
<td><code class="docutils literal"><span class="pre">30000</span></code> (30 seconds)</td>
<td><code class="docutils literal"><span class="pre">30000</span></code> (30 seconds) (at-least-once) / <code class="docutils literal"><span class="pre">100</span></code> (exactly-once)</td>
</tr>
<tr class="row-odd"><td>default.deserialization.exception.handler (Deprecated. Use deserialization.exception.handler instead.)</td>
<td>Medium</td>
<td colspan="2">Exception handling class that implements the <code class="docutils literal"><span class="pre">DeserializationExceptionHandler</span></code> interface.</td>
<td><code class="docutils literal"><span class="pre">LogAndContinueExceptionHandler</span></code></td>
<td><code class="docutils literal"><span class="pre">LogAndFailExceptionHandler</span></code></td>
</tr>
<tr class="row-even"><td>default.key.serde</td>
<td>Medium</td>
Expand All @@ -326,11 +326,10 @@ <h4><a class="toc-backref" href="#id45">num.standby.replicas</a><a class="header
set by the user or all serdes must be passed in explicitly (see also default.key.serde).</td>
<td><code class="docutils literal"><span class="pre">null</span></code></td>
</tr>
<tr class="row-even"><td>default.dsl.store</td>
<tr class="row-even"><td>default.dsl.store (Deprecated. Use dsl.store.suppliers.class instead.)</td>
<td>Low</td>
<td colspan="2">
[DEPRECATED] The default state store type used by DSL operators. Deprecated in
favor of <code>dsl.store.suppliers.class</code>
The default state store type used by DSL operators.
</td>
<td><code class="docutils literal"><span class="pre">"ROCKS_DB"</span></code></td>
</tr>
Expand Down Expand Up @@ -481,54 +480,59 @@ <h4><a class="toc-backref" href="#id45">num.standby.replicas</a><a class="header
The default of <code>-1</code> (meaning: use broker default replication factor) requires broker version 2.4 or newer.</td>
<td><code class="docutils literal"><span class="pre">-1</span></code></td>
</tr>
<tr class="row-even"><td>retry.backoff.ms</td>
<tr class="row-even"><td>repartition.purge.interval.ms</td>
<td>Low</td>
<td colspan="2">The frequency in milliseconds with which to delete fully consumed records from repartition topics. Purging will occur after at least this value since the last purge, but may be delayed until later.</td>
<td><code class="docutils literal"><span class="pre">30000</span></code> (30 seconds)</td>
</tr>
<tr class="row-odd"><td>retry.backoff.ms</td>
<td>Low</td>
<td colspan="2">The amount of time in milliseconds, before a request is retried.</td>
<td><code class="docutils literal"><span class="pre">100</span></code></td>
</tr>
<tr class="row-odd"><td>rocksdb.config.setter</td>
<tr class="row-even"><td>rocksdb.config.setter</td>
<td>Medium</td>
<td colspan="2">The RocksDB configuration.</td>
<td><code class="docutils literal"><span class="pre">null</span></code></td>
</tr>
<tr class="row-even"><td>state.cleanup.delay.ms</td>
<tr class="row-odd"><td>state.cleanup.delay.ms</td>
<td>Low</td>
<td colspan="2">The amount of time in milliseconds to wait before deleting state when a partition has migrated.</td>
<td><code class="docutils literal"><span class="pre">600000</span></code></td> (10 minutes)</td>
</tr>
<tr class="row-odd"><td>state.dir</td>
<tr class="row-even"><td>state.dir</td>
<td>High</td>
<td colspan="2">Directory location for state stores.</td>
<td><code class="docutils literal"><span class="pre">/${java.io.tmpdir}/kafka-streams</span></code></td>
</tr>
<tr class="row-even"><td>task.assignor.class</td>
<tr class="row-odd"><td>task.assignor.class</td>
<td>Medium</td>
<td colspan="2">A task assignor class or class name implementing the <code>TaskAssignor</code> interface.</td>
<td>The high-availability task assignor.</td>
</tr>
<tr class="row-odd"><td>task.timeout.ms</td>
<tr class="row-even"><td>task.timeout.ms</td>
<td>Medium</td>
<td colspan="2">The maximum amount of time in milliseconds a task might stall due to internal errors and retries until an error is raised. For a timeout of <code>0 ms</code>, a task would raise an error for the first internal error. For any timeout larger than <code>0 ms</code>, a task will retry at least once before an error is raised.</td>
<td><code class="docutils literal"><span class="pre">300000</span></code></td> (5 minutes)</td>
</tr>
<tr class="row-even"><td>topology.optimization</td>
<tr class="row-odd"><td>topology.optimization</td>
<td>Medium</td>
<td colspan="2">A configuration telling Kafka Streams if it should optimize the topology and what optimizations to apply. Acceptable values are: <code>StreamsConfig.NO_OPTIMIZATION</code> (<code>none</code>), <code>StreamsConfig.OPTIMIZE</code> (<code>all</code>) or a comma separated list of specific optimizations: <code>StreamsConfig.REUSE_KTABLE_SOURCE_TOPICS</code> (<code>reuse.ktable.source.topics</code>), <code>StreamsConfig.MERGE_REPARTITION_TOPICS</code> (<code>merge.repartition.topics</code>),
<code>StreamsConfig.SINGLE_STORE_SELF_JOIN</code> (<code>single.store.self.join</code>). </td>
<td><code class="docutils literal"><span class="pre">"NO_OPTIMIZATION"</span></code></td>
</tr>
<tr class="row-odd"><td>upgrade.from</td>
<tr class="row-even"><td>upgrade.from</td>
<td>Medium</td>
<td colspan="2">The version you are upgrading from during a rolling upgrade.
See <a class="reference internal" href="#streams-developer-guide-upgrade-from"><span class="std std-ref">Upgrade From</span></a></td>
<td><code class="docutils literal"><span class="pre">null</span></code></td>
</tr>
<tr class="row-even"><td>windowstore.changelog.additional.retention.ms</td>
<tr class="row-odd"><td>windowstore.changelog.additional.retention.ms</td>
<td>Low</td>
<td colspan="2">Added to a windows maintainMs to ensure data is not deleted from the log prematurely. Allows for clock drift.</td>
<td><code class="docutils literal"><span class="pre">86400000</span></code></td> (1 day)</td>
</tr>
<tr class="row-odd"><td>window.size.ms</td>
<tr class="row-even"><td>window.size.ms</td>
<td>Low</td>
<td colspan="2">Sets window size for the deserializer in order to calculate window end times.</td>
<td><code class="docutils literal"><span class="pre">null</span></code></td>
Expand Down
4 changes: 2 additions & 2 deletions 40/streams/developer-guide/dsl-api.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -764,10 +764,10 @@ <h4 class="anchor-heading"><a id="streams_concepts_globalktable" class="anchor-l
<td>
<p class="first">Manually trigger repartitioning of the stream with desired number of partitions. (<a class="reference external" href="/{{version}}/javadoc/org/apache/kafka/streams/kstream/KStream.html#repartition--">details</a>)</p>

<code><span class="pre">repartition()</span></code> is similar to <code><span class="pre">through()</span></code> however Kafka Streams will manage the topic for you.
Kafka Streams will manage the topic for <code><span class="pre">repartition()</span></code>.
Generated topic is treated as internal topic, as a result data will be purged automatically as any other internal repartition topic.
In addition, you can specify the desired number of partitions, which allows to easily scale in/out downstream sub-topologies.
<code><span class="pre">repartition()</span></code> operation always triggers repartitioning of the stream, as a result it can be used with embedded Processor API methods (like <code><span class="pre">transform()</span></code> et al.) that do not trigger auto repartitioning when key changing operation is performed beforehand.
<code><span class="pre">repartition()</span></code> operation always triggers repartitioning of the stream, as a result it can be used with embedded Processor API methods (like <code><span class="pre">process()</span></code> et al.) that do not trigger auto repartitioning when key changing operation is performed beforehand.

<pre class="line-numbers"><code class="language-java">KStream&lt;byte[], String&gt; stream = ... ;
KStream&lt;byte[], String&gt; repartitionedStream = stream.repartition(Repartitioned.numberOfPartitions(10));</code></pre>
Expand Down
1 change: 1 addition & 0 deletions 40/toc.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,7 @@
<li><a href="#monitoring">6.7 Monitoring</a>
<ul>
<li><a href="#remote_jmx">Security Considerations for Remote Monitoring using JMX</a>
<li><a href="#group_coordinator_monitoring">Group Coordinator Monitoring</a>
<li><a href="#tiered_storage_monitoring">Tiered Storage Monitoring</a>
<li><a href="#kraft_monitoring">KRaft Monitoring</a>
<li><a href="#selector_monitoring">Selector Monitoring</a>
Expand Down
29 changes: 22 additions & 7 deletions 40/upgrade.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@

<script id="upgrade-template" type="text/x-handlebars-template">

<h4><a id="upgrade_4_0_0" href="#upgrade_4_0_0">Upgrading to 4.0.0</a></h4>
<h4><a id="upgrade_4_0_1" href="#upgrade_4_0_1">Upgrading to 4.0.1</a></h4>

<h5><a id="upgrade_clients_4_0_0" href="#upgrade_clients_4_0_0">Upgrading Clients to 4.0.0</a></h5>
<h5><a id="upgrade_clients_4_0_1" href="#upgrade_clients_4_0_1">Upgrading Clients to 4.0.1</a></h5>

<p><b>For a rolling upgrade:</b></p>

Expand All @@ -32,7 +32,7 @@ <h5><a id="upgrade_clients_4_0_0" href="#upgrade_clients_4_0_0">Upgrading Client
or <a href="https://cwiki.apache.org/confluence/x/y4kgF">KIP-1124</a>.</li>
</ol>

<h5><a id="upgrade_servers_4_0_0" href="#upgrade_servers_4_0_0">Upgrading Servers to 4.0.0 from any version 3.3.x through 3.9.x</a></h5>
<h5><a id="upgrade_servers_4_0_1" href="#upgrade_servers_4_0_1">Upgrading Servers to 4.0.1 from any version 3.3.x through 3.9.x</a></h5>

<p>Note: Apache Kafka 4.0 only supports KRaft mode - ZooKeeper mode has been removed. As such, <b>broker upgrades to 4.0.0 (and higher) require KRaft mode and
the software and metadata versions must be at least 3.3.x</b> (the first version when KRaft mode was deemed production ready). For clusters in KRaft mode
Expand All @@ -56,7 +56,18 @@ <h5><a id="upgrade_servers_4_0_0" href="#upgrade_servers_4_0_0">Upgrading Server
has a boolean parameter that indicates if there are metadata changes (i.e. <code>IBP_4_0_IV1(23, "4.0", "IV1", true)</code> means this version has metadata changes).
Given your current and target versions, a downgrade is only possible if there are no metadata changes in the versions between.</li>
</ol>

<h5><a id="upgrade_servers_401_notable" href="#upgrade_servers_401_notable">Notable changes in 4.0.1</a></h5>
<ul>
<li>
The filename for rotated <code>state-change.log</code> files has been updated from <code>stage-change.log.[date]</code> to <code>state-change.log.[date]</code> in the log4j2.yaml configuration file.
See <a href="https://issues.apache.org/jira/browse/KAFKA-19576">KAFKA-19576</a> for details.
</li>
<li>
Kafka Streams include a critical fix to upgrade from <code>KStreams#transformValues()</code> (remove with 4.0.0 release)
to <code>KStreams#processValues()</code>.
For more details, see the <a href="/{{version}}/documentation/streams/developer-guide/dsl-api.html#transformers-removal-and-migration-to-processors">migration guide</a>.
</li>
</ul>
<h5><a id="upgrade_servers_400_notable" href="#upgrade_servers_400_notable">Notable changes in 4.0.0</a></h5>
<ul>
<li>
Expand All @@ -68,7 +79,7 @@ <h5><a id="upgrade_servers_400_notable" href="#upgrade_servers_400_notable">Nota
</li>
<li>
Apache Kafka 4.0 only supports KRaft mode - ZooKeeper mode has been removed. About version upgrade,
check <a href="/{{version}}/documentation.html#upgrade_4_0_0">Upgrading to 4.0.0 from any version 3.3.x through 3.9.x</a> for more info.
check <a href="/{{version}}/documentation.html#upgrade_4_0_1">Upgrading to 4.0.1 from any version 3.3.x through 3.9.x</a> for more info.
</li>
<li>
Apache Kafka 4.0 ships with a brand-new group coordinator implementation (See <a href="https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=217387038#KIP848:TheNextGenerationoftheConsumerRebalanceProtocol-GroupCoordinator">here</a>).
Expand Down Expand Up @@ -336,10 +347,10 @@ <h5><a id="upgrade_servers_400_notable" href="#upgrade_servers_400_notable">Nota
<ul>
<li>
All public API, deprecated in Apache Kafka 3.6 or an earlier release, have been removed, with the exception of <code>JoinWindows.of()</code> and <code>JoinWindows#grace()</code>.
See <a href="https://issues.apache.org/jira/browse/KAFKA-17531">KAFKA-17531</a> for details.
See <a href="https://issues.apache.org/jira/browse/KAFKA-17531">KAFKA-17531</a> for details.
</li>
<li>
The most important changes are highlighted in the <a href="/{{version}}/documentation/streams/upgrade-guide.html#streams_api_changes_400">Kafka Streams upgrade guide</a>.
The most important changes are highlighted in the <a href="/{{version}}/documentation/streams/upgrade-guide.html#streams_api_changes_400">Kafka Streams upgrade guide</a>.
</li>
<li>
For a full list of changes, see <a href="https://issues.apache.org/jira/browse/KAFKA-12822">KAFKA-12822</a>.
Expand Down Expand Up @@ -416,6 +427,10 @@ <h5><a id="upgrade_servers_400_notable" href="#upgrade_servers_400_notable">Nota
<li> See <a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-890%3A+Transactions+Server-Side+Defense">KIP-890</a> and
<a href="https://cwiki.apache.org/confluence/display/KAFKA/KIP-1050%3A+Consistent+error+handling+for+Transactions">KIP-1050</a> for more details </li>
</ul>
<li>
The filename for rotated <code>state-change.log</code> files incorrectly rotates to <code>stage-change.log.[date]</code> (changing state to stage). This issue is corrected in 4.0.1.
See <a href="https://issues.apache.org/jira/browse/KAFKA-19576">KAFKA-19576</a> for details.
</li>
</ul>
</li>
</ul>
Expand Down