8260555: Change the default TIMEOUT_FACTOR from 4 to 1 #26749

lkorinth · 2025-08-12T17:01:41Z

This changes the timeout factor from 4 to 1. Most of the changes add timeouts to individual test cases so that I am able to run them with a timeout factor of 0.7 (some margin to the checked in factor of one)

In addition to changing the timeout factor, I am also using a library call to parse the timeout factor from the Java properties (I cannot use the library function everywhere as jtreg does not allow me to add @library notations to non test case files).

My approach has been to run all tests, and afterwards updating those that fail due to the timeout factor. The amount of updated test cases is huge, and my strategy has been to quadruple the timeout if I could not directly see that less was needed. In a few places, I have added a bit more timeout so that it will work with the 0.7 timeout factor.

These fixes have been created when I have ploughed through test cases:
JDK-8352719: Add an equals sign to the modules statement
JDK-8352709: Remove bad timing annotations from WhileOpTest.java
JDK-8352074: Test MemoryLeak.java seems not to test what it is supposed to test
CODETOOLS-7903937: JTREG uses timeout factor on socket timeout but not on KEEPALIVE
CODETOOLS-7903961: Make default timeout configurable

After the review, I will update the copyrights.

I have run testing tier1-8. The last time with a timeout factor of 1 instead of 0.7.

I got 4 timing related faults:

runtime/Thread/TestThreadDumpMonitorContention.java
This is probably: https://bugs.openjdk.org/browse/JDK-8361370
sun/tools/jhsdb/BasicLauncherTest.java
I am unsure about this one, it has not failed on my runs before, even with a timeout factor of 0.7, maybe I was unlucky.
gc/stress/TestReclaimStringsLeaksMemory.java
I updated this to 480 seconds, I finish this fairly fast (~14s) on my not very fast computer, but the Macs that fail are old x86-based ones.
sun/security/ssl/X509KeyManager/CertChecking.java
This is a new test that I got on last rebase. I have added a timeout of 480 to it.

In addition to these four tests, I have another one "java/lang/ThreadLocal/MemoryLeak.java" that earlier failed with a timeout factor of 0.7 but did not fail in the last run. I will not update that test case, because the extra time spent is strange and should be looked at. I have created JDK-8352074 on that test case, and I will probably create another bug describing the timeout I got.

From the review of the cancelled "8356171: Increase timeout for test cases as preparation for change of default timeout factor", I have taken a few actions:

in testing(md|html): interpreted mode -> forced compilation mode
in MTTest.java: changed 1200 -> 400 (was 300 to begin with)

I am now re-running tier 1-8.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8260555: Change the default TIMEOUT_FACTOR from 4 to 1 (Enhancement - P4)

Reviewers

Serguei Spitsyn (@sspitsyn - Reviewer) 🔄 Re-review required (review applies to ac47dbdc)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/26749/head:pull/26749
$ git checkout pull/26749

Update a local copy of the PR:
$ git checkout pull/26749
$ git pull https://git.openjdk.org/jdk.git pull/26749/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 26749

View PR using the GUI difftool:
$ git pr show -t 26749

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/26749.diff

Using Webrev

Link to Webrev Comment

bridgekeeper · 2025-08-12T17:02:22Z

👋 Welcome back lkorinth! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2025-08-12T17:03:59Z

@lkorinth This change is no longer ready for integration - check the PR body for details.

openjdk · 2025-08-12T17:05:18Z

@lkorinth The following labels will be automatically applied to this pull request:

build
client
compiler
core-libs
hotspot
i18n
javadoc
net
nio
security
serviceability

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

mlbridge · 2025-08-12T17:09:31Z

Webrevs

01: Full - Incremental (dbe42964)
00: Full (ac47dbdc)

plummercj · 2025-08-12T17:52:02Z

sun/tools/jhsdb/BasicLauncherTest.java
I am unsure about this one, it has not failed on my runs before, even with a timeout factor of 0.7, maybe I was unlucky.

@lkorinth Can you send me a link to the failure?

sendaoYan · 2025-08-13T01:47:43Z

test/hotspot/jtreg/compiler/c1/TestPinnedIntrinsics.java

@@ -25,7 +25,7 @@
 * @test


Should we need to update the copyright year for the touched files

After the review, I will update the copyrights.

It is IMO easier to review big changes without the noise.

sendaoYan · 2025-08-13T01:51:44Z

test/hotspot/jtreg/compiler/c2/TestStressRecompilation.java

@@ -26,7 +26,7 @@
 * @bug 8245801
 * @requires vm.debug
 * @summary Test running with StressRecompilation enabled.
- * @run main/othervm -Xcomp -XX:+IgnoreUnrecognizedVMOptions -XX:+StressRecompilation
+ * @run main/othervm/timeout=480 -Xcomp -XX:+IgnoreUnrecognizedVMOptions -XX:+StressRecompilation


I think the default value(120s) will be enough? On my machine this test use 11.546 senonds to finish.

> grep "elapsed time" tmp/compiler/classUnloading/methodUnloading/TestOverloadCompileQueues.jtr -rn 55:elapsed time (seconds): 0.581 66:elapsed time (seconds): 0.575 116:elapsed time (seconds): 3.088 162:elapsed time (seconds): 0.001 173:elapsed time (seconds): 11.546

I have only (to my knowledge) updated test cases that has timed out for me. We have some not very modern test machines that is slower. That in combination with a debug build, in combination with a timeout factor of 0.7 might have made the test time out. Unfortunately I no longer have the logs for this failure so I can not check if the machine was failing because it was low on memory etc. I still think it is reasonable to keep the old timeout of 480. I have no intuitive feeling for how expensive -XX:+StressRecompilation is.

sspitsyn

I've reviewed the Serviceability related tweaks and I'm okay with them in general.
But I'm curious if you do not see any timeouts with this anymore.

turbanoff · 2025-08-13T09:44:33Z

test/langtools/tools/lib/toolbox/ToolBox.java

-    private static final int RETRY_DELETE_MILLIS = isWindows() ? (int)(500 * timeoutFactor): 0;
-    private static final int MAX_RETRY_DELETE_MILLIS = isWindows() ? (int)(15 * 1000 * timeoutFactor) : 0;
+    private static final int RETRY_DELETE_MILLIS = isWindows() ? 500 : 0;
+    private static final int MAX_RETRY_DELETE_MILLIS = isWindows() ?  60 * 1000 : 0;


Suggested change

private static final int MAX_RETRY_DELETE_MILLIS = isWindows() ? 60 * 1000 : 0;

private static final int MAX_RETRY_DELETE_MILLIS = isWindows() ? 60 * 1000 : 0;

lkorinth · 2025-08-13T13:18:57Z

sun/tools/jhsdb/BasicLauncherTest.java
I am unsure about this one, it has not failed on my runs before, even with a timeout factor of 0.7, maybe I was unlucky.

@lkorinth Can you send me a link to the failure?

I sent it to you on email.

erikj79 · 2025-08-13T13:25:48Z

make/RunTests.gmk

@@ -936,7 +936,7 @@ define SetupRunJtregTestBody
  JTREG_ALL_OPTIONS := $$(JTREG_JAVA_OPTIONS) $$(JTREG_VM_OPTIONS)

  JTREG_AUTO_PROBLEM_LISTS :=
-  JTREG_AUTO_TIMEOUT_FACTOR := 4
+  JTREG_AUTO_TIMEOUT_FACTOR := 1 # IT MAKES NO SENCE TO CHANGE IT. Fix individual test cases that time out instead.


I'm not sure about this comment, but if we keep it, please move it to the line above and break lines as appropriate.

I updated it to "Please reach consensus before changing this. It was not easy changing it to a 1. " I also did not break the comment as it was shorter than line 933 above it. Is it acceptable now?

lkorinth · 2025-08-13T13:50:49Z

I've reviewed the Serviceability related tweaks and I'm okay with them in general. But I'm curious if you do not see any timeouts with this anymore.

I only got the four timeouts described in the description, I got a few other failures as well that was not timeout related. I sent you a link to the test results in an email.

dougxc · 2025-08-14T19:30:30Z

I have run testing tier1-8. The last time with a timeout factor of 1 instead of 0.7.

Would you mind also running tier9 to avoid surprises there.

8260555: Change the default TIMEOUT_FACTOR from 4 to 1

ac47dbd

sendaoYan reviewed Aug 13, 2025

View reviewed changes

sspitsyn approved these changes Aug 13, 2025

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Aug 13, 2025

turbanoff reviewed Aug 13, 2025

View reviewed changes

erikj79 reviewed Aug 13, 2025

View reviewed changes

After suggestions from Erik and Andrey

dbe4296

openjdk bot removed the ready Pull request is ready to be integrated label Aug 13, 2025

	private static final int MAX_RETRY_DELETE_MILLIS = isWindows() ? 60 * 1000 : 0;
	private static final int MAX_RETRY_DELETE_MILLIS = isWindows() ? 60 * 1000 : 0;

8260555: Change the default TIMEOUT_FACTOR from 4 to 1 #26749

Are you sure you want to change the base?

8260555: Change the default TIMEOUT_FACTOR from 4 to 1 #26749

Conversation

lkorinth commented Aug 12, 2025 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented Aug 12, 2025

Uh oh!

openjdk bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openjdk bot commented Aug 12, 2025

Uh oh!

mlbridge bot commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

plummercj commented Aug 12, 2025

Uh oh!

sendaoYan Aug 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lkorinth Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

sendaoYan Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

lkorinth Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

sspitsyn left a comment

Choose a reason for hiding this comment

Uh oh!

turbanoff Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

lkorinth Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

lkorinth commented Aug 13, 2025

Uh oh!

erikj79 Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

lkorinth Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

lkorinth commented Aug 13, 2025

Uh oh!

dougxc commented Aug 14, 2025

Uh oh!

Uh oh!

lkorinth commented Aug 12, 2025 •

edited by openjdk bot

Loading

openjdk bot commented Aug 12, 2025 •

edited

Loading

mlbridge bot commented Aug 12, 2025 •

edited

Loading

sendaoYan Aug 13, 2025 •

edited

Loading