Skip to content

Commit d2a998b

Browse files
committed
[GR-59732] Make ProgressReporter Top 10 printing strictly package-based.
PullRequest: graal/21792
2 parents 4ed66aa + e60154e commit d2a998b

File tree

7 files changed

+316
-202
lines changed

7 files changed

+316
-202
lines changed

docs/reference-manual/native-image/BuildOutput.md

Lines changed: 36 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -23,58 +23,60 @@ Below is the example output when building a native executable of the `HelloWorld
2323
================================================================================
2424
GraalVM Native Image: Generating 'helloworld' (executable)...
2525
================================================================================
26-
[1/8] Initializing... (2.8s @ 0.15GB)
27-
Java version: 25+13, vendor version: GraalVM CE 25-dev+13.1
26+
[1/8] Initializing... (2.0s @ 0.19GB)
27+
Java version: 26+9, vendor version: GraalVM CE 26-dev+9.1
2828
Graal compiler: optimization level: 2, target machine: x86-64-v3
29-
C compiler: gcc (linux, x86_64, 12.2.0)
29+
C compiler: gcc (linux, x86_64, 15.2.1)
3030
Garbage collector: Serial GC (max heap size: 80% of RAM)
3131
--------------------------------------------------------------------------------
32-
Build resources:
33-
- 13.24GB of memory (42.7% of system memory, using available memory)
34-
- 16 thread(s) (100.0% of 16 available processor(s), determined at start)
35-
[2/8] Performing analysis... [****] (4.5s @ 0.54GB)
36-
3,158 types, 3,625 fields, and 14,804 methods found reachable
37-
1,012 types, 36 fields, and 377 methods registered for reflection
38-
57 types, 57 fields, and 52 methods registered for JNI access
32+
Build resources:
33+
- 28.45GB of memory (42.5% of system memory, using available memory)
34+
- 20 thread(s) (100.0% of 20 available processor(s), determined at start)
35+
[2/8] Performing analysis... [******] (3.4s @ 0.40GB)
36+
3,297 types, 3,733 fields, and 15,247 methods found reachable
37+
1,066 types, 36 fields, and 415 methods registered for reflection
38+
58 types, 59 fields, and 52 methods registered for JNI access
3939
0 downcalls and 0 upcalls registered for foreign access
4040
4 native libraries: dl, pthread, rt, z
41-
[3/8] Building universe... (0.8s @ 0.99GB)
42-
[4/8] Parsing methods... [*] (0.6s @ 0.75GB)
43-
[5/8] Inlining methods... [***] (0.3s @ 0.32GB)
44-
[6/8] Compiling methods... [**] (3.7s @ 0.60GB)
45-
[7/8] Laying out methods... [*] (0.8s @ 0.83GB)
46-
[8/8] Creating image... [**] (3.1s @ 0.58GB)
47-
5.32MB (24.22%) for code area: 8,702 compilation units
48-
7.03MB (32.02%) for image heap: 93,301 objects and 5 resources
49-
8.96MB (40.83%) for debug info generated in 1.0s
50-
659.13kB ( 2.93%) for other data
51-
21.96MB in total image size, 21.04MB in total file size
41+
[3/8] Building universe... (1.0s @ 0.60GB)
42+
[4/8] Parsing methods... [*] (0.4s @ 0.62GB)
43+
[5/8] Inlining methods... [****] (0.2s @ 0.59GB)
44+
[6/8] Compiling methods... [**] (3.7s @ 0.66GB)
45+
[7/8] Laying out methods... [*] (0.7s @ 0.60GB)
46+
[8/8] Creating image... [**] (2.3s @ 0.65GB)
47+
5.24MB (21.86%) for code area: 8,788 compilation units
48+
7.67MB (32.01%) for image heap: 90,323 objects and 55 resources
49+
9.43MB (39.34%) for debug info generated in 0.3s
50+
11.05MB (46.13%) for other data
51+
23.96MB in total image size, 13.31MB in total file size
5252
--------------------------------------------------------------------------------
5353
Top 10 origins of code area: Top 10 object types in image heap:
54-
4.03MB java.base 1.14MB byte[] for code metadata
55-
927.05kB svm.jar (Native Image) 927.31kB java.lang.String
56-
111.71kB java.logging 839.68kB byte[] for general heap data
57-
63.38kB org.graalvm.nativeimage.base 736.91kB java.lang.Class
58-
47.59kB jdk.proxy1 713.13kB byte[] for java.lang.String
59-
35.85kB jdk.proxy3 272.85kB c.o.s.c.h.DynamicHubCompanion
60-
27.06kB jdk.internal.vm.ci 250.83kB java.util.HashMap$Node
61-
23.44kB org.graalvm.sdk 196.52kB java.lang.Object[]
62-
11.42kB jdk.proxy2 182.77kB java.lang.String[]
63-
8.07kB jdk.graal.compiler 154.26kB byte[] for embedded resources
64-
1.39kB for 2 more packages 1.38MB for 884 more object types
54+
791.32kB java.base/java.util 1.41MB byte[] for code metadata
55+
363.66kB java.base/java.lang 1.21MB byte[] for string data
56+
323.39kB java.base/java.text 838.53kB java.base/java.lang.String
57+
241.87kB java.base/java.util.stream 633.02kB o.g.n.~e/c.o.s.c.h.Dyna~anion
58+
229.23kB java.base/java.util.regex 431.58kB heap alignment
59+
214.23kB java.base/java.util.concurrent 428.26kB java.base/java.lang.Class
60+
166.60kB o.g.n.~e/c.o.svm.core.code 323.23kB java.base/j.util.HashMap$Node
61+
153.78kB java.base/java.time.format 284.47kB byte[] for general heap data
62+
152.90kB java.base/java.math 232.06kB java.base/java.lang.Object[]
63+
142.02kB o.g.n.~e/c.o.s.c.genscavenge 183.10kB java.base/j.u.HashMap$Node[]
64+
2.32MB for 146 more packages 1.70MB for 966 more object types
6565
--------------------------------------------------------------------------------
6666
Recommendations:
67+
FUTR: Use '--future-defaults=all' to prepare for future releases.
6768
HEAP: Set max heap for improved and more predictable memory usage.
6869
CPU: Enable more CPU features with '-march=native' for improved performance.
6970
--------------------------------------------------------------------------------
70-
0.8s (4.6% of total time) in 35 GCs | Peak RSS: 1.93GB | CPU load: 9.61
71+
0.9s (6.1% of total time) in 54 GCs | Peak RSS: 1.82GB | CPU load: 13.25
7172
--------------------------------------------------------------------------------
7273
Build artifacts:
74+
/home/janedoe/helloworld/gdb-debughelpers.py (debug_info)
7375
/home/janedoe/helloworld/helloworld (executable)
7476
/home/janedoe/helloworld/helloworld.debug (debug_info)
7577
/home/janedoe/helloworld/sources (debug_info)
7678
================================================================================
77-
Finished generating 'helloworld' in 17.0s.
79+
Finished generating 'helloworld' in 14.2s.
7880
```
7981

8082
## Build Stages

substratevm/src/com.oracle.svm.core/src/com/oracle/svm/core/SubstrateOptions.java

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -809,6 +809,9 @@ public static boolean hasColorsEnabled(OptionValues values) {
809809
@Option(help = "Print GC warnings as part of build output", type = OptionType.User)//
810810
public static final HostedOptionKey<Boolean> BuildOutputGCWarnings = new HostedOptionKey<>(true);
811811

812+
@Option(help = "Write code breakdown information into CSV file", type = OptionType.User)//
813+
public static final HostedOptionKey<Boolean> BuildOutputCodeBreakdownFile = new HostedOptionKey<>(false);
814+
812815
@BundleMember(role = BundleMember.Role.Output)//
813816
@Option(help = "Print build output statistics as JSON to the specified file. " +
814817
"The output conforms to the JSON schema located at: " +

substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/CodeBreakdownProvider.java

Lines changed: 8 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -24,40 +24,22 @@
2424
*/
2525
package com.oracle.svm.hosted;
2626

27-
import java.security.CodeSource;
2827
import java.util.Collection;
29-
import java.util.Collections;
30-
import java.util.HashMap;
3128
import java.util.Map;
29+
import java.util.stream.Collectors;
3230

3331
import org.graalvm.nativeimage.ImageSingletons;
3432

3533
import com.oracle.svm.hosted.code.CompileQueue.CompileTask;
36-
import com.oracle.svm.hosted.meta.HostedMethod;
3734

3835
class CodeBreakdownProvider {
39-
private Map<String, Long> codeBreakdown;
36+
private Map<Class<?>, Long> codeBreakdown;
4037

4138
CodeBreakdownProvider(Collection<CompileTask> compilationTasks) {
42-
Map<String, Long> nameToSizeMap = new HashMap<>();
43-
for (CompileTask task : compilationTasks) {
44-
String key = null;
45-
Class<?> javaClass = task.method.getDeclaringClass().getJavaClass();
46-
Module module = javaClass.getModule();
47-
if (module.isNamed()) {
48-
key = module.getName();
49-
if ("org.graalvm.nativeimage.builder".equals(key)) {
50-
key = "svm.jar (Native Image)";
51-
}
52-
} else {
53-
key = findJARFile(javaClass);
54-
if (key == null) {
55-
key = findPackageOrClassName(task.method);
56-
}
57-
}
58-
nameToSizeMap.merge(key, (long) task.result.getTargetCodeSize(), Long::sum);
59-
}
60-
codeBreakdown = Collections.unmodifiableMap(nameToSizeMap);
39+
codeBreakdown = Map.copyOf(compilationTasks.stream().collect(
40+
Collectors.groupingBy(
41+
compileTask -> compileTask.method.getDeclaringClass().getJavaClass(),
42+
Collectors.summingLong(compileTask -> compileTask.result.getTargetCodeSize()))));
6143
}
6244

6345
/**
@@ -66,31 +48,10 @@ class CodeBreakdownProvider {
6648
*
6749
* @return the code breakdown
6850
*/
69-
public static Map<String, Long> getAndClear() {
51+
public static Map<Class<?>, Long> getAndClear() {
7052
CodeBreakdownProvider singleton = ImageSingletons.lookup(CodeBreakdownProvider.class);
71-
Map<String, Long> map = singleton.codeBreakdown;
53+
var map = singleton.codeBreakdown;
7254
singleton.codeBreakdown = null;
7355
return map;
7456
}
75-
76-
private static String findJARFile(Class<?> javaClass) {
77-
CodeSource codeSource = javaClass.getProtectionDomain().getCodeSource();
78-
if (codeSource != null && codeSource.getLocation() != null) {
79-
String path = codeSource.getLocation().getPath();
80-
if (path.endsWith(".jar")) {
81-
// Use String API to determine basename of path to handle both / and \.
82-
return path.substring(Math.max(path.lastIndexOf('/') + 1, path.lastIndexOf('\\') + 1));
83-
}
84-
}
85-
return null;
86-
}
87-
88-
private static String findPackageOrClassName(HostedMethod method) {
89-
String qualifier = method.format("%H");
90-
int lastDotIndex = qualifier.lastIndexOf('.');
91-
if (lastDotIndex > 0) {
92-
qualifier = qualifier.substring(0, lastDotIndex);
93-
}
94-
return qualifier;
95-
}
9657
}

substratevm/src/com.oracle.svm.hosted/src/com/oracle/svm/hosted/HeapBreakdownProvider.java

Lines changed: 51 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ protected void calculate(BeforeImageWriteAccessImpl access, boolean resourcesAre
114114
}
115115
long objectSize = o.getSize();
116116
totalObjectSize += objectSize;
117-
classToDataMap.computeIfAbsent(o.getClazz(), c -> new HeapBreakdownEntry(c)).add(objectSize);
117+
classToDataMap.computeIfAbsent(o.getClazz(), HeapBreakdownEntry::of).add(objectSize);
118118
if (reportStringBytesConstant && o.getObject() instanceof String string) {
119119
byte[] bytes = getInternalByteArray(string);
120120
/* Ensure every byte[] is counted only once. */
@@ -140,24 +140,24 @@ protected void calculate(BeforeImageWriteAccessImpl access, boolean resourcesAre
140140
long heapAlignmentSize = getTotalHeapSize() - totalObjectSize;
141141
assert heapAlignmentSize >= 0 : "Incorrect heap alignment detected: " + heapAlignmentSize;
142142
if (heapAlignmentSize > 0) {
143-
HeapBreakdownEntry heapAlignmentEntry = new HeapBreakdownEntry("", "heap alignment", "#glossary-heap-alignment");
143+
HeapBreakdownEntry heapAlignmentEntry = HeapBreakdownEntry.of("", "heap alignment", "#glossary-heap-alignment");
144144
heapAlignmentEntry.add(heapAlignmentSize);
145145
entries.add(heapAlignmentEntry);
146146
}
147147

148148
/* Extract byte[] for Strings. */
149149
if (stringByteArrayTotalSize > 0) {
150-
addEntry(entries, byteArrayEntry, new HeapBreakdownEntry(BYTE_ARRAY_PREFIX + "java.lang.String"), stringByteArrayTotalSize, stringByteArrayTotalCount);
150+
addEntry(entries, byteArrayEntry, HeapBreakdownEntry.of(BYTE_ARRAY_PREFIX + "string data"), stringByteArrayTotalSize, stringByteArrayTotalCount);
151151
}
152152
/* Extract byte[] for code info. */
153153
List<Integer> codeInfoByteArrayLengths = CodeInfoTable.getCurrentLayerImageCodeCache().getTotalByteArrayLengths();
154154
long codeInfoSize = codeInfoByteArrayLengths.stream().map(l -> objectLayout.getArraySize(JavaKind.Byte, l, true)).reduce(0L, Long::sum);
155-
addEntry(entries, byteArrayEntry, new HeapBreakdownEntry(BYTE_ARRAY_PREFIX, "code metadata", "#glossary-code-metadata"), codeInfoSize, codeInfoByteArrayLengths.size());
155+
addEntry(entries, byteArrayEntry, HeapBreakdownEntry.of(BYTE_ARRAY_PREFIX, "code metadata", "#glossary-code-metadata"), codeInfoSize, codeInfoByteArrayLengths.size());
156156
/* Extract byte[] for metadata. */
157157
int metadataByteLength = ImageSingletons.lookup(RuntimeMetadataDecoder.class).getMetadataByteLength();
158158
if (metadataByteLength > 0) {
159159
long metadataSize = objectLayout.getArraySize(JavaKind.Byte, metadataByteLength, true);
160-
addEntry(entries, byteArrayEntry, new HeapBreakdownEntry(BYTE_ARRAY_PREFIX, "reflection metadata", "#glossary-reflection-metadata"), metadataSize, 1);
160+
addEntry(entries, byteArrayEntry, HeapBreakdownEntry.of(BYTE_ARRAY_PREFIX, "reflection metadata", "#glossary-reflection-metadata"), metadataSize, 1);
161161
}
162162
ProgressReporter reporter = ProgressReporter.singleton();
163163
long resourcesByteArraySize = 0;
@@ -177,19 +177,19 @@ protected void calculate(BeforeImageWriteAccessImpl access, boolean resourcesAre
177177
}
178178
}
179179
if (resourcesByteArraySize > 0) {
180-
addEntry(entries, byteArrayEntry, new HeapBreakdownEntry(BYTE_ARRAY_PREFIX, "embedded resources", "#glossary-embedded-resources"), resourcesByteArraySize, resourcesByteArrayCount);
180+
addEntry(entries, byteArrayEntry, HeapBreakdownEntry.of(BYTE_ARRAY_PREFIX, "embedded resources", "#glossary-embedded-resources"), resourcesByteArraySize, resourcesByteArrayCount);
181181
}
182182
}
183183
reporter.recordJsonMetric(ImageDetailKey.RESOURCE_SIZE_BYTES, resourcesByteArraySize);
184184
/* Extract byte[] for graph encodings. */
185185
if (graphEncodingByteLength >= 0) {
186186
long graphEncodingSize = objectLayout.getArraySize(JavaKind.Byte, graphEncodingByteLength, true);
187187
reporter.recordJsonMetric(ImageDetailKey.GRAPH_ENCODING_SIZE, graphEncodingSize);
188-
addEntry(entries, byteArrayEntry, new HeapBreakdownEntry(BYTE_ARRAY_PREFIX, "graph encodings", "#glossary-graph-encodings"), graphEncodingSize, 1);
188+
addEntry(entries, byteArrayEntry, HeapBreakdownEntry.of(BYTE_ARRAY_PREFIX, "graph encodings", "#glossary-graph-encodings"), graphEncodingSize, 1);
189189
}
190190
/* Add remaining byte[]. */
191191
assert byteArrayEntry.byteSize >= 0 && byteArrayEntry.count >= 0;
192-
addEntry(entries, byteArrayEntry, new HeapBreakdownEntry(BYTE_ARRAY_PREFIX, "general heap data", "#glossary-general-heap-data"), byteArrayEntry.byteSize, byteArrayEntry.count);
192+
addEntry(entries, byteArrayEntry, HeapBreakdownEntry.of(BYTE_ARRAY_PREFIX, "general heap data", "#glossary-general-heap-data"), byteArrayEntry.byteSize, byteArrayEntry.count);
193193
assert byteArrayEntry.byteSize == 0 && byteArrayEntry.count == 0;
194194
setBreakdownEntries(entries);
195195
}
@@ -209,26 +209,23 @@ private static byte[] getInternalByteArray(String string) {
209209
}
210210
}
211211

212-
public static class HeapBreakdownEntry {
213-
final HeapBreakdownLabel label;
212+
public abstract static class HeapBreakdownEntry {
214213
long byteSize;
215214
int count;
216215

217-
public HeapBreakdownEntry(HostedClass hostedClass) {
218-
this(hostedClass.toJavaName(true));
216+
public static HeapBreakdownEntry of(HostedClass hostedClass) {
217+
return new HeapBreakdownEntryForClass(hostedClass.getJavaClass());
219218
}
220219

221-
public HeapBreakdownEntry(String name) {
222-
label = new SimpleHeapObjectKindName(name);
220+
public static HeapBreakdownEntry of(String name) {
221+
return new HeapBreakdownEntryFixed(new SimpleHeapObjectKindName(name));
223222
}
224223

225-
HeapBreakdownEntry(String prefix, String name, String htmlAnchor) {
226-
label = new LinkyHeapObjectKindName(prefix, name, htmlAnchor);
224+
public static HeapBreakdownEntry of(String prefix, String name, String htmlAnchor) {
225+
return new HeapBreakdownEntryFixed(new LinkyHeapObjectKindName(prefix, name, htmlAnchor));
227226
}
228227

229-
public HeapBreakdownLabel getLabel() {
230-
return label;
231-
}
228+
public abstract HeapBreakdownLabel getLabel(int maxLength);
232229

233230
public long getByteSize() {
234231
return byteSize;
@@ -253,6 +250,41 @@ void remove(long subByteSize, int subCount) {
253250
}
254251
}
255252

253+
static class HeapBreakdownEntryFixed extends HeapBreakdownEntry {
254+
255+
private final HeapBreakdownLabel label;
256+
257+
HeapBreakdownEntryFixed(HeapBreakdownLabel label) {
258+
this.label = label;
259+
}
260+
261+
@Override
262+
public HeapBreakdownLabel getLabel(int unused) {
263+
return label;
264+
}
265+
}
266+
267+
static class HeapBreakdownEntryForClass extends HeapBreakdownEntry {
268+
269+
private final Class<?> clazz;
270+
271+
HeapBreakdownEntryForClass(Class<?> clazz) {
272+
this.clazz = clazz;
273+
}
274+
275+
@Override
276+
public HeapBreakdownLabel getLabel(int maxLength) {
277+
if (maxLength >= 0) {
278+
String moduleNamePrefix = ProgressReporterUtils.moduleNamePrefix(clazz.getModule());
279+
int maxLengthClassName = maxLength - moduleNamePrefix.length();
280+
String truncatedClassName = ProgressReporterUtils.truncateFQN(clazz.getTypeName(), maxLengthClassName);
281+
return new SimpleHeapObjectKindName(moduleNamePrefix + truncatedClassName);
282+
} else {
283+
return new SimpleHeapObjectKindName(clazz.getTypeName());
284+
}
285+
}
286+
}
287+
256288
public interface HeapBreakdownLabel {
257289
String renderToString(LinkStrategy linkStrategy);
258290
}

0 commit comments

Comments
 (0)