Optimize DialogueFeignClient small response reader

[schlosna]

Before this PR

Small responses <= 8KiB would always allocate 8KiB ByteBuffer as InputStreamReader creates a StreamDecoder that allocates a fixed 8192 byte ByteBuffer. This allocation becomes a scalability bottleneck for high throughput RPCs with small responses (think something returning timestamps, locks, authorization results, etc.)

See https://github.com/openjdk/jdk/blob/4c03e5938df0a9cb10c2379af81163795dd3a086/src/java.base/share/classes/sun/nio/cs/StreamDecoder.java#L248

After this PR

==COMMIT_MSG==
Avoid InputStreamReader / HeapByteBuffer overhead for small (less than 8KiB) inputs, see FasterXML/jackson-core#1081 and FasterXML/jackson-benchmarks#9 (comment) for benchmarks showing between 2x and 10x speedup handling deserialization of small values.

==COMMIT_MSG==

Possible downsides?

[changelog-app]

Generate changelog in changelog/@unreleased

What do the change types mean?

• feature: A new feature of the service.
• improvement: An incremental improvement in the functionality or operation of the service.
• fix: Remedies the incorrect behaviour of a component of the service in a backwards-compatible way.
• break: Has the potential to break consumers of this service's API, inclusive of both Palantir services
  and external consumers of the service's API (e.g. customer-written software or integrations).
• deprecation: Advertises the intention to remove service functionality without any change to the
  operation of the service itself.
• manualTask: Requires the possibility of manual intervention (running a script, eyeballing configuration,
  performing database surgery, ...) at the time of upgrade for it to succeed.
• migration: A fully automatic upgrade migration task with no engineer input required.

Note: only one type should be chosen.

How are new versions calculated?

• ❗The break and manual task changelog types will result in a major release!
• 🐛 The fix changelog type will result in a minor release in most cases, and a patch release version for patch branches. This behaviour is configurable in autorelease.
• ✨ All others will result in a minor version release.

Type

• Feature
• Improvement
• Fix
• Break
• Deprecation
• Manual task
• Migration

Description

Optimize DialogueFeignClient small response reader

Check the box to generate changelog(s)

• Generate changelog entry

[stale]

This PR has been automatically marked as stale because it has not been touched in the last 14 days. If you'd like to keep it open, please leave a comment or add the 'long-lived' label, otherwise it'll be closed in 7 days.

[changelog-app]

✅ Successfully generated changelog entry!

What happened?

Your changelog entries have been stored in the database as part of our migration to ChangelogV3.

Need to regenerate?

Simply interact with the changelog bot comment again to regenerate these entries.

[Copilot]

Pull Request Overview

This PR optimizes the DialogueFeignClient by avoiding InputStreamReader and HeapByteBuffer overhead for small responses (less than 8KiB). The optimization reads the entire response into memory as a string for small payloads, which can be more efficient than stream-based reading for small data.

Key changes:

• Added optimization for small response bodies in the asReader() method
• Updated URL decoding to use StandardCharsets.UTF_8 directly instead of string literal
• Removed unused UnsupportedEncodingException import

[Copilot]

The try-with-resources block closes the InputStream after creating the StringReader, but the StringReader contains the full content and doesn't need the stream to remain open. However, this creates a potential issue if asInputStream() returns the same underlying stream instance that might be needed elsewhere. Consider ensuring this optimization only applies when it's safe to consume the entire stream.

Suggested change

	try (InputStream inputStream = asInputStream()) {
	InputStream inputStream = asInputStream();
	try {

[schlosna]

when the response is 8KiB or less we want to fully consume the response input stream and release the response after converting to an immutable String & StringReader for consumption by Jackson deserialization

[aldexis]

Is there a guarantee that if asReader() has been called, nothing will then call asInputStream()? I imagine it doesn't make much sense to call both, but I'm not very familiar with this API.

[schlosna]

adjusted the logic per below

[aldexis]

Is there any way to allocate the string more directly? inputStream.readAllBytes() will allocate a byte[] and new String is going to copy that byte array into a new one, leading to twice the allocation. I'm not clear whether that is in fact more efficient than the current behavior if the length is more than 4096

[schlosna]

unfortunately there is not a more direct way to allocate the String without extra array copy outside of JDK.

The relevant bits of benchmark FasterXML/jackson-benchmarks#9 (comment) show that StringReader version is beneficial or equivalent up to 32KiB, so would prefer to just keep the 8KiB limit here

Benchmark                                       (mode)  (size)               (type)  Mode  Cnt   Score    Error  Units
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE     128  INPUT_STREAM_READER  avgt    3   0.475 ±  0.006  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE     128        STRING_READER  avgt    3   0.144 ±  0.002  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE     512  INPUT_STREAM_READER  avgt    3   0.786 ±  0.035  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE     512        STRING_READER  avgt    3   0.388 ±  0.096  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    1024  INPUT_STREAM_READER  avgt    3   1.105 ±  0.004  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    1024        STRING_READER  avgt    3   0.698 ±  0.004  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    2048  INPUT_STREAM_READER  avgt    3   1.537 ±  0.088  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    2048        STRING_READER  avgt    3   1.456 ±  3.178  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    4096  INPUT_STREAM_READER  avgt    3   3.047 ±  0.010  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    4096        STRING_READER  avgt    3   3.113 ±  0.040  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    7168  INPUT_STREAM_READER  avgt    3  10.650 ±  2.855  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    7168        STRING_READER  avgt    3  11.025 ± 23.020  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    8192  INPUT_STREAM_READER  avgt    3  13.048 ±  0.202  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE    8192        STRING_READER  avgt    3  12.708 ±  0.513  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE   16384  INPUT_STREAM_READER  avgt    3  32.922 ±  0.386  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE   16384        STRING_READER  avgt    3  32.588 ±  0.890  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE   32768  INPUT_STREAM_READER  avgt    3  72.577 ± 11.414  us/op
JsonArbitraryFieldNameBenchmark.parse  NO_CANONICALIZE   32768        STRING_READER  avgt    3  72.497 ± 13.592  us/op

[schlosna]

Also worth noting that when InputStreamReader is returned, feign.jackson.JacksonDecoder#decode must wrap it in a BufferedReader with a char[1] buffer as InputStreamReader doesn't support mark/reset, while StringReader does.

[aldexis]

Is there a guarantee that if asReader() has been called, nothing will then call asInputStream()? I imagine it doesn't make much sense to call both, but I'm not very familiar with this API.

[aldexis]

Other question: Should we guard against malicious/bad responses that return an incorrect length? (e.g. I imagine there is a corner case where the length is set to 0, which doesn't make sense)
Here we'll buffer the entire response in memory. If the content length header lied to us, we might buffer a lot more than we expected, since we just read the entire thing in memory. Should we just read the first length bytes only?

[schlosna]

improved the handling here in cases where content-length is incorrect

[autorelease3]

Released 8.24.0