Skip to content

H2: record time to first upstream H2 Data #41709

@mandarjog

Description

@mandarjog

Title: H2: record time to first upstream H2 data

Description:
We have several measurements in Envoy like
%RESPONSE_DURATION%

Total duration in milliseconds of the request from the start time to the first byte read from the upstream host.

I want to add %RESPONSE_DATA_DURATION%

Total duration in milliseconds of the request from the start time to the first DATA byte from the upstream host.

For streaming GRPC, the above two are different, since some headers can be/are sent by the upstream right away.

** Why **
This measurement is aligned with Time To first Token (TTFT) measurements that show how long it takes to start generating response. The two measurements that we do have duration and response_duration do not capture this important metric. Specifically response_duration is unusually low for a model generating response.

** Proposal **

  1. Keep the code change similar to measurements like response_duration

  2. Add onFirstUpstreamDataRxByteReceived to StreamInfo and a measurement to UpstreamTiming

  3. Set the above measurements in CodecBridge::decodeData

  4. Add Accesslog formatter key / calculation for the RESPONSE_FIRST_UPSTREAM_DATA_DURATION (modeled on RESPONSE_FIRST_UPSTREAM_BYTE_DURATION.

  5. Add upstream cluster histogram ?

  6. If there is already a way to do this, then let me know.

  7. I see that other measurements like upstream_timing_.first_upstream_rx_byte_received_ = time_source.systemTime(); use systemTime, should I use that, or use MonotonicTime?

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions