-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Description
Title: H2: record time to first upstream H2 data
Description:
We have several measurements in Envoy like
%RESPONSE_DURATION%
Total duration in milliseconds of the request from the start time to the first byte read from the upstream host.
I want to add %RESPONSE_DATA_DURATION%
Total duration in milliseconds of the request from the start time to the first DATA byte from the upstream host.
For streaming GRPC, the above two are different, since some headers can be/are sent by the upstream right away.
** Why **
This measurement is aligned with Time To first Token (TTFT) measurements that show how long it takes to start generating response. The two measurements that we do have duration and response_duration do not capture this important metric. Specifically response_duration is unusually low for a model generating response.
** Proposal **
-
Keep the code change similar to measurements like
response_duration -
Add
onFirstUpstreamDataRxByteReceivedto StreamInfo and a measurement toUpstreamTiming -
Set the above measurements in
CodecBridge::decodeData -
Add Accesslog formatter key / calculation for the
RESPONSE_FIRST_UPSTREAM_DATA_DURATION(modeled on RESPONSE_FIRST_UPSTREAM_BYTE_DURATION. -
Add upstream cluster histogram ?
-
If there is already a way to do this, then let me know.
-
I see that other measurements like
upstream_timing_.first_upstream_rx_byte_received_ = time_source.systemTime();use systemTime, should I use that, or use MonotonicTime?