Changing the video codec from VP8 to H.264 #3345
Conversation
fkwp
added
the
PR-Improvement
fkwp
changed the title
src/livekit/options.ts
Outdated
| videoSimulcastLayers: [ | ||
| VideoPresets.h180, | ||
| VideoPresets.h360, | ||
| VideoPresets.h540, | ||
| ] as VideoPreset[], |
There was a problem hiding this comment.
I might be misunderstanding the livekit docs. But can we actually hand it three presets here? Livekit docs stated up to 2 custom layers. Or are those standard preset no custom layers?
videoSimulcastLayers: [VideoPreset]
custom video simulcast layers for camera tracks, defaults to h180, h360, h540.
You can specify**up to two custom layers** that will be used instead of the LiveKit default layers.
There was a problem hiding this comment.
nope its limited to three in total including the base layer.
fkwp
force-pushed
the
fkwp/change_video_codec
branch
from
3c08780 to
717bd1c
Compare
| @@ -29,7 +29,7 @@ window.setLKLogLevel = setLKLogLevel; | |||
| initRageshake().catch((e) => { | |||
| logger.error("Failed to initialize rageshake", e); | |||
| }); | |||
| setLKLogLevel("info"); | |||
| setLKLogLevel("debug"); | |||
There was a problem hiding this comment.
Is this on purpose? or a left over of a dev setup?
There was a problem hiding this comment.
this is part of debugging why h264 with simulcast is not working on iOS devices.
src/livekit/options.ts
Outdated
| VideoPresets, | ||
| } from "livekit-client"; | ||
|
|
||
| const VideoPresetsH264 = { |
There was a problem hiding this comment.
what makes this specific H264 presets? Aren't those just regular presets that could also be used with other encordings?
There was a problem hiding this comment.
this PR is in draft -- this is part of debugging why h264 with simulcast is not working on iOS devices.
using h264 specific presets help finding out if the h264 macro-block constraint is the problem here.
src/livekit/options.ts
Outdated
| VideoPresetsH264.h144, | ||
| VideoPresets.h360, |
There was a problem hiding this comment.
why the mixture of VideoPresetsH264 and VideoPresets?
|
What is the status on this one? Can the OP get a description what this is blocked by. (I do remember some device specific issues where some resolution just were not send at all?) |
H264 video is still broken on iOS — other participants won’t receive video from an iOS device. This happens when the iOS device is the first to join a LiveKit room and starts streaming H264. However, it's working if a non-iOS client is already streaming H264 before the iOS device joins. Note Using VP8 on the other client does not fix the issue — it must be H264. |
|
Test on safari on mac os works. |
| screenShareEncoding: ScreenSharePresets.h1080fps30.encoding, | ||
| stopMicTrackOnMute: false, | ||
| videoCodec: "vp8", | ||
| videoCodec: "h264", |
There was a problem hiding this comment.
| videoCodec: "h264", | |
| videoCodec: platform === "ios" || !useH264Setting ? "vp8" : "h264", |
Adding a dev setting for using h264 would allow merging this and would provide us more data on how this behaves in real live.
Let me know if I should update this PR.
There was a problem hiding this comment.
@fkwp any thoughts on this? This would also be nice to have in the 15 release ?
Recent research has confirmed that the number of devices lacking H.264 decoding capabilities is negligible.
In contrast, many modern devices come equipped with native H.264 encoders and decoders. By switching to the H.264 codec, we can significantly reduce CPU usage. Especially mobile devices will benefit, where it can lead to better battery life.
Furthermore, we add a 540p simulcast video layer.
fixes #3166 #2579 #2373
Current state
H264 video is still broken on iOS — other participants won’t receive video from an iOS device. This happens when the iOS device is the first to join a LiveKit room and starts streaming H264.
However, it's working if a non-iOS client is already streaming H264 before the iOS device joins. Note Using VP8 on the other client does not fix the issue — it must be H264.