Find default hardware encoder in VideoEncoder #1124

Dan-Flores · 2025-12-11T05:59:56Z

To improve ease of use of the VideoEncoder on GPU, this PR adds logic to find and use a hardware enabled codec if one is available.

A boolean arg is added to findCodec to flip the function to search for encoders. This means it has the following behaviors:

For video container formats where the default codec has a hardware equivalent (ex. libx264, h264_nvenc), the function will correctly return the hardware encoder.
For video container formats where the default codec does NOT have a hardware equivalent(ex. the mpeg codec used foravi), no hardware encoder will be used by default.

…o auto_select_gpu_encoder

Dan-Flores · 2025-12-11T06:01:58Z

test/test_encoders.py

-            ),
-        ],
-    )
+    @pytest.mark.parametrize("format", ("mov", "mp4", "mkv"))


These formats all default to h264_nvenc, so IN_GITHUB_CI is no longer necessary to avoid av1_nvenc.

By removing codec we have lost important testing coverage:

manually passing a codec that is not None

passing a non-default codec for that given container format

testing on av1

Let's put all these back. To test the new behavior that this PR is enabling, all we have to do is to specify a codec=None case. Something like this (with comments to help understanding the intent):

@pytest.mark.parametrize( ("format", "codec"), [ ("mov", None), # should default to h264_nvenc ("mov", "h264_nvenc"), ("mp4", "hevc_nvenc"), # non-default ("avi", "h264_nvenc"), pytest.param( ("mkv", "av1_nvenc"), marks=pytest.mark.skipif( IN_GITHUB_CI, reason="av1_nvenc is not supported on CI" ), ), ], )

Note also that I separated "format" and "codec" into their own parameter input - that's the more conventional way of passing couples of parameters with pytest. You'll have to update the function's signature to accept both format and codec separately.

Good point, I've added back these tests so we are explicitly testing hevc_nvenc and av1_nvenc.

test/test_encoders.py

NicolasHug

Thanks @Dan-Flores , made a first pass

NicolasHug · 2025-12-11T10:25:20Z

src/torchcodec/_core/Encoder.cpp

+        avCodec = hwCodec.value();
+      }
+    }
    TORCH_CHECK(avCodec != nullptr, "Video codec not found");


Not from this PR but we might as well address it here:

We have this TORCH_CHECK just above, and we also have a similar one in the first if branch above

TORCH_CHECK( avCodec != nullptr, "Video codec ", codec, " not found. To see available codecs, run: ffmpeg -encoders");

Let's just have a single TORCH_CHECK after this whole if/else block? The error message mentioning ffmpeg -encoders is good.

NicolasHug · 2025-12-11T10:27:47Z

src/torchcodec/_core/CudaDeviceInterface.cpp

+  }
+
+  return std::nullopt;
+}


This is 99% the same code as the existing findCodec(). Let's just extend findCodec and add an optional bool decoder parameter who's default is true.

NicolasHug · 2025-12-11T10:30:31Z

src/torchcodec/_core/Encoder.cpp

    TORCH_CHECK(
        avFormatContext_->oformat != nullptr,
        "Output format is null, unable to find default codec.");
    avCodec = avcodec_find_encoder(avFormatContext_->oformat->video_codec);


I think we should logically call avcodec_find_encoder after we have tried findHardwareEncoder. The reason is: for whatever reason, avcodec_find_encoder could return an error (like if there's no CPU codec for a given configuration) but findHardwareEncoder might be able to find the right one. So I thikn we should logically do:

if onCUDA: codec = findHardwareEncoder() # first try NVENC specific stuff if codec is still not found: codec = avcodec_find_encoder()

Thanks for the suggestion, I've updated the code to conditionally call avcodec_find_encoder iff there is no hardware encoder for the default codec.

NicolasHug · 2025-12-11T10:31:35Z

src/torchcodec/_core/Encoder.cpp

    avCodec = avcodec_find_encoder(avFormatContext_->oformat->video_codec);
+    // If frames are on a CUDA device, try to substitute the default codec
+    // with its hardware equivalent
+    if (frames_.device().is_cuda() && deviceInterface_) {


This frames_.device().is_cuda() && deviceInterface_ is a bit imprecise because when frames are on CUDA, we should always have defined a device interface. If it's not defined, then it's a bug. This should be:

if frames on cuda: assert device_interface exists # continue here

NicolasHug · 2025-12-11T10:39:06Z

test/test_encoders.py

-            ),
-        ],
-    )
+    @pytest.mark.parametrize("format", ("mov", "mp4", "mkv"))


By removing codec we have lost important testing coverage:

manually passing a codec that is not None

passing a non-default codec for that given container format

testing on av1

Let's put all these back. To test the new behavior that this PR is enabling, all we have to do is to specify a codec=None case. Something like this (with comments to help understanding the intent):

@pytest.mark.parametrize( ("format", "codec"), [ ("mov", None), # should default to h264_nvenc ("mov", "h264_nvenc"), ("mp4", "hevc_nvenc"), # non-default ("avi", "h264_nvenc"), pytest.param( ("mkv", "av1_nvenc"), marks=pytest.mark.skipif( IN_GITHUB_CI, reason="av1_nvenc is not supported on CI" ), ), ], )

Note also that I separated "format" and "codec" into their own parameter input - that's the more conventional way of passing couples of parameters with pytest. You'll have to update the function's signature to accept both format and codec separately.

NicolasHug

Thanks @Dan-Flores . Left some Qs but approving to unblock.

NicolasHug · 2025-12-12T09:35:21Z

test/test_encoders.py

    @needs_ffmpeg_cli
    @pytest.mark.needs_cuda
-    # TODO-VideoEncoder: Auto-select codec for GPU encoding
+    # @pytest.mark.parametrize("format", ("mov", "mp4", "mkv"))


NicolasHug · 2025-12-12T09:36:19Z

test/test_encoders.py

-        format, codec = format_codec
        device = "cuda"
-        qp = 1  # Lossless (qp=0) is not supported on av1_nvenc, so we use 1
+        qp = 1  # Use near lossless encoding to reduce noise and support av1_nvenc


QQ would we be able to assert closer expected results (stricter tolerance) if we set qp = 0 for the other codecs? If we can, then it might be worth it

No, at qp = 0 or qp = 1 the highest assertion we can make is 96% of frames match.

NicolasHug · 2025-12-12T09:38:13Z

test/test_encoders.py

-        if codec == "av1_nvenc":
-            ffmpeg_cmd.extend(["-rc", "constqp"])  # Set rate control mode for AV1
-        ffmpeg_cmd.extend(["-qp", str(qp)])  # Use lossless qp for other codecs
+        ffmpeg_cmd.extend(["-qp", str(qp)])


There are two if codec == "av1_nvenc": that were removed, above and below. Just checking if that was by design, and why?

yes this is intentional, I realized constqp is the default rate control mode when qp is set for the nvenc codecs, so its not necessary to set it here.

Dan-Flores added 2 commits December 10, 2025 18:32

add findHardwareEncoder, remove codec from test params

b2bf92d

Merge branch 'main' of https://github.com/meta-pytorch/torchcodec int…

45efb9f

…o auto_select_gpu_encoder

meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Dec 11, 2025

Dan-Flores commented Dec 11, 2025

View reviewed changes

test/test_encoders.py Show resolved Hide resolved

Dan-Flores changed the title ~~Find default hardware encoder~~ Find default hardware encoder in VideoEncoder Dec 11, 2025

Dan-Flores marked this pull request as ready for review December 11, 2025 06:19

NicolasHug reviewed Dec 11, 2025

View reviewed changes

Dan-Flores added 3 commits December 11, 2025 21:22

remove findHardwareEncoder

84fc4fb

add back codec param

217c490

reorder codec selection, combine torch check

1c19b1c

NicolasHug approved these changes Dec 12, 2025

View reviewed changes

remove unused param comment

d3e1e32

Dan-Flores merged commit 1e171fa into meta-pytorch:main Dec 12, 2025
84 of 87 checks passed

Find default hardware encoder in VideoEncoder #1124

Find default hardware encoder in VideoEncoder #1124

Conversation

Dan-Flores commented Dec 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

NicolasHug left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

NicolasHug left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Dan-Flores commented Dec 11, 2025 •

edited

Loading