Remove WebCodecs tests with PRECONDITION_FAILED

[nt1m]

Test List

• ✅ full-cycle-test.https.any.html?vp9_p2
• ✅ full-cycle-test.https.any.worker.html?vp9_p2
• reconfiguring-encoder.https.any.html?h264_annexb
• reconfiguring-encoder.https.any.html?h264_avc
• ✅ reconfiguring-encoder.https.any.html?vp9_p2
• reconfiguring-encoder.https.any.worker.html?h264_annexb
• reconfiguring-encoder.https.any.worker.html?h264_avc
• ✅ reconfiguring-encoder.https.any.worker.html?vp9_p2
• temporal-svc-encoding.https.any.html?h264
• temporal-svc-encoding.https.any.worker.html?h264

Rationale

Support for different video codecs is out of scope for WebCodecs.

[nt1m]

@nairnandu @jgraham Can you find someone on the Blink / Gecko side to review? Thanks!

[jgraham]

@ChunMinChang does this make sense from Gecko's point of view? (I think if the spec doesn't mandate specific codec support we can't really have tests that depend on specific codecs in Interop without significant extra buy-in/work).

[ChunMinChang]

WebCodecs doesn't ask the user-agent to support specific codecs. Instead, it has an API that allows users to check if one codec is supported. An idiom way for WPT to run a codec test is to call isConfigSupported first to determine if the test should continue.

Removing av1 or not won't violate the spec, but what's the reason to remove av1? Can't using isConfigSupported first solve this issue?

[nairnandu]

@dalecurtis could you review this proposal for Chromium?

[nt1m]

I would be ok with updating tests to pass if isConfigSupported is false. Though I don't think codec support reflects the quality of the WebCodecs implementations.

[dalecurtis]

We wrote the tests to check isConfigSupported with assert_implements_optional, but may have missed some. As @ChunMinChang notes, tests should bail if they detect the UA doesn't support the codec.

[nt1m]

Seems like those tests indeed use assert_implements_optional, unfortunately, that reports a status of PRECONDITION_FAILED, and for scoring purposes, that counts the test as failing.

I guess this is just #319 then.

[nt1m]

OK I submitted a PR to fix assert_implements_optional being counted as failing: web-platform-tests/results-analysis#178

cc @DanielRyanSmith

[chrishtr]

@nt1m can this issue be closed now given your previous comment?

[nt1m]

AV1 tests are now passing, but we still have a small subset of tests that have PRECONDITION_FAILED for specific codecs.