Upgrade to 7.x

[ackava]

Place holder issue to upgrade to 7.x, this library is great but there are various bugs in 5.x which were resolved later and can work with latest high def files.

Questions to address.

1. Video Conversion is a heavy process, for most file uploads, if conversion can be done in browser, it will help reducing server resource costs. And any such URL endpoint can easily accommodate SharedArrayBuffer with required headers. Time required to upload video and convert it on server is less than converting with single thread. There seems no benefit of single threaded ffmpeg. So is it alright to drop single threaded core because the performance is very poor?

2. Are there any blocking issues from moving core-mt to ffmpeg 7?

[ghnp5]

I was just wondering this.
I've just realized that a "serious" issue/bug was only happening on the browser's ffmpeg (wasm), while when I test on my computer (ffmpeg 7), it worked perfectly.

Unfortunately, this bug can only be "resolved" if I re-encode the video (rather than a simple -c copy) which will require users to wait several minutes until their videos are processed.

Or, alternatively, we'd use ffmpeg 7.

But, I've absolutely no idea how easy it is to upgrade this project to ffmpeg 7,
and the repo author may no longer be as active (based on README asking for maintainers).

Does anyone know what would be necessary to do, to upgrade to ffmpeg 7?

---

It may be as simple as updating the Dockerfile:

# We cannot upgrade to n6.0 as ffmpeg bin only supports multithread at the moment.
ENV FFMPEG_VERSION=n5.1.4

And then we just have to use multithread mode?

I can give it a try tomorrow! Hopefully I'll not run into many issues.

[ghnp5]

Hey! While I went very far with this, unfortunately I had to park it, because:

1. I lost all those days upgrading to v7, "100% sure" that it would fix the issue that motivated me to upgrade.... only for the issue to be STILL THERE, once I have v7 functional (for x264, I mean)...

2. When I reverted back to v5, I found a way to fix that issue. So, I released my project with v5 + a few fixes/improvements in the JS code, including a "no-simd" version, for it to work in Safari.

3. So I saved all the work I did anyway, and later (no estimated date), without pressure, I'd try to see if I can finish the upgrade.

4. There were a few glitches/issues still, like randomly it entering a loop without coming back, for certain videos.

5. Because it's multi-threading (ST not supported in v7), the browser support would be less too.

6. Due to browsers requiring extra security headers (below), I'd have to put the loading of the WASM file inside an iframe or so, because otherwise it would break any scripts/etc that are external to the website. (affects websites with Ads, for example)

ffmpeg wasm - DataCloneError: Worker.postMessage: The WebAssembly.Memory object cannot be serialized. The Cross-Origin-Opener-Policy and Cross-Origin-Embedder-Policy HTTP headers can be used to enable this.

Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp

7. 2 codecs still need to be fixed: x265 and vorbis

For x265, it seems it works "better" (or at least builds, but might not work well after) with an older version of emscripten/emsdk.
I was in the middle of understanding which version actually breaks it, so I could look at the changes and see if it gives me a hint how to fix. https://github.com/emscripten-core/emscripten/blob/main/ChangeLog.md

But that's when I eventually parked this, so I didn't find the version exactly that broke it. But it would be around v3.1.50~52... It worked perfectly with v3.1.40. It was very weird errors that meant nothing to me.

8. Exceptions/crashes of ffmpeg, or even the abort() call, need to be caught properly by JS.
   I couldn't figure out a way to catch them, and it might also be because I'm not doing the "awaiting" properly, since I'm basically doing a setInterval to see when Module.ret finally changes.

Searching for _emscripten_thread_crashed, _emscripten_proxy_main exceptions, PROXY_TO_PTHREAD exceptions, etc... might help.

EXPORT_EXCEPTION_HANDLING_HELPERS and getExceptionMessage may also give a hint.

9. In one of the mp4 videos I was testing, if I go far in the middle of trimming it (using ffmpeg wasm), it seems the file won't output, and the logs were showing something like "size = 0Kb, speed = 20x" ... not sure what happened there.
   The problem is that the setInterval never triggered the final call, because ffmpeg was there in an endless loop. Maybe the output file was locked, or so.

So, in summary... too many issues outstanding, and eventually I found that it wasn't going to fix the issue I wanted to fix, and was able to fix it in v5.

It would be nice to use the latest version of ffmpeg, but v5 is working good enough, and probably better than v7 would, anyway.

[rimmer]

Thanks for the answer!

7. 2 codecs still need to be fixed: x265 and vorbis

I actually had some progress in x265 itself. To avoid it adding -march param one can use -DX64=1 -DX86_64=1 flags in x265.sh.

BASE_FLAGS=(
  -DCMAKE_TOOLCHAIN_FILE=$EM_TOOLCHAIN_FILE
  -DX64=1
  -DX86_64=1
  -DENABLE_LIBNUMA=OFF
  -DENABLE_SHARED=OFF
  -DENABLE_CLI=OFF
)

This comes from wide-video/ffmpeg-wasm/blob/main/scripts/build-x265.sh, which is a project similar to this one, but they seem to be building ffmpeg 7. It's not that well documented. so I'd prefer ffmpeg.wasm instead,I also didn't try to build wide-video/ffmpeg-wasm with the upstream x265 and ffmpeg.

[ghnp5]

Oh, thank you very much. I'll take that as a note, and bookmark that port for v7, in case I come back to it later!

[rimmer]

@ghnp5 but, yep, clearly if v5 works for you, then there's no reason to invest much time.

[dmitri31337]

Hello;

Checking in on this to see if was implemented as I figured there was significant performance improvements related to taking advantage of avx512 registers in recent intel processors around 7.0