New Wasm workloads: Transformers.js ML sentiment analysis and Speech-to-Text

[danleh]

These could be replacements for the tfjs workloads (even though the model file size issue remains). Run via transformersjs-bert-wasm and transformersjs-whisper-wasm.

TODOs: Evaluate startup/model loading performance, take CPU profile, decide whether Whisper task is too long-running, compress model files on-disk for repo size, modify NPM server to handle those?

[netlify]

✅ Deploy Preview for webkit-jetstream-preview ready!

Name	Link
🔨 Latest commit	fb0ab2b
🔍 Latest deploy log	https://app.netlify.com/projects/webkit-jetstream-preview/deploys/68a5de260f9cca0008bc9aaa
😎 Deploy Preview	https://deploy-preview-148--webkit-jetstream-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[eqrion]

I am getting the following error when running the deploy preview in Firefox:

Uncaught (in promise) Error: Unexpected fetch: blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/bb376a7e-5209-4208-ac76-defa885759a2
    fetch blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53:23
    getBinary https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm line 688 > injectedScript:38
    init blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53:45
    doRun https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:57
    <anonymous> https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:80
[ac76acf2-8e08-42c9-a0b5-087158373d53:23:11](blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53)
    start https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:257
    AsyncFunctionThrow self-hosted:804
    (Async: async)
    onclick https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:453
    (Async: EventHandlerNonNull)
    prefetchResources https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:451
    InterpretGeneratorResume self-hosted:1332
    AsyncFunctionNext self-hosted:800
    (Async: async)
    initialize https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:405
    initialize https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:53
    onload https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:1

[danleh]

I am getting the following error when running the deploy preview in Firefox:

Uncaught (in promise) Error: Unexpected fetch: blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/bb376a7e-5209-4208-ac76-defa885759a2
    fetch blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53:23
    getBinary https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm line 688 > injectedScript:38
    init blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53:45
    doRun https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:57
    <anonymous> https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:80
[ac76acf2-8e08-42c9-a0b5-087158373d53:23:11](blob:https://deploy-preview-148--webkit-jetstream-preview.netlify.app/ac76acf2-8e08-42c9-a0b5-087158373d53)
    start https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:257
    AsyncFunctionThrow self-hosted:804
    (Async: async)
    onclick https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:453
    (Async: EventHandlerNonNull)
    prefetchResources https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:451
    InterpretGeneratorResume self-hosted:1332
    AsyncFunctionNext self-hosted:800
    (Async: async)
    initialize https://deploy-preview-148--webkit-jetstream-preview.netlify.app/JetStreamDriver.js:405
    initialize https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:53
    onload https://deploy-preview-148--webkit-jetstream-preview.netlify.app/?test=transformersjs-whisper-wasm:1

My bad, the dynamic import of ONNX runtime didn't work with the blob/preloading in the browser, fixed.

[kmiller68]

Seems like both Bert and Whisper spend a majority of the time in whatever function index 7460 is (I assume some matrix multiplication function). Although, it does look like about 70-75% vs 50-55% in 7460, respectively, so Whisper is doing other things. They also look pretty similar in where they spend their time within that function.

The fact that Whisper does other things (and IIUC, is more popular) makes it somewhat more interesting. On the other hand, it seems like the dominant function in either case is the same, and running faster is a significant benefit.