Add support for hand tracking microgestures

[simonedevit]

This pull request adds the support for hand tracking microgestures, a Meta OpenXR extension - compatible with Quest 2/Pro/3/3S - that adds five boolean inputs per hand (thumb tap + D-pad–like swipes).

WebXROculusHandController has been created starting from WebXRGenericHandController as base and mapping the input profile with theoculus-hand profile: https://github.com/immersive-web/webxr-input-profiles/pull/274/files.

---

Behaviour

As today, all the input controller profiles are retrieved from the WebXR controller repository, unfortunately, the oculus-hand controller is missing. I made the PR immersive-web/webxr-input-profiles#278 to add it, but i don't know if/when it will be approved.

As-is

1. Starting a WebXR session with a compatible Quest device, xrController.inputSource.profiles is correctly populated with oculus-hand profile:

   ['oculus-hand', 'generic-hand', 'generic-hand-select', 'generic-trigger']

2. All input controller profiles are retrieved from:
   https://immersive-web.github.io/webxr-input-profiles/packages/viewer/dist/profiles/profilesList.json

   (but oclus-hand is missing).

3. Get generic-hand because it is the first available profile:
   https://immersive-web.github.io/webxr-input-profiles/packages/viewer/dist/profiles/generic-hand/profile.json

4. Map generic-hand.

To-be (after pull request is merged)

1. The user create an XR experience disabling the online controller repository (see "Considerations" below):

scene.createDefaultXRExperienceAsync({
        inputOptions: {
            disableOnlineControllerRepository: true
        }
 })

2. Starting a WebXR session with a compatible Quest device, xrController.inputSource.profiles is correctly populated with oculus-hand profile:

['oculus-hand', 'generic-hand', 'generic-hand-select', 'generic-trigger']

3. The oculus-hand profile is retrieved from Babylon.js itself (local).

---

Note

Forcing the input profile:

scene.createDefaultXRExperienceAsync({
        inputOptions: {
            forceInputProfile: 'oculus-hand'
        }
 })

is not a valid option because in any case the generic-hand profile will be retrieved.

---

Test

Tested with Oculus Quest 3.
Playground (valid after PR build): https://playground.babylonjs.com/#F41V6N#2275.

---

Considerations

When the oculus-hand profile will be added to online repository, it shouldn't introduce regressions because the oculus-hand it will be retrieved and consumed automatically. The oculus-hand profile is 1:1 mapping with generic-hand profile with the adding of specific swipe components (menu - only left hand, swipe-left, swipe-right, swipe-forward, swipe-backward, tap-thumb).

---

Discussion

Forum link: https://forum.babylonjs.com/t/hand-tracking-microgestures/60860.

[bjsplat]

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

[bjsplat]

Snapshot stored with reference name:
refs/pull/17255/merge

Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17255/merge/index.html

To test a playground add it to the URL, for example:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17255/merge/index.html#WGZLGJ#4600

Links to test babylon tools with this snapshot:

https://playground.babylonjs.com/?snapshot=refs/pull/17255/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/17255/merge
https://gui.babylonjs.com/?snapshot=refs/pull/17255/merge
https://nme.babylonjs.com/?snapshot=refs/pull/17255/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/17255/merge#BCU1XR#0

[bjsplat]

Interaction tests

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17255/merge/testResults/interactionplaywright/index.html

[bjsplat]

Devhost visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17255/merge/testResults/devhostplaywright/index.html

[bjsplat]

Visualization tests for WebGPU

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17255/merge/testResults/webgpuplaywright/index.html

[bjsplat]

WebGL2 visualization test reporter:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17255/merge/testResults/webgl2playwright/index.html

[docEdub]

Nice PR!

I tested the example playground and it works well.

The only question I have is how does the "menu" button get triggered and why isn't it handled in the example playground?

[simonedevit]

The "menu" button is activated by the left-hand pinch gesture.

com.oculus.vrshell-20251009-095654-0.mp4

Here the updated playground: https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/17255/merge/index.html#F41V6N#2277.