microgesture support

[Utopiah]

The Quest Browser 38.1 support hand microgestures. They follow the OpenXR specification https://registry.khronos.org/OpenXR/specs/1.1/html/xrspec.html#XR_META_hand_tracking_microgestures and there is a demo page at https://micro-gestures-demo.glitch.me/ and the result show up in the GamePad API https://immersive-web.github.io/webxr-samples/controller-state.html

It could be interesting, assuming that others, e.g. Apple for the VisionPro or Google for AndroidXR do follow OpenXR specs for WebXR, to bring this to a similar level than hand-tracking-controls namely extending events beyond pinchstarted, pinchended and pinchmoved with, possibly (following OpenXR naming) tap (or maybe thumb_tap), thumb_location (that I initially forgot, as I somehow didn't manage to test it) and swipe_left, swipe_right, swipe_forward, and swipe_backward new events for example.

[Utopiah]

Related meta-quest/immersive-web-emulation-runtime#23

[dmarcos]

Yeah. it's on my radar. thanks

[vincentfretin]

If you didn't see it yet, @mrxz made a Connect Four game, the gestures are indeed quite nice
https://x.com/noerihuisman/status/1921680416257216544

[Utopiah]

Neat! I wasn't aware of it so thanks for sharing.

IMHO due to OpenXR support (and to be pragmatic, the market share of Quest...) it could arguably be into core rather than an outside component.

[mrxz]

Definitely a feature worth having in the core. Note that the OpenXR extension is a vendor extension by META, so there's no reason to assume other vendors will implement it. It's quite likely that other vendors would introduce their own OpenXR extension with their own gestures/naming/etc...

In our case the right abstraction is WebXR. The micro-gestures are implemented as additional buttons on the gamepad of the oculus-hand profile (see immersive-web/webxr-input-profiles#274). So adding support is pretty straightforward.

The only question is what the right component structure would be. On one hand (pun not intended) the pattern used for controllers could be continued (meta-quest-controls uses tracked-controls, oculus-hand-tracking-controls uses hand-tracking-controls). Alternatively a component could be created specifically for microgestures, say oculus-hand-microgestures that can be added to entities with hand-tracking-controls.

[dmarcos]

I would just add it to hand-tracking-controls. Maybe a property micogesturesEnabled to toggle. I assume other vendors will implement same / similar gestures. can split per vendor later if it makes sense

[dmarcos]

Would be cool to have the Connect Four game or similar as example