GestureHandler only works on DOMElement, not Mesh

[steveblue]

When I apply a drag gesture using the GestureHandler to a Node that has a DOMElement as child it works and the DOMElement is draggable. However, if I try to do the same thing with a Mesh the events don't fire in Engine v0.7.0. In the Guides there is only an example using DOMElement, which makes me wonder if this is possibly a design flaw? Is there is a plan to support GestureHandlers on Mesh?

[michaelobriena]

Yes this is the case. WebGL has no default eventing scheme. There are two approaches to implementing it.

1. Picking. (Me not fully understanding this ) Essentially we store some unique value on the meshes buffer so that when it is selected we can look up what was hit. Pros: fast and easier to implement | Cons: user does not know where on the mesh it was clicked makes rotation around center impossible.

2. Ray casting. Shoot a ray from the camera and figure out what it hits. Pros: accurate and you know where on a mesh you hit | Cons: much harder to do.

We have a PR in for the picking version but unfortunately this seems to not be worth putting in because of the limited usecases. Feel free to merge that branch into a fork of yours if need be.

We are looking at the ray casting version but it is not the highest of priorities for us right now.

[steveblue]

+1 Ray casting

I understand if this is not the highest priority, but it should be. I think without an Event scheme on GL Objects Mixed Mode is half baked. It's great that both DOMElement and Mesh are in the same coordinate space BUT if you can't interact with them using the same API (GestureHandler), then what is the point?

[michaelobriena]

@steveblue while I understand not having GL hit detection limits what is possible, we have little business need for it right now and therefore are focusing on other projects. Please realize that this is a small team of people working on many projects other than just our rendering engine and we have given this tech to the community for free.

Mixed Mode is just one feature of our rendering engine. Not everything written in Famous needs to be mixed content.

Also, there are plenty of ways to use webgl without specific mesh hit detection.

• DOM decoration (shiny buttons)
• Art
• Lighting
• Anything non interactive

If you write a ray caster feel free to submit a pull request and we can review & merge.

[alexanderGugel]

@steveblue There are quite some generic (= rendering agnostic) ray casters out there. If you really need this sort of functionality, you can use one of those modules. While I haven't looked into the details, all of them seem to be abstract enough to be integrated into custom projects.

It's pretty much a solved problem. The PR Mike was referring to earlier in the discussion can be found here: #275. You might get some merge conflicts, but nothing to serious usually.

[oldschooljarvis]

Also +1 for ray casting.

Correct me if I'm wrong, but part of the allure of Mixed Mode is to have a unified API where you can treat DOM and GL in a reasonably uniform fashion. If Famous DOM has event/interaction architecture and Famous GL doesn't, then the GL side of things really becomes a sort of second-class citizen at that point.

Admittedly in most cases, GL input events can be simulated fairly easily by simply placing an invisible DOMElement over a GL element to listen for events. However, there's also many cases where input events on non-rectangular forms are desirable, and I suppose that actually highlights a somewhat glaring deficiency that both modes share (especially when image alpha opacity is involved).

That said, I can sort of understand if such things aren't a business priority right now. At the same time, Engine serves as the foundation for most of those aforementioned projects, and if Engine isn't both rock-solid and robust, it doesn't bode well for either said projects nor adoption in general.

With regards to ray casting, one approach might be to first implement it in a generalized sense as utility functions integrated into Engine, similar to math helpers. Then, use that to implement ray casting-based input events. Ray helpers are incredibly useful when working in a 3D context, so much so that most 3D libraries include their own by default. I see Alex linked a bunch of generic libraries, and those are certainly better than nothing - but if users have to discover, select, and integrate third party libraries just to work with rays, then it's just another barrier keeping people from being productive with Famous.

I apologize if I'm being overly negative here, please don't take it personally. I understand you guys are working hard with limited resources; I just want to see Famous succeed too.

[steveblue]

I've created a new Ray class and added some operations to Vec3 on this branch. https://github.com/infamous/engine/tree/infamous-feat-raycast Raycasting solves the issue of gestures for GL and is way less intrusive than #275 plus this gives developers the added bonus of raycasting, which could be used for all sorts of applications beyond gesture handling. @michaelobriena @alexanderGugel

[CompSciFutures]

Let's see a demo

On 26 Aug 2015, at 5:10 pm, Steve Belovarich notifications@github.com wrote:

I've created a new Ray class and added some operations to Vec3 on this branch. https://github.com/infamous/engine/tree/infamous-feature-raycast Raycasting solves the issue of gestures for GL and is way less intrusive than #275 plus this gives developers the added bonus of raycasting, which could be used for all sorts of applications beyond gesture handling.

―
Reply to this email directly or view it on GitHub.

[steveblue]

@Aprender here is a demo

http://stephenbelovarich.com/play/raycast/

Check the console for Vec3 that return from the hit detection.

    var gestureNode = scene.addChild()
        .setOrigin(0.5, 0.5, 0.5)
        .setMountPoint(0.5, 0.5, 0.5)
        .setAlign(0.5, 0.5, 0.5)
        .setSizeMode(1, 1, 1)
        .setAbsoluteSize(window.innerWidth,window.innerHeight);

    var gestureElem = new DOMElement(gestureNode);
    gestureElem.setProperty('background-color', 'transparent');

    var gestures = new GestureHandler(gestureNode);
    function callback(e) {
        ray = new Ray([e.center.x, e.center.y, -1000],[0, 0, 1]);
        hit = ray.intersectSphere(meshNode.getPosition(),100);
        console.log(hit);
        if(hit){
            sphere.setBaseColor(new Color('#FFAA00'));
        } else {
            sphere.setBaseColor(new Color('#454545'));
        }

        hit = ray.intersectBox(boxNode.getPosition(),100);
        console.log(hit);
        if(hit){
            mesh.setBaseColor(new Color('#AAFF00'));
        } else {
            mesh.setBaseColor(new Color('#454545'));
        }
    }
    gestures.on('drag', callback);

[CompSciFutures]

@steveblue finally found some spare time to look at this. V cool, nice elegant simple solution to a thorny problem.

+1 This should be in the engine.