Adding offsetPos to Touch.Event

[joakin]

Hi!

Thanks for the nice library 💟

I was using it to make a small drawing app, and used Mouse and Touch separately, as I didn't want to deal with polyfills at this time.

When using Mouse, I used .offsetPos, really easy to do, great success.

When I went in to add touch compatibility, I had a harder time. I tried using .clientPos which is what the docs mention you should be using probably most of the time, but that was giving me the coordinates for the viewport instead of relative to the element like offsetPos does on Mouse. So when I scrolled, the coordinates would be off and painting would be broken by whatever scroll offset happened.

I did a bunch of digging and MDN docs looking to see if I would be able to handle this within Elm and a custom decoder, and I found out a way to get the equivalent of offsetPos for the Touch.Event by decoding the offset of the element the events were bound on from event.target.

It was quite hard to figure out how to use custom decoders with the package, so I ended up copying parts of the source on Touch.elm and manually using onWithOptions.

In the end I managed to get it to work, you can see an example here:

• My custom onTouch fn that uses my decoders Drawing.elm#L213-L215
• The decoders I used with the ones from your library go get the offset position: Drawing.elm#L410-L452

I tested it on Chrome, Safari iOS and Chrome Android, so it seems like the main mobile browsers work fine with it.

I was wondering if you would be interested in a PR that adds .offsetPos to the touch events, as I found it very useful, and I imagine that it would be very used for many things like it is on the Mouse package.

Have a nice day!

[mpizenberg]

I'm glad it was helpful and thank you for the feedback!

I was using it to make a small drawing app, and used Mouse and Touch separately, as I didn't want to deal with polyfills at this time.

I initially did the same thing, eventually not having relative pos in the Touch API was so annoying that I decided to switch to Pointer. By the way, in Firefox, I have a "ReferenceError: customElements is not defined". Also you might be interested in setCapture that enables continuation of mouse events when the pointer moves out of the dom element. It says there is not much browser support but I don't see the compatibility table. For elm 0.18, you can do something like:

attribute "onmousedown" "if ('setCapture' in document.body) { event.target.setCapture(); };"

In 0.19 it won't work anymore but there will be alternatives.

When I went in to add touch compatibility, I had a harder time.

I can believe that ^^. I tried something similar at the beginning. Reading debois/elm-dom code was of great help, especially the boundingClientRect decoder. It also made me realize that getting the relative position was a bit complicated if I wanted it to work all the time (nested scrollable elements). I eventually ended up choosing to stick to the official API. Your experience is clearly making me reconsider this. At least docs can be improved to mention all this as a first step, that's for sure!

It was quite hard to figure out how to use custom decoders with the package, so I ended up copying parts of the source on Touch.elm and manually using onWithOptions.

Ok, it's kind of the "advanced" section because events handling in elm is not the easiest, so I was expecting this to be a little harder. Yet this also means that there is some docs work to do!

The decoders I used with the ones from your library go get the offset position: Drawing.elm#L410-L452

I believe your solution is only working if the parent of your drawing element has a position of (0,0). It needs to be done recursively to the top of the tree like in boundingClientRect in debois/elm-dom.

I was wondering if you would be interested in a PR that adds .offsetPos to the touch events, as I found it very useful, and I imagine that it would be very used for many things like it is on the Mouse package.

Thanks a lot for your time writing this feedback! I was thinking changing the docs would be first. It's a lot less exciting for PR but extremely valuable too! PS, the current (slightly) active branch is 0.19.

Good day to you too!

[joakin]

I initially did the same thing, eventually not having relative pos in the Touch API was so annoying that I decided to switch to Pointer.

Ah, I'll definitely have to try it out!

By the way, in Firefox, I have a "ReferenceError: customElements is not defined"

Right! I haven't used the customElements polyfill so I guess it is expected that it is broken. I have to look into that but I didn't want to bother with polyfills or transpiling JS just yet. Thanks for letting me know.

Also you might be interested in setCapture [...]

Oh interesting, that is definitely an annoying default behavior for sure. I also see the link to setPointerCapture, I have to do some learning about pointer events...

It also made me realize that getting the relative position was a bit complicated if I wanted it to work all the time (nested scrollable elements).

Oh I didn't think of that. Every time I have to do something with positioning of elements it is a pain without getBoundingClientRect and those more modern APIs. I see the next version will have support for them so it should be a lot easier to deal with these cases...

I eventually ended up choosing to stick to the official API. Your experience is clearly making me reconsider this. At least docs can be improved to mention all this as a first step, that's for sure!

Yeah all of this is full of quirks to be honest. For example, there was some chatting in #api-design in slack about some mouse API and for example apparently offsetPos with Mouse is relative to the target, so if you have nested elements it also won't work well (you get the position relative to the nested element, instead of the one you bound the event to), see https://ellie-app.com/32NBmNrf2fha1, and mdn offsetX mentions that it uses event.target.

As a default, that can be surprising. Given using currentTarget and manually calculating position like I did would involve the same problems you were mentioning with nested scrollable elements, I don't think it can be fixed in the library. Maybe it would be worth adding the caveats to offsetPos in the Mouse docs too. This is a separate issue that could be improved.

I believe your solution is only working if the parent of your drawing element has a position of (0,0). It needs to be done recursively to the top of the tree like in boundingClientRect in debois/elm-dom.

Right! I completely forgot about this. I think you are right. Thanks for the feedback, makes sense.

I was thinking changing the docs would be first. It's a lot less exciting for PR but extremely valuable too! PS, the current (slightly) active branch is 0.19.

👍 I see, just looked at it on the alpha site. Makes sense.

So, a couple of questions:

• I can look into writing a bit more on how to use custom decoders in the docs. Where do you think we should add more detail? I can think of these options:
  • Write an example in examples/, link to it from the modules that have the advanced section about custom event decoding
  • In the readme at the after or before Examples, adding a small example. Maybe link to README for more info in modules.
  • In one module, like Touch. Seems suboptimal though since it is something you can do in all the modules.
  • Add an example in a comment on each module. More verbose and direct, but more work and duplication.
• Would you be interested in a PR that adds some information about the caveats of offsetPos in Mouse?

[mpizenberg]

offsetPos with Mouse is relative to the target

Oh yeah, this one bit me too! I was drawing some SVG stuff and it was flickering somehow. It took time for me to realize that the SVG I was drawing was becoming the "target" of the event and the coordinates would be wrong half events. Now for drawing purposes, I often add the css rule "pointer-events: none" (which is different from pointer events, naming ...) that prevents an element from becoming a target of mouse events. Cf updated ellie: https://ellie-app.com/33yPZFdFc9Qa1.

Would you be interested in a PR that adds some information about the caveats of offsetPos in Mouse?

Yep, definitely!

I can look into writing a bit more on how to use custom decoders in the docs.

Custom decoders are usually useful for stuff that is not well supported yet or browser-dependent (and thus not in this lib). I had made some effort to give a simple example in the Mouse and Pointer modules. Let me know what you think of those.

I did not bother making examples for Drag and Wheel. Here, we could probably add a link to the Mouse module for an example of how to use eventDecoder? Or a full fledged example if you have one in mind.

For the Touch module, the example is a bit more contrieved unfortunately because of the TouchEvent API that gives you a list of touches and not just one touch. So the example supposes that you want to modify all the touches. In your case, you are only interested in a property at the top level of the event. With the currently exposed API, you could have done this:

touchCoordinates : EventWithOffset -> ( Float, Float )
touchCoordinates { event, targetOffset } =
    List.head event.changedTouches
        |> Maybe.map (.pagePos >> fromCoordinates targetOffset)
        |> Maybe.withDefault ( 0, 0 )

fromCoordinates : ( Float, Float ) -> ( Float, Float ) -> ( Float, Float )
fromCoordinates ( x0, y0 ) ( x, y ) =
    ( x - x0, y - y0 )

type alias EventWithOffset =
    { event : Touch.Event
    , targetOffset : ( Float, Float )
    }

decodeEventWithOffset : Decoder EventWithOffset
decodeEventWithOffset =
    Decode.map2 EventWithOffset
        Touch.eventDecoder
        offsetDecoder

offsetDecoder : Decoder ( Float, Float )
offsetDecoder =
    Decode.field "target" <|
        Decode.map2 (,)
            (Decode.field "offsetTop" Decode.float)
            (Decode.field "offsetLeft" Decode.float)

Don't hesitate to try it, I'm just conjecturing it works but didn't write any actual code. If that's the case and you find it more understandable, what we could do is having two examples for touch customized decoders. One at the touch level, (the one already provided in touchListDecoder) and one at the event level like above.

What do you think of this plan?