Multi-touch gestures in the Factor UI

Recently I acquired a MacBook Pro. The new models come with a multi-touch keypad and I've found it extremely useful in Safari to be able to go back, go forward and zoom with the mouse. However, I missed the ability to do this in other applications, especially the Factor UI. The Factor UI already supports vertical and horizontal scrolling gestures, and I wanted to be able to use the other gestures as well.

Apparently, Apple's official stance is that no multitouch API will be made public until 10.6. I was slightly discouraged by this but some more Googling turned up a blog post by Elliott Harris detailing the undocumented API for receiving these events.

While normally I shy away from relying on undocumented functionality in this case the API is dead-simple and it is almost an oversight on Apple's part not to document it. And if they break it, well, it will be easy to update Factor too.

Here are the changes I had to make. First, I added some new UI gestures to extra/ui/gestures/gestures.factor; these are cross-platform and theoretically the Windows and X11 UI backends could produce them too, perhaps as a result of button presses on those "Internet" keyboards:

TUPLE: left-action ;        C: <left-action> left-action
TUPLE: right-action ;       C: <right-action> right-action
TUPLE: up-action ;          C: <up-action> up-action
TUPLE: down-action ;        C: <down-action> down-action

TUPLE: zoom-in-action ;  C: <zoom-in-action> zoom-in-action
TUPLE: zoom-out-action ; C: <zoom-out-action> zoom-out-action

Next, I edited extra/ui/cocoa/views/views.factor with some new methods for handling the new multitouch gestures, and translating them to Factor gestures:

{ "magnifyWithEvent:" "void" { "id" "SEL" "id" }
    [
        nip
        dup -> deltaZ sgn {
            {  1 [ T{ zoom-in-action } send-action$ ] }
            { -1 [ T{ zoom-out-action } send-action$ ] }
            {  0 [ 2drop ] }
        } case
    ]
}

{ "swipeWithEvent:" "void" { "id" "SEL" "id" }
    [
        nip
        dup -> deltaX sgn {
            {  1 [ T{ left-action } send-action$ ] }
            { -1 [ T{ right-action } send-action$ ] }
            {  0
                [
                    dup -> deltaY sgn {
                        {  1 [ T{ up-action } send-action$ ] }
                        { -1 [ T{ down-action } send-action$ ] }
                        {  0 [ 2drop ] }
                    } case
                ]
            }
        } case
    ]
}

Note that I'm throwing away useful information for the sake of simplicity; the zoom gesture gives you a precise zoom amount, not just +1/-1. The swipe gestures seem to be completely discrete though. I haven't implemented rotation gestures yet because I haven't figured out what to use them for.

With the above code written, the UI now sends multi-touch gestures to gadgets however no gadgets used them yet. I fired up the gesture logger, "gesture-logger" run, and tested that the new actions actually get sent. They were, except the first time I messed up the code and got the signs the wrong way round.

With that fixed, I could proceed to add gesture handlers to the various UI tools. I edited various source files:

browser-gadget "multi-touch" f {
    { T{ left-action } com-back }
    { T{ right-action } com-forward }
} define-command-map

inspector-gadget "multi-touch" f {
    { T{ left-action } &back }
} define-command-map

workspace "multi-touch" f {
    { T{ zoom-out-action } com-listener }
    { T{ up-action } refresh-all }
} define-command-map

I think its pretty clear from the above what the default gesture assignments are:

• In the browser tool, swipe left/right navigate the help history.
• In the inspector, swipe left goes back.
• In any tool, a pinch (zoom out) closes the current tool, leaving only the listener visible. A swipe up reloads any changed source files (I'm not sure if I like this yet).

15 minutes of Google, 20 minutes of hacking, and now Factor supports the fanciest feature of Apple's latest hardware.