Multi-touch gestures in the Factor UI

Recently I acquired a MacBook Pro. The new models come with a multi-touch keypad and I've found it extremely useful in Safari to be able to go back, go forward and zoom with the mouse. However, I missed the ability to do this in other applications, especially the Factor UI. The Factor UI already supports vertical and horizontal scrolling gestures, and I wanted to be able to use the other gestures as well.

Apparently, Apple's official stance is that no multitouch API will be made public until 10.6. I was slightly discouraged by this but some more Googling turned up a blog post by Elliott Harris detailing the undocumented API for receiving these events.

While normally I shy away from relying on undocumented functionality in this case the API is dead-simple and it is almost an oversight on Apple's part not to document it. And if they break it, well, it will be easy to update Factor too.

Here are the changes I had to make. First, I added some new UI gestures to extra/ui/gestures/gestures.factor; these are cross-platform and theoretically the Windows and X11 UI backends could produce them too, perhaps as a result of button presses on those "Internet" keyboards:

TUPLE: left-action ;        C: <left-action> left-action
TUPLE: right-action ;       C: <right-action> right-action
TUPLE: up-action ;          C: <up-action> up-action
TUPLE: down-action ;        C: <down-action> down-action

TUPLE: zoom-in-action ;  C: <zoom-in-action> zoom-in-action
TUPLE: zoom-out-action ; C: <zoom-out-action> zoom-out-action

Next, I edited extra/ui/cocoa/views/views.factor with some new methods for handling the new multitouch gestures, and translating them to Factor gestures:

{ "magnifyWithEvent:" "void" { "id" "SEL" "id" }
    [
        nip
        dup -> deltaZ sgn {
            {  1 [ T{ zoom-in-action } send-action$ ] }
            { -1 [ T{ zoom-out-action } send-action$ ] }
            {  0 [ 2drop ] }
        } case
    ]
}

{ "swipeWithEvent:" "void" { "id" "SEL" "id" }
    [
        nip
        dup -> deltaX sgn {
            {  1 [ T{ left-action } send-action$ ] }
            { -1 [ T{ right-action } send-action$ ] }
            {  0
                [
                    dup -> deltaY sgn {
                        {  1 [ T{ up-action } send-action$ ] }
                        { -1 [ T{ down-action } send-action$ ] }
                        {  0 [ 2drop ] }
                    } case
                ]
            }
        } case
    ]
}

Note that I'm throwing away useful information for the sake of simplicity; the zoom gesture gives you a precise zoom amount, not just +1/-1. The swipe gestures seem to be completely discrete though. I haven't implemented rotation gestures yet because I haven't figured out what to use them for.

With the above code written, the UI now sends multi-touch gestures to gadgets however no gadgets used them yet. I fired up the gesture logger, "gesture-logger" run, and tested that the new actions actually get sent. They were, except the first time I messed up the code and got the signs the wrong way round.

With that fixed, I could proceed to add gesture handlers to the various UI tools. I edited various source files:

browser-gadget "multi-touch" f {
    { T{ left-action } com-back }
    { T{ right-action } com-forward }
} define-command-map

inspector-gadget "multi-touch" f {
    { T{ left-action } &back }
} define-command-map

workspace "multi-touch" f {
    { T{ zoom-out-action } com-listener }
    { T{ up-action } refresh-all }
} define-command-map

I think its pretty clear from the above what the default gesture assignments are:

• In the browser tool, swipe left/right navigate the help history.
• In the inspector, swipe left goes back.
• In any tool, a pinch (zoom out) closes the current tool, leaving only the listener visible. A swipe up reloads any changed source files (I'm not sure if I like this yet).

15 minutes of Google, 20 minutes of hacking, and now Factor supports the fanciest feature of Apple's latest hardware.

Improvements to io.monitors; faster refresh-all

Factor's io.monitors library previously supported Mac OS X, Windows and Linux. Now it also supports BSD, but in a much more restricted fashion than the other platforms. Basically you cannot monitor directories, just individual files. This is because kqueue() only provides very limited functionality in this regard. However, having something is better than nothing, and the functionality provided on BSDs is still useful for monitoring log files and such.

On Linux, inotify doesn't directly support monitoring recursive directory hierarchies so Factor's monitors didn't support recursive monitoring, but a mailing list post by Robert Love discusses how to solve this issue in user-space. I used his solution to implement recursive monitors on Linux.

Another oddity relating to inotify is that if you add the same directory twice to the same inotify, you get the same watch ID both times, and events are only reported once. This means that the previous implementation where there was one global inotify instance shared by all monitors wasn't really as general as one would hope, because you couldn't run two programs that monitor overlapping portions of the file system. I thought of several possible fixes but in the end just changed the monitors API to accommodate this case. All monitor operations must now be wrapped in a with-monitors combinator. On Linux, it creates an inotify instance and stores it in a dynamically-scoped variable, so that subsequent calls to <monitor use this inotify. Independent inotifies in different threads don't interact at all. On Mac OS X, BSD and WIndows, with-monitors just calls the quotation without doing any special setup.

Another issue I fixed was that on Mac OS X, monitors would only work when used from the UI because no run loop was running otherwise. I made a run loop run all of the time and this allows monitors to work in the terminal listener.

Now that monitors are working better, I was able to use them to make refresh-all. This word finds all changed source files in the vocabulary roots and reloads them. It does this by comparing cached CRC32 checksums with the actual checksum of the file. Previously it would also compare modification times, but I took that code out because filesystem meta-data queries got moved out of the VM and into the native I/O code, which isn't available during bootstrap. A side-effect of this is that refresh-all became much slower, because it had to read all files. Using monitors I was able to make this faster than it has ever been. A thread waiting on a monitor is started on startup. Then, the source tree only has to be checksummed in its entirety the first time refresh-all is used in a session. Subsequently, only files for which the monitor reported changes have to be scanned. So refresh-all runs instantly if there are no changes, and so on.

The golden rule of writing comments

This is something I've wanted to say for a while, and I think many (most?) programmers don't realize it:

Never comment out code.

Comments are for natural-language descriptions of code, or pseudocode maybe.

If you comment out some code, then the parser isn't checking the syntax, the compiler isn't checking semantics, and the unit tests are not unit testing it.

So the code may as well not work. Why have non-working code in your program, especially if it's not called anywhere?

But perhaps the code is there so that you can see the what the program used to do. In that case, just fire up your favorite graphical GIT/SVN/P4/whatever frontend and check out an older revision.

Multi-methods and hooks

For a while now Factor has had 'hooks', which are generic words dispatching on a dynamically scoped variable. Hooks can be used with variables which are essentially global: the current OS, current CPU, UI backend, etc -- or variables which are truly context-specific, such as the current database connection.

I added support for hooks to the extra/multi-methods library, which is going in the core soon. While doing this I was able to significantly generalize the concept. Suppose we want to define a piece of functionality which depends on both the operating system and processor type.

We can begin with defining an ordinary generic word:

GENERIC: cache-size ( -- l1 l2 )

Notice that it is defined as taking no inputs from the stack.

I don't really know the APIs involved here, but suppose that Linux gives us a way to get this info that works across all CPUs. So we define a method specializing on the os dynamic variable (which is really treated like global):

METHOD: cache-size { { os linux } } ... read something from /proc ... ;

Now suppose on Mac OS X we use different APIs per CPU:

METHOD: cache-size { { os macosx } { cpu ppc } } ... ;

METHOD: cache-size { { os macosx } { cpu x86.32 } } ... ;

METHOD: cache-size { { os macosx } { cpu x86.64 } } ... ;

But perhaps on ARM CPUs, we just use an instruction to read the cache size without any OS-specific calls at all:

METHOD: cache-size { { cpu arm } } ... ;

Now in this case, you have an issue where if you're on Linux and ARM, the method that ends up being called depends on the order in which they were defined. If you wanted to explicitly resolve this ambiguity, you would define a new method on { { os linux } { cpu arm } }; because it is more specific than the other two, it is always called first.

The powerful thing about this new implementation of hooks is that not only can you dispatch on multiple variables, but you can add methods to any old generic which dispatches on a variable and the original designer of the generic does not have to explicitly take this into account.

For example, Factor's compiler currently has a large number of hooks dispatching on the CPU type (as an aside, Phil Dawes wrote an excellent introduction to the Factor compiler recently). If those hooks need to be further refined by OS, as is often the case with FFI-related components, the method implementation on the hook needs to perform its own dispatch; this is the "double dispatch" pattern and design patterns are something to be avoided if one wants to write quality code. When multi-methods go in the core, the compiler will simply define a series of generic words taking no inputs from the stack, and each method will specialize on the CPU, and maybe an OS too.

Another new capability is dispatching off stack values and variables in the same method. Among other things, this will be useful in eliminating a case of double dispatch in the core right now, where the <client> word for opening a client socket has to dispatch off the address type on the stack, and then call another hook which dispatches on the I/O backend stored in a variable. This can be combined into a single generic word where some methods dispatch on stack values, and others dispatch on the I/O backend variable.

The other nice thing about this is that the multi-method GENERIC: word unifies and generalizes four words in the core, GENERIC:, GENERIC# for dispatching on a value further down on the stack, HOOK: for dispatching on a variable, and MATH: which performs double dispatch on numbers only.

One of my goals for Factor 1.0 is to get the object system "right", and with the new-style slot accessors, inheritance, and singletons, we're almost there. All that remains to be done is to merge the multi-methods code. The code is still not quite ready to go in the core, though. The only feature that single dispatch has and multiple-dispatch lacks is call-next-method, which is easy to implement. A bigger hurdle to clear is performance; right now multi-methods are implemented in a naive way, where the dispatch time is O(mn) with m methods and n stack positions and variables to dispatch on. This can be improved significantly and I will find time reading the literature on optimizing method dispatch over the next few weeks.

GC fixes and improvements

With some advice from Daniel Ehrenberg, I have made a few fixes and improvements to Factor's garbage collector.

First of all, there was a potential memory leak situation I overlooked. Suppose that there is a compiled definition in the code heap which references a very large object in the data heap, and neither the compiled definition or the large object is referenced from anywhere else. Because the code heap was only ever collected when it filled up, it was possible that this large object would never be reclaimed, and it would incur unnecessary collection cycles and memory pressure as a result. This could even result in unbounded heap growth if this was done in a loop. For example, the following test case would crash Factor even though it should have run in constant space:

: leak-step 800000 f <array> 1quotation call drop ;

: leak-loop 1000 [ leak-step ] times ;

The fix is to always collect the code heap when collecting the oldest generation. Only collecting the code heap when it fills up is simply unsound because the code heap can reference objects in the data heap. If this was not the case then collecting them independently would be okay, but it isn't.

The next thing I did was improve how allocation of large objects is handled. Previously, if a new object was too large to fit in the nursery, the entire heap would grow, and every time the heap grew it would increase the size of all generations. This is generally not what you want, because if the nursery is too large, we lose locality, and if the accumulation space is too large we waste time copying objects back and forth that should really be in tenured space.

Now, the nursery and accumulation space have a fixed size that can be changed on startup with command line switches, but never changes while Factor is running. If an attempt is made to allocate an object larger than the nursery, it is directly allocated in tenured space. Presumably, if you're making a 2 megabyte array, you're going to do something with it, and hold on top it for at least a few collection cycles, rather than discard it immediately, so it makes sense to avoid the copying altogether and stick it in tenured space. On retarded microbenchmarks this change might result in worse performance, but on more realistic workloads it should be better; having a small nursery helps with locality and avoiding the inevitable copying of the large array from the nursery to accumulation space and then to tenured space is good too.

Future improvements to the GC will include:

• Shrinking the heap when memory usage lowers and remains low for several collections (Zed Shaw experimented with implementing this but didn't finish due to lack of time).
• Allocating chunks from the OS in small increments, say 1mb, instead of allocating one large heap. This would make heap growth more efficient (no need to copy everything over) and it would also allow full use of the entire address space (and not half). However it will require rethinking Factor's card marking write barrier, which currently assumes a contiguous heap.
• Using mark/sweep/compact for the oldest generation.
• Incremental marking and sweeping.

Daniel and I will be working on all of this over the summer.

Inheritance is done

I've implemented the last major missing piece of functionality in the new inheritance system, and that is method dispatch which is compatible with inheritance.

To make inheritance as useful as delegation, there needs to be an easy way to call the next applicable method. With delegation, this was done entirely in the language itself:

M: my-gadget graft*
    dup delegate graft* ... do more stuff ... ;

However the delegate method would not see the current object on the stack; hence the "slicing problem". With inheritance, there is a special word:

M: my-gadget graft*
    dup call-next-method ... do more stuff ... ;

Now call-next-method is a parsing word. It expands to this:

M: my-gadget graft*
    dup \ my-gadget \ graft* (call-next-method) ... do more stuff ... ;

At compile time, (call-next-method) expands into code which checks if the top of the stack is of the correct type for the current method; if not, it throws an error. If the check succeeds, it calls the next applicable method.

While I added this feature primarily for use with tuple inheritance, it also works in two other instances:

• Predicate class inheritance
• Union classes

The first case was a real pain to deal with before today, because there there was no way to call the next method on a parent predicate class. Suppose you had two predicate classes:

PREDICATE: foo < word ... ;

PREDICATE: bar < foo ... ;

And a generic word:

GENERIC: blah

M: foo blah ... ;

M: bar blah ... ;

If bar wanted to do something and then call the foo method, the only way was with a design pattern:

: foo-blah ... ;

GENERIC: blah

M: foo blah foo-blah ;

M: bar blah ... foo-blah ;

This is smelly. Now you can just do:

GENERIC: blah

M: foo blah ... ;

M: bar blah ... call-next-method ;

The situation with unions is similar. However, because unions give you what amounts to multiple inheritance, there is a complication.

Since next method is computed statically, and this gives different semantics than CLOS. Suppose we have the following code:

TUPLE: a ;
TUPLE: b ;
TUPLE: c ;

UNION: x a b ;
UNION: y a c ;

UNION: z x y ;

We can represent the subtype relationships here as follows:

         Z
        / \
       X   Y
      /\   /\
    B    A    C

Suppose we have a generic word:

GENERIC: funky* ( obj -- )

M: z funky* "z" , drop ;

M: x funky* "x" , call-next-method ;

M: y funky* "y" , call-next-method ;

M: a funky* "a" , call-next-method ;

M: b funky* "b" , call-next-method ;

M: c funky* "c" , call-next-method ;

: funky [ funky* ] { } make ;

Now the following two results are straightforward:

( scratchpad ) T{ b } funky .
{ "b" "x" "z" }
( scratchpad ) T{ c } funky .
{ "c" "y" "z" }

However, suppose you have an instance of a on the stack. The next method after a is either x or y, and this depends on the phase of the moon; the two classes overlap but neither is a subset of the other, so we have some ambiguity here. However, for the sake of argument, suppose the next method is x. Now, statically, the next method after x is z. However, given that we have an a on the stack, the method on y is also applicable, so if we computed it dynamically, the next method would be y and then z. So to summarize, right now, we get one of the following two results:

( scratchpad ) T{ a } funky .
{ "a" "x" "z" }
( scratchpad ) T{ a } funky .
{ "a" "y" "z" }

However, it would be more desirable to get one of the following:

( scratchpad ) T{ a } funky .
{ "a" "x" "y" "z" }
( scratchpad ) T{ a } funky .
{ "a" "y" "x" "z" }

However, doing so would require expanding (call-next-method) into a type dispatch, not a direct call. Implementing this efficiently and updating all occurrences of (call-next-method) as classes are redefined is still an open problem, and one that I will tackle later. The current static behavior will suffice for now.

Now that the code is done, I still need to write documentation and update the core to actually use inheritance in place of delegation. After that, I will return to the web framework and let things settle down a bit. Once I'm happy with the stability of both the core and the state of the web framework, I will tackle multi-methods; what this really means is efficient implementation of multi-methods, given that an inefficient implementation already exists in extra/. This presents its own set of unique challenges but I look forward to solving that problem.

Once Factor has multi-methods, our object system will be comparable in power to CLOS, however I believe it will offer genuine improvements and innovations in several respects. Furthermore, the implementation will be a lot simpler than PCL.

Converting a table to a tree

I found an interesting Haskell snippet in a reddit comment.

From the comment:

Here's a slightly longer function I created a while ago:

listToForest :: Eq a => [[a]] -> Forest a
listToForest = map toTree . groupBy ((==) `on` head) . filter (/= [])
               where toTree = Node . (head . head) <*> (listToForest . map tail)

The function turns a table of results, such as that which might come back from a database query, into a tree. e.g.

[[A, B, C],
 [A, B, D],
 [A, E, F]]

Becomes:

    A 
   / \
  B   E
 / \   \
C   D   F

I decided to try implementing this in Factor, and I came up with three versions.

All versions represent a tree as its root node, and each node is a hashtable mapping the key of the child to the child node.

The first version does not use any utility words other than what's in the standard library; it consists of a single, rather long word:

: list>forest ( list -- forest )
    [ empty? not ] subset
    H{ } clone [ [ >r unclip r> [ ?push ] change-at ] curry each ] keep
    [ list>forest ] assoc-map ;

The second version uses a couple of utilities:

: push-at ( value key assoc -- )
    [ ?push ] change-at ;

: classify ( seq classifier -- assoc )
    H{ } clone [ [ slip push-at ] 2curry each ] keep ;

: list>forest ( list -- forest )
    [ empty? not ] subset [ unclip ] classify [ list>forest ] assoc-map ;

The third version has a different implementation of classify which uses extra/fry syntax sugar:

: push-at ( value key assoc -- )
    [ ?push ] change-at ;

: classify ( seq classifier -- assoc )
    H{ } clone [ '[ @ , push-at ] each ] keep 

: list>forest ( list -- forest )
    [ empty? not ] subset [ unclip ] classify [ list>forest ] assoc-map ;

If this was a one-off thing I'd use the first version. If I was writing production code, I would use one of the second two, and I might even put the utilities in a shared vocabulary since they're generally useful. In the last version, the only stack manipulation operator is keep, and everything else just happens to be on the stack in the correct order. This is how good stack code is written.

Some explanation:

• ?push is like push except it makes a new vector if the input is f
• slip calls a quotation underneath the top of the stack.
• push-at adds a value to the sequence stored at a key in a hashtable.
• classify takes a sequence and a quotation which decomposes a sequence element into a key and a value. It builds an association mapping all the unique keys to corresponding values.
• unclip takes a sequence and outputs the rest of the sequence followed by the first element.
• Haskell's groupBy is a generalization of my classify; it works with an equivalence relation rather than a "fibration" into a set with an implied equivalence relation (wrong term, but I'm not sure what the correct term is).
• My classify runs in linear time and I suspect groupBy is quadratic, but I could be wrong.

I'd be interested in seeing a version of this function in idiomatic Common Lisp or Scheme. I suspect it would be somewhat longer, but not horribly so.