Sunday, August 12, 2007

Mixins

Consider typical message-passing OOP (with made up syntax):
class Rectangle {
state w, h;
method area() { ... }
method paint() { ... }
}

class Ellipse {
state major, minor;
method area() { ... }
method paint() { ... }
}

We can add new types of shapes easily, but we cannot define new operations on existing shapes without modifying existing code. In a language with algebraic data types, the situation is reversed (again, made up syntax):
data Shape = Rectangle w h | Ellipse major minor
area shape =
case shape of
Rectangle w h -> ...
Ellipse major minor -> ...

draw shape =
case shape of
Rectangle w h -> ...
Ellipse major minor -> ...

Here, we can define new operations on existing shapes but we cannot define new shapes and have the existing operations work with them.

Generic functions/words give us the best of both worlds. Here's the Factor syntax:
GENERIC: area ( shape -- )
GENERIC: draw ( shape -- )

TUPLE: rectangle w h ;
M: rectangle area ... ;
M: rectangle draw ... ;

TUPLE: ellipse major minor ;
M: ellipse area ... ;
M: ellipse draw ... ;

We can define new shapes implementing the area and draw generic words, and we can define new generic words which implement methods for rectangle and ellipse, without coupling anything together.

Now let's go back to our first message-passing OOP example, and suppose this language supports multiple inheritance of mixins. There are many definitions of what a mixin is, but for now, assume a mixin is a class with no state, just behavior.
class Shape {
abstract method triangulate();

method paint() {
... triangulate the shape with triangulate(),
then render it with a standard algorithm ...
}
}

class Rectangle < Shape {
state w, h;
method area() { ... }
method triangulate() { ... }
}

class Ellipse < Shape {
state major, minor;
method area() { ... }
method triangulate() { ... }
}

We can define new shapes which mixin the Shape mixin, but we cannot add additional mixins to the Rectangle and Ellipse data types.

In traditional Factor, the rough equivalent of the above would be to use a union class:
GENERIC: area ( shape -- )
GENERIC: triangulate ( shape -- triangle-seq )
GENERIC: draw ( shape -- )

TUPLE: rectangle w h ;
M: rectangle area ... ;
M: rectangle triangulate ... ;

TUPLE: ellipse major minor ;
M: ellipse area ... ;
M: ellipse triangulate ... ;

UNION: shape rectangle ellipse ;
M: shape draw ... triangulate, then draw the triangles ... ;

Now we can certainly define new union classes, and make either rectangles or ellipses instances of these union classes, but we cannot extend the shape union class with new shapes.

Until now, the accepted workaround ("design pattern") would be to not define a shape class at all, and instead define a draw method on the object class:
M: object draw ... triangulate, then draw the triangles ... ;

M: beizer-curve draw ... override default implementation ... ;

However, this was crufty. Defining methods on the object class is clutter. Plus, if you lose the shape union, you no longer have a way to distinguish shapes from other objects.

Enter Factor's new mixin classes feature.

Just like generic words generalize message passing OOP and case-based pattern-matching, Factor's mixin classes generalize multiple behavior inheritance and union classes.

We can define a new mixin as follows:
MIXIN: shape

We can specialize generic words on this mixin:
M: shape draw ... default implementation in terms of triangulate ...

But here's the kicker: unlike a union, where all members must be listed up-front, we can declare an existing class to be an instance of an existing mixin:
INSTANCE: rectangle shape

INSTANCE: ellipse shape

I successfully used mixins to remove some repeated code between the various implementations of sequences and associative mappings. Until now, there was no way to specialize a generic word on all sequences; now this is possible:
GENERIC: foo

M: sequence foo ... ;

A mixin is essentially a suite of methods; there is some similarity between Haskell typeclasses and Factor mixins. I'm still discovering the various applications of mixins; after Factor 0.90 is released, I intend to apply them to our I/O stream implementations and remove some boilerplate there.

So to recap, Factor's object system allows the following operations without forcing unnecessary coupling:
  • Defining new operations over existing types
  • Defining existing operations over new types
  • Importing existing mixin method suites into new types
  • Importing new method suites into existing types
  • Defining new operations in existing mixin method suites
  • Defining new mixin method suites which implement existing operations

This is a lot more general than mainstream message passing OOP and allows a wider range of problems to be expressed directly in terms of method dispatch.

Saturday, August 11, 2007

Going to Austin

I will be in Austin, Texas from the 18th of August to the 26th. I'll be hanging out and hacking code with Doug and Eduardo. If anyone else in the Austin area wants to meet up, send me an e-mail.

Friday, August 10, 2007

Smalltalk is dying

I just saw that ObjectArts has discontinued development of Dolphin Smalltalk.

It seems Smalltalk is going the way of the dodo bird. Basically, we're left with a rapidly dwindling number of commercial implementations which are way overpriced considering their relatively low level of polish and functionality; and the only open source option is Squeak, which is not production-ready and never will be. Then there are various dead and semi-dead pre-alpha-stage projects. One commercial Smalltalk which stands out from the rest is Ambrai, but it is in early development, and it is unlikely a proprietary Mac OS X-only product will attract a significant base of library contributions. Perhaps Pepsi will get somewhere, who knows.

A pity, really, considering how computing might look today if things went differently in the early days of Smalltalk.

Update: I wasn't talking about popularity at all. It doesn't matter if Smalltalk isn't popular; for some reason, popular languages tend to suck (Java, Ruby, COBOL, etc.) The problem with Smalltalk is that the implementations are in terrible shape. The closest thing to a usable open source implementation is Squeak, and it leaves a lot to be desired. Common Lisp is an example of a less popular language which doesn't suffer from this problem; open source Lisp implementations are fantastic.

New suite of benchmarks

I spent a bit of time cleaning up various benchmarks and putting them over a common framework. Now you can do "benchmark" run to run a suite of benchmarks. The results are collected and printed in a table:
Benchmark                    Run time (ms) GC time (ms)
benchmark.continuations 473 3
benchmark.empty-loop 480 0
benchmark.fib1 194 0
benchmark.fib2 855 0
benchmark.fib3 1418 0
benchmark.fib4 3334 60
benchmark.fib5 2126 11
benchmark.iteration 8829 23
benchmark.mandel 4421 77
benchmark.nsieve 2221 3
benchmark.nsieve-bits 65679 239
benchmark.partial-sums 26293 183
benchmark.raytracer 26031 242
benchmark.recursive 36740 233
benchmark.reverse-complement 15637 167
benchmark.ring 9124 79
benchmark.sort 1934 38
benchmark.spectral-norm 32142 991
benchmark.sum-file 287 0

I want to set up a continuous testing cluster with a number of machines. The machines would perform the following tasks:
  • Build Factor
  • Run unit tests
  • Run benchmarks
  • Create deployment images for various modules and upload them to a "Factor application central" web site

Tuesday, August 07, 2007

Named local variables and lexical closures in Factor

I previously wrote about adding named parameters to Factor as an extension library. Well, now I've beefed up this code with support for lexical closures, and threw it in extra/locals.

Hopefully, this module can now replace the less efficient rewrite-closures library by Eduardo Cavazos.

Here is an example word using locals:
:: add-test | x y z | x y + z + ;
1 2 3 add-test .
6

The syntax is simple; to define a word which uses locals, you use :: instead of :. Then you follow the word name with a list of locals enclosed in | characters, and use these locals anywhere in the body of the word.

Closure conversion is performed:
:: map-test | seq inc | seq [ inc + ] map ;
{ 10 20 } 5 map-test .
{ 15 25 }

Here, we're using the "inc" local inside the quotation passed to map. The :: word performs all the necessary rewriting to make this work. The key to making this efficient is compiled curry.

You can also define lambdas, which are like quotations except inputs are placed in local variables:
:: map-test | seq inc | seq [| elt | elt inc + ] map ;
{ 10 20 } 5 map-test .
{ 15 25 }

The lambda [| elt | elt inc + ] is equivalent to the quotation [ inc + ], and in both cases closure conversion is performed by the :: word.

Lambdas can be used in normal colon definitions, too:
: foo-bar 1 2 [| x y | x y * ] lambda call ;
foo-bar .
3

The lambda word converts a lambda into code for producing a quotation. If the input to lambda is literal, the code transformation is done at compile time.

So how does the closure conversion actually work? Consider the following code:
[| a b | a [| c | c b + ] map ]

The inner quotation refers to the free variable b. We can rewrite it as follows:
[| a b | a b [| c b | c b + ] curry map ]

That is, we add all free variables to the list of inputs, then fetch these free variables right before pushing the quotation; after the quotation is pushed, we use curry to partially apply the quotation to the free variables. All lambdas and quotations are rewritten this way, as we lift free variables up. In the above example, we have already arrived at a form with no free variables anywhere. Now, we convert to point-free form. This is done in a brute-force way; within a lambda, all locals are stored on the retain stack. Here is the final result of closure conversion applied to the above example:
[
>r
>r r>
r>
dup
>r
>r r>
dup
>r
[ >r >r r> r> dup >r >r r> dup >r + r> drop r> drop ] curry
map r>
drop r>
drop
]

Looks rather complicated, but most of the stack shuffling is optimized away by the compiler.

This vocabulary is not in the darcs repository yet, because it depends on compiled curry, which I'm still in the process of debugging. However, it should work with the latest code from darcs, except words using closures won't compile. I put the code up on LispPaste.

In only 121 lines, I was able to add efficient lexical scoping to Factor. Even though I encourage people to try to express everything in a clean way using the stack only, sometimes locals are handy (complex math formulas, bizarro native APIs taking 11 parameters each, etc). Having more options is always good. Plus, this library makes a great demo of how to extend the language in a radical way without altering the core implementation!

Monday, August 06, 2007

"er" words

Take a look at the new implementation of subset:
: pusher ( quot -- quot accum )
V{ } clone [ [ push-if ] 2curry ] keep ; inline

: subset ( seq quot -- subseq )
over >r pusher >r each r> r> like ; inline

The idea of having a pusher word was originally suggested by Eduardo Cavazos. This word takes a predicate quotation and yields two values, the first being a quotation taking an object as input, and the second being a vector. If you call the quotation with an object for which the predicate yields true, the object is pushed on the vector. The generated quotation "closes over" the vector. Now the only thing subset has to do is pass the pusher quotation to each, and hide the pusher vector on the retain stack until after the iteration is done. When iteration has completed, the pusher vector contains all elements which passed the predicate.

I think there is a very interesting idiom here, and it is worth exploring this idea more.

Also, note that while from the outside, subset is referentially transparent, it uses mutable state internally. While it may be possible to write something just as simple without mutation, I think in many cases, being able to mutate considerably simplifies code; certain algorithms are just more naturally expressed with mutation, at least in a stack language, and integrating abstractions such as monads or uniqueness types into a stack language is still an open research problem (but Christopher Diggins is working on it...) There is something to be said for allowing different programming styles in a language, instead of forcing a single style on the programmer (as in Haskell and other purely functional languages, or even Smalltalk's "everything-is-message-passing-OOP".)

Efficient partial function application (aka compiled curry)

Note: the code described here is not in darcs yet, but will make it there within the new few days after I fix a few residual bugs.

Ever since quotations became a first-class data type, Factor has had a curry word:
  3 [ + ] curry .
[ 3 + ]

This is essentially a partial function application. (I'm aware that currying and function application is not the same, but this word was named before I was aware of that fact, and anyway, curry reads better than papply or partial.)

Previously, this was an expensive word to use. Not only did it allocate a new quotation and copy the existing quotation's elements, but words which called quotations produced by curry did not compile, and had to run in the interpreter; this is due to the compiler's design, which "lifts" all quotations up to their call site.

Now, this has all changed. For interpreted code, curry now runs in constant time, not linear time proportional to the quotation's length, because it allocates a small object holding the object together with the quotation. These objects are instances of a curry class, but they print like quotations and are equal to quotations having the same elements, so they're essentially identical:
3 [ + ] curry [ 3 + ] = .
t

When a curried quotation is called by call in the interpreter, the VM simply pushes the curried object first, then calls the quotation.

In compiled code, we can do even better now. The compiler knows about curry and applies a rewrite rule to the dataflow graph. Essentially, it replaces all occurrences of curry's output value with a pair of values, being the object and quotation; it also "widens" all stack shuffling from the point where the curry is created to where it is called. If the curry is passed to an ordinary word, the compiler inserts a real call to curry which allocates memory; however, the key idea behind this rewrite rule is that curry instantiation is "lazy", and is not done at all if the curry is simply passed downward to a combinator.

For example, consider this example:
: foo ( x y -- x' y' ) 3 [ + ] curry 2apply ;

This word adds 3 to the top two stack elements. The compiler first inlines the 2apply word:
: foo ( x y -- x' y' ) 3 [ + ] curry tuck >r >r call r> r> call ;

Now it applies the rewrite rule:
: foo ( x y -- x' y' ) 3 [ + ] abc--bcabc >r >r >r call r> r> r> call ;

The compiler represents stack shuffles in symbolic form, internally; I'm writing abc--bcabc to mean a shuffle with that effect, for which no word exists in the library.

Now, it lifts the quotation up to the call site:
: foo ( x y -- x' y' ) 3 [ + ] abc--bcabc >r >r >r drop + r> r> r> drop + ;

Finally, it removes the literal value [ + ], since it is now dead; it travels around the stack without ever being used for anything, and is dropped:
: foo ( x y -- x' y' ) 3 ab--bab >r >r + r> r> + ;

So in this case, the compiler will produce the exact same code for the following two snippets:
3 [ + ] curry 2apply
3 tuck >r >r + r> r> +

While [ + ] curry 2apply is not a useful thing to write, since the direct expansion is simpler, it does demonstrate how the compiler is able to remove the object allocation from this code.

So, curry is efficient now, both in the interpreter and compiler. But what does this mean?

The role played by curry here is somewhat like a Lisp lambda which closes over one free variable. We can use curry to package up a bunch of values, pass them to the combinator, and fiddle with them from inside a quotation. Until now, you either had to pay a performance penalty for using curry, or you would use stack juggling tricks. For example, consider the each and 2each combinators. Here is the current implementation:
: (each) ( seq quot i -- seq quot i )
[ rot nth-unsafe swap call ] 3keep ; inline

: each ( seq quot -- )
over length [ (each) ] repeat 2drop ; inline

: (2each) ( quot seq seq i -- quot seq seq i )
[ 2nth-unsafe rot dup slip ] 3keep ; inline

: 2each ( seq1 seq2 quot -- )
-rot 2dup min-length [ (2each) ] repeat 3drop ; inline

Here is an implementation using curry:
: (each) ( seq quot -- n quot' )
>r [ length ] keep r>
[ >r nth-unsafe r> call ] 2curry ; inline

: each ( seq quot -- )
(each) each-integer ; inline

: (2each) ( seq1 seq2 quot -- n quot' )
>r [ min-length ] 2keep r>
[ >r 2nth-unsafe r> call ] 3curry ; inline

: 2each ( seq1 seq2 quot -- )
(2each) each-integer ; inline

Note that each-integer is the new version of repeat, with a slightly altered stack effect.

In the old code, the quotation passed to repeat has to manually save and restore various values that it needs; now that curry is efficient, we just package those values up with the quotation before passing it along to each-integer.

It is somewhat ironic that while Factor is as "concatenative" language, until now concatenating quotations together was discouraged, at least where performance was important! But now, we can easily write an efficient compose word which takes a pair of quotations:
: compose [ >r call r> call ] 2curry ; inline

For example,
[ 2 2 ] [ + ] compose .
[ [ 2 2 ] [ + ] >r call r> call ]

It is pretty clear that this quotation has the same effect when called as [ 2 2 + ]. The difference between compose, which only works on quotations and produces a funny result, and append, which works on all sequences, is that since compose is built from curry, it runs in O(1) time, and the compiler is able to eliminate its runtime cost altogether.

For example, consider the assoc-each combinator. It is implemented in terms of the assoc-find combinator, since assoc-find is the only primitive combinator each assoc instance must implement. On each iteration, we want to call the quotation, but always output f, forcing assoc-find to continue the iteration until the end. Formerly, assoc-each was implemented as follows. First, we had a utility word, assoc-find-with, which called assoc-find while retaining a parameter on the stack. It was in implemented using a assoc-with word which was also used for assoc-each-with, and so on:
: assoc-with 2swap [ >r -rot r> call ] 2keep ; inline

: assoc-find-with ( obj assoc quot -- key value ? )
swap [ assoc-with rot ] assoc-find
>r >r 2nip r> r> ; inline

: assoc-each ( assoc quot -- )
swap [ rot call f ] assoc-find-with 3drop ; inline

This is ugly as hell, and essentially boilerplate. Here is the new way:
: assoc-each ( assoc quot -- )
[ f ] compose assoc-find 3drop ; inline

That's it; no need to define -with words, and no unnecessary stack shuffling. We exploit the "concatenative" nature of Factor here -- composition of sequences is the same as composition of functions!

Which brings me to the next topic; all the "manually curried" combinators you know and love, such as each-with, map-with, and so forth, have been moved to a deprecated vocabulary. They're going away soon. Updating code is easy.
swap [ swap XYZ ] each-with  ===  [ XYZ ] curry each
[ XYZ ] each-with === [ XYZ ] curry* each

Same for map-with, subset-with, and so on. The curry* word is just partial application on the second stack element; it is defined in terms of curry.

All the sequence and assoc combinators have been greatly simplified. In fact, I was able to move the 2swap word out of the core and into extra/shuffle; stack shuffling has been simplified by curry to such an extent that in particular, 2swap is simply not needed anymore. Other more complex shuffle words such as roll and pick are used less frequently now, as well.

By lowering the cost of an abstraction, I was able to simplify code by a non-trivial amount. There's a general theme here: the goal of a compiler is to make idiomatic code run fast. It is not good enough to only accelerate hand-unrolled, tuned code chock-full of irrelevant detail and declarations. Nobody wants to write unnecessarily complex code just to please the compiler.