Callbacks working on AMD64, Objective C exceptions

I've been debugging callbacks and now they work on AMD64. I also cleaned up the x86 assembler and extended it to support the full range of x86 addressing modes. On the Mac side, Objective C exceptions are now caught by the Factor runtime and reported to Factor code in the same manner as Factor exceptions; this makes debugging Cocoa code much more pleasant.

I would like to port Factor to MacIntel, but I don't have such a box; if any reader of this weblog does and would like to offer me an ssh account, please let me know! Perhaps it won't even take any work; the x86 compiler backend exists, and Factor does run on PowerPC Macs. The only issues I can forsee are in the FFI; if the OS X ABI is sufficiently different from Linux/x86, changes will need to be made.

Callbacks working on PowerPC and x86

The title says it all. You can now pass function pointers to C libraries which invoke Factor code. I still have to update the AMD64 backend for the new functionality, but PowerPC and x86 callback support is pretty much done, modulo a few bugs.

Working on callbacks; another server problem fixed

I've started implementing callbacks from C code to Factor. They are defined like so:

: foo-callback ( -- xt )
    "void" { "int" "int" } [ + . ] alien-callback ;

Once compiled, the foo-callback word will push an address on the stack. This address is a compiled code block which uses C calling conventions and may be passed to C library functions expecting a function pointer as an argument. I took care of the interaction with continuations by invoking callbacks inside a nested "context" with initially-empty stacks. Continuations may be saved and restored while a callback is in progress, and the only restriction is that a continuation saved inside a callback must only ever return to the top level while it is inside that callback. Another potential issue is that if the callback quotation throws an unhandled exception, the C stack will unwind all the way up the top-level exception handler, thus the interveined C code may leak memory or have other problems.

Callbacks are not complete yet -- there's some corner cases that are not supported, and I have not got things working on anything other than PowerPC. On x86, callbacks which do not take arguments work.

Once callbacks are working, I will continue hacking away at the Cocoa bindings. I'm still trying to come up with a clean way to integrate message sends with Factor syntax. The current scheme of defining words like [initWithFrame:backingStore:defer:] is temporary. At present, I'm leaning towards something like this:

[ foo #initWithFrame bar #backingStore 0 #defer ] send

Where the #... words would be automatically-generated when Cocoa classes are imported, simply by splitting selector names. I still need to work out how this will end up being compiled efficiently. I certainly do not want all Objective C message sends to be dynamically looked up; so under the hood, it will still rely on one compiled optimized alien call word per class/method.

In other news, we found and fixed another stability issue on the factorcode.org server. This time the problem was in the continuation-based web framework.

QuickTime and PDFKit demos

I have cleaned up the Cocoa bridge and started putting together a library of utility words in library/cocoa/. The WebKit demo is much shorter now because the bulk of its code has been moved to the library. I have put together two new demos, a QuickTime player and PDF viewer.

Both demos also use the new Cocoa event loop code, which integrates Cocoa event handling with Factor's multitasking. You can continue entering expressions in the listener and such while the Cocoa demos are running. However, you cannot run the SDL-based UI after one of these Cocoa demos. When the UI gets a Cocoa backend this will no longer be an issue.

Improved Cocoa bridge; fixed some I/O bugs

The Cocoa bridge now uses runtime introspection to discover methods defined on classes. While the current implementation is far from perfect, it saves me from the pain of declaring methods by hand, and it works well enough to run a minimal web browser demo:

• Source code
• Screenshot

The Cocoa bridge is far from complete. It needs to support Objective C exceptions, autorelease pools, subclassing, messages that return structs, and various other odds and ends. Subclassing is the hardest one, since it requires callbacks to be implemented first. However, 100% coverage of the Cocoa feature set is not a goal, since I'll only be using this binding as a backend for the Factor UI, not directly for application development (but then, who knows).
Over the last week or so the factorcode.org server went through some rough times. I was running the HTTP server and IRC bot in the same Factor instance, and it started to crash on a regular basis. Turns out there were two race conditions in the I/O code:

• If one thread posted a write request to an I/O port and another thread closed the port before the write request was serviced, the port would actually be closed before the former request was completed, because incredibly enough, I implemented the callback queue as a stack. Making it a real queue fixed the problem


• The second problem was trickier. If one thread closed a stream with pending output, and another thread wrote more output to the stream before the originally pending output was complete, the same problem would occur, in that the port would be closed before all I/O was complete. This time, the fix was not FIFO -vs- LIFO, but simply marking a port as closed before pending output is flushed; so once a close operation is initiated, no more output other than the final flush can begin.

Another problem was a scalability issue. The file responder used Chris Double's cont-responder framework, which is very nice and clean. It had one problem, however, in that it would queue up the entire response in a string before sending it to the client -- this was a problem if several people begin downloading a large file at the same time! This is also now fixed.
I'm guessing the factorcode.org server is the highest-volume deployment of the Factor HTTP server to date, so it has uncovered some bugs I'd never notice during local testing. Hopefully, there won't be too many more.