Thursday, March 15, 2007

Improved unit testing framework

I noticed a time-waster in my workflow recently. After making changes to code, I run unit tests. The old test framework would run all tests, collecting errors, printing a report at the end. However this report was not granular enough. It would only print which file failed and what the error was, not the specific test that failed. And there was no way to get a traceback after the fact. So I would often invoke test-modules, wait for the tests to finish, look at the report of failed files, and run each one individually to see what test failed, and where it failed, by using :c.

The situation is better now. More detailed information is collected at the end of a test run -- here I intentionally broke the ^ word to return incorrect results, it throws an error when the exponent is zero, otherwise always returns 3:

Now I click "Traceback", which is a presentation of a continuation. The UI presents the continuation in a window:

The code to display a continuation in this manner was taken from the UI single stepper tool. It is a generic gadget now, used by both the single stepper and for presenting continuations.

The best part? The entire unit test framework is 59 lines of code. The gadget to present a continuation (keep in mind this gadget is reusable) is 31 lines. So in total, it only takes 90 lines of code to build a useful unit testing tool in Factor.

Despite the small size, I can't think of many ways I could improve it. Some unit test frameworks out there can generate fancy HTML reports, but if I wanted that, all I have to do is HTTP server module and wrapping the call to test-modules in with-html-stream. Then I'd get clickable links for browsing word definitions in the output.

No comments: