Factor is now five years old

Time flies! It is hard to believe that another year has gone by and it is now five years since I began to work on the project that would become Factor.

Community

Really, the most important thing that has happened in the last year is that there are some new contributors around. Big shout-out to James Cash, William Schlieper, Joe Groff, Phil Dawes, Aaron Schaefer, Alfredo Beaumont and anybody else who joined the project within the last year. Along with the long-time contributors who were there from the very beginning -- Doug Coleman, Eduardo Cavazos, Chris Double, Daniel Ehrenberg, and everyone else who has helped out over the years -- you people make Factor fun. I'm honored to be working with such high-caliber programmers as the ones in the Factor community: they are the most interesting, intelligent and creative people I know in the software trade. I never anticipated having many people show any interest in Factor at all, but nowadays it is common to see more than 50 people in the #concatenative channel of irc.freenode.net.

Switch to Git

At the end of last year, we switched to git from darcs for revision control. I'd like to think the changeover was a complete success -- git has many powerful features and the five-minute-long "darcs push" and "darcs pull" waits are a thing of the past.

Continuous integration

Instead of having manual releases once in a while and keeping the repository in various states of breakage between releases, we've switched to a model of continuous builds and automatic binary packages. We have binary packages for 11 platforms. For those that prefer to build from source, known-good automatically-updated clean branches are available. Stability has improved with continuous integration, since unit tests are being run all the time; and the days of checking out Factor from the repository only to have bootstrap fail are long gone.

Soon we will push ahead and add even more platforms; in the near future we will be adding the missing 64-bit and PowerPC systems and the total number of supported platforms with binary packages will increase to 17. Write once, run anywhere.

Language changes

Some of the language changes over the last year have helped make Factor more expressive and maintainable; idiomatic Factor from today is a huge improvement over the state of affairs a year ago. The main improvements are enumerated below, but there were countless others:

• Eduardo's new accessors
• Eduardo's cleave combinators
• Inheritance
• Tuple slot declarations
• Various improvements to the lexical scope and closures library, locals

While language evolution was a big focus of mine for the first few months of the year, I have taken a break from major language changes over the last six months ago to focus on libraries, the compiler, as well as stability. This focus paid off as many bugs and performance problems were fixed; very soon I will start moving forward with the language again, with multiple dispatch at the top of the list.

Library improvements

Many notable new libraries were added, including:

• Doug Coleman's database abstraction layer with O/R mapper and SQLite, Postgres backends
• Chris Double's parsing expression grammars
• Daniel Ehrenberg's Unicode library
• Joe Groff's BLAS binding
• The new concurrency library by yours truly
• The new web framework by yours truly, still a work in progress

The I/O system alone saw some major progress:

• File system change monitors
An awesome process launching that beats anything I've seen in any language
• Secure sockets support

The implementation

Along with improvements to the language and library, we've had major progress on the low-level plumbing. Since the goal is to make Factor ready for high-load production applications, I've had to learn a lot about garbage collection and compilers over the last five years, and especially the last year alone.

The biggest change in the last year is that Factor is no longer a dual interpreted/compiled implementation. The interpreter has been replaced with a non-optimizing compiler and the optimizing compiler saw some major improvements (this, this, and this). There have also been countless improvements to the GC, and algorithmic tuning of the core libraries.

The web site

We now have a community blog aggregator, and a pastebin for the IRC channel. Since we eat our own dogfood, both of those are, naturally, written in Factor and running on the Factor HTTP server, along with the main Factor website.

Line counts

Every year, I've given some line number counts for the Factor codebase. Here are the latest stats:

• Factor VM: 12050 lines of C, 391 lines of assembly
• Factor core: 9077 lines of Factor
• Factor basis library: 64311 lines of Factor
• Factor extra library: 50825 lines of Factor
• Unit tests: 24308 lines of Factor
• Documentation: 242426 words of help markup

Compared to last year,

• The VM has grown by about 2000 lines of code; this isn't much considering there have been many improvements made, and the interpreter was replaced by a simple JIT
• Parts of core and extra were split off and moved to a new basis library which includes useful libraries, tools, the UI, and the compiler; the code that remains in the core is really the fundamentals required to parse and execute Factor code and not much else
• The amount of library code in basis and extra has increased considerably

Line counts are often meaningless, but in this case we see a very good trend; the lowest levels of Factor, that is the VM and the core library, are certainly not bloating at all; on the other hand, many more libraries are being added, and more and more Factor is starting to complete with more established languages in terms of library coverage.

Last year I made the claim that Factor 1.0 would be released this year. This was perhaps too optimistic. While slapping a "1.0" sticker onto the tree could be theoretically done at any time, I want to release a 1.0 that I and everyone else in the community is truly happy with. I remember how underwhelming Java 1.0 was; I don't want to make that mistake. With continuous builds, anyone can try out and test Factor and stability is improving all the time, so there is no need to rush forward and declare my job done prematurely. Having said that, I tentatively give myself another year until 1.0.

The rate of progress over the last year has been amazing. I'm looking forward to yet another year of Factor.

As a side note, I began jEdit development some time in August 1998. So this month jEdit turns 10 years old! I stopped maintaining jEdit in late 2005 and passed the torch to a new development team who remain active to this day; a new release was made a few weeks ago. With 2,000 downloads daily, and more than 4.5 million downloads in all, jEdit's success continues to amaze me. It is still my preferred text editor and I learned almost everything I know about software engineering from working on jEdit, with much of the rest coming from my experience working on Factor.