Converting modules to the new module system

I think I can now say the move to the new module system is done, at least for the core. The last remaining component I had to update was the documentation, which has been neglected as of late. In the end I documented some 400 new words and updated the documentation for hundreds more.

What remains to be done is updating all contributed modules. A few modules such as the HTTP server have been updated already, but a lot remains to be done.

Tomorrow, I'm leaving to New Zealand. I will be there until mid-July. I'll be hanging out with Chris Double in Auckland, and my old school friends in Wellington (where I lived from 1992 to 2002). I'll also stop by in Napier and perhaps the east coast of the North Island; if anybody wants to meet up, let me know by e-mail! During this time I will be taking a break from programming (and everything else); this should give the Factor contributors some time to catch up with the new module system. When I come back, I will take care of any remaining modules myself; I hope most modules will be updated by then.

While I am away, Daniel Ehrenberg will be in change of harvesting patches; if you have patches in an HTTP-hosted repository, please let him know so that he can pull them, and if you plan on submitting patches by e-mail, send them to the Factor-talk mailing list so that he can apply them to his repository. His repository is found at

http://littledan.onigirihouse.com/repos

This ensures that when I return, I can just pull all patches from him and continue working.

Here are some general guidelines which should help contributors update their modules:

• Try to rethink the module's division into source files. Prior to the new module system this could be arbitrary, but now it is in one-to-one correspondence with vocabulary names. Try not to have too many vocabularies, but on the other hand and try to avoid having vocabularies which group unrelated functionality together.


• Many core words were moved around between vocabularies, especially in the UI. If you find that a source file's USING: form is mostly out of date, you can use the following trick. Paste the following word definition in the listener:
  

  USING: parser namespaces vocabs kernel assocs sequences hashtables sorting prettyprint ;
  
  : USING?
      use get [
          dictionary get
          [ nip vocab-words eq? ] assoc-find-with
          2drop
      ] map [ ] subset prune natural-sort in get swap vocabs. ;
      parsing

  
  Then, remove the source file's USING: form entirely, and add USING? at the end. Load the source file -- you will be prompted with a bunch of restarts where you can pick vocabularies to add to the search path. Then at the end of the file, the USING? word will execute and print the USING: form that you need based on the restarts you chose.


• Circular dependencies between vocabularies are highly discouraged because the vocabulary loader might not behave as you would expect; it will not go into an infinite loop, but it also might not load one element in the cycle. This might be fixed later, but for now, avoid circularity because it is bad design anyway.


• Many modules had OS-specific components which were loaded conditionally. Each component would define words in the same vocabulary. This no longer works because source files are mapped to vocabularies, but instead you can use hooks. Enter \ HOOK: help in the listener.


• For application modules which have a main entry point, the entry point should be defined on the topmost vocabulary. For example, consider the tetris module, which has already been updated. It has this layout:
  

  tetris
  tetris.board
  tetris.game
  tetris.gl
  tetris.piece
  tetris.tetromino

  
  The tetris vocabulary depends on all the others, and provides a MAIN: hook.


• When updating modules, it is also a good idea to give the module a general code cleanup, round of testing, maybe write some documentation and unit tests for it too.

I hope these guidelines help.

The new module system will be the last major shakeup of the source tree for quite some time. When I come back from New Zealand, I will spend a few months working on behind the scenes stuff:

• compilation of code which constructs quotations with curry
• compilation of code which uses continuations
• removing the interpreter
• UI stuff, tools
• Finishing stand-alone application deployment
• merging Dan's Unicode library

None of this should break code in a major way like the module system did.