Tuesday, October 30, 2007
Playing around with the UI
I'm working on a graphical front-end for Factor's stand-alone app deployment tool. I implemented checkbox and radio button gadgets and threw together a prototype. Check out how the UI is constructed in the source code; it's pretty neat.
Sunday, October 28, 2007
Improved usability of profiler tool
Factor 0.90 introduced a profiler:
The problem was that the
Today I changed the implementation to be much more clever. Now, words are always compiled with profiler prologues. The trick is that unless the profiler is enabled, the entry point of the word points at just after the profiler prolog. When the profiler is enabled, the entire code heap is scanned, and pointers to words are updated to point to before the prolog.
Seems like a pretty obvious idea in hindsight. I know I'll end up using the profiler a lot more now. I also want to experiment with using it to implement new tools, such as test coverage. Basically you'd run unit tests for a certain vocabulary with the profiler enabled, then look at what percentage of words in that vocabulary were called. This won't test branch coverage, but Factor words tend to be short, so a coarse word-oriented coverage tool can still be useful here.
enable-profiler
[ ... ] profile
disable-profiler
The problem was that the
enable-profiler/disable-profiler calls would recompile all words to add/remove a prologue which incremented the word's call count. The point here was that we only want a profiler prolog when we're actually profiling, and not all the time, because of the time overhead in testing the global profiling flag on every word invocation. However, in practice this meant that I rarely used the profiler, because recompiling all words takes a minute or two.Today I changed the implementation to be much more clever. Now, words are always compiled with profiler prologues. The trick is that unless the profiler is enabled, the entry point of the word points at just after the profiler prolog. When the profiler is enabled, the entire code heap is scanned, and pointers to words are updated to point to before the prolog.
Seems like a pretty obvious idea in hindsight. I know I'll end up using the profiler a lot more now. I also want to experiment with using it to implement new tools, such as test coverage. Basically you'd run unit tests for a certain vocabulary with the profiler enabled, then look at what percentage of words in that vocabulary were called. This won't test branch coverage, but Factor words tend to be short, so a coarse word-oriented coverage tool can still be useful here.
SourceForge mailing list archive problems
It seems that the SourceForge Factor-talk mailing list archive has stopped archiving posts. The last archived post is from 11 days ago, but there were numerous posts since then.
The Factor project's entire Internet presence was formerly hosted on SourceForge. At some point I moved the web site, then downloads, over to my server. Hopefully soon, the mailing list will follow suit, and I can stop using SourceForge forever.
jEdit was SourceForge project #588. It was okay back then. Now there are 160,000 projects! In the last several years, I've experienced many "SourceForge moments":
Of course this is a free service, blah blah, so I shouldn't complain. I'm glad there are other choices, like linode.com hosting, which is really cheap and trouble-free.
I'll bet that in the next few years, SourceForge will experience a huge outage and tens of thousands of projects will lose their data forever...
The Factor project's entire Internet presence was formerly hosted on SourceForge. At some point I moved the web site, then downloads, over to my server. Hopefully soon, the mailing list will follow suit, and I can stop using SourceForge forever.
jEdit was SourceForge project #588. It was okay back then. Now there are 160,000 projects! In the last several years, I've experienced many "SourceForge moments":
- CVS servers which are slow and almost always down
- Anonymous CVS which lags behind developer CVS by days
- Mysterious download corruption
- Weird outages based on the first letter of the project (eg, "all projects starting with G-R are inaccessible") Mailing lists where posts take days to appear
- Not being able to log in on some pages
- Ridiculously low disk quotas for projects
- Download counters took them at least a year to implement properly
Of course this is a free service, blah blah, so I shouldn't complain. I'm glad there are other choices, like linode.com hosting, which is really cheap and trouble-free.
I'll bet that in the next few years, SourceForge will experience a huge outage and tens of thousands of projects will lose their data forever...
Wednesday, October 24, 2007
Going to Minneapolis
I'm back from Montreal and already looking forward to my next trip. I'm going to Minneapolis (in Minnesota) from November 18th to November 22nd. I'm going to meet the world-famous littledan.
Friday, October 19, 2007
Going to Montreal on Monday
I'm going to Montreal on Monday night to attend the MSLUG meeting. I'm not going to OOPSLA because I don't have the time. If you want to hang out before or after the MSLUG meeting, though, let me know. I look forward to meeting Christopher Diggins and Zach Beane.
Tuesday, October 16, 2007
IBM developerworks is lame
I saw an article on IBM Developerworks about Cusp, an IDE for Lisp. Written by a student of Computer Science who obviously first saw Lisp last semester, it made me chuckle a little bit.
"Its strength is in the processing of lists: AI and symbol mathematics."
"You'll notice that Lisp is not at all like other general programming languages. For example, in most general programming languages, you perform multiplication just as you would on paper: int times = 5 * 5;."
(Sorry, but I don't write "int times = 5 * 5;" on paper... perhaps he meant 5×5?)
"Note that the above method actually defines a type string. Up to now, you've been using Lisp as a largely typeless language. Though the double quotes implicitly types data as strings, the above method explicitly types both the input and output to the concat2 function as strings."
(Me no speak good English? Does anybody proofread this stuff or do they nod and smile, looking all impressed at this kid's Lisp knowledge?)
The lack of attention to detail also becomes apparent when he can't even get a simple recursive function right near the end.
The article misses out on all the important features that Cusp offers which make it a serious alternative to SLIME with Emacs: cross-referencing, code completion, navigation, etc. Instead we get a "AI using recursion and linked lists" article.
Most of the links at the end are not very useful either. You have a link to the wikipedia article on Lisp, a paper from 1979, and about 30 Eclipse links. There's also a link to Paul Graham, who nowadays is more of a crazy rich guy with money than a programmer.
Ironically, the one relevant link, Programming Lisp with Cusp, is actually about a thousand times more informative and better written than the Developerworks article is.
Now, the paranoid Lisp zealots like to chalk things like this up to semi-deliberate attempts at smearing Lisp. However, I know that Developerworks articles on Java are just as bad, if not worse. Hell, any Developerworks article, on any topic, is usually extremely useless, misleading and poorly written.
This is just another case of a "by idiots, for idiots" type situation. if you want to learn how to program like IBM's finest enterprise drones, look no further than IBM Developerworks.
If they want to be taken seriously, they need to put some basic editorial standards in place. There needs to be a distinction between a blog and a site like this; the latter should not just be a mass blog orgy.
"Its strength is in the processing of lists: AI and symbol mathematics."
"You'll notice that Lisp is not at all like other general programming languages. For example, in most general programming languages, you perform multiplication just as you would on paper: int times = 5 * 5;."
(Sorry, but I don't write "int times = 5 * 5;" on paper... perhaps he meant 5×5?)
"Note that the above method actually defines a type string. Up to now, you've been using Lisp as a largely typeless language. Though the double quotes implicitly types data as strings, the above method explicitly types both the input and output to the concat2 function as strings."
(Me no speak good English? Does anybody proofread this stuff or do they nod and smile, looking all impressed at this kid's Lisp knowledge?)
The lack of attention to detail also becomes apparent when he can't even get a simple recursive function right near the end.
The article misses out on all the important features that Cusp offers which make it a serious alternative to SLIME with Emacs: cross-referencing, code completion, navigation, etc. Instead we get a "AI using recursion and linked lists" article.
Most of the links at the end are not very useful either. You have a link to the wikipedia article on Lisp, a paper from 1979, and about 30 Eclipse links. There's also a link to Paul Graham, who nowadays is more of a crazy rich guy with money than a programmer.
Ironically, the one relevant link, Programming Lisp with Cusp, is actually about a thousand times more informative and better written than the Developerworks article is.
Now, the paranoid Lisp zealots like to chalk things like this up to semi-deliberate attempts at smearing Lisp. However, I know that Developerworks articles on Java are just as bad, if not worse. Hell, any Developerworks article, on any topic, is usually extremely useless, misleading and poorly written.
This is just another case of a "by idiots, for idiots" type situation. if you want to learn how to program like IBM's finest enterprise drones, look no further than IBM Developerworks.
If they want to be taken seriously, they need to put some basic editorial standards in place. There needs to be a distinction between a blog and a site like this; the latter should not just be a mass blog orgy.
Friday, October 12, 2007
New addition to planet-factor blogroll
Samuel Tardieu's weblog is now syndicated on planet-factor. Welcome to the community, Sam. Parsing Sam's atom feed actually exposed a bug in Factor's XML parser; Daniel Ehrenberg fixed it on very short notice. Thanks Dan!
New toys to play with
Doug Coleman sent me his old Cingular 8525 (Windows CE) phone. And my friend Erik in Ottawa gave me his ancient Dell Windows laptop in exchange for a Mac Mini that I haven't used since I bought the G5. Win-win situation: I get a Windows box for development, Erik gets a (faster!) computer which doesn't suffer from spyware/adware/etc. :-)
I'm now going to be in charge of the Windows CE port of Factor, and I can help out a bit with the Windows XP/Vista port too! Doug can now focus on more interesting things. I've been pestering him about finishing the regexp library for a while now... :-)
I installed Cygwin, CeGCC and Microsoft's ActiveSync on the Windows laptop, then I got a command prompt going on the phone using the directions in Chris Double's weblog. Note that Chris doesn't mention this, but Windows CE does not ship with a registry editor; to edit the registry you need to install a third-party registry editor. I used TRE.
Doug originally got the phone cheap with an AT&T contract; I used the Hermes Unlocker tool to CID/SIM unlock the phone (which is illegal, voids the warranty, etc.)
While Factor 0.90 worked on Windows CE and ARM, there have been a few changes since then. First of all is the new two-tiered compilation; I have to implement the ARM backend for the new non-optimizing compiler. Then I have to make some (minor) changes to the ARM optimizing compiler backend.
Once it is in a working state, my first task will be removing the dependency on CMD.EXE. It is a pain to install (requires editing the registry) and is rather unusable on the small screen of the phone. Instead I'll borrow code from PythonCE, which implements a simple input/output console using the Win32 API. This will increase the size of Factor's VM on Windows CE somewhat, however it won't depend on the hard to install CMD.EXE.
Eventually I want to get the Factor UI running on the phone, with a cut-down UI listener and workspace. This way I can experiment with new input methods and interesting ways of writing code on the phone. I think Factor is concise and powerful enough that on-board development will be a practical possibility (of course you'll always be able to develop your Factor apps on a desktop machine and copy them over to your phone, but where's the fun in that?)
The phone only has 64 Mb of memory. While Factor's semispace GC is very efficient on systems with virtual memory and overcommit, on a phone with no swap having that semispace just hang around there sucking up RAM is really, really painful. I will be implementing an incremental mark and sweep GC at some point, which will improve interactive performance on desktop machines and reduce memory usage on the phone.
I'm now going to be in charge of the Windows CE port of Factor, and I can help out a bit with the Windows XP/Vista port too! Doug can now focus on more interesting things. I've been pestering him about finishing the regexp library for a while now... :-)
I installed Cygwin, CeGCC and Microsoft's ActiveSync on the Windows laptop, then I got a command prompt going on the phone using the directions in Chris Double's weblog. Note that Chris doesn't mention this, but Windows CE does not ship with a registry editor; to edit the registry you need to install a third-party registry editor. I used TRE.
Doug originally got the phone cheap with an AT&T contract; I used the Hermes Unlocker tool to CID/SIM unlock the phone (which is illegal, voids the warranty, etc.)
While Factor 0.90 worked on Windows CE and ARM, there have been a few changes since then. First of all is the new two-tiered compilation; I have to implement the ARM backend for the new non-optimizing compiler. Then I have to make some (minor) changes to the ARM optimizing compiler backend.
Once it is in a working state, my first task will be removing the dependency on CMD.EXE. It is a pain to install (requires editing the registry) and is rather unusable on the small screen of the phone. Instead I'll borrow code from PythonCE, which implements a simple input/output console using the Win32 API. This will increase the size of Factor's VM on Windows CE somewhat, however it won't depend on the hard to install CMD.EXE.
Eventually I want to get the Factor UI running on the phone, with a cut-down UI listener and workspace. This way I can experiment with new input methods and interesting ways of writing code on the phone. I think Factor is concise and powerful enough that on-board development will be a practical possibility (of course you'll always be able to develop your Factor apps on a desktop machine and copy them over to your phone, but where's the fun in that?)
The phone only has 64 Mb of memory. While Factor's semispace GC is very efficient on systems with virtual memory and overcommit, on a phone with no swap having that semispace just hang around there sucking up RAM is really, really painful. I will be implementing an incremental mark and sweep GC at some point, which will improve interactive performance on desktop machines and reduce memory usage on the phone.
Wednesday, October 10, 2007
Announcing Planet Factor
I put together an Atom/RSS aggregator in Factor. It is running live on the experimental HTTP server. Parroting the idea behind Planet Lisp, now there's Planet Factor.
It took me about an hour of work in total to put this together. Most of the parts were already there - Chris Double's Atom/RSS library, which uses Daniel Ehrenberg's XML library is the cornerstone here, and of course the HTTP server itself.
A thread fetches the feeds every 30 minutes. Entries are then concatenated and sorted by date, and stored in a global variable. The Planet Factor web app together with the news box on the Factor home page access this variable and present the results in a nice way.
The Planet Factor source is relatively short. (And on a somewhat related note, that pastebin is a Factor web app too...) This is my favorite part:
Ignoring the diagnostic message logging, you basically have
Where
It took me about an hour of work in total to put this together. Most of the parts were already there - Chris Double's Atom/RSS library, which uses Daniel Ehrenberg's XML library is the cornerstone here, and of course the HTTP server itself.
A thread fetches the feeds every 30 minutes. Entries are then concatenated and sorted by date, and stored in a global variable. The Planet Factor web app together with the news box on the Factor home page access this variable and present the results in a nice way.
The Planet Factor source is relatively short. (And on a somewhat related note, that pastebin is a Factor web app too...) This is my favorite part:
dup [
second
dup "Fetching " diagnostic
dup news-get feed-entries
swap "Done fetching " diagnostic
] parallel-map
Ignoring the diagnostic message logging, you basically have
[ news-get ] parallel-map
Where
news-get makes a network connection, downloads the XML and parses it. While the XML parsing is not parallelized, the network I/O is. Not many languages make it this easy to fire off overlapping I/O operations...
Passing structs by value on AMD64
The Linux/AMD64 ABI is a bit crazy when it comes to passing structs by value. The rules are as follows:
There are additional rules for C99 complex numbers and long double types, but Factor's FFI doesn't support those so I don't have to worry about that.
For a long time I didn't properly implement these semantics. They're implemented now. I don't think anybody has run into this issue, however when Mac OS X Leopard is released with 64-bit Cocoa, it would certainly come up since many Cocoa APIs rely on passing structs (assuming Apple uses the same or similar ABI as Linux on AMD64, which is likely).
I think the FFI is one of Factor's strongest points. Many popular languages have weak FFIs (or are weak in other respects). On the other hand, when implementing an obscure language, one must work twice as hard as everybody else to be taken seriously.
- If the structure is greater than 16 bytes, it must be passed on the C stack, even if there are enough registers.
- If the structure is 16 bytes or less, it is broken up into 8 byte chunks. Each chunk is considered separately.
- If at least one of the fields in a chunk is an integer, the whole chunk is passed in an integer register.
- Otherwise, if the chunk is either two float fields, or a double field, it is passed in an SSE register. In particular, this means that if your structure consists of two floats, they must be packed into a single SSE register.
There are additional rules for C99 complex numbers and long double types, but Factor's FFI doesn't support those so I don't have to worry about that.
For a long time I didn't properly implement these semantics. They're implemented now. I don't think anybody has run into this issue, however when Mac OS X Leopard is released with 64-bit Cocoa, it would certainly come up since many Cocoa APIs rely on passing structs (assuming Apple uses the same or similar ABI as Linux on AMD64, which is likely).
I think the FFI is one of Factor's strongest points. Many popular languages have weak FFIs (or are weak in other respects). On the other hand, when implementing an obscure language, one must work twice as hard as everybody else to be taken seriously.
Subscribe to:
Posts (Atom)