HTTPS support in http.server, some notes on HTTP request/response parsing

Generic servers in Factor

The io.servers.connection vocabulary supersedes the io.server vocabulary, adding support for a connection count limit, SSL, and more configuration flags.

The best way to explain how it works is with an example:

USING: accessors io io.servers.connection ;

<threaded-server>
    "hello-world" >>name
    100 >>max-connections
    1080 >>insecure
    1090 >>secure
    <secure-config>
        "cert/server.pem" >>key-file
    >>secure-config
    [ "Hello, world." print ] >>handler
start-server

This is a server which sends "Hello, world." to the client, then closes the connection. It accepts standard TCP/IP connections on port 1080, and SSL connections on port 1090. For SSL, it identifies itself to the client with the certificate in certs/server.pem. We set a limit of 100 concurrent connections, and connections are logged to resource:logs/hello-world/*.log.

Here is another example:

USING: accessors io io.servers.connection io.launcher ;

<threaded-server>
    "telnet" >>name
    10666 >>insecure
    [
        <process>
            "/bin/sh -i" >>command
            input-stream get >>stdin
            output-stream get >>stdout
            +stdout+ >>stderr
        run-process
    ] >>handler
start-server

Please do not run this on a machine facing the Internet; it starts a server on port 10666 which spawns a bash shell, with I/O redirected to the client connection. This example won't work on Windows, or with SSL connections, because those types of streams cannot be stored in the stdin and stdout slots of a process tuple yet; however that will be fixed eventually, and then code such as the above might serve as a prototype of a Factor re-implementation of inetd.

So that's io.server.connections for you: a lot of functionality wrapped up in a small package. Eduardo Cavazos is working on a DNS server in Factor right now, and at some point we're going to isolate some UDP/IP abstractions from his code which are not specific to DNS, and move those to io.servers.packet.

HTTPS support in http.server

Using io.servers.connection, I implemented https support in the HTTP server. This was pretty easy, now that all the components are in place. The main changes I made were to the authentication system in the web framework. It now redirects you to the SSL port when a page requiring login is requested, however this can be disabled (although I would not encourage this).

For web applications not using the authentication framework, there is another useful utility: any HTTP responder can be wrapped in a <secure-only responder; this upgrades the connection SSL if a non-SSL request is made.

To improve performance, I added session resumption support to io.sockets.secure, for both client and server connections; this can be disabled in the <secure-config> instance passed to with-secure-context. The SSL library still needs support for client authentication and it also needs to run on Windows; once those two items are done, it will pretty much be feature-complete. I might write bindings to the OpenSSL cyphers allowing them to be used stand-alone, too, however this is low-priority since I don't need that for anything right now.

Parsing HTTP requests and responses

I also replaced the hand-coded code for parsing HTTP requests and responses with new code that uses Parser Expression Grammars (PEGs). While the new code was not a lot shorter, it does have a more declarative flavor, and is also more correct.

During the course of this exercise, I learned how far modern web browsers deviate from the specifications, particularly in the area of cookies. Cookies are specified by a pair of RFCs, RFC2109 and RFC2965, however the latter seems to be almost universally ignored, and browser and server developers seem to treat the former as more of a set of ideas than a formal spec.

Mark Nottingham blogged about this a little under two years ago, and seemingly, not much has changed since then. There is also this bug in Firefox.

After I mentioned these difficulties in the #concatenative IRC channel, Zed Shaw joined the ensuing discussion and noted that the RFCs themselves are quite vague and seemingly contradict themselves.

The general design of cookies makes it easy to expose vulnerabilities such as this one in the Tomcat application server. I hope my code is secure but I will need to review it later in more detail to be sure.