Labs/Bespin/DesignDocs/JSServer
Contents
JavaScript Server
by Kevin Dangoor, May 4 2009
Rationale
We would like to see a JavaScript implementation of the Bespin Server built up. Many people at this point don't think of JavaScript as a server-side programming language, though there have been many attempts to bring JavaScript to the server with varying degrees of success. Here are the reasons why having the Bespin server in JavaScript would be useful for the Bespin project (in decreasing order of importance):
- Plugins that feature both client and server components
- Code sharing between client and server
- Better support in Bespin
- Familiarity (just about everyone coming to the Bespin project knows JavaScript. Python is less common)
- Possible speed improvements
The following sections will dive into these advantages in more detail.
Plugins
Many features for Bespin can be implemented purely on the client side. Certain features require access to the server in order to work properly or well. For example, any cross-domain access needs to go through the server. Another example is the Quick Open feature which allows you to incrementally search through the filenames to find matching files. To work quickly enough, the server needs to maintain a list of the files in the project that allows for quick searching. This was easy enough to do, but there was no way this could be implemented as a plugin.
The problem faced in making plugins with the current Python server is that there is no way, at the time of this writing, to reliably write secure, performant Python code. The C Python runtime just wasn't made for it, and the PyPy project is still to experimental and slow.
Because of their heritage in the browser, JavaScript interpreters have excellent support for sandboxing of code. Even Rhino, which is not used in any major browsers, offers excellent control over script execution.
Code Sharing
Over time, we anticipate having the Bespin project code packaged up in a variety of ways for a variety of uses. One example would be to use Appcelerator Titanium or something similar to package up Bespin for use on the desktop. We could do that today, but you would immediately lose the version control system support (as one example of a feature with a significant server side component). Even Quick Open currently relies on server code which would need to be duplicated in new JS code.
If the server was implemented as a collection of JavaScript modules, some of those would be runnable in the browser, and many could be used in Titanium.
Better support in Bespin
I believe that Bespin's JavaScript editing interface is likely to have more features than its Python editing ability.
Familiarity
A few people coming to the Bespin project have commented that they have never used Python before, and there has been very little in the way of contribution to the server, compared to the client side. I think that it will be easier for people to contribute to the project if the server is in JavaScript.
Speed
Though all dynamic languages have seen new projects appearing to speed up their execution, it is unlikely that any will see the same level of investment that is going on in JavaScript right now. I put this as the lowest priority consideration, however, because plenty of good apps have been written on top of interpreters that are not speed demons.
Implementation
The goal of this porting effort is to try to quickly bring the JavaScript server inline with the Python server, have them march along side-by-side until the JS server is stable enough, and then move bespin.mozilla.com over to the JS server.
To make this possible, the likely best approach to building the JS server will be to start it off as a near line-by-line port of the Python. This will make it much easier to integrate changes from the Python code as they are made. Additionally, the unit tests should also be ported from Python to ensure that the server's behavior remains intact.
In January 2009, I started an effort to create a "standard library" for server side JavaScript. The idea behind that project is that JavaScript should have as much built-in support for creating applications as Python, Ruby and Java do. As of early May 2009, this effort has produced a cross-interpreter module standard and is close to standardizing files and objects for handling binary data. Additionally, there is a fair amount of support for Jack, which is a direct descendant of Ruby's Rack and Python's WSGI. This is helpful, because the Python server is built around WSGI.
The Source
As of this writing, the authoritative source for the Bespin JS server is on hg.mozilla.org
Currently, Irakli Gozalishvil is actively working on the JS backend and he maintains a fork at GitHub.
We are closely monitoring the Hg-Git project to allow us to easily collaborate seamlessly via both Mercurial and Git. As of the end of April, this project was not quite able to handle Bespin.
The Platform
We are starting development on the Rhino interpreter. Rhino implements the JavaScript language on top of Java. By using a JVM-based language, we are able to ensure that the code will easily run on all major operating systems and we will be able to leverage Java's libraries while the standard JS libraries are built out.
Once enough of the standard library is built, we can freely switch between Rhino, SpiderMonkey and v8 (and possibly even JSCore) as needs dictate.
In order to be able to run on our interpreter of choice, we will use only the features available in JavaScript 1.6, which appears to be universally supported by these interpreters.
Other Projects in Use
We have a number of projects that we depend on, but we do not yet have an automated process in place to install these dependencies. Further, many are under active development, so tracking the source is beneficial.
Narwhal provides a multi-interpreter standard library implementation and includes Rhino.
Jack builds on Narwhal to provide some useful utilities for building webapps. It includes a simple web server.
JsDoc Toolkit will be used for documenting the code.
Helma NG may be a source of useful code and inspiration, but at this time is too tied to Rhino for our purposes. Helma NG is a good package and is following along with the ServerJS work, which is why it is good to follow. Helma NG is also available on GitHub.
jsUnity is an actively developed JS test framework that should be seriously considered. The author appears accommodating of requests from ServerJS. Also worth noting: Ondrej Zara wrote a test runner that is somewhat similar to what we get from Nose. We can run tests file-by-file, but we will ultimately need a test runner that can discover and run our entire collection of tests.
Database Storage Thoughts
It is not 100% clear at this time what the best approach for storing users and their social graph will be. Some options:
- Helma's filestore is a simple way to get going. However, it likely doesn't feature all of the parts of the RDBMS we're using (cascading deletes?) and may not scale up to the level we need.
- Direct JDBC is an option. ServerJS will almost certainly come up with an interface like JDBC. The Nitro web framework (built on Jack) already features an interface on top of JDBC and may be a fine approach. Note that the drawback is that working at this level will make it harder to use one database engine during development and another in production, as we do today in Python.
- ActiveRecord.js: an ORM in JavaScript. Tom Robinson appeared to have made some preliminary Rhino support. It is unclear what level the Rhino support is at.
Apache Derby is likely a good choice for a development database. We are presently using MySQL in production.
Project Infrastructure
To start with, all discussion related to the Bespin JS Server will take place on the bespin-core mailing list.
