Friday, March 30, 2012

Building a client / server Dart App – Part 1 – server side.


Update: 05/May/2012: This post^ provides a simple, up-to-date (at the time of writing) example
on how to do simple dart client and server. What you may read below is still valid, but some api’s may have moved on a little. Please let me know if you find any obvious bugs.
One of the features I liked about node.js was that you could write the entire project in 1 language – JavaScript on both the client and the server.
Dart can do the same.  On the client side, we can write our app in Dart and convert to JavaScript.  Server side, again, we can write our app in Dart, and run it in the DartVM (command line dart).
This post will demonstrate a simple “blog” project which takes some input on the client side, sends it to the server where it is saved in a MongoDB database, and can then retrieve the blogposts back out of the database – as shown in the image below. (all the code, running on 29/03/2012 with dartium + dart 5549 is available on github^).
running the blog-server app

Prerequisites

Dart SDK^ (essential)
Dart Editor^ (helpful)
Dartium^ (chromium with the dart VM) (also helpful)
To run a server side dart application, simply run
dart my-app.dart
or even better – use the Dart Editor and hit the “run” button.

Server Side

We will start with the server.  There are a few libraries from github that are required, namely CrimsonHttp, log4dart and mongo-dart:
At the time of writing, there is no generic way to import libraries on the server side, other than using absolute or relative paths.  To this end, we will need the following structure (assuming that the project is the folder you are creating everything in) so that each bit can reference each other using relative paths:
project\dart-blog-server                       <– this is where we will place the application code
project\dart-blog-server\client         <– this is where our static client side files will go
project\crimson
project\log4dart <– this is used by crimsonHttp and can log to console or file
project\mongo-dart
Dart uses async, non-blocking IO, and has an HttpServer class available to it.  Our example will use CrimsonHttp^, which wraps the built in HttpServer (from dart:io library), and adds some basic functionality, such as providing static file handling and routing.  It’s still a plaything at the moment, but it serves our purpose for now.
We need to create a .dart file – I’ve called mine “dartwatch-blog-server.dart” – it will contain the following declarations at the top:
#library("dartwatch:blogserver");              //any reasonable name
#import("../crimson/core/crimson.dart");       //the core crimsonHttp server library
#import("../crimson/handlers/handlers.dart");  //builtin handlers library
#import("../mongo-dart/lib/mongo.dart");       //mongodb drivers.
#import("dart:json");
#import("dart:io");
Once we have that much, we can start by getting our app to respond to http requests using CrimsonHttp.

CrimsonHttp

We will start by getting that running and responding to the browser. The code below describes creates a BlogServer class which contains a run() method. That run method instantiates a CrimsonHttpServer, and a module for that server. We then add a number of handlers to that module: two Route and one StaticFile.
The Route endpoint handler takes a path and an http method (GET, POST etc…) that it will try and match against the http request, and a third parameter which is a method which will be called if the route matches the request.
The StaticFile endpoint handler takes a local (ie, server side) path, and will try and serve up static files from that folder if they match the filename on the request. For example, if you put “myFile.html” in ./client on the server, it would serve it in response to http://localhost:8083/myFile.html.
Finally, we start the server listening to localhost on port 8083.
The main() function start running when you execute your dart script, and starts all this happpening.
class BlogServer {

  //todo - add mongo-db support

  void run() {

    CrimsonHttpServer server = new CrimsonHttpServer();
    CrimsonModule module = new CrimsonModule(server);
    module.handlers
               .addEndpoint(new Route("/post/all", "GET", getPosts))  //return all posts - closure
               .addEndpoint(new Route("/post", "POST", addPost)) //add a post - closure
               .addEndpoint(new StaticFile("./client"));

    server.modules["*"] = module;  //this is the default module.

    server.listen("127.0.0.1", 8083);
  }

  Future addPost(HttpRequest req,res,data) {
    //todo - fill in the blanks
  }

  Future Future getPosts(req,res,CrimsonData data) {
    //todo - fill in the blanks
  }

}

void main() {
new BlogServer().run();
}
The code above isn’t finished yet – there are three TODO’s – add support for mongoDB, and fill in the two empty functions to get the posts, and to add a post.

mongo-dart

The mongoDb bit is quite simple – within the BlogServer class, we will add a private variable called _posts which represents a mongoDb collection of blog posts. Then we’ll add a default constructor which will connect to the database, and open the collection of posts (creating it if it doesn’t exist). You will need to ensure you have started mongod (the mongodb server, from the command line) before running it now.
class BlogServer
  //done - add mongo-db support
  MCollection _posts;

  //constructor
  BlogServer() {
    Db db = new Db("dart-blog-server");
    db.open();                             //open the connection to mongodb
    _posts = db.collection("posts");

    // uncomment the following line to remove all the posts at app startup
    //_posts.remove();

  }

  //...snip... other BlogServer code

}
MongoDB stores its data in a form of JSON (BSON – Binary json). This means that we can effortlessly read and write Maps converted from JSON back and forth to mongodb.

Reading and writing data

Finally, we need to fill in our two methods. The Route handlers require that the handlers take a method which takes an httpRequest, httpResoponse, and a CrimsonData object. It should also return a Future (which is a way of allowing async calls to promise that they will return a value at some point), or it can return null (which means that they have finished executing).
We’ll look at the addPost() method first. The first real line of code creates a Completer object. A completer itself creates a Future (which is the object that the method returns). When we are finished, as a result of async operations, we tell the completer that we are done – this triggers the Future to return a value to the calling code (the calling code in this case is the CrimsonHttp Route class).
Because our addPost method is called in response to an http POST – we need to get the data that is being sent from the browser from the request body. To do this, we read it from the request inputStream, which requires us to add an onClosed() handler, which gets called when we have read all the data – which will look something like
{"posted": "2012-03-29 00:00:00.000", "text":"some text sent from the client"}
Our data goes into a StringBuffer, which we read out into the postData string variable. We then use the JSON.parse function (from the built in dart:json library) to convert it from a string into a Map. We can then use the _posts.insert() function on the mongodb collection to insert that posts. Then we tell the completer that we are done.
The flow of the method is thus:
1. Create completer
2. Start reading input stream
3. Method returns to caller, returning future from the completer
4. Finish reading input stream
5. Save data from input stream
6. Tell completer we’ve finished
7. Completer tells the caller that we’re finished (using a handler method).
Remember, the caller in this context is the Route class which has decided to call this function based upon the http request (eg: POST: http://localhost:8083/post )
class BlogServer {

//...snip.... other code

  Future addPost(HttpRequest req,res,data) {
    print("adding post: ${req}");
    Completer completer = new Completer();

    StringBuffer body = new StringBuffer();
    StringInputStream input = new StringInputStream(req.inputStream);
    input.onData = () => body.add(input.read());
    input.onClosed = () {
      String postdata = body.toString();
      print("postdata: ${postdata}");
      if (postdata != null) {

        Map blogPost = JSON.parse(postdata);
        print(blogPost);
        Map savedBlogPost = new Map();
        savedBlogPost["text"] = blogPost["text"];      //explicitly read and written for purpose of example
        savedBlogPost["posted"] = blogPost["posted"];  //explicitly read and written for purpose of example
        _posts.insert(savedBlogPost);  //  add the post to mongodb collection
        completer.complete(data);      //  finally, tell our completer that we're done.
      }
    };

    return completer.future;
  }
//...snip - other code
The last method on the server to complete is the getPosts method. This takes a similar form, in that it also returns a Future from a completer. It simply queries for all the records in the collection (by providing an empty {} query to the mongodb _posts.find({}) method), and converts them back to a JSON string. The find() method returns a cursor, which we can iterate through using the .each() function. In the each function, we add each item to a list of posts.
When all the items have been received, we call the .then() function to convert them back to JSON and write them to the response – it takes the form of:
//pseudocode:
cursor.each(   ( data ) {
  //this will get called for each item of data returned
}).then( (throwaway_value) {
  //this will then be called when we've retrieved all items of data.
});
In the then() function, after we have written our data to the response, we call the completer.complete to let the caller know that we’re done.
Future getPosts(req,res,CrimsonData data) {
    Completer completer = new Completer();

    List postList = new List();

    Cursor cursor = _posts.find({});   //no query - find all.

    cursor.each((Map post) {
      //for each post (which is a map) add it to the list

      //the mongo-dart ObjectId isn't supported by the JSON.stringify()
      //so we'll just extract it, convert it to a string, and put it back again.
      var id = post["_id"];
      post.remove("_id");
      post["_id"] = id.toString();

      //finally, add the post to the list of posts.
      postList.add(post);

    }).then((dummy) {
      //when we've got them all from the db, return them
      print(postList);
      try {
        String postAsString = JSON.stringify(postList);   //convert the List of Maps to a string
        res.outputStream.writeString(postAsString);     //write the string to the response
        completer.complete(data);                      //tell the caller we've completed.
      }
      catch (var ex, var stack) {
        print(ex);                   //some simple error handling.
        print(stack);
        completer.completeException(ex);   //we can tell our caller that we had an error
      }
    });

    return completer.future;
  }
That’s it. You can now run this on the server side using the dart.exe (or dart binary for mac / linux from the dart-sdk). If you browse to http://localhost:8083/posts/all you will see the app connect to mongodb and try to retrieve all the posts (of which there are none at present).
You can grab all the code for this project on github
Part 2 will create the browser client in dart, which will be served from the StaticFile handler, and call the two routes to add and read our posts from mongodb.

1 comment:

  1. Mongo DB is my current struggle right now as I am still adapting to its prefix and paradigm. Unlike the SQL major which has a dedicated sql server support org support, Mongo DB's support resources are hard to come by. So thanks for this one.

    ReplyDelete