Thursday, June 27, 2013

RESTful CRUD APIs with AppEngine, Cloud Endpoints and Dart code generation.

Dart and Web UI is an ideal platform for writing single page web apps.  These are typically made up of static files (html, css, js/dart) that are served up and execute and render in the browser.  Once running, they make requests back to the server for data, treating the server as a datasource API.

The combination of REST and JSON provides a great way for a server to expose an API to browser applications, and Dart can easily access JSON web services.

I was looking for a way to serve my client-side Dart application, and find a way to add server-side persistence in a scalable and open way.  AppEngine immediately sprang to mind.

AppEngine doesn't support server-side Dart at the moment (guess: don't hold your breath until after Dart reaches v1.0), but:


  1. It does support Java, Python, Go and PHP.  
  2. You can generate a "Cloud Endpoints" from your Java classes to provide a RESTful JSON based CRUD API
  3. The generated cloud endpoint implements Google's discovery API, and you can use the Dart discovery API client generator to generate a Dart pub package strongly typed to your CRUD API (thanks to the great work by Gerwin Sturm and Adam Singer).
  4. You can use this generated Dart in the browser to seamlessly access the AppEngine API. 

And that's what this blog post is about.

I'm going to use Java, Eclipse and the Google Eclipse Plugin for the AppEngine bits, and you can definitely use Python to achieve the same thing (I'm not sure about cloud endpoint support for Go or PHP yet).

Creating an AppEngine cloud endpoint 

There are plenty of tutorials about how to create an AppEngine project in eclipse using the Google plugin, so I'm not going to repeat them here.  Just follow the "Web Application Project" Wizard and uncheck the "Use GWT" option.

Once you have a project, you can go ahead and create a class that you want to persist.  In this example, it's a Message class for a guestbook (in a retro Geocities way).  The snippet below shows the basic class without all the getters and setters.


// java
public class Message {
  private String id;
  private String name;
  private String messageText;

  public Message() { }

  // snip getters and setters
}

To make this message an AppEngine persistent object, we just need to add a couple of annotations, namely @Entity and @Id.  I'm going to create my own ID values, but you can autogenerate them if you prefer.

Our class now looks like this:

// java
@Entity
public class Message {
  @Id
  private String id;
  private String name;
  private String messageText;

  public Message() { }
 
  // snip getters and setters
}

This is all we need to get a persistent server-side object.  Now we use the code generator that comes with the Google Eclipse Plugin to create an appengine cloud endpoint.

  • Right click your class and select "Create Cloud Endpoint Class" (as shown in the screenshot below)
Generating a Cloud Endpoint
That will create a couple of files. One for the data persistence side (EMF.java), and the other that exposes CRUD operations for the Message class (MessageEndpoint.java)

That's all there is to it!

Lets start the local AppEngine and access our API.  Run the project as a "Web Application" and that will start the local AppEngine server.


Accessing the API

There are a number of APIs exposed by appengine, and they live under the _ah/api URL path.  First up is the Discovery API, accessible via:




This exposes a list of all the REST APIs that our server is exposing.  There is only one at present - the MessageEndpoint API.   Let's discover that by navigating to:

http://localhost:8888/_ah/api/discovery/v1/apis/messageendpoint/v1/rest (referenced by the "discoveryLink" field.  That returns a host of information about the message endpoint API:


This JSON contains all the information needed to code generate strongly typed code for accessing the API.  Key parts include our Message class definition:

    "Message" : {
      "id" : "Message",
      "type" : "object",
      "properties" : {
        "id" : {
          "type" : "string"
        },
        "messageText" : {
          "type" : "string"
        },
        "name" : {
          "type" : "string"
        }
      }

and CRUD operations you can perform on it, such as getMessage, by using a GET request to message/{id}

   "getMessage" : {
      "id" : "messageendpoint.getMessage",
      "path" : "message/{id}",
      "httpMethod" : "GET",
      "parameters" : {
        "id" : {
          "type" : "string",
          "required" : true,
          "location" : "path"
        }
      },
      "parameterOrder" : [ "id" ],
      "response" : {
        "$ref" : "Message"
      }
    }

The complete list of operations are:
  • getMessage => GET: /message/{id}
  • insertMessage => POST: message/
  • listMessage => GET: message/
  • removeMessage => DELETE: message/{id}
  • updateMessage => PUT: message

We can try that by using the command line CURL command to try POSTing some data to insert a message, and GET the list of messages:

The following curl command POSTs some JSON data to the URL provided (all one line)
curl --header "Content-Type:application/json" --data "{'id':'msg1','name':'Chris','messageText':'Awesome Guestbook'}" http://localhost:8888/_ah/api/messageendpoint/message
Output of CURL POST command
I've inserted a couple of messages, with IDs "msg1" and "msg2".  We can list these with a simple GET request:

GETting the list of messages

Now we're ready to hook AppEngine up with Dart.

Using the Dart Discovery API Client Generator

Google Developer Experts Adam Singer and Gerwin Sturm spent some time earlier in the year building a code generation tool to build Dart packages from Google APIs using the Google's discovery APIs.  We're going to use that now.  The next part of this blog assumes that you have dart and git on your path (accessible from the command line).

First, we need to get the generator code, so we'll clone it from github by running the following command:

git clone https://github.com/dart-gde/discovery_api_dart_client_generator.git
Now you can run the generate.dart, passing in the URL of the discovery API for the messageendpoint:

cd discovery_api_dart_client_generator  
dart bin\generate.dart -u 
http://localhost:8888/_ah/api/discovery/v1/apis/messageendpoint/v1/rest
This outputs a Dart pub package in the output/dart_messageendpoint_v1_api_client subfolder.  We can import that into a Dart project.

Using the generated pub package in a Dart project.

We need to jump through a couple of hoops now, as we need to host the client-side Dart files within the AppEngine server.  AppEngine serves files from the /war folder within the Java project, and you can get your Dart files in there in a number of ways, for example, exporting a folder via the Java classpath output, symlinking directly within the war folder, or adding a build step to pub deploy the files into the war folder.

For the sake of ease with this demo, though, I'm just going to go right ahead and create a new Dart web project right inside the /war folder.  Typically, you'd want to only include the output of pub deploy, though.


Now that we've got a Dart project, let's check that we can access it through appengine, via the url: http://localhost:8888/client/web/client.html - if you're checking that URL in chromium, it won't do much because we've not generated the dart2js code yet.

In the Dart editor, let's create a launch that uses the appengine server instead of the built in Dart server.  That way it will run in Dartium, connect to the debugger, but be served from appengine:


Now that we can launch our app from the Dart Editor, into Dartium, served from AppEngine, let's look at accessing our new API.

Edit the pubspec.yaml to add the generated pub package.  I'm using a path url (which hard codes this app to my machine).  In the real world, you'd be better using a git location, so that you can share development among your team.


name: client
description: A sample web application
dependencies:
  browser: any
  google_messageendpoint_v1_api:
    path: c:\work\dart\projects\discovery_api_dart_client_generator\output\dart_messageendpoint_v1_api_client

Run a pub install if it's not run already, and you should get the package installed:

Installed endpoint package
In the screenshot above, you can see that there are three libraries created: browser, client and console.  The client library is shared between browser and console (which expose the relevant dart:html or dart:io APIs and re-export client).  Because we're working in the browser, we'll need to import the browser package using the following line:


import 'package:google_messageendpoint_v1_api/messageendpoint_v1_api_browser.dart';

We can now reference our Message class.  Let's write some simple code to create a message, and then retrieve a list of message objects.  This is going to output (using print()) to the browser console.  We're not building a UI here (that is left for a later blog post).

First, we create an instance of the endpoint class and set the root URL.  Remember those 5 CRUD methods on the API from earlier?  We also get great autocomplete from that:



Let's try listing the messages, which performs the same GET that we used with the CURL command earlier:


import 'dart:html';
import 'package:google_messageendpoint_v1_api/messageendpoint_v1_api_browser.dart';

void main() {
  // create an instance of the endpoint
  Messageendpoint endpoint = new Messageendpoint();
  endpoint.rootUrl = "http://localhost:8888/"; // set the root URL
  
  endpoint.listMessage().then((CollectionResponse_Message messageCollection) {
    print(messageCollection.items); // GET : message/
    print(messageCollection.items[0].messageText); // "Awesome Guestbook" 
  });
}


Let's try adding a new message, saving it back to the server, and retrieving it by ID:


  // create a message
  var message = new Message.fromJson({});
  message.id = "msg3";
  message.messageText = "I am a message";
  message.name = "Chris";
  
  // save it back to the server
  endpoint.insertMessage(message).then((Message savedMessage) {    
    // message is saved.  Retrieve it by ID
    endpoint.getMessage("msg3").then((Message loadedMessage) {
      print(loadedMessage.messageText); // "I am a message "
    });
  });

Summary

For now, that's it.   In this blog post you've seen how to

  • create a persistent Java class on AppEngine, 
  • generate a cloud endpoint to expose the Java class CRUD as a RESTful API
  • use the discovery API to generate a Dart pub package strongly typed to the Java class
  • use the generated pub package to read and write to the API.

There's a lot more that you can do with the discovery API and the code generated Dart, including adding OAuth tokens to authenticate requests, or setting the quota user and IP to enforce per-user limits.

You can find a whole host of generated APIs for most of the public google services on github, but I think the most powerful feature is being able to generate APIs for your own appengine apps.

References & further reading

3 comments:

  1. Thanks for this great tutorial!

    In the "Accessing the API" section, you show a curl command that you used to manually insert a message. Although it's correct in the screen capture, the text above that is missing the /v1 so (at least for me) does not work. For anyone following the steps closely, copy-pasting this and getting an Error 404 may be confusing.

    Also, despite the link you provided, the comment "I'm going to create my own ID values, but you can autogenerate them if you prefer" wasn't particularly helpful to me. I wasn't able to find anything at that link to explain how to autogenerate IDs for an annotated entity. Further poking around led me to @GeneratedValue, but I can't seem to get that to work with curl. If you know the secret, please share.

    ReplyDelete