English

Google App Engine

Using the Datastore

Storing data in a scalable web application can be tricky. A user could be interacting with any of dozens of web servers at a given time, and the user's next request could go to a different web server than the one that handled the previous request. A web server may depend on data that is spread out across dozens of machines, possibly in different locations around the world.

Thanks to Google App Engine, you don't have to worry about any of that. App Engine's infrastructure takes care of all of the distribution, replication and load balancing of data behind a simple API—and you get a powerful query engine as well.

A Complete Example Using the Datastore

Here is a new version of myapp/hello/hello.go that stores greetings in the datastore. The rest of this page discusses the new pieces.

package hello

import (
    "appengine"
    "appengine/datastore"
    "appengine/user"
    "http"
    "template"
    "time"
)

type Greeting struct {
    Author  string
    Content string
    Date    datastore.Time
}

func init() {
    http.HandleFunc("/", root)
    http.HandleFunc("/sign", sign)
}

func root(w http.ResponseWriter, r *http.Request) {
    c := appengine.NewContext(r)
    q := datastore.NewQuery("Greeting").Order("-Date").Limit(10)
    greetings := make([]Greeting, 0, 10)
    if _, err := q.GetAll(c, &greetings); err != nil {
        http.Error(w, err.String(), http.StatusInternalServerError)
        return
    }
    if err := guestbookTemplate.Execute(w, greetings); err != nil {
        http.Error(w, err.String(), http.StatusInternalServerError)
    }
}

var guestbookTemplate = template.Must(template.New("book").Parse(guestbookTemplateHTML))

const guestbookTemplateHTML = `
<html>
  <body>
    {{range .}}
      {{with .Author}}
        <p><b>{{html .}}</b> wrote:</p>
      {{else}}
        <p>An anonymous person wrote:</p>
      {{end}}
      <pre>{{html .Content}}</pre>
    {{end}}
    <form action="/sign" method="post">
      <div><textarea name="content" rows="3" cols="60"></textarea></div>
      <div><input type="submit" value="Sign Guestbook"></div>
    </form>
  </body>
</html>
`

func sign(w http.ResponseWriter, r *http.Request) {
    c := appengine.NewContext(r)
    g := Greeting{
        Content: r.FormValue("content"),
        Date:    datastore.SecondsToTime(time.Seconds()),
    }
    if u := user.Current(c); u != nil {
        g.Author = u.String()
    }
    _, err := datastore.Put(c, datastore.NewIncompleteKey(c, "Greeting", nil), &g)
    if err != nil {
        http.Error(w, err.String(), http.StatusInternalServerError)
        return
    }
    http.Redirect(w, r, "/", http.StatusFound)
}

Replace myapp/hello/hello.go with this, then reload http://localhost:8080/ in your browser. Post a few messages to verify that messages get stored and displayed correctly.

Storing the Submitted Greetings

For the guestbook application, we want to store greetings posted by users. Each greeting includes the author's name, the message content, and the date and time the message was posted so we can display messages in chronological order.

To represent this data we create a Go struct named Greeting:

type Greeting struct {
    Author  string
    Content string
    Date    datastore.Time
}

Now that we have a data type for greetings, the application can create new Greeting values and put them into the datastore. The new version of the sign handler does just that:

func sign(w http.ResponseWriter, r *http.Request) {
    c := appengine.NewContext(r)
    g := Greeting{
        Content: r.FormValue("content"),
        Date:    datastore.SecondsToTime(time.Seconds()),
    }
    if u := user.Current(c); u != nil {
        g.Author = u.String()
    }
    _, err := datastore.Put(c, datastore.NewIncompleteKey(c, "Greeting", nil), &g)
    if err != nil {
        http.Error(w, err.String(), http.StatusInternalServerError)
        return
    }
    http.Redirect(w, r, "/", http.StatusFound)
}

This creates a new Greeting value, settings its Author field to the current user, its Content field with the data posted by the user, and its Date field to the current time.

Finally, datastore.Put saves our new value to the datastore. We pass it a new, incomplete key so that the datastore will create a new key for this record automatically.

Retrieving the Stored Greetings With datastore.Query

The datastore package provides a Query type for querying the datastore and iterating over the results.

The new version of the root handler queries the datastore for greetings:

func root(w http.ResponseWriter, r *http.Request) {
    c := appengine.NewContext(r)
    q := datastore.NewQuery("Greeting").Order("-Date").Limit(10)
    greetings := make([]Greeting, 0, 10)
    if _, err := q.GetAll(c, &greetings); err != nil {
        http.Error(w, err.String(), http.StatusInternalServerError)
        return
    }
    if err := guestbookTemplate.Execute(w, greetings); err != nil {
        http.Error(w, err.String(), http.StatusInternalServerError)
    }
}

First the function constructs a Query value that requests Greeting objects in Date-descending order, with a limit of 10 objects.

q := datastore.NewQuery("Greeting").Order("-Date").Limit(10)

Then it calls q.GetAll(c, &greetings), which runs the query and appends the query results to the greetings slice.

greetings := make([]Greeting, 0, 10)
if _, err := q.GetAll(c, &greetings); err != nil {
    http.Error(w, err.String(), http.StatusInternalServerError)
    return
}

Finally, the guestbookTemplate.Execute function renders an HTML page containing these greetings and writes it out to the http.ResponseWriter. For more details on the templating language, see the template package documentation.

For a complete description of the Datastore API, see the Datastore reference.

Clearing the Development Server Datastore

The development web server uses a local version of the datastore for testing your application, using temporary files. The data persists as long as the temporary files exist, and the web server does not reset these files unless you ask it to do so.

If you want the development server to erase its datastore prior to starting up, use the --clear_datastore option when starting the server:

dev_appserver.py --clear_datastore myapp/

Next...

We now have a working guest book application that authenticates users using Google accounts, lets them submit messages, and displays messages other users have left. Because App Engine handles scaling automatically, we will not need to restructure our application as it gets popular.

Continue to Uploading Your Application.