English

Google App Engine

Using Google Data APIs on App Engine

Jeff Scudder
December 9, 2008

I'd like to expand on my earlier article entitled Retrieving Authenticated Google Data Feeds with Google App Engine by exploring how your app can read, write, and sync data to from Google services. For this article, I've written an app that allows you to invite your friends to events, like birthday parties! This app features integration with Google Calendar, using the Google Calendar Data API.

Once you have created an event, you can choose to add it to Google Calendar, and it will appear on your default calendar and the calendars of everyone who is invited. Once the event is associated with Google Calendar, any changes you make in the invitation app will be mirrored in Google Calendar.

We'll explore this design in the following sections:

Creating an Event in the Datastore

We begin by creating a data model to store an event.

from google.appengine.ext import db


class Event(db.Model):
    title = db.StringProperty(required=True)
    description = db.TextProperty()
    time = db.DateTimeProperty()
    location = db.TextProperty()
    creator = db.UserProperty()

And we need to track the attendees for each event, so we create an Attendee model:

class Attendee(db.Model):
    email = db.StringProperty()
    event = db.ReferenceProperty(Event)

When one of our users wants to create an event, they fill out a form on the event creation page. The essential part of this form handler creates attendees and events and stores them in the datastore.

import datetime
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
from google.appengine.api import users


class CreateEvent(webapp.RequestHandler):
    ...

    def post(self):
        # Create an event in the datastore.
        new_event = Event(title=self.request.get('name'),
                          creator=users.get_current_user(),
                          # Take the time string passing in by JavaScript in the
                          # form and convert to a datetime object.
                          time=datetime.datetime.strptime(
                              self.request.get('datetimestamp'), '%d/%m/%Y %H:%M'),
                          description=self.request.get('description'),
                          location=self.request.get('location'))
        new_event.put()

        # Associate each of the attendees with the event in the datastore.
        attendee_list = []
        if self.request.get('attendees'):
            attendee_list = self.request.get('attendees').split(',')
            if attendee_list:
                for attendee in attendee_list:
                    new_attendee = Attendee(email=attendee.strip(), event=new_event)
                    new_attendee.put()
      ...

Listing Events From the Datastore

Now that the event is in the datastore, we can display a list of the events which a user has created or has been invited to attend. An HTTP GET request for the event list page will search for some of the users events:

class EventsPage(webapp.RequestHandler):
    def get(self):
        """Displays the events the user has created or is invited to."""
        # Find the events which were created by this user and those which the user
        # is invited to.
        invited_events = []
        owned_events = []
        ...

        if users.get_current_user():t = Thread(target=threadproc)
            t.start()
            threads.append(t)
            time.sleep(DELAY_BETWEEN_THREAD_START)
            owned_query = Event.gql('WHERE creator = :1 ORDER BY time',
                users.get_current_user())
            owned_events = owned_query.fetch(5)

            invited_query = Attendee.gql('WHERE email = :1',
                users.get_current_user().email())
            for invitation in invited_query.fetch(5):
                invited_events.append(invitation.event)
      ...

You can see in the above code that we request five events that the user has created and five to which they have been invited. To show more events, this app could add paging but paging is beyond the scope of this article, so we will leave it out for now.

Using the Google Calendar Data API

Now that we can create and display events, we can hook up our app to Google Calendar using the Google Calendar Data API. The data API allows us to create, edit, and delete events (as well as other things which we won't be using like creating calendars).

In order for our app to modify the user's Google Calendar, the app needs to request their permission. If the user grants permission for this app to edit the user's calendar, our application will receive an AuthSub token which will be included in requests to the Google Calendar API. All of this is handled for you by the Google Data Python Client library.

To use the gdata-python-client in your app, you'll need to copy the atom and gdata directories from this client package into the root directory of your app. The atom and gdata directories are found under src. Once those directories are included, you'll need to add the following imports to the app:

import atom
import gdata.service
import gdata.auth
import gdata.alt.appengine
import gdata.calendar
import gdata.calendar.service

Obtaining the User's Authorization

Now we are ready to ask the user to authorize our app to access Google Calendar. This will only need to be done the first time that the user tries to post to Google Calendar. The auth token which the app receives can be used over and over (until the user revokes it) so the app automatically stores the token in the datastore for future reuse.

In this app, I've added the auth token management logic to the page which displays the user's events.

If there is no auth token which belongs to this user, we create a token_request_url which the user must visit if they want to post to Google Calendar. Once the user visits this URL and grants the app access to their calendar, they are redirected back to this events list page.

If the user has been redirected to this page after granting access, there will be an auth token in the page's URL, so we extract this token if it is present, upgrade it to a multi-use token, and store it in the datastore. For an explanation on the AuthSub protocol, tokens, and redirects, refer to the documentation for the AuthSub authorization protocol.

class EventsPage(webapp.RequestHandler):

    def __init__(self):
        # Create a Google Calendar client to talk to the Google Calendar service.
        self.calendar_client = gdata.calendar.service.CalendarService()
        # Modify the client to search for auth tokens in the datastore and use
        # urlfetch instead of httplib to make HTTP requests to Google Calendar.
        gdata.alt.appengine.run_on_appengine(self.calendar_client)

    def get(self):
        """Displays the events the user has created or is invited to."""
        # For brevity, left out the code from above which found events in the datastore.
        ...
        token_request_url = None

        # Find an AuthSub token in the current URL if we arrived at this page from
        # an AuthSub redirect.
        auth_token = gdata.auth.extract_auth_sub_token_from_url(self.request.uri)
        if auth_token:
            self.calendar_client.SetAuthSubToken(
                self.calendar_client.upgrade_to_session_token(auth_token))

        # Check to see if the app has permission to write to the user's
        # Google Calendar.
        if not isinstance(self.calendar_client.token_store.find_token(
                'http://www.google.com/calendar/feeds/'),
                gdata.auth.AuthSubToken):
            token_request_url = gdata.auth.generate_auth_sub_url(self.request.uri,
               ('http://www.google.com/calendar/feeds/default/',))

Creating an Event in Google Calendar

Now that the app has permission to access Google Calendar, we can create a Google Calendar event using the data for an event which the user has already created. The app will display a link to the event in Google Calendar once it is added, so we need to add a new property to our Event model to store the URL of the Google Calendar event.

When the app edits a Google Calendar event, it needs to send the updated information to the event's edit URL. The edit URL has version information, so that the Google server can detect if someone has edited the entry since you last saw it. If you are updating from an outdated version of the Google Calendar event, the server will alert you. Since the app needs this edit URL to make changes to the event, we need to add a property to store it as well.

The new event model for the app looks like this:

class Event(db.Model):
    title = db.StringProperty(required=True)
    description = db.TextProperty()
    time = db.DateTimeProperty()
    location = db.TextProperty()
    creator = db.UserProperty()
    edit_link = db.TextProperty()
    gcal_event_link = db.TextProperty()
    gcal_event_xml = db.TextProperty()

Now that the app can save the links to edit the Google Calendar event and display it, we add a form handler to create a new event in Google Calendar.

class EventsPage(webapp.RequestHandler):
    ...

    def post(self):
        """Adds an event to Google Calendar."""
        event_id = self.request.get('event_id')

        # Fetch the event from the datastore and make sure that the current user
        # is an owner since only event owners are allowed to create a calendar
        # event.
        event = Event.get_by_id(long(event_id))

        if users.get_current_user() == event.creator:
            # Create a new Google Calendar event.
            event_entry = gdata.calendar.CalendarEventEntry()
            event_entry.title = atom.Title(text=event.title)
            event_entry.content = atom.Content(text=event.description)
            start_time = '%s.000Z' % event.time.isoformat()
            event_entry.when.append(gdata.calendar.When(start_time=start_time))
            event_entry.where.append(
                gdata.calendar.Where(value_string=event.location))
            # Add a who element for each attendee.
            attendee_list = event.attendee_set
            if attendee_list:
                for attendee in attendee_list:
                  new_attendee = gdata.calendar.Who()
                  new_attendee.email = attendee.email
                  event_entry.who.append(new_attendee)

            # Send the event information to Google Calendar and receive a
            # Google Calendar event.
            try:
                cal_event = self.calendar_client.InsertEvent(event_entry,
                    'http://www.google.com/calendar/feeds/default/private/full')
                edit_link = cal_event.GetEditLink()
                if edit_link and edit_link.href:
                    # Add the edit link to the event to use for making changes.
                    event.edit_link = edit_link.href
                alternate_link = cal_event.GetHtmlLink()
                if alternate_link and alternate_link.href:
                    # Add a link to the event in the Google Calendar HTML web UI.
                    event.gcal_event_link = alternate_link.href
                    event.gcal_event_xml = str(cal_event)
                event.put()
            # If adding the event to Google Calendar failed due to a bad auth token,
            # remove the user's auth tokens from the datastore so that they can
            # request a new one.
            except gdata.service.RequestError, request_exception:
                request_error = request_exception[0]
                if request_error['status'] == 401 or request_error['status'] == 403:
                    gdata.alt.appengine.save_auth_tokens({})
                # If the request failure was not due to a bad auth token, reraise the
                # exception for handling elsewhere.
                else:
                    raise
        else:
            self.response.out.write('I\'m sorry, you don\'t have permission to add'
                                    ' this event to Google Calendar.')

The app can now take an existing entry from the datastore, and create a corresponding Google Calendar event!

Note the creation of the When element which uses a date format as described here.

Deleting Events

We can also allow the application to delete the Google Calendar event when the user chooses to delete their event in this app. The deletion handler will display a confirmation page and delete the event if the user posts by pressing the "yes" button.

class DeleteEvent(webapp.RequestHandler):
    def get(self):
        event_id = self.request.get('event_id')
        self.response.out.write('Are you sure?')
        self.response.out.write('<form action="/delete_event" method="post">'
            '<input name="event_id" value="%s" type="hidden">'
            '<input value="Yes, Delete this event." type="submit"></form>' % (
                event_id))

    def post(self):
        event_id = self.request.get('event_id')
        if event_id:
            event = Event.get_by_id(int(event_id))
            if event and users.get_current_user() == event.creator:
                # Create a Google Calendar client to use the Google Calendar service.
                calendar_client = gdata.calendar.service.CalendarService()
                # Modify the client to search for auth tokens in the datastore and use
                # urlfetch instead of httplib to make HTTP requests to Google Calendar.
                gdata.alt.appengine.run_on_appengine(calendar_client)
                # If we have an edit link, delete the event from Google Calendar.
                if event.edit_link:
                    calendar_client.DeleteEvent(str(event.edit_link))
                # Delete the event object from the datastore.
                event.delete()
        self.response.out.write('Deleted %s' % event_id)

Editing Events

In the same way we can copy any changes made to the event entity over to the Google Calendar event.

The application has an edit page which allows a user to change the details of an event which they own. If the event has been added to Google Calendar, we should update the Google Calendar event as well.

class EditEvent(webapp.RequestHandler):
    ...

    def post(self):
        """Changes the details of an event and updates Google Calendar."""
        ...
        event_id = self.request.get('event_id')
        if event_id:
            event = Event.get_by_id(int(event_id))
            if event and users.get_current_user() == event.creator:
                calendar_client = gdata.calendar.service.CalendarService()
                gdata.alt.appengine.run_on_appengine(calendar_client)
                # If this Event is in Google Calendar, send an update to Google Calendar
                if event.edit_link and event.gcal_event_xml:
                    # Reconstruct the Calendar entry, and update the information.
                    cal_event = gdata.calendar.CalendarEventEntryFromString(
                        str(event.gcal_event_xml))
                    # Modify the event's Google Calendar entry
                    cal_event.title = atom.Title(text=self.request.get('name'))
                    cal_event.content = atom.Content(text=self.request.get('description'))
                    start_time = '%s.000Z' % datetime.datetime.strptime(
                        self.request.get('datetimestamp'), '%d/%m/%Y %H:%M').isoformat()
                    cal_event.when = [gdata.calendar.When(start_time=start_time)]
                    cal_event.where = [gdata.calendar.Where(
                        value_string=self.request.get('location'))]
                    # Add a who element for each attendee.
                    if self.request.get('attendees'):
                        attendee_list = self.request.get('attendees').split(',')
                        if attendee_list:
                            cal_event.who = []
                            for attendee in attendee_list:
                                cal_event.who.append(gdata.calendar.Who(email=attendee))
                    # Send the updated Google Calendar entry to the Google server.
                    try:
                        updated_entry = calendar_client.UpdateEvent(str(event.edit_link),
                                                                    cal_event)
                        # Change the properties of the Event object.
                        event.edit_link = updated_entry.GetEditLink().href
                        event.gcal_event_xml = str(updated_entry)
                        event.title = self.request.get('name')
                        event.time = datetime.datetime.strptime(
                            self.request.get('datetimestamp'), '%d/%m/%Y %H:%M')
                        event.description = self.request.get('description')
                        event.location = self.request.get('location')
                        event.put()
                        self.response.out.write('Done')
                        # Continued below.
      ...

In the above, the app assumes that the datastore contains the most recent version of the Google Calendar entry. However, it is possible that the event has been changed in Google Calendar so we risk overwriting changes with our edit form.

Google Calendar has a mechanism in place to prevent silently overwriting new data from a stale entry (as do other Google Data APIs.) If Google Calendar detects that we are using an outdated versison of the data, the server will send a 409 Conflict message. The HTTP 409 response code is defined in RFC 2616 as:

The request could not be completed due to a conflict with the current state of the resource. This code is only allowed in situations where it is expected that the user might be able to resolve the conflict and resubmit the request. The response body SHOULD include enough information for the user to recognize the source of the conflict. Ideally, the response entity would include enough information for the user or user agent to fix the problem;...

When the client library sees a 409 Conflict message, the client raises an exception which we can catch and update the event with fresh data from Google Calendar.

When an editing conflict arises, this app will update the event in the datastore to hold that latest values from Google Calendar, and the user will need to re-edit the event. Now they will see the most recent information when editing the event.

                   except gdata.service.RequestError, request_exception:
                      request_error = request_exception[0]
                      # If the update failed because someone changed the Google Calendar
                      # event since creation, update the event and ask the user to
                      # repeat the edit.
                      if request_error['status'] == 409:
                          # Get the updated event information from Google Calendar.
                          updated_entry = gdata.calendar.CalendarEventEntryFromString(
                              request_error['body'])
                          # Change the properties of the Event object so that the next edit
                          # will begin with the new values.
                          event.edit_link = updated_entry.GetEditLink().href
                          event.gcal_event_xml = request_error['body']
                          event.title = updated_entry.title.text
                          event.description = updated_entry.content.text
                          event.location = updated_entry.where[0].value_string
                          # Edit time and attendees
                          ...
                          event.put()
                          self.response.out.write('Could not update because the event '
                                                  'has been edited in Google Calendar. '
                                                  'Event details have now been updated '
                                                  'with the latest values from Google '
                                                  'Calendar. Try again.')
                      # If the request failure was not due to an optimistic concurrency
                      # conflict reraise exception for handling elsewhere.
                      else:
                          raise
    ...

You may choose to use a different approach than the above, as you are free to handle these edit conflict errors however you like.

We could also reduce the chances of such a conflict by periodically checking our events to see if there have been any updated made in Google Calendar. The app could check the events when they are being displayed to see if there is a Calendar entry associated with the event, and check to see if there have been any modifications since the last saved version.

Suggestions and Ideas

In this article I've shown integrating with one Google Data service, but there are a host of Google Data APIs available and your app can use any and all of these at once. For example, our event invitation app could also modify a Google Spreadsheet which lists all of a users events. It could post on the user's blog using the Blogger API to promote an event which they have just created. Or, it could read a list of the user's contacts using the Google Contacts API to give them a list of people to pick from when inviting attendees.

For an example of a similar application, see: gdata-samples-calendar-contacts.appspot.com