English | Site Directory

Transactions

The App Engine datastore supports transactions. A transaction is an operation or set of operations that either succeeds completely, or fails completely. An application can perform multiple operations and calculations in a single transaction.

Using Transactions

A transaction is a datastore operation or a set of datastore operations that either succeed completely, or fail completely. If the transaction succeeds, then all of its intended effects are applied to the datastore. If the transaction fails, then none of the effects are applied.

Every datastore write operation is atomic. An attempt to create, update or delete an entity either happens, or it doesn't. An operation may fail due to a high rate of contention, with too many users trying to modify an entity at the same time. Or an operation may fail due to the application reaching a quota limit. Or there may be an internal error with the datastore. In all cases, the operation's effects are not applied, and the datastore API raises an exception.

An application can execute a set of statements and datastore operations in a single transaction, such that if any statement or operation raises an exception, none of the datastore operations in the set are applied. The application defines the actions to perform in the transaction using a Python function, then calls db.run_in_transaction() with the function as an argument:

from google.appengine.ext import db

class Accumulator(db.Model):
  counter = db.IntegerProperty()

def increment_counter(key, amount):
  obj = db.get(key)
  obj.counter += amount
  obj.put()

q = db.GqlQuery("SELECT * FROM Accumulator")
acc = q.get()

db.run_in_transaction(increment_counter, acc.key(), 5)

db.run_in_transaction() takes the function object, and positional and keyword arguments to pass to the function. If the function returns a value, db.run_in_transaction() will return the value.

If the function returns, the transaction is committed, and all effects of datastore operations are applied. If the function raises an exception, the transaction is "rolled back," and the effects are not applied.

If the function raises the Rollback exception, db.run_in_transaction() returns None. For any other exception, db.run_in_transaction() re-raises the exception.

What Can Be Done In a Transaction

The datastore imposes several restrictions on what can be done inside a single transaction.

All datastore operations in a transaction must operate on entities in the same entity group. This includes retrieving entities by key, updating entities, and deleting entities. Notice that each root entity belongs to a separate entity group, so a single transaction cannot create or operate on more than one root entity. For an explanation of entity groups, see Keys and Entity Groups.

An app cannot perform a query during a transaction. However, an app can retrieve datastore entities using keys during a transaction, and be guaranteed that the fetched entity is consistent with the rest of the transaction. You can prepare keys prior to the transaction, or you can build keys inside the transaction with key names or IDs.

An application cannot create or update an entity more than once in a single transaction.

All other Python code is allowed inside a transaction function. The transaction function should not have side effects other than the datastore operations. The transaction function may be called multiple times if a datastore operation fails due to another user updating entities in the entity group at the same time. When this happens, the datastore API retries the transaction a fixed number of times. If they all fail, db.run_in_transaction() raises a TransactionFailedError. You can adjust the number of times the transaction is retried using db.run_in_transaction_custom_retries() instead of db.run_in_transaction().

Similarly, the transaction function should not have side effects that depend on the success of the transaction, unless the code that calls the transaction function knows to undo those effects. For example, if the transaction stores a new datastore entity, saves the created entity's ID for later use, then the transaction fails, the saved ID does not refer to the intended entity because the entity's creation was rolled back. The calling code would have to be careful not to use the saved ID in this case.

Uses For Transactions

This example demonstrates one use of transactions: updating an entity with a new property value relative to its current value.

def increment_counter(key, amount):
  obj = db.get(key)
  obj.counter += amount
  obj.put()

This requires a transaction because the value may be updated by another user after this code fetches the object, but before it saves the modified object. Without a transaction, the user's request will use the value of counter prior to the other user's update, and the save will overwrite the new value. With a transaction, the application is told about the other user's update. If the entity is updated during the transaction, then the transaction is retried until all steps are completed without interruption.

Another common use for transactions is to update an entity with a named key, or create it if it doesn't yet exist:

class SalesAccount(db.Model):
  address = db.PostalAddressProperty()
  phone_number = db.PhoneNumberProperty()

def create_or_update(parent_obj, account_id, address, phone_number):
  obj = db.get(Key.from_path("SalesAccount", account_id, parent=parent_obj))
  if not obj:
    obj = SalesAccount(key_name=account_id,
                       parent=parent_obj,
                       address=address,
                       phone_number=phone_number)
  else:
    obj.address = address
    obj.phone_number = phone_number

  obj.put()

As before, a transaction is necessary to handle the case where another user is attempting to create or update an entity with the same string ID. Without a transaction, if the entity does not exist and two users attempt to create it, the second will overwrite the first without knowing that it happened. With a transaction, the second attempt will retry, notice that the entity now exists, and update the entity instead.

Create-or-update is so useful that there is a built-in method for it: Model.get_or_insert() takes a key name, an optional parent, and arguments to pass to the model constructor if an entity of that name and path does not exist. The get attempt and the create happen in one transaction, so (if the transaction is successful) the method always returns a model instance that represents an actual entity.

Tip: A transaction should happen as quickly as possible to reduce the likelihood that the entities used by the transaction will change, requiring the transaction be retried. As much as possible, prepare data outside of the transaction, then execute the transaction to perform datastore operations that depend on a consistent state. The application should prepare keys for objects used inside the transaction, then fetch the entities inside the transaction.