English

Google App Engine

Choosing a Datastore (Python)

App Engine offers two data storage options with different reliability and consistency guarantees:

  • In the High Replication Datastore (HRD), data is replicated across multiple data centers using a system based on the Paxos algorithm. This provides the highest level of availability for reads and writes, at the cost of higher latency on writes due to the propagation of data. Most queries are eventually consistent.
  • The Master/Slave Datastore designates one data center to hold the master copy of all data. Data written to the master data center is replicated asynchronously to all other (slave) data centers. Since only one data center is the master for writing at any given time, this option offers strong consistency for all reads and queries, at the cost of periods of temporary unavailability during data center issues or planned downtime.

Note: The Python 2.7 runtime does not support the Master/Slave Datastore.

This page compares the two options and describes how to specify one of them for your application.

Data Storage Options Compared

The High Replication Datastore (HRD) is a highly available, highly reliable storage solution that remains available for reads and writes during routine maintenance and unplanned events, and is extremely resilient in the face of catastrophic failure. The Master/Slave Datastore, by contrast, is suitable only for a limited class of applications that

  • do not require high availability of data
  • can tolerate spikes in Datastore latency

In the HRD, all strongly consistent operations (such as get by key or put) occur within a transaction. Entity groups are a unit of consistency as well as of transactionality: queries that require strongly consistent results must contain an ancestor filter, which restricts the results to a single entity group. Queries that span multiple entity groups are not guaranteed to return up-to-date results. For more information about using ancestor queries in this context, see Using the High Replication Datastore.

The following table summarizes the differences between the High-Resolution and Master/Slave Datastores:

High Replication Master/Slave
Performance
Put/delete latency 1/2x–1x 1x
Get latency 1x 1x
Query latency 1x 1x
Consistency
Put/get/delete Strong Strong
Most queries Eventual Strong
Ancestor queries Strong Strong
Occasional planned read-only period
No Yes
Unplanned downtime
Extremely rare; no data loss. Rare; possible to lose a small % of writes occurring near downtime (recoverable after event)
Python 2.7 support
Only supported Datastore Not supported

Selecting a Data Storage Option

The HRD is the default for all new applications. Existing applications must migrate to the HRD by copying entities to a new application. If you prefer to use the Master/Slave Datastore, you can select it as follows:

For a new application:
  1. On the Create an Application screen, click Edit.
  2. Select the Master/Slave radio button.
  3. Click Create Application.
  4. Warning! This option is irreversible; if you choose the Master/Slave Datastore and later want to migrate to the HRD, you'll need to create a new application and copy your current Datastore to the new application.

For an existing application:
  1. Create a new application using the steps above.
  2. Copy your current Datastore to the new application.