Google Code offered in: English - Español - 日本語 - 한국어 - Português - Pусский - 中文(简体) - 中文(繁體)
NDB is an experimental, innovative, and rapidly changing new feature for App Engine. Unfortunately, being on the bleeding edge means that we may make backwards-incompatible changes to NDB. We will inform the community when this feature is no longer experimental.
The NDB API provides persistent storage in a schemaless object datastore. It supports automatic caching, sophisticated queries, and atomic transactions. NDB is well-suited to storing structured data records.
An application creates entities, objects with data values stored as properties of an entity. When the application reads an entity, that entity is automatically cached; this gives fast (and inexpensive) reads for frequently-read entities. The application can perform queries over entities.
NDB saves data objects, known as entities. An entity has one or more properties, named values of one of several supported data types. For example, a property can be a string, an integer, or a reference to another entity.
NDB can group multiple operations in a single transaction. The transaction cannot succeed unless every operation in the transaction succeeds; if any of the operations fail, the transaction is automatically rolled back. This is especially useful for distributed web applications, where multiple users may be accessing or manipulating the same data at the same time.
NDB uses Memcache as a cache service for "hot spots" in the data. If the application reads some records often, NDB can read them quickly from cache.
An application uses NDB to define models. A model is a Python class that acts as a sort of database schema: it describes a kind of data in terms of the properties. The underlying Datastore is very flexible in how it stores data objects—two entities of the same kind can have different properties. An application can use NDB models to enforce type-checking but doesn't have to.
Each entity is identified by a key, an identifier unique within the application's datastore. The key can have a parent, another key. This parent can itself have a parent, and so on; at the top of this "chain" of parents is a key with no parent, called the root.
Entities whose keys have the same root form an entity group or group. If entities are in different groups, then changes to those entities might sometimes seem to occur "out of order". If the entities are unrelated in your application's semantics, that's fine. But if some entities' changes should be consistent, your application should make them part of the same group when creating them.
import cgi import urllib from google.appengine.ext import ndb from google.appengine.ext import webapp from google.appengine.ext.webapp.util import run_wsgi_app class Greeting(ndb.Model): """Models an individual Guestbook entry with content and date.""" content = ndb.StringProperty() date = ndb.DateTimeProperty(auto_now_add=True) @classmethod def query_book(cls, ancestor_key): return cls.query(ancestor=ancestor_key).order(-cls.date) class MainPage(webapp.RequestHandler): def get(self): self.response.out.write('<html><body>') guestbook_name = self.request.get('guestbook_name') ancestor_key = ndb.Key("Book", guestbook_name or "*notitle*") greetings = Greeting.query_book(ancestor_key).fetch(20) for greeting in greetings: self.response.out.write('<blockquote>%s</blockquote>' % cgi.escape(greeting.content)) self.response.out.write(""" <form action="/sign?%s" method="post"> <div><textarea name="content" rows="3" cols="60"></textarea></div> <div><input type="submit" value="Sign Guestbook"></div> </form> <hr> <form>Guestbook name: <input value="%s" name="guestbook_name"> <input type="submit" value="switch"></form> </body> </html>""" % (urllib.urlencode({'guestbook_name': guestbook_name}), cgi.escape(guestbook_name))) class SubmitForm(webapp.RequestHandler): def post(self): # We set the parent key on each 'Greeting' to ensure each guestbook's # greetings are in the same entity group. guestbook_name = self.request.get('guestbook_name') greeting = Greeting(parent=ndb.Key("Book", guestbook_name or "*notitle*"), content = self.request.get('content')) greeting.put() self.redirect('/?' + urllib.urlencode({'guestbook_name': guestbook_name})) application = webapp.WSGIApplication([ ('/', MainPage), ('/sign', SubmitForm) ]) def main(): run_wsgi_app(application) if __name__ == '__main__': main()
A model is a class that describes a type of entity,
including the types and configuration for its properties.
It's roughly analogous to a SQL Table.
An entity can be created by calling the model's class constructor and then
stored by calling the put()
method.
class Greeting(ndb.Model): """Models an individual Guestbook entry with content and date.""" content = ndb.StringProperty() date = ndb.DateTimeProperty(auto_now_add=True) class SubmitForm(webapp.RequestHandler): def post(self): # We set the parent key on each 'Greeting' to ensure each guestbook's # greetings are in the same entity group. guestbook_name = self.request.get('guestbook_name') greeting = Greeting(parent=ndb.Key("Book", guestbook_name or "*notitle*"), content = self.request.get('content')) greeting.put()
This sample code defines the model class Greeting
. Each
Greeting
entity has two properties: the text content of
the greeting and the date the greeeting was created.
To create and store a new greeting, the application creates a new
Greeting
object and calls its put()
method.
To make sure that greetings in a guestbook don't appear "out of order"
the application sets a parent key when creating a new Greeting
.
Thus, the new greeting will be in the same entity group as other
greetings in the same guestbook. The application uses this fact
when querying: it uses an ancestor query.
An application can query to find entities that match some filters.
class Greeting(ndb.Model): ... @classmethod def query_book(cls, ancestor_key): return cls.query(ancestor=ancestor_key).order(-cls.date) def get(self): guestbook_name = self.request.get('guestbook_name') ancestor_key = ndb.Key("Book", guestbook_name or "*notitle*") greetings = Greeting.query_book(ancestor_key).fetch(20) for greeting in greetings: self.response.out.write('<blockquote>%s</blockquote>' % cgi.escape(greeting.content))
A typical NDB query filters entities by kind. In this example,
query_book
generates a query that returns Greeting
entities.
A query can also specify filters on entity
property values and keys. As in this example, a query can specify an
ancestor, finding only entities that "belong to" some ancestor.
A query can specify sort order.
If a given entity has at least one (possibly null) value for every
property in the filters and sort orders and all the filter criteria
are met by the property values, then that entity is returned as a result.
Every query uses an index, a table that contains the results for the query in the desired order. The underlying Datastore automatically maintains simple indexes (indexes that use only one property). It defines its complex indexes in a configuration file, index.yaml. The development web server automatically adds suggestions to this file when it encounters queries that do not yet have indexes configured. You can tune indexes manually by editing the file before uploading the application. You can also update the indexes separately from uploading the application. If your datastore has many records, it takes a long time to create a new index for them; in this case, it's wise to update the index definitions before uploading code that uses the new index.
This index mechanism supports a wide range of queries and is suitable for most applications. However, it does not support some kinds of queries common in other database technologies. In particular, joins aren't supported.
Note: For a full discussion of writes to the underlying Datastore, see Life of a Datastore Write and Transaction Isolation in App Engine.
NDB writes data in steps:
The NDB function that writes the data (for example, put()
)
returns after the cache invalidation; the Apply phase happens
asynchronously.
If there is a failure during the Commit phase, there are automatic retries, but if failures continue, your application receives an exception. If the Commit phase succeeds but the Apply fails, the Apply is rolled forward to completion when one of the following occurs:
This behavior affects how and when data is visible to your application. The change may not be completely applied to the underlying Datastore a few hundred milliseconds or so after the NDB function returns. A non-ancestor query performed while a change is being applied may see an inconsistent state (i.e., part but not all of the change). For more information about the timing of writes and queries, see Transaction Isolation in App Engine.
NDB uses the App Engine Datastore. Each use of the Datastore counts toward the Datastore API Calls limit. Some NDB calls result in multiple calls to the API, and so use more of your resource.
Data sent to the datastore by the app counts toward the Data Sent to (Datastore) API limit. Data received by the app from the datastore counts toward the Data Received from (Datastore) API limit.
The total amount of data currently stored in the datastore for the app cannot exceed the Stored Data (billable) limit. This includes all entity properties and keys and the indexes necessary to support querying these entities. See How Entities and Indexes are Stored for a complete breakdown of the metadata required to store entities and indexes at the Bigtable level.
For more information on system-wide safety limits, see Limits, and the "Quota Details" section of the Admin Console.
In addition to system-wide safety limits, the following limits apply specifically to the use of the datastore:
Limit | Amount |
---|---|
maximum entity size | 1 megabyte |
maximum number of values in all indexes for an entity |
5,000 values |
|