Google Code offered in: English - Español - 日本語 - 한국어 - Português - Pусский - 中文(简体) - 中文(繁體)
The Blobstore API allows your application to serve data objects, called blobs, that are much larger than the size allowed for objects in the Datastore service. Blobs are useful for serving large files, such as video or image files, and for allowing users to upload large data files. Blobs are created by uploading a file through an HTTP request. Typically, your applications will do this by presenting a form with a file upload field to the user. When the form is submitted, the Blobstore creates a blob from the file's contents and returns an opaque reference to the blob, called a blob key, which you can later use to serve the blob. The application can serve the complete blob value in response to a user request, or it can read the value directly using a streaming file-like interface.
Google App Engine includes the Blobstore service, which allows applications to serve data objects limited only by the amount of data
that can be uploaded or downloaded over
a single HTTP connection. These objects are called
Blobstore values, or blobs.
Blobstore values are served as responses from request handlers and are created as uploads via web forms. Applications do not create blob data directly; instead, blobs are created indirectly, by a submitted web form or other HTTP
POST
request.
Blobstore values can be served to the user, or accessed by the application in a file-like stream, using the Blobstore API.
Note: Blobs as defined by the Blobstore service are not related to blob property values used by the datastore.
To prompt a user to upload a Blobstore value, your application presents a web form with a file upload field. The application generates the form's action URL by calling the Blobstore API. The user's browser uploads the file directly to the Blobstore via the generated URL. Blobstore then stores the blob, rewrites the request to contain the blob key, and passes it to a path in your application. A request handler at that path in your application can perform additional form processing.
To serve a blob, your application sets a header on the outgoing response, and App Engine replaces the response with the blob value.
Blobs can't be modified after they're created, though they can be deleted. Each blob has a corresponding blob info record, stored in the datastore, that provides details about the blob, such as its creation time and content type. You can use the blob key to fetch blob info records and query their properties.
Applications can use the Blobstore to accept large files as uploads
from users and to serve those files. Files are called blobs once they're
uploaded. Applications don't access blobs directly. Instead,
applications work with blobs through blob info entities
(represented by the
BlobInfo
class) in the datastore.
The user creates a blob by submitting an HTML form that includes one or more file input fields. Your application calls create_upload_url()
to get the destination (action) of this form, passing the function a URL path of a handler in your application. When the user submits the form, the user's browser uploads the specified files directly to the Blobstore. The Blobstore rewrites the user's request and stores the uploaded file data, replacing the uploaded file data with one or more corresponding blob keys, then passes the rewritten request to the handler at the URL path you provided to create_upload_url()
. This handler can do additional processing based on the blob key.
The application can read portions of a Blobstore value using a file-like streaming interface. See
The BlobReader
Class.
To create and upload a blob, follow this procedure:
blobstore.create_upload_url()
to create an upload URL for the form that the user will fill out, passing the application path to load
when the POST
of the form
is completed:
upload_url = blobstore.create_upload_url('/upload')
There is an asynchronous version,
create_upload_url_async()
.
It allows your application
code to continue running while Blobstore generates the upload URL.
enctype
must be set to multipart/form-data
. When the user submits the form, the
POST
is handled by the Blobstore API, which creates the blob. The API also creates an info record for the blob and stores the record in the datastore, and passes the rewritten request to
your application on the given path as a blob key:
self.response.out.write('<html><body>') self.response.out.write('<form action="%s" method="POST" enctype="multipart/form-data">' % upload_url) self.response.out.write("""Upload File: <input type="file" name="file"><br> <input type="submit" name="submit" value="Submit"> </form></body></html>""")
class UploadHandler(blobstore_handlers.BlobstoreUploadHandler): def post(self): upload_files = self.get_uploads('file') # 'file' is file upload field in the form blob_info = upload_files[0] self.redirect('/serve/%s' % blob_info.key())
The webapp framework provides the blobstore_handlers.BlobstoreUploadHandler
upload handler class to help you parse the form data. For more information, see the reference for BlobstoreUploadHandler
.
When the Blobstore rewrites the user's request, the MIME parts of the uploaded files have their bodies emptied, and the blob key is added as a MIME part header. All other form fields and parts are preserved and passed to the upload handler.
If you don't specify a content type, the Blobstore will try to infer it
from the file extension. If no content type can be determined, the newly created blob is assigned content type application/octet-stream
.
To serve blobs, you must include a blob download handler as a path in
your application. The application serves a blob by setting a header on the
outgoing response.
The following sample uses
the webapp
framework. When using webapp
, the handler should pass the blob key for the desired blob to self.send_blob()
.
In this example, the blob key is passed to the download handler as part of the URL. The download handler can get the blob key by any means you choose,
such as through another method or user action.
class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler): def get(self, resource): resource = str(urllib.unquote(resource)) blob_info = blobstore.BlobInfo.get(resource) self.send_blob(blob_info)
The webapp framework provides the download handler class
blobstore_handlers.BlobstoreDownloadHandler
to help you parse the form data. For more information, see the reference for
BlobstoreDownloadHandler
.
Blobs can be served from any application URL. To serve a blob in your application, you put a special header in the response containing the blob key. App Engine replaces the body of the response with the content of the blob.
The Blobstore supports serving part of a large value instead of the full value in response to a request. To serve a partial value, include the X-AppEngine-BlobRange
header in the outgoing response. Its value is a standard HTTP byte range.
The byte numbering is zero-based.
A blank X-AppEngine-BlobRange
instructs the API to ignore the range header and serve the full blob. Example ranges include:
0-499
serves the first 500 bytes of the value (bytes 0 through 499, inclusive).500-999
serves 500 bytes starting with the 501st byte.500-
serves all bytes starting with the 501st byte to the end of the value.-500
serves the last 500 bytes of the value.If the byte range is valid for the Blobstore value, the Blobstore sends a
206
Partial
Content
status code and the requested byte range to the client. If the range is not valid for the value, the Blobstore sends
416
Requested
Range
Not
Satisfiable
.
The Blobstore does not support multiple byte ranges in a single request
(for example, 100-199,200-299
),
whether or not they overlap.
The
webapp.blobstore_handlers.BlobstoreDownloadHandler
class includes features for setting this header using provided byte indices, and for deriving the byte range automatically from a range
header provided by the user.
In the following sample application, the application's main URL loads the form that asks the user for a file to upload, and the upload handler immediately calls the download handler to serve the data. This is to simplify the sample application. In practice, you would probably not use the main URL to request upload data, nor would you immediately serve a blob you had just uploaded.
#!/usr/bin/env python # import os import urllib from google.appengine.ext import blobstore from google.appengine.ext import webapp from google.appengine.ext.webapp import blobstore_handlers from google.appengine.ext.webapp import template from google.appengine.ext.webapp.util import run_wsgi_app class MainHandler(webapp.RequestHandler): def get(self): upload_url = blobstore.create_upload_url('/upload') self.response.out.write('<html><body>') self.response.out.write('<form action="%s" method="POST" enctype="multipart/form-data">' % upload_url) self.response.out.write("""Upload File: <input type="file" name="file"><br> <input type="submit" name="submit" value="Submit"> </form></body></html>""") class UploadHandler(blobstore_handlers.BlobstoreUploadHandler): def post(self): upload_files = self.get_uploads('file') # 'file' is file upload field in the form blob_info = upload_files[0] self.redirect('/serve/%s' % blob_info.key()) class ServeHandler(blobstore_handlers.BlobstoreDownloadHandler): def get(self, resource): resource = str(urllib.unquote(resource)) blob_info = blobstore.BlobInfo.get(resource) self.send_blob(blob_info) def main(): application = webapp.WSGIApplication( [('/', MainHandler), ('/upload', UploadHandler), ('/serve/([^/]+)?', ServeHandler), ], debug=True) run_wsgi_app(application) if __name__ == '__main__': main()
An application can call some Blobstore functions that work in the background.
Blobstore carries out the request while the application does other things.
To make the request, the application calls an asynchronous function.
The function immediately returns an RPC object;
this object represents the request.
When the application needs the result of the request, it calls the
RPC object's
get_result()
method.
If the service has not completed the request when the application calls
get_result()
,
the method waits until the request is complete (or has reached the
deadline, or an error occurs). The method returns the result object,
or raises an exception if an error occurred while carrying out the request.
For example, this code snippet
upload_url = blobstore.create_upload_url('/upload') slow_operation() self.response.out.write("""<form action="%s" method="POST" enctype="multipart/form-data">""" % upload_url)
becomes
upload_url_rpc = blobstore.create_upload_url_async('/upload') slow_operation() upload_url = upload_url_rpc.get_result() self.response.out.write("""<form action="%s" method="POST" enctype="multipart/form-data">""" % upload_url)
In this example, the application carries out the
slow_operation()
code at the same time that
Blobstore generates the upload URL.
App Engine allows you to create Blobstore blobs programmatically, providing a file-like API that you can use to read and write to blobs. Some common uses of this functionality include exporting data and generating reports—or any function that involves generating large binary data objects.
This is a low-level API.
The following sample shows how to create a new Blobstore file and get its blob key:
from __future__ import with_statement from google.appengine.api import files # Create the file file_name = files.blobstore.create(mime_type='application/octet-stream') # Open the file and write to it with files.open(file_name, 'a') as f: f.write('data') # Finalize the file. Do this before attempting to read it. files.finalize(file_name) # Get the file's blob key blob_key = files.blobstore.get_blob_key(file_name)
Space used for Blobstore values contributes to the Stored Data (billable) quota. Blob info entities in the datastore count towards datastore-related limits.
For more information on systemwide safety quotas, see Quotas, and the Quota Details section of the Administration Console.
In addition to systemwide safety quotas, a limit applies specifically to the use of the Blobstore: the maximum size of Blobstore data that can be read by the application with one API call is 32 megabytes.