English

Google App Engine

Overview of the Conversion API (Java)

Experimental!

Conversion is an experimental, innovative, and rapidly changing new feature for App Engine. Unfortunately, being on the bleeding edge means that we may make backwards-incompatible changes to Conversion. We will inform the community when this feature is no longer experimental.

Overview

The App Engine Conversion API converts documents between common filetypes using Google's infrastructure for efficiency and scale. The API enables conversions between HTML, PDF, text, and image formats, synchronously or asynchronously, with an option to perform optical character recognition (OCR). You can use it for functions such as:

  • Processing scanned documents
  • Building indexing and search pipelines
  • Generating printable files for download

Supported Filetypes

The API supports the following types of files:

  • html - CSS is supported as long as it is included within the HTML page; external CSS files are not supported. Paged Media CSS is supported.
  • txt
  • gif, jpg, bmp - When converting from text-based formats into *.png, the resulting document has one asset for each page of the text file. For conversions from *.pdf files, the contents of a page are determined by the pagination in the file. For *.html files, content is placed into *.pdf and then converted from *.pdf to *.png. For *.txt files, page size is determined by system defaults.
  • png files may be used as the output for a conversion, but not the input.
  • pdf

See Conversion Paths for more information about conversions between these filetypes.

This document has the following sections:

Getting Started

The Conversion API employs a few core concepts and data structures:

  • Asset: a piece of data, tagged with a MIME type and an identifier
  • Document: a collection of one or more assets.
  • Conversion: converts document to document, based on an output MIME type and optional conversion options.

We expect all input asset strings(e.g. HTML and TXT) in UTF-8. We support the most common fonts used across the web, and at least one font for sans-serif, serif and monospace.

Performing Conversions Synchronously

Synchronous conversions apply to situations in which the user expects the converted file immediately. Users must wait for the time it takes the API to perform the conversion, so when using the synchronous API, avoid time-intensive operations (such as OCR). For example, you would use synchronous conversions to dynamically generate a PDF invoice when a customer clicks on a download link.

import com.google.appengine.api.conversion.Asset;
import com.google.appengine.api.conversion.Conversion;
import com.google.appengine.api.conversion.ConversionResult;
import com.google.appengine.api.conversion.ConversionService;
import com.google.appengine.api.conversion.ConversionServiceFactory;
import com.google.appengine.api.conversion.Document;

// Create a conversion request from HTML to PNG.
Asset asset = new Asset(
    "text/html", "<b>some data<b>".getBytes(), "testfile.html");
Document document = new Document(asset);
Conversion conversion = new Conversion(document, "image/png");

ConversionService service =
    ConversionServiceFactory.getConversionService();
ConversionResult result = service.convert(conversion);

if (result.success()) {
  // Note: in most cases, we will return data all in one asset,
  // except that we return multiple assets for multi-page images.
  for (Asset asset : result.getOutputDoc().getAssets()) {
    doSomethingWithAsset(asset.getData());
  }
} else {
  handleError(result.getErrorCode());
}

Performing Conversions Asynchronously

The asynchronous API allows you to perform conversions in the background for jobs in which the user does not expect an immediate result. Asynchronous conversions are useful for time-intensive operations (such as OCR). For example, use synchronous conversions for background jobs that generate invoices and send email them to a number of customers.

import com.google.appengine.api.conversion.Asset;
import com.google.appengine.api.conversion.Conversion;
import com.google.appengine.api.conversion.ConversionResult;
import com.google.appengine.api.conversion.ConversionService;
import com.google.appengine.api.conversion.ConversionServiceFactory;
import com.google.appengine.api.conversion.Document;

// Create a conversion request from HTML to PNG.
Asset asset = new Asset(
    "text/html", "<b>some data</b>".getBytes(), "testfile.html");
Document document = new Document(asset);
Conversion conversion = new Conversion(document, "image/png");

ConversionService service =
    ConversionServiceFactory.getConversionService();
Future<ConversionResult> resultFuture = service.convertAsync(conversion);

// ... Do something else as you like ...

ConversionResult result = resultFuture.get();

if (result.success()) {
  // Note: in most cases, we will return data all in one asset.
  // We return multiple assets for multi-page images.
  for (Asset asset : result.getOutputDoc().getAssets()) {
    doSomethingWithAsset(asset.getData());
  }
} else {
  handleError(result.getErrorCode());
}

Additional Functions

Adding Assets

You can add more assets to a document (such as images in an HTML page) simply by defining the asset and calling the conversion function as follows:

import com.google.appengine.api.conversion.Asset;
import com.google.appengine.api.conversion.Conversion;
import com.google.appengine.api.conversion.ConversionResult;
import com.google.appengine.api.conversion.ConversionService;
import com.google.appengine.api.conversion.ConversionServiceFactory;
import com.google.appengine.api.conversion.Document;

// Create a conversion request from HTML to PNG.
Asset asset = new Asset(
    "text/html", "<b>some data</b>".getBytes(), "testfile.html");
Document document = new Document(asset);
Conversion conversion = new Conversion(document, "image/png");

ConversionService service =
    ConversionServiceFactory.getConversionService();
Future<ConversionResult> resultFuture = service.convertAsync(conversion);

// ... Do additional work

ConversionResult result = resultFuture.get();

if (result.success()) {
  // Note: in most cases, we will return data all in one asset.
  // We return multiple assets for multi-page images.
  for (Asset asset : result.getOutputDoc().getAssets()) {
    doSomethingWithAsset(asset.getData());
  }
} else {
  handleError(result.getErrorCode());
}

Note: The name of the additional asset (in this case, static/icon.gif) is used by the primary HTML file to reference the image, so it must be the same as the corresponding HTML tag src attribute. The API does not currently support conversion using external CSS files, but you can get around this by adding the CSS directly to the HTML page.

Performing Multiple Conversions in a Single Request

To perform multiple conversions in one request, simply pass a list of conversions into the Convert method, and get back the results in the same order:

List<Conversion> conversions = new ArrayList<Conversion>();
conversions.add(new Conversion(document1, "text/plain"));
conversions.add(new Conversion(document2, "text/html"));
List<ConversionResult> results = service.convert(conversions);

More Conversion Options

The Conversion API also allows you to provide more specific conversion options. You can specify:

  • The width of output images (*.png only)
  • A specific page or page range to output to an image (*.png only)
  • The language of source text for OCR operations (*.txt, *.html, and *.pdf only)
. The following code sample converts pages 2-10 of an HTML page into *.png files that are 1,000 pixels wide.
ConversionOptions options = ConversionOptions.Builder
    .withImageWidth(1000)
    .firstPage(2)
    .lastPage(10);
Conversion conversion = new Conversion(input, OUTPUT_TYPE, options);

Conversion Paths

Conversions can be performed in any direction between PDF, HTML, TXT, and image formats, and OCR will be employed if necessary. Note that while PNG, GIF, JPEG, and BMP image formats are supported as input formats, only PNG is available for output.

Please refer to the following table for the complete list of conversions:

Input MIME Type Output MIME Type Category
image/bmp, image/gif, image/jpeg, image/png text/html OCR
image/bmp, image/gif, image/jpeg, image/png application/pdf OCR
image/bmp, image/gif, image/jpeg, image/png text/plain OCR
application/pdf text/html OCR
application/pdf text/plain OCR
application/pdf image/png Printable generation
text/html application/pdf Printable generation
text/html image/png Printable generation
text/html text/plain HTML to TXT conversion
text/plain text/html TXT to HTML conversion
text/plain application/pdf Printable generation
text/plain image/png Printable generation

Limits

Each Conversion request counts toward the Conversion API Calls quota. At this stage, we offer a small free quota of 100 conversions per day.

In addition to quotas, the following limits also apply the following limits:

Limit Amount
file size 2 megabytes
maximum conversions per request 10 conversions
maximum deadline 60 seconds*

*Beware the 60s time limit, particularly when invoking OCR on large PDF documents.