How to log Google API Errors in Python

Errors are common for any application, it's just that we need to log and handle them appropriately. It looks like just about every error I get back from BigQuery is a apiclient.errors.HttpError, for example

<HttpError 409 when requesting https://www.googleapis.com/bigquery/v2/projects/some_id/datasets/some_dataset/tables?alt=json returned "Already Exists: Table some_id:some_dataset.some_table">

BigQuery is a REST API, the errors it uses follow standard HTTP error conventions.

In python, an HttpError has a resp.status field that returns the HTTP status code. This can be: notFound, duplicate, accessDenied, invalidQuery, backendError, resourcesExceeded, invalid, quotaExceeded, rateLimitExceeded, timeout, etc.

For example:

from googleapiclient.errors import HttpError
try:
   ...
except HttpError as err:
  # If the error is a rate limit or connection error,
  # wait and try again.
  if err.resp.status in [403, 500, 503]:
    time.sleep(5)
  else: raise

The response is also a JSON object, an even better way is to parse the JSON and read the error reason field:

if err.resp.get('content-type', '').startswith('application/json'):
    reason = json.loads(err.content).get('error').get('errors')[0].get('reason')

victor

posted on 09 Aug 18