Bigquery Table too large to be exported to a single file. Specify a uri including a * to shard export

When trying to export a BigQuery Table from Web UI to Google Storage bucket, You get the below error message:

Table too large to be exported to a single file. 
Specify a uri including a * to shard export

You can export up to 1 GB of table data to a single file. If you are exporting more than 1 GB of data, use a wildcard to export the data into multiple files. When you export data to multiple files, the size of the files will vary

Solution:

If you table is small (< 1GB) in size, then you do not have a problem, you can simply specify the bucket name and file name:

gs://my-bucket/file-name.json

If you table is big (> 1GB) in size, then include a * in the file name, so that google can export the table into multiple file chunks, each 1 GB in size, like given below:

gs://my-bucket/file-name-*.json

It will create multiple files like shown below:

gs://my-bucket/file-name-000000000000.json
gs://my-bucket/file-name-000000000001.json
gs://my-bucket/file-name-000000000002.json

nVector

posted on 15 May 20