01 Feb 21 · dan-irving · #bigquery ·   Bookmark   ×

BigQuery Auto Detect Schema tables - load job fails with "Could not parse 'Text' as bool"

Bigquery autodetect incorrectly assigns bool datatype to a string column and therefore the bq load job fails

Error while reading data, error message: Could not parse 'good' as bool for field order_Flag (position 26) starting at location 1689438

Solution:

When auto-detection is enabled, BigQuery only scans up to 100 rows of data in your file to use as a representative sample. BigQuery then examines each field and attempts to assign a data type to that field based on the values in the sample

To overcome this, you can pre create your tables and not use auto-detect for such tables

dan-irving

posted on 01 Feb 21

Enjoy great content like this and a lot more !

Signup for a free account to write a post / comment / upvote posts. Its simple and takes less than 5 seconds