How to schedule a shell script using Google Cloud Shell in BigQuery?

Google cloud shell is friendly tiny VM that you can use to run tasks. However, If you want to schedule a script you will need to just create a separate compute instance for scheduling the script.

First, Create a new compute instance:

• On the menu go to the Compute section and hover on Compute Engine and then click on VM Instances.
• Go to the menu bar above the instance section and there you will see a Create Instance button. Click it and fill in the configuration values that you want your new instance to have. The values that you select will determine your VM instance features. You can choose, among other values, the name, zone and machine type for your new instance.
• In the Machine type section click the drop-down menu tab to select an f1-micro instance
• In the Identity and API access section, give access scope to the Storage API so that you can read and write to your bucket in case you need to do so; the default access scope only allows you to read. Also, enable BigQuery API.

SSH into your new VM and create a CRON Job:

Once you have the instance created and access to the bucket, just create your cron job inside your new instance: In the user account under which the cron job will execute, run crontab -e and edit this file to run the cron job that will execute your some-task.sh script.

16 17 * * * gsutil cat gs://miptestauto/baby.sh | sh

crontab -l

Mike-Barn

posted on 23 Aug 18