Google Colab: Keep training data after a timeout

Google Colab is a great way to get started with Deep Learning. Using Colab anyone can learn to code using TensorFlow and Keras without having to invest in hardware. One way to circumvent losing data after the 12-hour timeout is to use rclone to store the model weight data externally, then later load it and continue training in a new session.

• First, we’ll have to install & configure rclone locally. A guide on how to do this for google cloud can be found on the official Rclone Website: https://rclone.org/drive/ — After we followed the guide, we’ll find the config file within the home directory: ~/.config/rclone/rclone.conf
• Within Colab we can now install and configure rclone using the config file created from step one, by adding the following Code Cell:

!curl https://rclone.org/install.sh | sudo bash
!mkdir /root/.config/rclone/
config = """[google]
type = drive
client_id = your-app-id
client_secret = your-app-secret
scope = drivetoken = {"access_token":"your-access-token","token_type":"Bearer","refresh_token":"your-refresh-token","expiry":"your-expiry"}
"""
with open('/root/.config/rclone/rclone.conf', 'w') as file:           file.write(config)

Now data can be transferred from and to the cloud using rclone command:

!rclone copy google:/model-content /content/

GSuite offers tons of storage to store datasets and machine learning model weights.