Easiest Way to Deploy Fast.ai models for Production with Google Cloud Functions cover image

Easiest Way to Deploy Fast.ai models for Production with Google Cloud Functions

Jian Jye • August 4, 2019

fastai google-cloud

If you are going through the Fast.ai course, one of the things you might be looking for would be an easy yet reliable option to deploy your models into production.

Google Cloud Functions together with its free tier is definitely one of the best options out there to get started. It being serverless also means that you don't get charged for anything if you are not using the services! That's perfect for a beginner's use case.

So how can we deploy our deep learning models to Google Cloud Functions?

Let's get started with this 7 simple steps.

Before that...

Source codes and sample model are available on Github.

This tutorial is based on Fast.ai - 2019 Lesson 2 Download notebook but its source codes are intentionally kept generic enough for most image classifications models.


1. Export the Trained Model

At the end of Lesson 2 - Download notebook, there is a section that teaches you to export your model via learn.export(). This command will generate a export.pkl model file in you respective folder.

Here we'll create a model.pkl model file by using learn.export('model.pkl').

If you could not get the model file, I have uploaded a sample model.pkl on Github.

Fast.ai - Lesson 2


2. Upload model.pkl Online

Since GCF does not come with a persistent storage, we will have to put the model somewhere else and fetch it as needed. We have 2 choices:

  1. Store it on a Google Cloud Storage bucket, then fetch the file privately
  2. Upload it to somewhere with a publicly downloadable link

For simplicity, we'll continue with option #2 though for best practises, you would want to go with option #1.

Note that if you put it on Cloud Storage, you can set the permission to allow public access for the particular object and generate a publicly accessible link for this tutorial.


3. Create Our Cloud Functions

Since Lesson 2 was about bears, let's create a Cloud Function called bears. We are going to need 512MB for memory as a large part of it would be used to load our large model file model.pkl.

Trigger would be for HTTP. We want to create an endpoint that in the future our web apps could interface with.

Finally, remember to choose Python 3.7 as the runtime.

Settings for our new Cloud Function


4. Setting Up the requirements.txt

Under Runtime, you will see 2 tabs for main.py and requirements.txt. Let's start with requirements.txt. This is where we specify the libraries we need for our Cloud Function.

We only needed 2 i.e. fastai and pytorch.

For pytorch we will be pulling the libraries via direct links. Since GCF does not come with GPU, we need the custom pytorch versions that do not come with CUDA or GCF will throw errors.

fastai>=1.0
# PyTorch 1.1.0 without CUDA
https://download.pytorch.org/whl/cpu/torch-1.1.0-cp37-cp37m-linux_x86_64.whl
https://download.pytorch.org/whl/cpu/torchvision-0.3.0-cp37-cp37m-linux_x86_64.whl

requirements.txt


5. Filling Up the main.py

There are only 24 lines of codes! Copy and paste the following codes into the main.py tab.

import json
import urllib.request
from fastai.vision import *

def handler(request):
    defaults.device = torch.device('cpu')
    // Here we download the model from the URL given in the model GET parameter and store it into /tmp folder
    model = request.args['model']
    urllib.request.urlretrieve(model, '/tmp/model.pkl')
    path = Path('/tmp')
    learner = load_learner(path, 'model.pkl')

    // Here we download the image from the URL given in the image GET parameter and store it into /tmp folder
    image = request.args['image']
    urllib.request.urlretrieve(image, '/tmp/image.jpg')

    // Load up the image and try to classify it
    img = open_image('/tmp/image.jpg')
    pred_class,pred_idx,outputs = learner.predict(img)

    // Sort the classification classes from highest score to lowest, then return the response as JSON
    return json.dumps({
        "predictions": sorted(
            zip(learner.data.classes, map(float, outputs)),
            key=lambda p: p[1],
            reverse=True
        )
    })

Then, for the Function to execute field, enter handler. This is the name of our function above.

main.py

We are going to leave the other stuff like Environment variables etc as default for now.


6. Deployment

Now click deploy and wait patiently has Google builds our Cloud Function, it may take a while.

While waiting, let's grab the HTTP URL for testing later. Click into our Cloud Function bears again, select the Trigger tab, and copy the URL.

Trigger URL


7. Prediction!

When you see the green tick beside the function name bears appears, it's now ready!

Give it a try at: https://<region-name>-<project-name>.cloudfunctions.net/bears?image=<image_url>&model=<model_url>

Hopefully everything works OK for you and you would see the right prediction scores as followed:

{"predictions": [["teddys", 0.9998297691345215], ["grizzly", 0.0001574010675540194], ["black", 1.285488178837113e-05]]}

In our case, we see that there is a 99% chance that our picture is a teddy bear!


Closing

That's all!

Now you can build a web app on top of this JSON output and extend its functions anyway you like!

References:

  1. Fast.ai Lesson 2 Download Notebook
  2. Deploying Pytorch Models to Serverless Environments
  3. How to serve deep learning models using TensorFlow 2.0 with Cloud Functions

Sign up for our newsletter