This article discusses an overview on how to build a machine learning model in a serverless manner with GCP. The brief explanation about machine learning concepts and how to implement it using BigQuery Machine Learning or TensorFlow and Keras are also will be covered here.

These following topics are covered in this article:

  1. Understand The Main Terms
  2. Build The Machine Learning Project
  3. Build a ML Model with BigQuery Machine Learning (BQML)
  4. Build a ML Model with TensorFlow and Keras

1. Understand The Main Terms

ML01

AI, ML and Deep Learning

2. Build a Machine Learning Project

ML02

Google Cloud Platform Dashboard

In this tutorial we aim to build a ML model using NYC taxicab dataset. Project in GCP is needed to build a ML model. If you don’t have one, you can sign up for free here.

 

First Step Launch AI Platform

Navigate to AI Platform on the side menu bar and select the Notebooks. If you familiar with Jupyter Notebook or have been using Google Colab, this Notebook is using exactly the same concepts.

ML03

Menu Selection for Notebooks

Click New Instance and select TensorFlow 2.x without GPU. Wait for a minute then click on open Jupyterlab to open Notebook environment.

ML04

How to launch the Notebook instances

Second Step : Cloning from Github

ML05

JupyterLab Environment— Terminal option on the bottom

You can start to write your own code or cloning a project from Github. For cloning a repository from Github you can use Terminal and type following command

git clone \ REPO_PATH

For this project you can clone from Google training code here.

There are two ways discussed on this tutorial to build ML model, by using BQML or using TensorFlow and Keras.

 

3. Build a ML model BigQuery Machine Learning (BQML)

In this project BMQL can be used for two things. First, to use BMQL to explore dataset, create ML datasets, create benchmark. Second, to use BigQuery ML to create first ML models.

ML06

Inside folder training-data-analyst

First Step Deal with the dataset using BQML

To deal with dataset In AI Platform, navigate to
training-data-analyst/quests/serverlessml/01_explore/solution
and open explore_data.ipynb.

Code for preparing the dataset can be found inside the explore_data.ipynb. Clear the output by clicking the clear button on Toolbar. Change the region, project, and bucket setting in the first cell based on your project. By clicking the Run button you will be able to see how to:

ML07

The data visualisation inside explore_data.ipynb

Second Step Build a ML model using BQML

BigQuery ML provides a fast way to build ML models on large structured and semi-structured datasets. To build our first models for taxifare prediction, navigate to
training-data-analyst/quests/serverlessml/02_bqml/solution
and open first_model.ipynb.

Clear the output by clicking the clear button on Toolbar. Change the region, project, and bucket setting in the first cell based on your project. By clicking the Run button you will be able to see how to:

ML08

RMSE resulted from DNN with BQML

Root Mean Square Error (RMSE) indicates the accuracy of a model. The lowest the RMSE the better the model performances.

4. Build a ML model with TensorFlow and Keras

First Step Learn how to read large datasets using TensorFlow

First we need to build data pipe line as the input of our Keras model and then construct the model. To build the data pipeline, navigate to
training-data-analyst/quests/serverlessml/03_tfdata/solution
and open input_pipeline.ipynb.

Clear the output by clicking the clear button on Toolbar. Change the region, project, and bucket setting in the first cell based on your project. By clicking the Run button you will be able to see how to:

 

Second Step Create Keras DNN and wide-and-deep model

The next step is build the DNN model using Keras to predict the fare amount for NYC taxi cab rides. Navigate to
training-data-analyst/quests/serverlessml/04_keras/solution
and open keras_dnn.ipynb.

Clear the output by clicking the clear button on Toolbar. Change the region, project, and bucket setting in the first cell based on your project. By clicking the Run button you will be able to see how to:

ML09

Visualisation of DNN Model in Keras

ML10

The Architecture of DNN we build with 465 trainable parameter

In this model, there are 4 layers of Neural Network. The first layer is input network with 5 feature nodes. Second layer has 32 hidden nodes with 192 parameter to train. Parameter means weights and biases. In the Third layer it has 8 hidden nodes with 264 connection. It connected to output layer with single output nodes and 9 parameters to train.

ML11

Loss and RMSE of the model

ML12

The training parameter

The number of iteration or epoch is only set into 5, increase the number of iteration is recommended. The other thing can be done to improve the model is through feature Engineering. After a training, the model can be deploy using gcloud ai-platform command which will take 5–10 minutes.

ML13

Code to deploy the ML model

Prediction can be done using gcloud ai-platform predict command. Before doing the prediction, the input should be written as a json file that consist of 5 input features as shown in figure Visualisation of DNN Model in Keras. The output shown that the fare will be $11.43.

ML14

#Serverless #AppliedMachineLearning #GCP #BigQueryML #DNN #TensorFlow #Keras

We also publish articles on Medium. Take a look and follow us to see when we publish new articles.

A note to our visitors

This website has updated its privacy policy in compliance with changes to European Union data protection law, for all members globally. We’ve also updated our Privacy Policy to give you more information about your rights and responsibilities with respect to your privacy and personal information. Please read this to review the updates about which cookies we use and what information we collect on our site. By continuing to use this site, you are agreeing to our updated privacy policy.