From a Deep Learning Model to a Web Application
How to deploy your deep learning or machine learning service for thousands of people using service into the web using just Python frameworks
Overview
It's very common to see Artificial Intelligence-based web applications in a variety of contexts. However, how to do it simply is still a mystery for many. Therefore, this article is a tutorial on one of the simplest ways to develop a web application that delivers a Deep Learning service.
In order to deploy a Deep Learning model into the web, we're going to use Flask as a web developer framework in Python, ngrok to access the app running in the jupyter notebook for experimentation, REST API to communicate with the application, PyTorch to develop the Deep Learning model. Once the application is ready, you can choose a server to host, such as Heruko, Railway, Render, AWS. In this article, we're going to use Railway since it's simple and has a limited free tier.
Our task is to create a web application that ingests an image file from the user and runs a pre-trained DenseNet to classify the image. The application must ask for the image and print the predicted class. We also want to show the percentage of confidence.
Firstly, let's go through the frameworks to develop the app, then the model creation and prediction function, and finally the web deployment.
1. Frameworks to develop the app
1.1 Flask
As said in the introduction Flaks is a web development python framework, we're going to use it in order to implement the app. Flask is a micro web framework written in Python. Even though, the "micro" Flaks is incredibly powerful and a simple, fast, robust tool for small projects. It's very famous for many web applications, REST APIs, and socket-based services. There are other web frameworks such as Django, however, I think that Flask is the best choice for beginners.
1.2 Ngrok
Ngrok appears to solve a problem that emerges when the application runs in jupyter notebook. The local address for the application is the local address where the notebook is running. Therefore, ngrok turns the local address, where the application will run initially, into another URL.
To use ngrok, you will need to create a free account on ngrok.com and copy your authtoken. Then, it's just necessary to paste it into this cell at the beginning of the notebook.
!ngrok authtoken YOUR_NGROK_AUTHTOKENThen,
# Create a Flask app object
app = Flask(__name__)
# Define a function to be called when the user accesses the root URL (/)
@app.route("/")
def home():
# application HTML response
return "<h1>That's my web application :)</h1>"
# You need ngrok to expose the Flask app outside of the notebook
run_with_ngrok(app)
# running the app
app.run()(Code partially borrowed from Neuromatch lecture)
Which returns locally the application UI. This usage of Flask with Jupyter Notebook by ngrok is super helpful for experimenting with the application UI and functionalities.
1.3 Interface (Jinja2)
We still need to create a way for the user to interact with the application. We will define an HTML template to create the interaction. Flask uses Jinja2 framework to acess HTML code. To simplify the process here, let's use this open-source code from NeuroMatch lecture*. The template allows the user to select an image from the computer in order to the model to classify it.
index_template = """
<html>
<head>
<!-- Load vue.js and axois.js -->
<script src="https://cdn.jsdelivr.net/npm/vue/dist/vue.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/axios/0.21.1/axios.min.js"></script>
</head>
<body>
<!-- The APP UI -->
<div id="app" style="width: 50%; margin: 200px auto">
<form id="imageForm" enctype="multipart/form-data" method="POST" style="text-align: center; display: block">
<label for="imageFile">Select image to classify:</label
><input id="imageFile" name="file" type="file" style="margin-left: 10px" />
<img v-if="image" :src="image" style="width: 250px; display: block; margin: 50px auto 10px" />
<div v-if="prediction" style="font-size: 32px; font-weight: bold; text-align: center">
{{ prediction }}
</div>
<input v-if="image" type="submit" value="Classify Image" style="margin: 20px 20px" />
</form>
</div>
<script>
<!-- The Vue application -->
var app = new Vue({
el: "#app",
data() {
return {
image: null,
prediction: null,
};
},
});
<!-- Calling the predict API when the form is submitted -->
document.getElementById("imageForm").addEventListener("submit", (e) => {
axios
.post("/predict", new FormData(document.getElementById("imageForm")), {
headers: {
"Content-Type": "multipart/form-data",
},
})
.then((response) => (app.prediction = response.data));
e.preventDefault();
});
<!-- Display the selected image -->
document.getElementById("imageFile").addEventListener("change", (e) => {
const [file] = document.getElementById("imageFile").files;
if (file) {
app.image = URL.createObjectURL(file);
}
});
</script>
</body>
</html>
"""The application must show the image to the user and return the class and the confidence of the given class. To execute this HTML template, we just need to change the function as follows. (Code partially borrowed from Neuromatch lecture)
# Serve the template with the interactive UI
@app.route("/")
def home():
return index_template
2. Deep Learning model
2.1 PyTorch and DenseNet 121
In order to implement the data transformations and the Deep Learning model, we're going to use PyTorch. The idea of this tutorial is to be really straightforward, hence, let's use a pre-trained densenet121 from torchvision models with 1000 classes. This network is based on dense connections of convolutional layers, for more details read the awarded original article. (Code partially borrowed from Neuromatch lecture).
import torch
from torchvision import models
import torchvision.transforms as transforms
# Load a pre-trainied DenseNet model from torchvision.models
model = models.densenet121(pretrained=True)
# Switch the model to evaluation mode
model.eval()
# Load the class labels from a file
class_labels_url = "https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt"
class_labels = urlopen(class_labels_url).read().decode("utf-8").split("\n")
# Define the transformation of the input image
transform = transforms.Compose([
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
])
If you want to change to your own model, you can load the trained model simply by: (Code by the author)
# Define the model
model = Architecture()
# Load the saved parameters
model.load_state_dict(torch.load('.../model.pth'))
# Switch the model to evaluation mode
model.eval()1.2 Prediction function
Now, we need to define a function to create an output for the application that goes directly through the UI.
def predict(model, transform, image, class_labels):
# Transform the image and convert it to a tensor
image_tensor = transform(image).unsqueeze(0)
## Switch the model to evaluation mode Pass the image through the model
with torch.no_grad():
output = model(image_tensor)
# Select the class with the higherst probability and look up the name
m = torch.nn.Softmax(dim=1)
class_prob = round(float(m(output).max()),3)*100
class_id = torch.argmax(output).item()
class_name = class_labels[class_id]
# Return the class name
return str(class_name) + " - Confidence: " + str(class_prob) + '%'(Code partially borrowed from Neuromatch lecture)
This function transforms the input image, passes it to the model, extracts the highest probability with the respective class and outputs the class name and the confidence, which is the Softmax layer output for the given class.
Therefore, this function should work this way: (Code partially borrowed from Neuromatch course)
# Load and display the image
image = Image.open(io.BytesIO(urlopen("https://unsplash.com/photos/2l0CWTpcChI/download?force=true&w=480").read()))
display(image)
# Classify the image
display(predict(model, transform, image, class_labels))
3. Incorporating the model into the application
Now, we need to incorporate the app developed with the pre-trained model and the prediction function.
Flask makes this task really easy.
app = Flask(__name__)
# Serve the template with the interactive UI
@app.route("/")
def home():
return index_template
# Classification API
@app.route("/predict", methods=['POST'])
def predict_api():
# Fetch the image from the request and convert it to a Pillow image
image_file = request.files['file']
image_bytes = image_file.read()
image = Image.open(io.BytesIO(image_bytes))
# Predict the class from the image
class_name = predict(model, transform, image, class_labels)
# Return the result
return class_name
# Run the app
run_with_ngrok(app)
## Uncomment below to run the app
app.run()(Code partially borrowed from Neuromatch lecture).
Looking at the code, there is a core structure that make the connection between the application and the model. The core structure is the predict_api function, which relates the prediction function to the UI HTML template by
@app.route(“/predict”, methods=[‘POST’])
Thus, we already have an entire application to classify images from the user showing the name of the class and the model confidence.
6. Deploy application into Web
Now, our application is finished, but we also want to deploy it into the web. To deploy the Flask application to the web, you can choose any cloud Platform-as-a-Service (PaaS) that enables developers to build, run, and operate applications entirely in the cloud using pre-configured environments. My recommendation is Railway as the simplest to deploy and has a free tier.
Once we finish the experimental phase in the notebooks, we can copy the code to a main.py file. We also need to create a requirement.txt file, in order to install all the dependencies into the cloud service, and a Procfile that executes the main.py to initialize the application during deployment.
The Procfile must contain the following:
web: gunicorn app:appThe final structure should be in GitHub as follows: https://github.com/Rodrigo-Motta/deploy_dl.git
Railway
This part is the easiest. You just need to create a free account and connect with a GitHub repository and click deploy. Thus, you can access your application over HTTP with the domain shown in the network settings.
For this specific project: web-production-784db.up.railway.app
(This URL may not work because sometimes I decide to turn off the application in Railway for some reason)
Conclusion
In this tutorial, we developed a really simple application that ingests an image from the user, runs a classification deep learning model, and shows the user the predicted class and the confidence given by the Softmax layer. For this task, we used Flask as the micro web framework, ngrok to access the application over the experiments inside a Jupyter Notebook, PyTorch to create the deep learning model, and Railway to deploy the application into the web.
Acknowledgments
This article was inspired by this amazing lecture: https://deeplearning.neuromatch.io/tutorials/Bonus_DeployModels/student/Bonus_Tutorial1.html. *The content is under Creative Commons Attribution 4.0 International License, which allows the copying and redistribution of the material in any medium or format for any purpose, even commercially. In the condition of giving appropriate credit.
