Building an AI powered REST API with Gradio and Huggingface Spaces – for free! – Tom Söderlund


Building an AI powered REST API with Gradio and Huggingface Spaces – for free! – Tom Söderlund

I want to use machine learning models in my web and mobile apps, but to do so I must host my ML app somewhere.

Hosting a pre-trained ML model is called inference. I just want to drop in some Python ML code and quickly get a REST API, but finding a simple way to do this has proven more difficult than I expected.

“I just want to drop in some Python ML code and quickly get a REST API”

There are plenty of hosting providers, including the big ones like Amazon AWS and Google GCP, but the process of using these are complex and often requires building your own Flask REST API.

Luckily, @prundin and @abidlabs had a solution: Gradio and Huggingface Spaces.

Create a new Space on Huggingface

Huggingface is like GitHub but for ML models and apps. A “Space” on Huggingface is an ML app that you can update via Git. Spaces are priced based on CPU type, and the simplest one is free!

Gradio is a library of user interface components (e.g. input fields) for ML apps.

Create a new Space by:

Local development setup

Go to your development folder and clone your space:

git clone https://huggingface.co/spaces/USER_NAME/SPACE_NAME

cd SPACE_NAME

Set up Python, Gradio, etc:

# Create a “safe” virtual Python environment (exit with command “deactivate”)
python3 -m venv env
source env/bin/activate

Create a .gitignore file to exclude the packages in `env` folder
echo "env/" >> .gitignore

# Install Gradio
pip3 install gradio

# Update the list of required packages (do this every time you add packages)
pip3 freeze > requirements.txt

Settings in the README file

The README.md contains some crucial settings for your app.

I struggled with some errors in my live environment on Huggingface, until I realized I had to lock down the Python version to be the same as I use locally. By running:

python3 --version

locally, and then adding the version number to README.md like this:

python_version: 3.9.13

Creating a simple user interface and REST API

Create a blank app.py file:

touch app.py

Edit app.py:

import gradio

def my_inference_function(name):
  return "Hello " + name + "!"

gradio_interface = gradio.Interface(
  fn=my_inference_function,
  inputs="text",
  outputs="text",
  examples=[
    ["Jill"],
    ["Sam"]
  ]
)
gradio_interface.launch()

Upload the files to Huggingface using Git:

git add .
git commit -m "Creating app.py"
git push

Testing your app on Huggingface

Believe it or not, but you now have a functional app with a REST API!

Open https://huggingface.co/spaces/USER_NAME/SPACE_NAME in your browser.

User interface: You can enter something in the name box and press Submit.

REST API interface: At the bottom of the page you have a link called “Use via API”. Click it for instructions, but you can now call your app with REST:

curl -X POST -H 'Content-type: application/json' --data '{ "data": ["Jill"] }' https://USER_NAME-SPACE_NAME.hf.space/run/predict

Testing your app locally

Locally, you can run:

python3 app.py

You can now test your app on: http://127.0.0.1:7860/

(You need to stop (Ctrl+C) and restart your app when you modify app.py)

Going further

You can now explore all the models on Huggingface, including Stable Diffusion 2 and GPT-Neo, and add them to your Spaces app.

See a complete ML example on: https://huggingface.co/spaces/tomsoderlund/swedish-entity-recognition

A complete ML example with REST API




Source link