Comment on page
Deploy PyCaret Models on edge with ONNX Runtime
Photo by Austin Distel on Unsplash
In this tutorial, I will show you how you can train machine learning models using PyCaret — an open-source, low-code machine learning library in Python—and convert them in ONNX format for deployment on an edge device or any other non-Python environment. For example, you can train machine learning models using PyCaret in Python and deploy them in R, Java, or C. The Learning Goals of this tutorial are:
👉 What is PyCaret and how to get started?
👉 What are different types of model formats (pickle, onnx, pmml, etc.)
👉 What is ONNX (pronounced as ONEX) and what are its benefits?
👉 Train machine learning model using PyCaret and convert it in ONNX for deployment on edge.
PyCaret is an open-source, low-code machine learning library and end-to-end model management tool built-in Python for automating machine learning workflows. PyCaret is known for its ease of use, simplicity, and ability to quickly and efficiently build and deploy end-to-end machine learning pipelines. To learn more about PyCaret, check out their GitHub.
PyCaret — An open-source, low-code machine learning library in Python
skl2onnx is an open-source project that converts scikit-learn models to ONNX. Once in the ONNX format, you can use tools like ONNX Runtime for high-performance scoring. This project was started by the engineers and data scientists at Microsoft in 2017. To learn more about this project, check out their GitHub.
You will need to install the following libraries for this tutorial. The installation will take only a few minutes.
**# install pycaret
**pip install pycaret
**# install skl2onnx
**pip install skl2onnx
**# install onnxruntime
**pip install onnxruntime
Before I introduce ONNX and the benefits, let’s see what are the different model formats available today for deployment.
This is the most common format and default way of saving model objects into files for many Python libraries including PyCaret. Pickle converts a Python object to a bitstream and allows it to be stored to disk and reloaded at a later time. It provides a good format to store machine learning models provided that the inference applications are also built-in python.
Predictive model markup language (PMML) is another format for machine learning models, relatively less common than Pickle. PMML has been around since 1997 and so has a large footprint of applications leveraging the format. Applications such as SAP **and PEGA CRM are able to leverage certain versions of the PMML. There are open-source libraries available that can convert scikit-learn models (PyCaret) to PMML. The biggest drawback of the PMML format is that it doesn’t support all machine learning models.