Machine Learning in Alteryx with PyCaret
👉 What is PyCaret and how to get started?
👉 What is Alteryx Designer and how to set it up?
👉 Train end-to-end machine learning pipeline in Alteryx Designer including data preparation such as missing value imputation, one-hot-encoding, scaling, transformations, etc.
👉 Deploy trained pipeline and generate inference during ETL.
PyCaret is an open-source, low-code machine learning library and end-to-end model management tool built-in Python for automating machine learning workflows. PyCaret is known for its ease of use, simplicity, and ability to quickly and efficiently build and deploy end-to-end machine learning pipelines. To learn more about PyCaret, check out their GitHub.
Alteryx Designer is a proprietary tool developed by **Alteryx** and is used for automating every step of analytics, including data preparation, blending, reporting, predictive analytics, and data science. You can access any data source, file, application, or data type, and experience the simplicity and power of a self-service platform with 260+ drag-and-drop building blocks. You can download the one-month free trial version of Alteryx Designer from here.
https://www.alteryx.com
For this tutorial, you will need two things. The first one being the Alteryx Designer which is a desktop software that you can download from here. Second, you need Python. The easiest way to get Python is to download Anaconda Distribution. To download that, click here.
Open Alteryx Designer and click on File → New Workflow
New Workflow in Alteryx Designer
On the top, there are tools that you can drag and drop on the canvas and execute the workflow by connecting each component to one another.
For this tutorial, I am using a regression dataset from PyCaret’s repository called insurance. You can download the data from here.
Sample Dataset
I will create two separate Alteryx workflows. First one for model training and selection and the second one for scoring the new data using the trained pipeline.
Let’s first read the CSV file from the **Input Data **tool followed by a **Python Script. **Inside the Python script execute the following code:
**# install pycaret
**from ayx import Package
Package.installPackages('pycaret')
**# read data from input data tool**
from ayx import Alteryx
data = Alteryx.read("#1")
**# init setup, prepare data**
from pycaret.regression import *
s = setup(data, target = 'charges', silent=True)
**# model training and selection
**best = compare_models()
**# store the results, print and save**
results = pull()
results.to_csv('c:/users/moezs/pycaret-demo-alteryx/results.csv', index = False)
Alteryx.write(results, 1)
**# finalize best model and save**
best_final = finalize_model(best)
save_model(best_final, 'c:/users/moezs/pycaret-demo-alteryx/pipeline')
This script is importing the regression module from pycaret, then initializing the setup function which automatically handles train_test_split and all the data preparation tasks such as missing value imputation, scaling, feature engineering, etc. compare_models trains and evaluates all the estimators using kfold cross-validation and returns the best model.
pull function calls the model performance metric as a Dataframe which is then saved as results.csv on a drive and also written to the first anchor of Python tool in Alteryx (so that you can view results on screen).
Finally, save_model saves the entire transformation pipeline including the best model as a pickle file.
Training Workflow
When you successfully execute this workflow, you will generate pipeline.pkl and results.csv file. You can see the output of the best models and their cross-validated metrics on-screen as well.
This is what results.csv contains:
These are the cross-validated metrics for all the models. The best model, in this case, is Gradient Boosting Regressor.
We can now use our pipeline.pkl to score on the new dataset. Since I do not have a separate dataset for ***insurance.csv without the label, ***what I will do is drop the target column i.e. charges, and then generate predictions using the trained pipeline.
Scoring Workflow
I have used the **Select Tool **to remove the target column i.e. charges . In the Python script execute the following code:
**# read data from the input tool**
from ayx import Alteryx**
**data = Alteryx.read("#1")
**# load pipeline
**from pycaret.regression import load_model, predict_model
pipeline = load_model('c:/users/moezs/pycaret-demo-alteryx/pipeline')
**# generate predictions and save to csv
**predictions = predict_model(pipeline, data)
predictions.to_csv('c:/users/moezs/pycaret-demo-alteryx/predictions.csv', index=False)
**# display in alteryx
**Alteryx.write(predictions, 1)
When you successfully execute this workflow, it will generate predictions.csv.
predictions.csv
PyCaret — Image by Author
PyCaret — Image by Author
There is no limit to what you can achieve using this lightweight workflow automation library in Python. If you find this useful, please do not forget to give us ⭐️ on our GitHub repository.
Machine Learning in KNIME with PyCaret A step-by-step guide on training and deploying end-to-end machine learning pipelines in KNIME using PyCarettowardsdatascience.com Easy MLOps with PyCaret + MLflow A beginner-friendly, step-by-step tutorial on integrating MLOps in your Machine Learning experiments using PyCarettowardsdatascience.com Write and train your own custom machine learning models using PyCaret towardsdatascience.com Build with PyCaret, Deploy with FastAPI *A step-by-step, beginner-friendly tutorial on how to build an end-to-end Machine Learning Pipeline with PyCaret and…*towardsdatascience.com Time Series Anomaly Detection with PyCaret A step-by-step tutorial on unsupervised anomaly detection for time series data using PyCarettowardsdatascience.com Supercharge your Machine Learning Experiments with PyCaret and Gradio A step-by-step tutorial to develop and interact with machine learning pipelines rapidlytowardsdatascience.com Multiple Time Series Forecasting with PyCaret A step-by-step tutorial on forecasting multiple time series using PyCarettowardsdatascience.com