# Analyze
## plot\_model
This function analyzes the performance of a trained model on the hold-out set. It may require re-training the model in certain cases.
### **Example**
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
lr = create_model('lr')
# plot model
plot_model(lr, plot = 'auc')
```
{% endcode %}
### **Change the scale**
The resolution scale of the figure can be changed with `scale` parameter.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
lr = create_model('lr')
# plot model
plot_model(lr, plot = 'auc', scale = 3)
```
{% endcode %}
### Save the plot
You can save the plot as a `png` file using the `save` parameter.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
lr = create_model('lr')
# plot model
plot_model(lr, plot = 'auc', save = True)
```
{% endcode %}
### Customize the plot
PyCaret uses [Yellowbrick](https://www.scikit-yb.org/en/latest/) for most of the plotting work. Any argument that is acceptable for Yellowbrick visualizers can be passed as `plot_kwargs` parameter.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
lr = create_model('lr')
# plot model
plot_model(lr, plot = 'confusion_matrix', plot_kwargs = {'percent' : True})
```
{% endcode %}
{% tabs %}
{% tab title="Before Customization" %}
{% endtab %}
{% tab title="After Customization" %}
{% endtab %}
{% endtabs %}
### Use train data
If you want to assess the model plot on the train data, you can pass `use_train_data=True` in the `plot_model` function.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
lr = create_model('lr')
# plot model
plot_model(lr, plot = 'auc', use_train_data = True)
```
{% endcode %}
#### Plot on train data vs. hold-out data
{% tabs %}
{% tab title="Train Data" %}
{% endtab %}
{% tab title="Hold-out Data" %}
{% endtab %}
{% endtabs %}
### **Examples by module**
#### **Classification**
| **Plot Name** | **Plot** |
| --------------------------- | ------------------- |
| Area Under the Curve | ‘auc’ |
| Discrimination Threshold | ‘threshold’ |
| Precision Recall Curve | ‘pr’ |
| Confusion Matrix | ‘confusion\_matrix’ |
| Class Prediction Error | ‘error’ |
| Classification Report | ‘class\_report’ |
| Decision Boundary | ‘boundary’ |
| Recursive Feature Selection | ‘rfe’ |
| Learning Curve | ‘learning’ |
| Manifold Learning | ‘manifold’ |
| Calibration Curve | ‘calibration’ |
| Validation Curve | ‘vc’ |
| Dimension Learning | ‘dimension’ |
| Feature Importance (Top 10) | ‘feature’ |
| Feature IImportance (all) | 'feature\_all' |
| Model Hyperparameter | ‘parameter’ |
| Lift Curve | 'lift' |
| Gain Curve | 'gain' |
| KS Statistic Plot | 'ks' |
{% tabs %}
{% tab title="auc" %}
{% endtab %}
{% tab title="confusion\_matrix" %}
{% endtab %}
{% tab title="threshold" %}
{% endtab %}
{% tab title="pr" %}
{% endtab %}
{% tab title="error" %}
{% endtab %}
{% tab title="class\_report" %}
{% endtab %}
{% tab title="rfe" %}
{% endtab %}
{% tab title="learning" %}
{% endtab %}
{% tab title="vc" %}
{% endtab %}
{% endtabs %}
{% tabs %}
{% tab title="feature" %}
{% endtab %}
{% tab title="manifold" %}
{% endtab %}
{% tab title="calibration" %}
{% endtab %}
{% tab title="dimension" %}
{% endtab %}
{% tab title="boundary" %}
{% endtab %}
{% tab title="lift" %}
{% endtab %}
{% tab title="gain" %}
{% endtab %}
{% tab title="ks" %}
{% endtab %}
{% tab title="parameter" %}
{% endtab %}
{% endtabs %}
#### Regression
| **Name** | **Plot** |
| --------------------------- | -------------- |
| Residuals Plot | ‘residuals’ |
| Prediction Error Plot | ‘error’ |
| Cooks Distance Plot | ‘cooks’ |
| Recursive Feature Selection | ‘rfe’ |
| Learning Curve | ‘learning’ |
| Validation Curve | ‘vc’ |
| Manifold Learning | ‘manifold’ |
| Feature Importance (top 10) | ‘feature’ |
| Feature Importance (all) | 'feature\_all' |
| Model Hyperparameter | ‘parameter’ |
{% tabs %}
{% tab title="residuals" %}
{% endtab %}
{% tab title="error" %}
{% endtab %}
{% tab title="cooks" %}
{% endtab %}
{% tab title="rfe" %}
{% endtab %}
{% tab title="feature" %}
{% endtab %}
{% tab title="learning" %}
{% endtab %}
{% tab title="vc" %}
{% endtab %}
{% tab title="manifold" %}
{% endtab %}
{% endtabs %}
#### Clustering
| **Name** | **Plot** |
| --------------------- | -------------- |
| Cluster PCA Plot (2d) | ‘cluster’ |
| Cluster TSnE (3d) | ‘tsne’ |
| Elbow Plot | ‘elbow’ |
| Silhouette Plot | ‘silhouette’ |
| Distance Plot | ‘distance’ |
| Distribution Plot | ‘distribution’ |
{% tabs %}
{% tab title="cluster" %}
{% endtab %}
{% tab title="tsne" %}
{% endtab %}
{% tab title="elbow" %}
{% endtab %}
{% tab title="silhouette" %}
{% endtab %}
{% tab title="distance" %}
{% endtab %}
{% tab title="distribution" %}
{% endtab %}
{% endtabs %}
#### Anomaly Detection
| **Name** | **Plot** |
| ------------------------- | -------- |
| t-SNE (3d) Dimension Plot | ‘tsne’ |
| UMAP Dimensionality Plot | ‘umap’ |
{% tabs %}
{% tab title="tsne" %}
{% endtab %}
{% tab title="umap" %}
{% endtab %}
{% endtabs %}
## evaluate\_model
The `evaluate_model` displays a user interface for analyzing the performance of a trained model. It calls the [plot\_model](#plot_model) function internally.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
juice = get_data('juice')
# init setup
from pycaret.classification import *
exp_name = setup(data = juice, target = 'Purchase')
# create model
lr = create_model('lr')
# launch evaluate widget
evaluate_model(lr)
```
{% endcode %}
{% hint style="info" %}
**NOTE:** This function only works in Jupyter Notebook or an equivalent environment.
{% endhint %}
## interpret\_model
This function analyzes the predictions generated from a trained model. Most plots in this function are implemented based on the SHAP (Shapley Additive exPlanations). For more info on this, please see
### Example
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost)
```
{% endcode %}
### Save the plot
You can save the plot as a `png` file using the `save` parameter.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, save = True)
```
{% endcode %}
{% hint style="info" %}
**NOTE:** When `save=True` no plot is displayed in the Notebook.
{% endhint %}
### Change plot type
There are a few different plot types available that can be changed by the `plot` parameter.
#### Correlation
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, plot = 'correlation')
```
{% endcode %}
By default, PyCaret uses the first feature in the dataset but that can be changed using `feature` parameter.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, plot = 'correlation', feature = 'Age (years)')
```
{% endcode %}
#### Partial Dependence Plot
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, plot = 'pdp')
```
{% endcode %}
By default, PyCaret uses the first available feature in the dataset but this can be changed using the `feature` parameter.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, plot = 'pdp', feature = 'Age (years)')
```
{% endcode %}
#### Morris Sensitivity Analysis
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, plot = 'msa')
```
{% endcode %}
#### Permutation Feature Importance
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, plot = 'pfi')
```
{% endcode %}
#### Reason Plot
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, plot = 'reason')
```
{% endcode %}
When you generate `reason` plot without passing the specific index of test data, you will get the interactive plot displayed with the ability to select the x and y-axis. This will only be possible if you are using Jupyter Notebook or an equivalent environment. If you want to see this plot for a specific observation, you will have to pass the index in the `observation` parameter.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, plot = 'reason', observation = 1)
```
{% endcode %}
Here the `observation = 1` means index 1 from the test set.
### Use train data
By default, all the plots are generated on the test dataset. If you want to generate plots using a train data set (not recommended) you can use `use_train_data` parameter.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# creating a model
xgboost = create_model('xgboost')
# interpret model
interpret_model(xgboost, use_train_data = True)
```
{% endcode %}
## dashboard
The `dashboard` function generates the interactive dashboard for a trained model. The dashboard is implemented using ExplainerDashboard ([explainerdashboard.readthedocs.io](https://explainerdashboard.readthedocs.io))
#### Dashboard Example
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
juice = get_data('juice')
# init setup
from pycaret.classification import *
exp_name = setup(data = juice, target = 'Purchase')
# train model
lr = create_model('lr')
# launch dashboard
dashboard(lr)
```
{% endcode %}
#### Video:
{% embed url="" %}
## check\_fairness
There are many approaches to conceptualizing fairness. The `check_fairness` function follows the approach known as group fairness, which asks: which groups of individuals are at risk for experiencing harm. `check_fairness` provides fairness-related metrics between different groups (also called sub-population).
#### Check Fairness Example
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
income = get_data('income')
# init setup
from pycaret.classification import *
exp_name = setup(data = income, target = 'income >50K')
# train model
lr = create_model('lr')
# check model fairness
lr_fairness = check_fairness(lr, sensitive_features = ['sex', 'race'])
```
{% endcode %}
#### Video:
{% embed url="" %}
## get\_leaderboard
This function returns the leaderboard of all models trained in the current setup.
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
diabetes = get_data('diabetes')
# init setup
from pycaret.classification import *
clf1 = setup(data = diabetes, target = 'Class variable')
# compare models
top3 = compare_models(n_select = 3)
# tune top 3 models
tuned_top3 = [tune_model(i) for i in top3]
# ensemble top 3 tuned models
ensembled_top3 = [ensemble_model(i) for i in tuned_top3]
# blender
blender = blend_models(tuned_top3)
# stacker
stacker = stack_models(tuned_top3)
# check leaderboard
get_leaderboard()
```
{% endcode %}
You can also access the trained Pipeline with this.
{% code lineNumbers="true" %}
```python
# check leaderboard
lb = get_leaderboard()
# select top model
lb.iloc[0]['Model']
```
{% endcode %}
![Output from lb.iloc\[0\]\['Model'\]](https://3811348345-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FjAq5m5T7Qtz03TnB0Wve%2Fuploads%2FUhx4lsLDXcS5wl30xS8Y%2Fimage.png?alt=media\&token=e6990e29-e67d-43d0-ab3b-256d997d11dc)
## assign\_model
This function assigns labels to the training dataset using the trained model. It is available for [Clustering](https://pycaret.gitbook.io/docs/get-started/modules), [Anomaly Detection](https://pycaret.gitbook.io/docs/get-started/modules), and [NLP](https://pycaret.gitbook.io/docs/get-started/modules) modules.
#### Clustering
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
jewellery = get_data('jewellery')
# init setup
from pycaret.clustering import *
clu1 = setup(data = jewellery)
# train a model
kmeans = create_model('kmeans')
# assign model
assign_model(kmeans)
```
{% endcode %}
#### Anomaly Detection
{% code lineNumbers="true" %}
```python
# load dataset
from pycaret.datasets import get_data
anomaly = get_data('anomaly')
# init setup
from pycaret.anomaly import *
ano1 = setup(data = anomaly)
# train a model
iforest = create_model('iforest')
# assign model
assign_model(iforest)
```
{% endcode %}
