CatBoost (Classification): Complete Guide — Principles, Examples, and Python Implementation

Summary

CatBoost (Categorical Boosting) is a gradient boosting algorithm developed by Yandex, designed from the ground up to natively handle categorical variables without requiring expensive manual preprocessing. Unlike XGBoost or LightGBM, which require prior encoding of categories, CatBoost incorporates an ordered target encoding mechanism that encodes each category based on the distribution of the target variable observed in the rows preceding it according to a random permutation. This approach eliminates the problem of target leakage (leakage of target information into the features) while preserving the predictive information of categorical variables. CatBoost excels particularly in classification on datasets rich in qualitative variables: customer data, financial transactions, application logs, or demographic data.

Mathematical Principle: Ordered Target Encoding

The heart of CatBoost lies in its method of encoding categorical variables, called ordered target encoding. To understand this mechanism, let’s examine it step by step.

Given a dataset containing n observations $(x_1, y_1), (x_2, y_2), \ldots, (x_n, y_n)$, where $x_i$ includes a categorical variable $C_i \in {c_1, c_2, \ldots, c_k}$ and $y_i \in {0, 1}$ is the binary label.

Step 1: Random Permutation

CatBoost generates several random permutations $\sigma_1, \sigma_2, \ldots, \sigma_m$ of the set of indices ${1, 2, \ldots, n}$. Each permutation defines an order in which the data is processed:

$\sigma_j : {1, 2, \ldots, n} \rightarrow {1, 2, \ldots, n}$

Step 2: Ordered Encoding

For each permutation $\sigma_j$ and for each row at position $p$ in that permutation, the categorical value $C_{\sigma_j(p)}$ is replaced by a statistic computed only on the preceding rows ${\sigma_j(1), \sigma_j(2), \ldots, \sigma_j(p-1)}$:

$$\hat{x}{\sigma_j(p)} = \frac{\sum$$}^{p-1} \mathbb{1}(C_{\sigma_j(q)} = C_{\sigma_j(p)}) \cdot y_{\sigma_j(q)} + a}{\sum_{q=1}^{p-1} \mathbb{1}(C_{\sigma_j(q)} = C_{\sigma_j(p)}) + a

where $\mathbb{1}(\cdot)$ is the indicator function, $a$ is a smoothing hyperparameter (prior count), and the denominator is the number of times that category appeared before position $p$.

Step 3: Combining Permutations

Encoded values are computed for multiple permutations and combined to form the final value used by the tree. This approach ensures that a row’s target is never used to encode its own features, thereby eliminating target leakage.

Why This Matters

Classic target encoding (mean encoding) computes $\hat{x}_i = E[y | C = c]$ on the entire training set, which creates information leakage: the same row used to compute the statistic is then predicted by the model. The result is massive overfitting on rare categories. CatBoost’s ordered encoding solves this problem elegantly.

Intuition: Why CatBoost Is a Game Changer

Imagine a customer dataset with a “city” column containing 500 distinct values. With classic one-hot encoding, you create 500 binary columns. The model must learn 500 separate weights, which is inefficient and harms generalization, especially for underrepresented cities.

CatBoost approaches the problem differently: instead of creating columns, it encodes each category with a numeric value that represents the target variable’s tendency for that category. If Paris has a conversion rate of 15% and Marseille 8%, this information is captured directly in the encoded feature. But instead of computing these rates on the entire dataset (which would cause leakage), CatBoost computes them on “previously seen data” according to a random permutation.

This intuition is simple but powerful: the category carries information about the target, and we can extract it without cheating. CatBoost combines this idea with the gradient boosting architecture (sequential addition of weak trees) to produce a model that is both precise and robust.

Python Implementation

Installation

pip install catboost

Binary Classification with Categorical Features

import numpy as np
import pandas as pd
from catboost import CatBoostClassifier, Pool
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report, roc_auc_score

# Synthetic data with categorical variables
np.random.seed(42)
n_samples = 5000

data = pd.DataFrame({
    'ville': np.random.choice(['Paris', 'Lyon', 'Marseille', 'Bordeaux', 'Lille'], n_samples),
    'secteur': np.random.choice(['tech', 'finance', 'santé', 'commerce', 'industrie'], n_samples),
    'abonnement': np.random.choice(['basique', 'standard', 'premium'], n_samples),
    'age': np.random.randint(18, 70, n_samples),
    'revenu': np.random.normal(35000, 12000, n_samples).clip(15000, 80000),
    'nb_visites': np.random.poisson(8, n_samples),
})

# Target correlated with features (15% positive)
proba = (
    0.1
    + 0.08 * (data['ville'] == 'Paris').astype(float)
    + 0.06 * (data['abonnement'] == 'premium').astype(float)
    + 0.04 * (data['secteur'] == 'tech').astype(float)
    + 0.002 * (data['revenu'] - 35000) / 10000
    + 0.01 * (data['nb_visites'] - 8) / 5
)
proba = np.clip(proba, 0.02, 0.95)
y = np.random.binomial(1, proba)

# Train/test split
X_train, X_test, y_train, y_test = train_test_split(
    data, y, test_size=0.2, random_state=42, stratify=y
)

# Identify categorical features
cat_cols = ['ville', 'secteur', 'abonnement']

# Create the CatBoost model
model = CatBoostClassifier(
    iterations=500,
    learning_rate=0.05,
    depth=6,
    l2_leaf_reg=3.0,
    loss_function='Logloss',
    eval_metric='AUC',
    cat_features=cat_cols,  # Categorical columns directly (no encoding needed)
    random_seed=42,
    verbose=100,
)

# Training with Pool
train_pool = Pool(X_train, label=y_train, cat_features=cat_cols)
val_pool = Pool(X_test, label=y_test, cat_features=cat_cols)

model.fit(
    train_pool,
    eval_set=val_pool,
    early_stopping_rounds=50,
)

# Predictions
preds = model.predict(X_test)
pred_proba = model.predict_proba(X_test)[:, 1]

print(f"ROC AUC : {roc_auc_score(y_test, pred_proba):.4f}")
print(classification_report(y_test, preds))

Comparison with XGBoost

XGBoost does not natively handle string-type categorical variables. You must encode them manually:

import xgboost as xgb
from sklearn.preprocessing import LabelEncoder

# Manual encoding required with XGBoost
le_ville = LabelEncoder()
le_secteur = LabelEncoder()
le_abonnement = LabelEncoder()

X_train_xgb = X_train.copy()
X_train_xgb['ville'] = le_ville.fit_transform(X_train_xgb['ville'])
X_train_xgb['secteur'] = le_secteur.fit_transform(X_train_xgb['secteur'])
X_train_xgb['abonnement'] = le_abonnement.fit_transform(X_train_xgb['abonnement'])

X_test_xgb = X_test.copy()
X_test_xgb['ville'] = le_ville.transform(X_test_xgb['ville'])
X_test_xgb['secteur'] = le_secteur.transform(X_test_xgb['secteur'])
X_test_xgb['abonnement'] = le_abonnement.transform(X_test_xgb['abonnement'])

xgb_model = xgb.XGBClassifier(
    n_estimators=500,
    learning_rate=0.05,
    max_depth=6,
    reg_lambda=3.0,
    eval_metric='logloss',
    random_state=42,
)
xgb_model.fit(X_train_xgb, y_train)

The difference is stark: CatBoost accepts categorical columns as-is, while XGBoost requires prior encoding that can be a source of errors and data leakage if not applied correctly.

Key Hyperparameters

Tuning Tip

Always start by increasing iterations with a moderate learning_rate (0.05) and enable early stopping. This is the most reliable way to achieve good performance without excessive fine-tuning. Then adjust depth and l2_leaf_reg to control complexity.

model = CatBoostClassifier(
    iterations=1500,
    learning_rate=0.03,
    depth=6,
    l2_leaf_reg=10,
    subsample=0.8,
    cat_features=cat_cols,
    loss_function='Logloss',
    eval_metric='AUC',
    random_seed=42,
)

Advantages and Limitations

Advantages

1. Native handling of categorical variables — No manual encoding. String-type columns are accepted directly, which simplifies pipelines and reduces processing errors.
2. Ordered target encoding — Avoids target leakage thanks to random permutations, providing reliable encoding even for rare categories.
3. State-of-the-art performance — CatBoost consistently ranks among the top algorithms in Kaggle competitions, often neck-and-neck with XGBoost and LightGBM.
4. Robustness against overfitting — The ordering mechanism and built-in regularizations allow good results with little tuning.
5. Interpretability — Provides feature importance, tree plots, and built-in visualization tools.
6. GPU support — Training can be significantly accelerated on GPU (task_type='GPU').

Limitations

1. Training speed — CatBoost is generally slower than LightGBM, especially on very large datasets (> 1 million rows). The ordered encoding adds a computational overhead.
2. Memory consumption — Computing ordered statistics and managing permutations consumes more RAM than competing approaches.
3. Less flexibility — Some advanced features present in XGBoost (complex custom objective functions) are not supported.
4. Smaller ecosystem — The community and community resources are less developed than those for XGBoost.

4 Real-World Use Cases

1. Customer Attrition Prediction (Churn)

A telecom operator wants to identify customers likely to cancel their subscription. The data includes many categorical variables: plan type, payment method, region, customer segment. CatBoost handles these categories natively and produces a risk score for each customer, enabling targeted retention actions.

2. Financial Fraud Detection

Financial transactions contain variables such as merchant type, country of origin, payment channel, and card type. These categorical features are often highly discriminating for fraud detection. CatBoost’s ordered target encoding effectively captures these signals without overfitting, even for rare combinations of categories.

3. Support Ticket Classification

A customer service team receives tickets with categorical fields: product category, priority level, contact channel, department concerned. CatBoost can automatically classify these tickets by type of resolution needed, accelerating routing to the competent teams.

4. Insurance Risk Scoring

Insurance companies use hundreds of categorical variables to assess risk: vehicle type, geographic zone, occupation, type of housing. CatBoost excels here because it effectively combines these categorical signals with continuous variables (age, claims history) in a single, interpretable model.

See Also

• Bellman-Ford Algorithm: Finding Shortest Paths with Negative Weights in Python
• Mastering Power Totient Sequences with Python: Complete Guide and Tips