Feature Engineering for Machine Learning: Encoding Categorical Variables in Python

Most machine learning algorithms speak only in numbers. They expect numerical input and produce numerical output. But real-world data is full of categories — country names, product types, customer segments, color labels, and job titles. Turning these categories into numbers without losing their meaning is one of the most important steps in feature engineering.

Encoding categorical variables well can make the difference between a model that barely outperforms a coin flip and one that delivers real business value. This post covers the most common encoding strategies in Python, when to use each one, and the tradeoffs involved.

Understanding Categorical Data

Categorical data comes in two flavors:

• Nominal: Categories with no inherent order — colors, cities, product categories. Red is not “greater than” blue.
• Ordinal: Categories with a natural ranking — education level (high school < bachelor’s < master’s < PhD), customer satisfaction (poor < fair < good < excellent).

The encoding approach differs for each. Confusing them is a common source of leakage and poor performance.

Libraries and Setup

We’ll use pandas for data handling and scikit-learn for encoding transformers. All examples assume these imports:

import pandas as pd
import numpy as np
from sklearn.preprocessing import LabelEncoder, OneHotEncoder, OrdinalEncoder
from sklearn.compose import ColumnTransformer
from sklearn.model_selection import cross_val_score

Ordinal Encoding — For Ordinal Categories

When your categories have a natural order, ordinal encoding maps them to integers that preserve that ranking.

# Sample data
df = pd.DataFrame({
    "education": ["high_school", "bachelor", "master", "phd"],
    "score": [78, 85, 92, 88],
})

# Define the order explicitly — this is crucial
education_order = ["high_school", "bachelor", "master", "phd"]
encoder = OrdinalEncoder(categories=[education_order])

df["education_encoded"] = encoder.fit_transform(df[["education"]])
print(df)

Output:

    education  score  education_encoded
0  high_school     78                0.0
1     bachelor     85                1.0
2       master     92                2.0
3          phd     88                3.0

When to use: Education level, customer tier, survey responses (Likert scales), or any variable where the distance between levels carries meaning.

Pitfall: Passing the wrong order silently produces a model that learns incorrect relationships. Always define and verify the category order against domain knowledge.

One-Hot Encoding — For Nominal Categories

For nominal categories without order, one-hot encoding creates a binary column for each category value. A row gets 1 in the column matching its category, 0 in all others.

df = pd.DataFrame({
    "city": ["jakarta", "surabaya", "bandung", "jakarta", "yogyakarta"],
    "price": [250, 180, 200, 275, 160],
})

encoder = OneHotEncoder(sparse_output=False, handle_unknown="ignore")
encoded = encoder.fit_transform(df[["city"]])

# Build a readable DataFrame
city_cols = encoder.get_feature_names_out(["city"])
df_encoded = pd.concat([
    df.drop(columns=["city"]),
    pd.DataFrame(encoded, columns=city_cols, dtype=int),
], axis=1)

print(df_encoded)

Output:

   price  city_bandung  city_jakarta  city_surabaya  city_yogyakarta
0    250             0             1              0                0
1    180             0             0              1                0
2    200             1             0              0                0
3    275             0             1              0                0
4    160             0             0              0                1

When to use: City names, product categories, payment methods — any nominal variable with a reasonable number of unique values.

Watch out for the curse of dimensionality: A column with 50 unique countries creates 50 new features. If your dataset is small, this can hurt performance through overfitting. For high-cardinality features (1000+ unique values), consider alternatives like target encoding or feature hashing.

Dummy Encoding (Drop First)

A common variant is dummy encoding, which drops the first category to avoid perfect multicollinearity with linear models:

pd.get_dummies(df["city"], drop_first=True)

This creates k-1 columns instead of k. Whether you need this depends on your model — tree-based models don’t care about multicollinearity, but linear models and neural networks do.

Label Encoding — For Target Variables Only

LabelEncoder assigns a unique integer to each category. Simple, but dangerous for features:

encoder = LabelEncoder()
df["city_encoded"] = encoder.fit_transform(df["city"])

This produces the same numeric mapping as ordinal encoding, but without a meaningful order. A model might interpret city 3 as “greater than” city 1, which is meaningless for nominal data.

Golden rule: Use LabelEncoder only for the target variable in classification tasks. For features, use OrdinalEncoder (ordinal) or OneHotEncoder (nominal).

Target Encoding (Mean Encoding)

Target encoding replaces each category with the mean of the target variable for that category. It’s powerful but requires careful handling to prevent data leakage.

from sklearn.model_selection import KFold

def target_encode(train, col, target, k=5):
    """Safe target encoding with cross-validation."""
    kf = KFold(n_splits=k, shuffle=True, random_state=42)
    train[f"{col}_encoded"] = np.nan

    for train_idx, val_idx in kf.split(train):
        fold_train = train.iloc[train_idx]
        fold_val = train.iloc[val_idx]

        means = fold_train.groupby(col)[target].mean()
        fold_val[f"{col}_encoded"] = fold_val[col].map(means)

    # Fill any missing with global mean
    global_mean = train[target].mean()
    train[f"{col}_encoded"] = train[f"{col}_encoded"].fillna(global_mean)

    return train

The cross-validation approach prevents the model from seeing the target value of a row when computing its own category mean — a subtle but critical form of leakage.

When to use: High-cardinality categorical features (postal codes, user IDs, product codes) where one-hot encoding would explode the feature space.

Tradeoff: Risk of overfitting if not regularized. Always combine with cross-validation or additive smoothing.

Frequency Encoding

Frequency encoding replaces each category with its count or proportion in the dataset:

freq = df["city"].value_counts(normalize=True)
df["city_freq"] = df["city"].map(freq)

Simple, no dimensionality increase, and often surprisingly effective. It captures the intuition that rare categories may behave differently from common ones.

When to use: As a quick baseline for high-cardinality features, or when you want a single numeric representation without leakage concerns.

Putting It All Together with ColumnTransformer

In practice, you’ll have a mix of column types. ColumnTransformer lets you apply different encodings to different columns in a single pipeline:

from sklearn.pipeline import Pipeline
from sklearn.ensemble import RandomForestClassifier

# Define column groups
ordinal_cols = ["education_level"]
onehot_cols = ["city", "payment_method"]
freq_cols = ["postal_code"]

# Build preprocessing
preprocessor = ColumnTransformer([
    ("ordinal", OrdinalEncoder(categories=[education_order]), ordinal_cols),
    ("onehot", OneHotEncoder(handle_unknown="ignore", sparse_output=False), onehot_cols),
    ("freq", "passthrough", freq_cols),  # handle separately
])

# Full pipeline
pipeline = Pipeline([
    ("preprocess", preprocessor),
    ("classifier", RandomForestClassifier(n_estimators=100, random_state=42)),
])

# Cross-validate the whole pipeline
scores = cross_val_score(pipeline, X_train, y_train, cv=5, scoring="accuracy")
print(f"CV accuracy: {scores.mean():.3f} ± {scores.std():.3f}")

This keeps your preprocessing and modeling in one object — no data leakage, no forgetting to apply the same transformation at inference time.

Encoding Decision Guide

Conclusion

Categorical encoding is not a one-size-fits-all problem. The right choice depends on your data’s cardinality, whether the categories have inherent order, and which model you’re using. A few key principles to carry forward:

• Ordinal data gets ordinal encoding — with explicit, domain-validated category order.
• Nominal data with few categories gets one-hot encoding — simple and interpretable.
• High-cardinality nominal features need special handling — target encoding or frequency encoding to avoid dimensionality explosion.
• Always use ColumnTransformer or pipelines — this prevents the most common form of data leakage.
• LabelEncoder is for targets, not features — treating nominal labels as ordered integers introduces false relationships into your model.

The time you invest in thoughtful feature engineering — starting with how you encode categories — pays returns in every downstream step of your machine learning workflow.

~ Kang Ifaz