🔐 AI/ML Security Lab – ☣️ Data Poisoning 🐍

Machine learning models rely on training data to identify patterns, learn relationships, and make predictions. But what happens when that training data is deliberately manipulated?

Data poisoning is a machine learning security threat where an attacker introduces corrupted or misleading information into the training process to influence a model's behaviour.

Unlike adversarial examples, which manipulate inputs during prediction, data poisoning targets the information a model learns from.

In this AI/ML Security Lab, we'll use Python and Scikit-learn to simulate a controlled data poisoning attack, measure its impact on classification accuracy, and explore how data integrity checks can help protect machine learning systems.

---

🎯 Understanding Data Poisoning

During training, a machine learning model learns relationships between input features and their corresponding labels.

For example, a classification model might learn to distinguish legitimate transactions from fraudulent ones based on historical data.

However, if some training labels are deliberately modified, the model may learn incorrect relationships.

Consider a simple example:

Original Training Data:

• Transaction A → Legitimate
• Transaction B → Fraudulent
• Transaction C → Legitimate

Poisoned Training Data:

• Transaction A → Fraudulent
• Transaction B → Legitimate
• Transaction C → Legitimate

The input data remains unchanged, but the labels associated with certain records have been altered.

This technique is known as label flipping, one of the simplest forms of data poisoning.

Depending on the dataset, poisoning strategy, and model, these modifications can reduce classification accuracy or influence individual predictions.

---

🟢 Building the Baseline Model

We'll begin by creating a synthetic classification dataset using Scikit-learn.

The dataset contains 2,000 samples with 10 numerical features and two target classes.

X, y = make_classification(
    n_samples=2000,
    n_features=10,
    n_informative=6,
    n_redundant=2,
    n_classes=2,
    flip_y=0,
    class_sep=1.5,
    random_state=42
)

Next, we'll split the dataset into training and testing sets and standardize the features.

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, random_state=42, stratify=y
)

scaler = StandardScaler()

X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

We'll then train a Logistic Regression classifier using the clean training labels.

baseline_model = LogisticRegression(max_iter=1000, random_state=42)
baseline_model.fit(X_train_scaled, y_train)

baseline_predictions = baseline_model.predict(X_test_scaled)
baseline_accuracy = accuracy_score(y_test, baseline_predictions)

print(f"Baseline Accuracy: {baseline_accuracy:.2%}")

This establishes the model's normal performance before introducing corrupted training data.

The clean test dataset will remain unchanged throughout the experiment, allowing us to compare all models under identical evaluation conditions.

---

🔴 Simulating a Data Poisoning Attack

Now we'll deliberately corrupt 30% of the training labels.

We'll use NumPy's random number generator to select training samples and reverse their binary labels.

rng = np.random.default_rng(42)

poison_rate = 0.30
poisoned_labels = y_train.copy()

poison_count = int(len(poisoned_labels) * poison_rate)
poison_indices = rng.choice(len(poisoned_labels), size=poison_count, replace=False)

poisoned_labels[poison_indices] = 1 - poisoned_labels[poison_indices]

Here, 1 - label reverses binary classification labels:

• Class 0 becomes Class 1.
• Class 1 becomes Class 0.

The input features remain unchanged.

Next, we'll train another Logistic Regression model using the poisoned labels.

poisoned_model = LogisticRegression(max_iter=1000, random_state=42)
poisoned_model.fit(X_train_scaled, poisoned_labels)

poisoned_predictions = poisoned_model.predict(X_test_scaled)
poisoned_accuracy = accuracy_score(y_test, poisoned_predictions)

The poisoned model now learns from a dataset containing incorrect labels.

This experiment demonstrates how manipulating training information can influence a model without changing its architecture or prediction code.

---

🔍 Observing the Impact

After training both models, we can compare their performance.

accuracy_drop = baseline_accuracy - poisoned_accuracy
changed_predictions = np.sum(baseline_predictions != poisoned_predictions)

print(f"Baseline Accuracy: {baseline_accuracy:.2%}")
print(f"Poisoned Accuracy: {poisoned_accuracy:.2%}")
print(f"Accuracy Change: {accuracy_drop:+.2%}")
print(f"Changed Predictions: {changed_predictions}")

Two measurements are particularly useful:

1. Classification Accuracy

This measures how frequently each model predicts the correct class on the clean test dataset.

2. Changed Predictions

This measures how many test samples receive different predictions from the baseline and poisoned models.

A poisoned model may produce different predictions even when its overall accuracy changes only slightly.

We can also visualize the accuracy comparison using Matplotlib.

comparison = pd.DataFrame({
    "Model": ["Baseline Model", "Poisoned Model"],
    "Accuracy": [baseline_accuracy, poisoned_accuracy]
})

plt.bar(comparison["Model"], comparison["Accuracy"])
plt.ylim(0, 1)
plt.ylabel("Accuracy")
plt.title("Impact of Data Poisoning on Model Accuracy")
plt.show()

Important: Data poisoning does not always produce a dramatic reduction in accuracy.

Some machine learning models can tolerate moderate amounts of random label noise. The impact depends on the model, dataset, poisoning rate, and attack strategy.

This is why evaluating both accuracy and individual prediction changes is important.

---

🛡️ Defending Against Data Poisoning

One approach to reducing data poisoning risks is to verify the integrity of training data before using it.

For this experiment, we'll assume that the original training labels are available as a trusted reference.

We can compare the poisoned labels against that reference to identify modifications.

trusted_labels = y_train.copy()

suspicious_indices = np.where(poisoned_labels != trusted_labels)[0]

print(f"Suspicious Labels Detected: {len(suspicious_indices)}")

Once the corrupted labels have been identified, we can restore their original values.

cleaned_labels = poisoned_labels.copy()
cleaned_labels[suspicious_indices] = trusted_labels[suspicious_indices]

This demonstrates a basic data integrity verification process.

However, real-world machine learning systems may not always have access to trusted reference labels.

Additional protections can include:

• Dataset versioning: Track changes to training datasets.
• Data provenance: Verify where training information originated.
• Access controls: Restrict unauthorized modifications.
• Data validation: Identify unexpected changes or inconsistencies.
• Human review: Investigate suspicious training records.

These measures help reduce the risk of compromised data entering the training pipeline.

---

🧪 Verifying the Defense

After restoring the original training labels, we'll retrain the classification model.

recovered_model = LogisticRegression(max_iter=1000, random_state=42)
recovered_model.fit(X_train_scaled, cleaned_labels)

recovered_predictions = recovered_model.predict(X_test_scaled)
recovered_accuracy = accuracy_score(y_test, recovered_predictions)

Now we can compare all three models.

results = pd.DataFrame({
    "Model": ["Baseline", "Poisoned", "Recovered"],
    "Accuracy": [baseline_accuracy, poisoned_accuracy, recovered_accuracy]
})

print(results)

The comparison includes:

Baseline Model: Trained using the original clean labels.

Poisoned Model: Trained using deliberately corrupted labels.

Recovered Model: Trained after restoring the original labels.

Because the recovered model uses the same clean training data and deterministic training configuration as the baseline, its accuracy should match the baseline model.

This demonstrates how restoring trusted training data can help recover model performance after a label-poisoning incident.

It also highlights an important distinction: the defense works here because we retained a trustworthy copy of the original labels. Without that reference, identifying poisoned records becomes considerably more challenging.

---

📌 Security Takeaways

• Data poisoning targets the training process by introducing corrupted or misleading information.
• Label-flipping attacks manipulate class labels without changing the input features.
• Poisoned training data can influence model accuracy and individual predictions.
• Comparing clean and poisoned models helps measure the effects of training data manipulation.
• Dataset integrity checks, provenance tracking, and access controls can help reduce poisoning risks.
• Restoring trusted training data and retraining can recover model performance.

---

🚀 Final Note

A machine learning model is only as trustworthy as the data and processes used to train it.

Even when an algorithm is functioning correctly, corrupted training information can influence its predictions and overall reliability.

By understanding data poisoning and implementing appropriate integrity checks, developers can build more robust and trustworthy machine learning systems.

---

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler


# -----------------------------------------
# Step 1: Generate the classification dataset
# -----------------------------------------
def load_dataset():
    """Generate a synthetic binary classification dataset."""

    X, y = make_classification(
        n_samples=2000,
        n_features=10,
        n_informative=6,
        n_redundant=2,
        n_classes=2,
        flip_y=0,
        class_sep=1.5,
        random_state=42
    )

    print(f"✅ Dataset generated | samples={len(X)} | features={X.shape[1]}")
    return X, y


# -----------------------------------------
# Step 2: Prepare and standardize the data
# -----------------------------------------
def prepare_data(X, y):
    """Split the dataset and standardize its features."""

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=0.25, random_state=42, stratify=y
    )

    scaler = StandardScaler()
    X_train_scaled = scaler.fit_transform(X_train)
    X_test_scaled = scaler.transform(X_test)

    print(f"✅ Data prepared | training={len(X_train)} | testing={len(X_test)}")

    return X_train_scaled, X_test_scaled, y_train, y_test


# -----------------------------------------
# Step 3: Train the baseline model
# -----------------------------------------
def train_model(X_train, y_train):
    """Train a Logistic Regression classifier."""

    model = LogisticRegression(max_iter=1000, random_state=42)
    model.fit(X_train, y_train)

    return model


# -----------------------------------------
# Step 4: Evaluate model performance
# -----------------------------------------
def evaluate_model(model, X_test, y_test, model_name):
    """Evaluate classification accuracy and predictions."""

    predictions = model.predict(X_test)
    accuracy = accuracy_score(y_test, predictions)

    print(f"📊 {model_name} Accuracy: {accuracy:.2%}")

    return predictions, accuracy


# -----------------------------------------
# Step 5: Simulate the data poisoning attack
# -----------------------------------------
def poison_training_labels(y_train, poison_rate=0.30):
    """Corrupt a percentage of training labels using label flipping."""

    rng = np.random.default_rng(42)
    poisoned_labels = y_train.copy()

    poison_count = int(len(poisoned_labels) * poison_rate)
    poison_indices = rng.choice(len(poisoned_labels), size=poison_count, replace=False)

    poisoned_labels[poison_indices] = 1 - poisoned_labels[poison_indices]

    print(f"\n🔴 Data Poisoning Attack")
    print(f"Poisoning Rate: {poison_rate:.0%}")
    print(f"Corrupted Labels: {poison_count}")

    return poisoned_labels, poison_indices


# -----------------------------------------
# Step 6: Observe the poisoning impact
# -----------------------------------------
def observe_impact(y_test, baseline_predictions, poisoned_predictions, baseline_accuracy, poisoned_accuracy):
    """Compare baseline and poisoned model performance."""

    accuracy_drop = baseline_accuracy - poisoned_accuracy
    changed_predictions = np.sum(baseline_predictions != poisoned_predictions)

    comparison = pd.DataFrame({
        "Model": ["Baseline Model", "Poisoned Model"],
        "Accuracy": [baseline_accuracy, poisoned_accuracy],
        "Training Data": ["Clean", "30% Labels Corrupted"]
    })

    print("\n🔍 Data Poisoning Impact")
    print(comparison.to_string(index=False))
    print(f"\nAccuracy Change (Baseline - Poisoned): {accuracy_drop:+.2%}")
    print(f"Changed Predictions: {changed_predictions}/{len(y_test)}")

    print("\nPoisoned Model Classification Report:")
    print(classification_report(y_test, poisoned_predictions, zero_division=0))

    print("Baseline Confusion Matrix:")
    print(confusion_matrix(y_test, baseline_predictions))

    print("\nPoisoned Confusion Matrix:")
    print(confusion_matrix(y_test, poisoned_predictions))

    return comparison


# -----------------------------------------
# Step 7: Visualize the accuracy comparison
# -----------------------------------------
def visualize_comparison(results, title):
    """Visualize model accuracy using a bar chart."""

    plt.figure(figsize=(9, 5))
    plt.bar(results["Model"], results["Accuracy"])

    plt.ylim(0, 1)
    plt.ylabel("Accuracy")
    plt.title(title)

    for index, accuracy in enumerate(results["Accuracy"]):
        plt.text(index, accuracy + 0.02, f"{accuracy:.2%}", ha="center")

    plt.tight_layout()
    plt.show()


# -----------------------------------------
# Step 8: Detect and restore poisoned labels
# -----------------------------------------
def apply_defense(poisoned_labels, trusted_labels):
    """Detect corrupted labels using a trusted reference."""

    suspicious_indices = np.where(poisoned_labels != trusted_labels)[0]
    cleaned_labels = poisoned_labels.copy()

    cleaned_labels[suspicious_indices] = trusted_labels[suspicious_indices]

    print("\n🛡️ Data Integrity Verification")
    print(f"Suspicious Labels Detected: {len(suspicious_indices)}")
    print(f"Labels Restored: {np.sum(cleaned_labels != poisoned_labels)}")

    return cleaned_labels


# -----------------------------------------
# Step 9: Verify model recovery
# -----------------------------------------
def verify_defense(X_train, X_test, cleaned_labels, y_test, baseline_accuracy, poisoned_accuracy):
    """Retrain the model and verify recovery after label restoration."""

    recovered_model = train_model(X_train, cleaned_labels)
    recovered_predictions, recovered_accuracy = evaluate_model(
        recovered_model, X_test, y_test, "Recovered Model"
    )

    results = pd.DataFrame({
        "Model": ["Baseline", "Poisoned", "Recovered"],
        "Accuracy": [baseline_accuracy, poisoned_accuracy, recovered_accuracy]
    })

    print("\n🧪 Defense Verification")
    print(results.to_string(index=False))
    print(f"\nRecovery Improvement: {recovered_accuracy - poisoned_accuracy:+.2%}")
    print(f"Recovered Baseline Accuracy: {np.isclose(recovered_accuracy, baseline_accuracy)}")

    return results


# -----------------------------------------
# Step 10: Run the complete security lab
# -----------------------------------------
def run_lab():
    """Execute the complete data poisoning security experiment."""

    print("🔐 Starting AI/ML Security Lab — Data Poisoning\n")

    # Build the baseline
    X, y = load_dataset()
    X_train, X_test, y_train, y_test = prepare_data(X, y)

    baseline_model = train_model(X_train, y_train)
    baseline_predictions, baseline_accuracy = evaluate_model(
        baseline_model, X_test, y_test, "Baseline Model"
    )

    # Simulate the attack
    poisoned_labels, poison_indices = poison_training_labels(y_train, poison_rate=0.30)

    poisoned_model = train_model(X_train, poisoned_labels)
    poisoned_predictions, poisoned_accuracy = evaluate_model(
        poisoned_model, X_test, y_test, "Poisoned Model"
    )

    # Observe the impact
    comparison = observe_impact(
        y_test, baseline_predictions, poisoned_predictions, baseline_accuracy, poisoned_accuracy
    )

    visualize_comparison(comparison, "Impact of Data Poisoning on Model Accuracy")

    # Apply the defense
    cleaned_labels = apply_defense(poisoned_labels, y_train)

    # Verify the defense
    results = verify_defense(
        X_train, X_test, cleaned_labels, y_test, baseline_accuracy, poisoned_accuracy
    )

    visualize_comparison(results, "Data Poisoning — Attack and Recovery Comparison")

    print("\n✅ AI/ML Security Lab completed successfully")


if __name__ == "__main__":
    run_lab()