Practical SVM¶
# Import the required libraries
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.svm import SVC
from sklearn.metrics import (
accuracy_score,
f1_score,
confusion_matrix,
classification_report,
ConfusionMatrixDisplay
)
1. Load the dataset¶
The Breast Cancer Wisconsin dataset contains:
- 569 observations
- 30 numerical features
- Two classes:
0 = malignant1 = benign
# Load the dataset
data = load_breast_cancer()
X = pd.DataFrame(data.data, columns=data.feature_names)
y = pd.Series(data.target, name="target")
print("Dataset shape:", X.shape)
print("\nClass names:", list(data.target_names))
print("\nClass distribution:")
print(y.value_counts().sort_index().rename(index={0: "malignant", 1: "benign"}))
X.head()
2. Split the data into training and test sets¶
We use an 80/20 stratified split.
stratify=y keeps approximately the same malignant/benign class proportions in both the training and test sets.
The test set is kept untouched while we select the SVM hyperparameters.
# 80/20 stratified train-test split
X_train, X_test, y_train, y_test = train_test_split(
X,
y,
test_size=0.20,
stratify=y,
random_state=42
)
print("Training observations:", len(X_train))
print("Test observations:", len(X_test))
print("\nTraining class distribution:")
print(y_train.value_counts().sort_index())
print("\nTest class distribution:")
print(y_test.value_counts().sort_index())
3. Create a scaling + SVM pipeline¶
Feature scaling is important for SVM because SVM calculations depend strongly on distances and feature magnitudes.
Using a Pipeline ensures that:
- the scaler is fitted only on the training portion of each cross-validation fold;
- validation data are not used to calculate the scaling parameters;
- data leakage is avoided.
# Create the machine-learning pipeline
pipeline = Pipeline([
("scaler", StandardScaler()),
("svc", SVC())
])
pipeline
# Hyperparameter combinations to test
param_grid = [
{
"svc__kernel": ["linear"],
"svc__C": [0.1, 1, 10, 100]
},
{
"svc__kernel": ["rbf"],
"svc__C": [0.1, 1, 10, 100],
"svc__gamma": [0.001, 0.01, 0.1, 1]
}
]
# 5-fold stratified cross-validation
cv = StratifiedKFold(
n_splits=5,
shuffle=True,
random_state=42
)
grid_search = GridSearchCV(
estimator=pipeline,
param_grid=param_grid,
scoring="f1_macro",
cv=cv,
n_jobs=-1,
return_train_score=True
)
grid_search.fit(X_train, y_train)
5. Best SVM configuration¶
Grid Search tests the candidate combinations using only the training data.
The test set is still untouched at this point.
print("Best parameters:")
print(grid_search.best_params_)
print("\nBest mean cross-validation Macro F1:")
print(f"{grid_search.best_score_:.4f}")
With the fixed split and search settings used here, the selected model should be:
[ \text{RBF kernel}, \quad C = 10, \quad \gamma = 0.01 ]
This does not mean these values are universally best. They are the best among the combinations tested for this dataset and this experimental setup.
6. View the top cross-validation results¶
This table shows which SVM configurations performed best during cross-validation.
# Convert Grid Search results to a readable table
results = pd.DataFrame(grid_search.cv_results_)
columns_to_show = [
"param_svc__kernel",
"param_svc__C",
"param_svc__gamma",
"mean_test_score",
"std_test_score"
]
top_results = (
results[columns_to_show]
.sort_values("mean_test_score", ascending=False)
.head(10)
.reset_index(drop=True)
)
top_results
7. Evaluate the selected model on the test set¶
Only after hyperparameter selection is complete do we evaluate the final model on the previously untouched test data.
# Predict the test observations
best_model = grid_search.best_estimator_
y_pred = best_model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
macro_f1 = f1_score(y_test, y_pred, average="macro")
cm = confusion_matrix(y_test, y_pred)
print(f"Test Accuracy: {accuracy:.4f} ({accuracy * 100:.2f}%)")
print(f"Macro F1: {macro_f1:.4f} ({macro_f1 * 100:.2f}%)")
print("\nConfusion Matrix:")
print(cm)
Expected confusion matrix¶
Rows represent the actual class and columns represent the predicted class:
| Actual | Predicted Malignant | Predicted Benign |
|---|---|---|
| Malignant | 41 | 1 |
| Benign | 1 | 71 |
Therefore:
- 112 out of 114 test observations are classified correctly.
- 1 malignant case is predicted as benign.
- 1 benign case is predicted as malignant.
# Graphical confusion matrix
disp = ConfusionMatrixDisplay(
confusion_matrix=cm,
display_labels=data.target_names
)
disp.plot()
plt.title("SVM Confusion Matrix — Test Set")
plt.show()
8. Classification report¶
The classification report shows precision, recall, and F1 score separately for the malignant and benign classes.
print(
classification_report(
y_test,
y_pred,
target_names=data.target_names,
digits=4
)
)
9. Summary¶
Using this workflow:
- Dataset: Breast Cancer Wisconsin
- Observations: 569
- Features: 30
- Split: 80% training / 20% testing
- Split type: stratified
- Preprocessing: StandardScaler
- Hyperparameter selection: Grid Search
- Validation method: 5-fold stratified cross-validation
- Search metric: Macro F1
- Best kernel: RBF
- Best C: 10
- Best gamma: 0.01
- Test accuracy: about 98.2%
- Test Macro F1: about 98.1%
- Test errors: 2 out of 114 observations