How to Evaluate a Binary Classifier: A Complete Guide

You trained a machine learning model to predict something binary: fraud or not fraud, churn or stay, disease or healthy. Now comes the question every data scientist faces: Is it actually good? That's where evaluation comes in. And here's the thing — most people do it wrong. They stop at accuracy, declare victory, and deploy. Then the model underperforms in production because they missed something crucial about their data or their use case. This guide walks you through the full evaluation toolkit: metrics, curves, and the thinking behind each one. By the end, you'll know exactly what to measure and why. The Confusion Matrix: What It Really Tells You Before metrics come numbers. Before numbers comes the confusion matrix — a simple 2x2 table that breaks down everything your model did. True Positives (TP): Your model said "yes" and was right. False Positives (FP): Your model said "yes" but was wrong. True Negatives (TN): Your model said "no" and was right. False Negatives (FN): Your model