Exploratory Data Analysis and Preprocessing

NoNo model, however sophisticated, fixes bad input data — the practical groundwork covered here is usually where a data science project actually spends most of its time, well before any model gets trained¹. The term "exploratory data analysis" itself, and much of the philosophy behind treating data exploration as a discipline in its own right rather than a quick check before the real analysis, goes back to Tukey's foundational book of the same name, which argued for looking at data — plotting it, summarising it robustly, actively hunting for the unexpected — before committing to any formal model at all².

Data Cleaning and Missing Data

RealReal datasets have gaps, and how they're handled matters more than it first appears. Deleting every row with a missing value (listwise deletion) is simple but can silently discard a large fraction of the dataset if missingness is common, and worse, it can bias the remaining data if the missingness itself isn't random — if higher earners are more likely to skip an income question, deleting those rows systematically under-represents high earners in whatever's left. Imputation — filling gaps with a plausible value (the column mean or median, a value predicted from other columns, or a more sophisticated model-based estimate) — keeps the row but introduces its own assumption: that the imputed value is a reasonable stand-in for the truth. Neither choice is free; the right one depends on how much data is missing, and whether there's a discoverable reason it's missing.

Feature Engineering

AA model can only use what's actually in its input columns — feature engineering is the deliberate process of constructing new, more useful columns from the raw ones. A timestamp on its own is a fairly weak feature; extracting the day of week, or whether it's a public holiday, or the number of days since a customer's last purchase, often carries far more of the signal a model actually needs. Normalisation and standardisation rescale numeric features (to a fixed 0-1 range, or to zero mean and unit variance respectively) — necessary for any model sensitive to the raw scale of its inputs, like gradient descent-based methods, where a feature ranging in the millions can dominate one ranging between 0 and 1 purely because of scale, not because it's actually more informative.

Train, Validation, and Test Splits

AA model evaluated on the same data it was trained on will look better than it actually is — it's had the chance to simply memorise that specific data, which says nothing about how it performs on data it hasn't seen. The standard discipline: split the data into a training set (used to fit the model), a validation set (used to tune choices like hyperparameters — see Neural Network Architectures for why this matters for network width, learning rate, and similar choices), and a held-out test set, touched exactly once, at the very end, to report a final, unbiased performance number. Reusing the test set to make design decisions — even innocuous ones, like "let's just check if this new feature helps" — quietly turns it into a second validation set, and the final reported number stops being a trustworthy estimate of real-world performance.

Cross-Validation

AA single train/validation split wastes data (whatever's in the validation set isn't used for training) and its result can vary noticeably depending on which specific rows happened to land in which split, especially with a smaller dataset. k-fold cross-validation fixes both problems: split the training data into k equal folds, train k separate times, each time using a different fold as the validation set and the remaining k-1 as training, then average the k performance scores. Every row gets used for both training and validation across the k rounds, just never in the same round, and averaging across folds gives a far more stable estimate of performance than any single split could.

Round	Fold 1	Fold 2	Fold 3	Fold 4	Fold 5
1	Validate	Train	Train	Train	Train
2	Train	Validate	Train	Train	Train
3	Train	Train	Validate	Train	Train
4	Train	Train	Train	Validate	Train
5	Train	Train	Train	Train	Validate

The final reported score is the average of the five "Validate" rounds' results — every row contributed to validating the model exactly once, and to training it four times out of five.

Class Imbalance

WhenWhen one class vastly outnumbers another — fraud detection, where fraudulent transactions might be 0.1% of the data — a model can achieve deceptively high accuracy by simply predicting the majority class every time, learning nothing useful about the minority class at all. This is exactly the situation Supervised Learning's precision/recall/F1 metrics exist to catch, since accuracy alone hides the problem completely. Common responses include oversampling the minority class (duplicating or synthetically generating more examples of it), undersampling the majority class, or adjusting the model's own training to weight misclassifying a rare-class example more heavily than a common-class one.

References