Classification vs. Regression in Keras: Building the Right Model for the Right Task

A hands-on walkthrough of binary classification, multiclass softmax, and scalar regression — with working Keras code, loss-function matching rules, and the sanity checks that catch silent bugs before they waste your time.

Download the entire book using the link below:

[embed]Understanding Deep Learning in PDF Download | OnePageCode Understanding Deep Learning in PDF Download by OnePageCode on Patreon. Join OnePageCode's community for exclusive…www.patreon.com

Workflow orientation and chapter goals

Practical machine learning is best thought of as a repeatable workflow rather than a grab-bag of tricks. For every supervised task you will follow the same high-level pipeline: preprocess the raw data into numeric inputs the model can consume, choose a minimal model whose final outputs match the type of target you have, compile that model with a loss and metrics appropriate to the task, fit while monitoring validation performance, then evaluate and sanity‑check predictions. Keep that pipeline as your baseline: preprocessing → simple model → evaluation. Only after a simple, correct baseline behaves sensibly should you consider more complex architectures.

A few conceptual terms tie the whole workflow together. Inputs (sometimes called features) are the numeric representations of your raw examples. Targets (labels) are the values the model must predict: scalars for regression, single class indices or one‑hot vectors for classification, or multiple independent indicators for multilabel problems. Predictions are the model’s outputs—probabilistic scores, logits, or numeric estimates—depending on your final activation. The loss is the scalar function used for training; it must match the prediction form and the target encoding. Mismatches here are a frequent source of silent errors: using a softmax final layer together with binary target vectors, or training a single‑unit sigmoid with categorical_crossentropy for non‑binary tasks, will produce confusing behavior or poor gradients.

A small preview of the three worked tasks helps make these distinctions concrete.

Email spam vs not‑spam (binary classification). Imagine a tiny bag‑of‑words representation for each message: a fixed vocabulary of words and a vector that counts or indicates which words appeared. The input is this numeric vector; the target is a single binary label (spam or not). A minimal model yields one output unit that produces a score interpretable as the probability of the positive class. That means a sigmoid activation on the final unit together with binarycrossentropy as the loss and accuracy or AUC as monitoring metrics. Beware of two common mistakes here: using a softmax with two outputs when a single sigmoid unit suffices, and feeding binary targets into categoricalcrossentropy—both are avoidable by matching the output form and loss to the target encoding.

Fruit type from tabular features (multiclass classification). Suppose each fruit example has a few continuous measurements—mass, hue, firmness—and the goal is to predict which fruit species it is out of several mutually exclusive classes. The inputs are the tabular feature vectors; targets are categorical (one class per sample). The minimal model ends with a softmax layer with as many units as there are classes, producing a probability distribution across classes. During training you provide one‑hot vectors (or integer class indices with the correct loss function), and you use categoricalcrossentropy as the loss and accuracy as a key metric. If you represent targets as one‑hot vectors, categoricalcrossentropy expects that format with softmax outputs; if you instead provide integer class indices to categorical_crossentropy you must ensure you are using the proper API variant that accepts sparse indices. Confusing labels (an individual target value) with the set of classes (the complete label space) is a frequent source of bugs—make explicit whether your targets are scalars, one‑hot vectors, or independent binary indicators.

Predicting courier delivery time (scalar regression). In a delivery example, inputs might include distance, number of stops, and traffic indicator; the target is a continuous scalar: delivery time in minutes. The minimal model ends with a single linear output (no activation), and you train with a regression loss such as meansquarederror (MSE) or meanabsoluteerror (MAE). Use MAE or MSE as your evaluation metric rather than accuracy, which makes no sense for continuous predictions. Also take care with feature scaling: compute normalization statistics (mean, standard deviation, min/max for each feature) on the training split only, apply those same transforms to validation and test data, and never normalize the target values unless you reverse that transform when reporting real‑world metrics. Accidentally adding a nonlinearity such as relu on the final regression unit will clip predictions and harm your model.

Across these three examples there are recurring practical points worth underscoring. First, preprocessing matters: simple vectorizers and normalizers enable performant baselines. For text represented as bag‑of‑words, the vectorizer’s vocabulary determines which tokens are represented; tokens not in the vocabulary will be silently ignored by a simple vectorizer, so review how your vectorization handles unknown words. For numerical features, always scale using training statistics and apply the same transform to validation and test splits. Second, the model’s final layer and its loss must agree with the task type: sigmoid + binarycrossentropy for single‑label binary problems, softmax + categoricalcrossentropy for mutually exclusive multiclass problems (with one‑hot targets or appropriate sparse indices), and linear + MSE/MAE for scalar regression. Third, resist the temptation to jump immediately to complex architectures. Start with a simple, correctly specified model that reflects the task, inspect validation curves and simple prediction examples, and only then iterate.

Finally, always sanity‑check predictions. For classification, inspect predicted probabilities and a confusion matrix or a few example predictions to confirm the model assigns sensible probabilities among classes. For regression, compare predicted and true values on a scatter plot, and ensure reported metrics (MAE/MSE) reflect the original target units. These small inspections catch the kinds of mistakes that automatic metrics alone can miss—wrong loss functions, misencoded targets, or improperly normalized inputs—before you invest time in scaling up model complexity.

Glossary: classification and regression

A supervised learning problem revolves around three concrete objects: the input sample x, the model’s prediction ypred, and the target annotation ytrue. The input x is one data point presented to the model (an image, a piece of text, a row of tabular features). The prediction ypred is the numeric output the model produces for x; it is what your code will compare against the annotation. The target ytrue (also called the ground truth or label) is the trusted annotation that describes what the correct output should be for that sample. A loss is a scalar function L(ytrue, ypred) that measures how far the prediction is from the target; training drives model parameters to reduce this loss on training examples.

Classes and labels require careful vocabulary. A class is an element of a set of possible categories for a task (for example, {spam, not-spam}). A label is the per-sample annotation that selects one or more classes from that set (for example, a particular email has the label spam). Confusing the set of classes with a specific label for a sample is a common source of mistakes; always separate the global concept (the set of classes) from the local one (the label assigned to a single sample).

Binary classification, categorical (multiclass) classification, multilabel classification, and scalar regression are distinct problem types with distinct target spaces and implications for model outputs and loss functions.

Binary classification. The problem has two mutually exclusive classes, often viewed as positive versus negative or yes versus no. Example: spam vs not-spam. The set of classes contains exactly two items; each sample’s label picks one of them. For a binary task you can represent the target as a single binary value per sample (0 or 1) or as a two-dimensional one-hot vector, but conceptually the task is to decide between two exclusive alternatives.

Categorical (multiclass) classification. The set of classes has more than two categories and, crucially, each sample belongs to exactly one class. Example: fruit recognition with classes {apple, banana, orange}. For each image there is exactly one true class. Targets for multiclass problems are commonly represented as one-hot vectors (a 1 in the position of the true class, zeros elsewhere) or as integer class indices; either representation must be handled consistently by the model and loss function. The exclusivity—exactly one true class per sample—is the defining feature that separates multiclass from multilabel.

Multilabel classification. Multiple labels can be true for the same sample; labels are not mutually exclusive. An image of a fruit basket could be labeled {apple: 1, banana: 1, orange: 0} if it contains both an apple and a banana. Multilabel tasks require the model to output a score per class independently; treating a multilabel problem as if it were multiclass (forcing exactly one active class) will systematically discard valid combinations of labels and produce poor results.

Scalar regression. The target y_true is a continuous numeric value rather than a discrete class. Example: predicting delivery time in minutes. Outputs are real numbers, and losses quantify squared or absolute deviation (mean squared error, mean absolute error) rather than categorical mismatch. Regression outputs should be unconstrained real values unless the domain demands otherwise; adding a nonlinear activation that clips outputs (for example, a rectified linear unit that forbids negative predictions) can introduce systematic bias if the true target range includes those values.

The mapping between outputs and targets is the crucial plumbing of a supervised system: the model must produce outputs in a form that the loss function can meaningfully compare to the provided targets. For binary problems that mapping is usually a single score per sample that corresponds to the target 0/1. For multiclass problems that mapping is a vector of scores across classes paired with an integer index or one-hot target that identifies the correct class. For multilabel problems the mapping is a vector of independent scores—one per possible label—paired with a binary vector target indicating which labels apply. For regression the mapping is a scalar (or vector of scalars) directly comparable to the continuous target.

Common mistakes arise when that mapping is inconsistent. Treating multiclass as multilabel (or vice versa) is one such error: forcing a single-label representation on a naturally multilabel dataset throws away legitimate combinations; forcing an independent-label architecture on a mutually exclusive multiclass problem loses the mutually exclusive structure and can make training harder. Another frequent slip is confusing the vocabulary: remember that a label is a per-sample annotation, while classes denote the global set of categories.

Concrete examples make these distinctions tangible. In the spam-versus-not-spam example, the classes set is {spam, not-spam}. Each email receives a label that is either spam or not-spam. Here the model’s prediction might be a single score ypred that correlates with the probability of spam; the target ytrue is a single bit for each sample. In the fruit example with {apple, banana, orange}, each image’s label picks exactly one class; representing targets as one-hot vectors is the standard choice when using losses that operate on full-class score vectors. In the delivery-time example the target is a number of minutes; the model’s y_pred must be a real-valued estimate and the loss should measure numeric distance (for example MSE or MAE).

Always keep the loss in mind as the formal bridge between predictions and targets: it quantifies the error you optimize. Choosing an inappropriate loss because the output form and the target form are mismatched is a fundamental modeling error. Inspect your target encoding and your model output shape before training; they must line up semantically so the loss function can do its job.

Binary classification: tiny spam detector

We build a tiny, deterministic spam detector that demonstrates the full binary classification pipeline: a simple bag-of-words vectorizer, a compact dense network ending with a single sigmoid unit, training with binary_crossentropy, and evaluation with accuracy on a held-out validation split.

The code below implements the example end-to-end. It deliberately keeps everything explicit and small so you can inspect the preprocessing, the model output shape, and the mapping from predicted probability to a binary decision.

import numpy as np
import keras
from keras import layers
# 1) Tiny corpus and labels (1=spam, 0=ham)
texts = [
    "win a free prize now",
    "schedule the project meeting",
    "limited time offer click now",
    "lunch meeting with team",
    "urgent: claim your free gift",
    "weekly report and invoice attached",
    "click to win big rewards",
    "reminder: project deadline",
    "exclusive offer only today",
    "let's discuss the roadmap",
    "free tickets available now",
    "please review the latest contract"
]
labels = np.array([1,0,1,0,1,0,1,0,1,0,1,0], dtype=np.float32)
# 2) Simple bag-of-words vectorizer
vocab = [
    'win','free','offer','click','now','urgent','claim','exclusive','prize','gift',
    'meeting','project','report','invoice','lunch','reminder','deadline','discuss','roadmap','review','contract'
]
index = {w:i for i,w in enumerate(vocab)}
X = np.zeros((len(texts), len(vocab)), dtype=np.float32)
for r, txt in enumerate(texts):
    for tok in txt.lower().split():
        if tok in index:
            X[r, index[tok]] += 1.0
# 3) Train/val split (keep it simple and deterministic)
X_train, y_train = X[:9], labels[:9]
X_val,   y_val   = X[9:], labels[9:]
# 4) Define a small binary classifier
model = keras.Sequential([
    layers.Input(shape=(X.shape[1],)),
    layers.Dense(8, activation='relu'),
    layers.Dense(1, activation='sigmoid')
])
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy'])
# 5) Train and evaluate
history = model.fit(X_train, y_train, epochs=40, batch_size=3, validation_data=(X_val, y_val), verbose=0)
val_loss, val_acc = model.evaluate(X_val, y_val, verbose=0)
preds = model.predict(X_val, verbose=0)
# 6) Checks
assert 'loss' in history.history and 'accuracy' in history.history
assert preds.min() >= 0.0 and preds.max() <= 1.0
assert float(val_acc) >= 0.9, f"Validation accuracy too low: {val_acc}"
print('history_keys', sorted(history.history.keys())[:2])
print('val_accuracy', round(float(val_acc), 3))
print('val_preds', preds.ravel().round(3).tolist())

What this code does and why each piece matters

1. The texts array and labels provide a deterministic toy dataset where labels are 1 for spam and 0 for ham. Using a fixed dataset and an explicit split makes results reproducible without relying on random sampling.
2. The bag-of-words vectorizer constructs a fixed vocabulary and counts token occurrences per text into a dense matrix X. This explicit vectorization step is crucial: models expect numeric tensors, and this simple approach shows how text becomes a feature vector. Tokens not present in the predefined vocabulary are silently ignored by this vectorizer; that’s a common behavior to remember when using small static vocabularies.
3. The model has a single-unit final layer with activation='sigmoid'. For binary classification with scalar outputs, use a single sigmoid unit (not a softmax) and compile with loss='binary_crossentropy'. This pairing yields outputs interpretable as probabilities in [0, 1], which the assertions check.
4. Metrics=['accuracy'] reports the proportion of examples where the model’s binary decision matches the target. Accuracy is appropriate here because the task is binary classification and the labels are 0/1. For the loss, binarycrossentropy is the correct choice; using categoricalcrossentropy here would be a mismatch and can produce incorrect training behavior.
5. The code uses a simple deterministic train/validation split (first nine examples for training, last three for validation) so training and validation behavior are stable every run.

Interpreting predicted probabilities and thresholding

The model.predict call produces probability scores between 0 and 1 for each validation example. To convert these probabilities into class decisions, apply a threshold: common practice is to classify as spam when probability >= 0.5 and not-spam otherwise. For example, given preds:

[0.98, 0.02, 0.95]

the thresholded decisions would be [1, 0, 1]. The 0.5 threshold gives a direct probabilistic intuition: values near 1 indicate strong model confidence in the positive (spam) class, values near 0 indicate confidence in the negative (ham) class, and values near 0.5 are uncertain.

Common mistakes to avoid

Never forget to use a sigmoid activation for a single-unit binary output; without it, outputs are unconstrained real numbers and do not represent probabilities. Pairing the wrong loss with the output is another frequent error—binarycrossentropy belongs with a single sigmoid unit, while categoricalcrossentropy is intended for multi-class outputs (softmax) and one-hot targets or sparse integer targets. Also be mindful that a small, fixed vocabulary means tokens outside it are ignored; if your dataset contains many out-of-vocabulary words, the simple bag-of-words above will silently lose information.

Multiclass classification: fruit type from tabular features

We want a simple, end-to-end multiclass classifier: three mutually exclusive fruit types (apple, banana, orange) predicted from three tabular features (mass, hue, firmness). Multiclass classification requires a final softmax layer that outputs a probability distribution across classes, and the targets must be one-hot encoded when you pair softmax with categorical_crossentropy. Numeric features must be scaled using only the training set statistics; applying validation-set statistics would leak information and change the model’s input distribution.

The following code builds a tiny, deterministic dataset, normalizes features with training statistics, one-hot encodes the labels, defines a small dense network with a softmax output, trains it with categorical_crossentropy, and evaluates accuracy on a held-out validation split.

import numpy as np
import keras
from keras import layers
# 1) Tabular features: [mass_g, hue_0to1, firmness_0to1]
# Classes: 0=apple, 1=banana, 2=orange
X = np.array([
    [180, 0.05, 0.8],  # apple
    [170, 0.06, 0.75], # apple
    [160, 0.07, 0.78], # apple
    [120, 0.15, 0.4],  # banana
    [110, 0.16, 0.35], # banana
    [130, 0.14, 0.45], # banana
    [200, 0.10, 0.6],  # orange
    [210, 0.11, 0.62], # orange
    [195, 0.09, 0.58], # orange
    [175, 0.05, 0.82], # apple (val)
    [115, 0.15, 0.38], # banana (val)
    [205, 0.10, 0.61]  # orange (val)
], dtype=np.float32)
y_idx = np.array([0,0,0, 1,1,1, 2,2,2, 0,1,2], dtype=np.int32)
# 2) Split train/val
X_train, y_train_idx = X[:9], y_idx[:9]
X_val,   y_val_idx   = X[9:], y_idx[9:]
# 3) Scale numeric features using train stats
mean = X_train.mean(axis=0)
std = X_train.std(axis=0) + 1e-6
X_train_norm = (X_train - mean) / std
X_val_norm   = (X_val   - mean) / std
# 4) One-hot encode labels
num_classes = 3
y_train = np.eye(num_classes, dtype=np.float32)[y_train_idx]
y_val   = np.eye(num_classes, dtype=np.float32)[y_val_idx]
# 5) Define a small softmax classifier
model = keras.Sequential([
    layers.Input(shape=(X.shape[1],)),
    layers.Dense(12, activation='relu'),
    layers.Dense(num_classes, activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])
# 6) Train and evaluate
history = model.fit(X_train_norm, y_train, epochs=60, batch_size=3, validation_data=(X_val_norm, y_val), verbose=0)
val_loss, val_acc = model.evaluate(X_val_norm, y_val, verbose=0)
preds = model.predict(X_val_norm, verbose=0)
# 7) Checks: accuracy and softmax properties
row_sums = preds.sum(axis=1)
assert np.allclose(row_sums, 1.0, atol=1e-5)
assert float(val_acc) >= 0.9, f"Validation accuracy too low: {val_acc}"
print('val_accuracy', round(float(val_acc), 3))
print('val_pred_classes', preds.argmax(axis=1).tolist())
print('softmax_row_sums_close_to_1', np.round(row_sums, 6).tolist())

Line-by-line intent and important behaviors:

1. The dataset X contains three features per sample: mass in grams, hue scaled to [0, 1], and firmness in [0, 1]. y_idx holds integer class indices (0, 1, 2).
2. We split the first nine rows for training and keep three rows for validation to check generalization.
3. mean and std are computed from Xtrain only and then applied to both Xtrain and X_val. This ensures the validation data is normalized using training statistics; using validation statistics would leak information and produce inconsistent preprocessing at inference time.
4. One-hot encoding: categoricalcrossentropy with a softmax output expects targets shaped like one-hot vectors. The snippet constructs ytrain and yval with np.eye(numclasses)[indices]. If you instead fed integer indices into categoricalcrossentropy while your model outputs one-hot-like softmax vectors, you would get a shape mismatch or incorrect behavior—use sparsecategorical_crossentropy if you prefer integer labels.
5. Model definition: the final Dense layer has activation='softmax' and num_classes units. Softmax converts raw logits into a probability distribution across classes for each sample.
6. model.compile uses loss='categorical_crossentropy' and metrics=['accuracy'], the canonical pairing for softmax multiclass models with one-hot targets.
7. After prediction, preds is an array of probability vectors. The code asserts that each row sums to 1 (softmax property) and that validation accuracy is at least 0.9; these two checks verify correct output shape/behavior and that the simple model can separate these small synthetic clusters.

Interpreting softmax outputs: a prediction vector like [0.85, 0.10, 0.05] should be read as 85% probability for class 0 (apple), 10% for class 1 (banana), 5% for class 2 (orange). The predicted class decision is the argmax of this vector; in this example argmax([0.85, 0.10, 0.05]) == 0, so the model predicts apple with 85% confidence.

Common mistakes highlighted by this workflow:

1. Forgetting to one-hot encode targets when using categoricalcrossentropy with softmax. If you prefer to keep integer labels, use sparsecategoricalcrossentropy instead of categoricalcrossentropy.
2. Normalizing validation (or test) inputs with their own mean/std instead of the training set mean/std. Always compute feature scaling parameters from the training split and apply them to other splits.
3. Using an activation like relu on the final layer for multiclass classification instead of softmax; relu will not produce a normalized probability distribution and is inappropriate for categorical_crossentropy.

This tiny example demonstrates the required mapping from task to model: multiclass (categorical) classification uses one-hot targets, a softmax output layer, and categorical_crossentropy loss; feature scaling is done using training statistics so the model sees consistent, zero-centered inputs during training and evaluation.

Scalar regression: delivery time prediction

We build a small, deterministic scalar-regression example that predicts delivery time (minutes) from two tabular features: distance_km and number of stops. The target is generated by a known linear rule so we can validate the model’s behavior and metrics precisely. We split into train and validation, compute normalization statistics from the training split only, train a compact dense network whose final layer is linear, use mean-squared error as the loss, and report mean absolute error as the evaluation metric.

The complete runnable pipeline appears below. Read the code first to see the concrete operations; the paragraphs that follow explain the important decisions and checks.

import numpy as np
import keras
from keras import layers
# 1) Features and target: delivery time (min) from distance_km and stops
# Deterministic formula: time = 8 + 4*distance_km + 3*stops
X = np.array([
    [1.0, 0],
    [2.5, 1],
    [3.0, 0],
    [5.0, 2],
    [8.0, 1],
    [0.5, 0],
    [6.0, 3],
    [4.0, 1],
    [7.5, 2],
    [2.0, 2],   # val
    [9.0, 0],   # val
    [1.2, 1]    # val
], dtype=np.float32)
y = (8 + 4*X[:,0] + 3*X[:,1]).astype(np.float32)
y = y.reshape(-1, 1)
# 2) Train/val split
X_train, y_train = X[:9], y[:9]
X_val,   y_val   = X[9:], y[9:]
# 3) Normalize using train stats
mean = X_train.mean(axis=0)
std = X_train.std(axis=0) + 1e-6
X_train_n = (X_train - mean) / std
X_val_n   = (X_val   - mean) / std
# 4) Define a small regression MLP (linear output)
model = keras.Sequential([
    layers.Input(shape=(2,)),
    layers.Dense(8, activation='relu'),
    layers.Dense(1)  # linear output
])
model.compile(optimizer='adam', loss='mse', metrics=['mae'])
# 5) Train and evaluate
history = model.fit(X_train_n, y_train, epochs=120, batch_size=3, validation_data=(X_val_n, y_val), verbose=0)
val_loss, val_mae = model.evaluate(X_val_n, y_val, verbose=0)
preds = model.predict(X_val_n, verbose=0)
# 6) Checks: MAE threshold and monotonicity wrt distance
assert float(val_mae) < 1.0, f"Validation MAE too high: {val_mae}"
# Monotonic sanity: increasing distance with same stops should increase time
x_a = np.array([[3.0, 1]], dtype=np.float32)
x_b = np.array([[6.0, 1]], dtype=np.float32)
x_a_n = (x_a - mean) / std
x_b_n = (x_b - mean) / std
pa = float(model.predict(x_a_n, verbose=0)[0,0])
pb = float(model.predict(x_b_n, verbose=0)[0,0])
assert pb > pa, 'Prediction should increase with distance when stops equal.'
print('val_mae', round(float(val_mae), 3))
print('val_preds', np.round(preds.ravel(), 2).tolist())
print('sanity_pa_pb', round(pa,2), round(pb,2))

Dataset and deterministic target The synthetic dataset in X has two columns: distance in kilometers and number of stops, with 12 examples total. The target y is produced by the explicit linear formula time = 8 + 4distance_km + 3stops. Because the rule is deterministic and linear, a small model should easily learn the mapping and allow us to reason about behavior (for example, that predictions should increase with distance when stops are fixed).

Train/validation split and the normalization rule After assigning the first 9 rows to training and the remaining 3 to validation, we compute the mean and standard deviation only on X_train. Those training statistics are then used to normalize both the training inputs and the validation inputs:

1. mean = X_train.mean(axis=0)
2. std = X_train.std(axis=0) + 1e-6
3. X_n = (X - mean) / std

Using training split statistics is essential. If you normalize validation (or test) data using its own mean/std, you leak information about that split’s distribution into model evaluation and risk giving misleading validation metrics. The tiny epsilon 1e-6 prevents division by zero for degenerate features.

Model output: a single linear unit The network ends with layers.Dense(1) without an activation. This is deliberate: regression targets are scalar, unbounded (or at least potentially negative/positive), and require a linear output to allow the network to predict any real value. Adding a nonlinear activation (for example, relu) on the final layer would clip the output and bias predictions—an easy, common mistake.

Loss and metric choices for regression The model is compiled with loss='mse' and metrics=['mae'].

1. MSE (mean squared error) as the training loss penalizes larger errors more heavily and is a standard choice for regression optimization.
2. MAE (mean absolute error) is a more interpretable metric for reporting model performance; it reports the average absolute difference between predictions and ground-truth targets in the same units as the target (here, minutes).

Do not use accuracy for regression tasks; accuracy is only meaningful for classification. Also avoid using classification losses like categoricalcrossentropy or binarycrossentropy for scalar regression.

Interpreting MAE: the units matter If the model reports val_mae = 0.7, that means the average absolute error is 0.7 minutes. Converting to seconds gives 0.7 * 60 = 42 seconds. Because MAE is in the same units as the target, it is straightforward to interpret and communicate to stakeholders.

Sanity checks and behavioral assertions After training and evaluating, the code performs two automated sanity checks:

assert float(val_mae) < 1.0 ensures the model’s average absolute error on the validation set is below 1 minute. Because the data follows a simple linear rule and the model has sufficient capacity, this is a reasonable expectation for the tiny dataset; the assertion will alert you if something went wrong.

A monotonicity check: create two inputs that differ only in distance (3 km vs 6 km) while keeping stops equal. Normalize them using the training mean/std and compare predictions pa and pb. We assert pb > pa because increasing distance, all else equal, should increase the predicted delivery time. Such domain-driven sanity checks are useful to catch implementation errors (for example, mixing up feature columns) even when numeric metrics look plausible.

Common pitfalls highlighted by this example A few concrete mistakes to avoid, shown by contrast with the correct pipeline above:

1. Do not add a nonlinear activation to the final regression layer (for example, Dense(1, activation='relu')), as that can clip or artificially constrain predictions.
2. Do not normalize the target values unless you carefully undo the transformation at inference time; normalizing inputs is the typical operation. If you do normalize targets, remember to invert the scaling when interpreting predictions.
3. Use regression-appropriate loss and metrics (MSE, MAE) rather than accuracy or classification losses.

Practical note about deterministic examples and randomness This snippet relies on small data and default optimizer randomness. For reproducible tiny examples, fix random seeds in the full experimental script. The provided code omits explicit seed-setting for brevity, but the key teaching points—the use of train statistics for normalization, a linear final unit for scalar regression, and MSE/MAE as loss/metric—remain unchanged regardless of initialization.

The print statements at the end show the validation MAE, the rounded validation predictions, and the two predictions used in the monotonicity check so you can inspect numeric results: valmae, valpreds, and sanitypapb.

Evaluating models and next steps

A reliable end-to-end workflow for supervised problems reduces to three disciplined steps: explicit preprocessing, a deliberately simple model, and careful evaluation on held-out data. Before you reach for a more complex architecture, verify that these fundamentals are correct and that a small baseline actually learns. Most stubborn errors in projects stem from mismatches between the task, the model outputs, and the loss, or from evaluating only on training data.

Match the task to the output activation, loss, and metric Always choose the final layer activation and loss together so they implement the intended learning objective.

1. Binary classification (single true/false label per sample): use a single output unit with sigmoid activation and binary_crossentropy loss. Report accuracy for coarse checks, but prefer precision/recall or AUC when class imbalance matters.
2. Multiclass (exactly one class per sample, from K classes): use K outputs with softmax activation and categoricalcrossentropy loss when your targets are one-hot vectors. If you feed integer class indices instead of one-hot vectors, use sparsecategorical_crossentropy so the loss interprets indices correctly.
3. Multilabel (each sample may belong to zero, one, or many classes independently): use K outputs with sigmoid activation (one probability per class) and binary_crossentropy applied per output. Treat each output as an independent binary problem; do not use softmax here.
4. Regression (predicting a continuous scalar): use a linear output (no activation) and a regression loss such as meansquarederror (MSE) or meanabsoluteerror (MAE). Do not add an activation like relu on the final unit — that will clip predictions and bias the learner.

Common mismatches to avoid: applying softmax with categoricalcrossentropy to binary targets, feeding integer labels to categoricalcrossentropy without using sparse categorical loss, or using accuracy as the primary metric for regression problems. These combinations lead to confusing training signals or meaningless metrics.

Preprocessing must be explicit and computed from training data only Preprocessing is not a one-off convenience; it directly changes the problem the model sees. Compute any normalization, scaling, tokenization, or vocabulary mappings on the training split and apply the exact transformation to validation and test splits. Do not recompute normalization statistics on the validation set — doing so leaks information and invalidates your generalization estimate.

If you use a simple vectorizer (bag-of-words), remember that tokens not in the training vocabulary will be silently ignored by many vectorizers. That’s acceptable for small experiments but should be an explicit decision, not an accidental source of errors.

Validation splits, curves, and sanity checks Always set aside a validation split that you do not touch during hyperparameter tuning or early development. The validation set estimates generalization: track validation loss and validation metrics each epoch and inspect their curves. Typical patterns to watch for are:

1. Converging training loss with diverging or rising validation loss indicates overfitting.
2. Training and validation losses that both plateau far above acceptable levels indicate underfitting or an optimization problem.
3. Validation metric improving while validation loss worsens often signals a mismatch between the training objective and the reported metric; the metric may not reflect improvements the loss optimizes for.

Beyond curves, perform quick sanity checks on model outputs before extensive experimentation. Useful checks include:

1. Prediction range: sigmoid outputs should lie in [0, 1]; softmax outputs for a sample should sum to 1 (within floating-point tolerance). Regression outputs should plausibly match the target scale.
2. Monotonic or known-behavior tests: if a feature is known to increase the target monotonically, verify that higher feature values produce higher average predictions.
3. Class balance baseline: check that a trivial baseline (predict majority class or mean target) is computed and recorded — your model should beat it. If a sophisticated model does worse than a simple baseline, the implementation likely has a bug.

Operational checklist before extensive experiments Treat the following as a short pre-flight checklist you run before launching long training runs or complex architectures:

Preprocessing verified: confirm tokenization, vocabulary, normalization, and train-derived statistics are being applied consistently to validation and test data.

Output and loss matched to task: ensure the final layer activation and chosen loss implement the intended task (refer to the mapping above).

Appropriate metric: pick a metric that reflects the task’s priorities (accuracy, AUC, precision/recall, MSE/MAE).

Validation split held out: confirm you have a held-out validation set and that it is not used to compute preprocessing statistics or for frequent manual tuning without proper resampling.

Quick sanity checks: verify prediction ranges, softmax normalization, baseline performance, and monotonic relations where applicable.

Pitfalls that lead to wasted time Do not report only training metrics. Without validation results, you have no estimate of generalization and will be blind to overfitting. Similarly, avoid subtle output/loss mismatches: a model that minimizes categorical_crossentropy on binary labels will behave strangely and produce misleading metrics. When demonstrating tiny reproducible examples, fix random seeds; otherwise, hidden randomness can make results nondeterministic and debugging harder.

What to inspect during and after training Inspect both the numeric final metrics and the training history plots. Validation loss and validation metric curves carry complementary information: loss evaluates the optimization objective while metrics reflect task performance. Look for consistent improvements on validation data rather than just training. For regression, report MAE or MSE on validation and test sets; for classification, prefer metrics appropriate to class balance and business goals.

Ready to study deeper theory Once a simple, well-preprocessed baseline matches task-to-output mappings, shows meaningful validation improvements above the trivial baseline, and passes the sanity checks, you are ready to study deeper modeling choices: regularization, capacity control, alternative optimizers, architectural motifs, and the statistical theory behind generalization. The practical discipline of explicit preprocessing, careful output/loss selection, and rigorous validation provides the foundation on which more advanced methods become useful rather than brittle.