🚀 Mastering Semantic Segmentation: My Deep Learning Journey in a Kaggle Competition

In this article, I share my experience working on a Kaggle semantic segmentation challenge as part of a deep learning project. This hands-on task pushed my understanding of computer vision, custom dataset creation, and model performance tuning — core skills every aspiring AI practitioner should master.

🧠 What is Semantic Segmentation?

Semantic segmentation is the process of classifying each pixel in an image into a predefined class. It’s fundamental in applications such as medical imaging, autonomous driving, and satellite image analysis. Unlike simple image classification, segmentation requires detailed spatial predictions.

📁 Dataset Preparation: Building a Custom Dataset Loader

One of the first tasks was creating a custom Tensorflow dataset class. This class was responsible for loading image-mask pairs, resizing them, normalizing the data, and augmenting them for improved generalization. Here’s a simplified version of the dataset loading logic:

[embed]

This setup gave me the flexibility to experiment with different image sizes and data augmentations.

🧪 Visualizing the Dataset

Before training any deep learning model, it’s crucial to verify your data visually. For semantic segmentation, this means ensuring that the input images and their masks are correctly aligned and that the masks are properly encoded.

To make this process effective, I wrote a single utility that:

1. Converts grayscale label masks to color using a class-wise id2color map.
2. Overlays the color mask onto the original image.
3. Displays the original image, the raw or colorized mask, and the overlay in one row.

Here’s the full visualization code:

[embed]

⚙️ DeepLabV3+ with EfficientNetB4 for Semantic Segmentation

To build a high-performing semantic segmentation model, I implemented a custom version of DeepLabV3+ using EfficientNetB4 as the backbone. This architecture is well-regarded for balancing efficiency and accuracy, making it ideal for tasks requiring precise pixel-level predictions.

The core design leverages two key components:

1. A powerful encoder (EfficientNetB4) for extracting hierarchical features.

2. The Atrous Spatial Pyramid Pooling (ASPP) module to capture multi-scale context before decoding.

The full model is defined as follows:

[embed]

At the heart of DeepLabV3+ lies the ASPP block, which uses parallel convolutions with different dilation rates to effectively capture objects at multiple scales. Here’s the implementation of that module:

[embed]

Together, these two components create a powerful encoder-decoder architecture capable of segmenting fine structures while maintaining global context, crucial for tasks like road scene understanding, medical imaging, and satellite image analysis.

🎯 Loss Function and Evaluation Metric

For segmentation tasks, especially in cases of class imbalance or multi-class predictions, using robust loss functions and metrics is key to training reliable models. I implemented a custom Dice + Focal loss combination for optimization, and used Mean IoU (Intersection over Union) as the evaluation metric.

🔥 Custom Loss: Dice Coefficient + Categorical Focal Loss

To balance between region-based overlap (Dice) and hard-to-classify pixel weighting (Focal Loss), I combined both into a hybrid loss. This improves performance, especially in datasets with underrepresented classes.

[embed]

🧠 Why this works:

• Dice Loss focuses on the overlap between prediction and ground truth.
• Focal Loss adds weight to difficult or minority class pixels.
• Combined, they guide the model to both generalize and focus on edge cases.

📊 Evaluation Metric: Mean IoU (Intersection over Union)

To evaluate the model’s segmentation quality, I used Mean IoU, a standard metric in semantic segmentation that compares predicted masks with ground truth across all classes.

[embed]

🚀 Training the Model

To train the DeepLabV3+ model, I designed a robust training loop that includes learning rate scheduling, model checkpointing, and early stopping to prevent overfitting and optimize convergence.

⚙️ Key Components of the Training Setup

• Learning Rate Scheduler: I used ExponentialDecay to gradually reduce the learning rate during training, improving stability.
• ModelCheckpoint: Saves the best-performing model based on validation Dice score.
• EarlyStopping: Halts training if performance doesn’t improve after a few epochs, avoiding overfitting.

📦 Complete Training Code

[embed]

📌 Notes:

• Using val_dice_score as the primary validation metric ensures the model prioritizes segmentation performance over simple accuracy.
• The EarlyStopping patience of 6 means the training halts if no validation improvement is seen for 6 consecutive epochs.
• The ModelCheckpoint ensures that the best-performing model is retained for later inference or deployment.

📈 Plotting Training Curves

To better understand how the model learns over time, I visualized both Mean IoU and Dice Loss curves for the training and validation sets.

These plots are critical for diagnosing overfitting, underfitting, or potential training instabilities.

✅ Mean IoU Curve

• The Mean IoU steadily increases on both training and validation sets, indicating that the model is learning to segment more accurately over time.
• Around epoch 20, a temporary dip in validation IoU suggests a learning fluctuation, possibly due to batch variation or early learning rate decay.
• Beyond epoch 25, the validation IoU stabilizes and tracks closely with the training IoU, which is a strong sign of good generalization.

Mean IoU

✅ Dice Loss Curve

• Dice loss shows a clear downward trend, which is ideal. Lower loss indicates better overlap between predicted and true segmentation masks.
• The validation loss closely follows the training loss early on, then diverges slightly around epoch 25, common in deep learning due to validation variability.
• The flat trend after 30 epochs suggests the model has converged, and further training may yield diminishing returns.

Dice Loss

🧠 Takeaways:

• The training behavior is stable and well-regularized, with no signs of overfitting.
• The combination of early stopping and model checkpointing helped prevent performance degradation.
• These visualizations gave me confidence in model performance and were used to determine the optimal epoch for final model export.

🧪 Inference

After training, I ran inference on the test dataset using the trained model weights. Predictions were thresholded and converted into binary masks.

[embed]

Preparing Kaggle Submission

Kaggle required submission in Run-Length Encoding (RLE) format. I converted each predicted mask to RLE and wrote it into a submission CSV file.

[embed]

📊 Final Results & Score

I submitted the CSV to Kaggle and received a Dice score of 88.626% (you can insert your real score here). The score was reflective of all the effort and optimizations applied throughout the pipeline.

Kaggle Leaderboard

✨ Lessons Learned

1. Data visualization is critical: A few simple plots can save hours of debugging.
2. Custom dataset creation teaches flexibility: Especially useful when datasets aren’t perfectly packaged.
3. Loss functions matter: Dice loss helped overcome class imbalance.
4. Training curves are gold: They provide immediate feedback on what’s going wrong.
5. Practice > Theory: Nothing beats learning through building.

🔚 Conclusion

This project gave me valuable experience in solving real-world semantic segmentation problems. It strengthened my skills in dataset handling, model architecture design, metric tracking, and Kaggle competition submission workflows.

If you’re venturing into computer vision or semantic segmentation, I highly recommend trying out a project like this. The learning curve is steep, but the view from the top is worth it!