Deep Learning Models for Geospatial Intelligence: Practical Insights from Project Experimentation

The IBM-NASA open-source model, Prithvi-EO-2.0 for environmental mapping tasks (IBM).

Introduction

Geospatial intelligence (GEO.INT.) has rapidly evolved with the adoption of deep learning (DL), enabling automated understanding of satellite, aerial, and drone imagery at scale. From detecting vehicles and aircraft to segmenting buildings, roads, and terrain, DL models have become the backbone of modern geospatial analytics.

In practice, however, choosing the right model is not just about benchmark accuracy — it is about data availability, spatial resolution, object scale, and operational constraints. Through hands-on projects involving satellite imagery, tiling pipelines (e.g., TorchGeo, TerraTorch), and object detection workflows, I’ve seen how different architectures behave under real-world conditions.

Geospatial intelligence is currently undergoing significant transformation with the advent of geospatial ‘pre-trained’ foundation models for Earth observation and can function efficiently in Gen-AI tasks with minimal labeled data that I plan to discuss in a different blog. In this case , I’ll briefly walk through my personal experience during client pilots and projects on reference deep learning models that are still in use in geospatial intelligence focusing mainly on semantic segmentation and object detection tasks, focusing on:

• Primary function
• Training Methodology
• Advantages and Limitations
• Practical geospatial examples

Geospatial Semantic Segmentation Models

Semantic segmentation is essential for tasks where we need pixel-level understanding, such as mapping infrastructure, terrain classification, or disaster assessment.

Google Earth’s semantic segmentation with labeled image of the patches of Houston, Texas(USA)

1. U-Net

U-Net and its improved variant U-Net++) is one of the most widely used architectures in geospatial segmentation due to its simplicity and strong performance on high-resolution imagery.

At its core, U-Net follows an encoder–decoder architecture. The encoder extracts features through convolutional layers, while the decoder reconstructs the segmentation mask. The key innovation is the use of skip connections, which preserve spatial detail — critical when working with satellite imagery where object boundaries matter.

From my experience, U-Net performs extremely well in building footprint extraction or road segmentation, especially when working with limited labelled datasets. It is also very stable during training and relatively easy to tune. However, U-Net struggles when dealing with large spatial context, such as wide-area land cover classification. Since it relies heavily on local features, it may miss broader patterns unless large tiles or additional context are introduced.

Most Applicable use case: High-resolution segmentation with limited data e.g., buildings, roads and infrastructure.

Vessel Detection Masking for Unet34 Deep Learning Model Training

2. DeepLabV3+

DeepLabV3+ improves on traditional CNN segmentation by introducing Atrous (dilated) convolutions and the Atrous Spatial Pyramid Pooling (ASPP) module.

Architecture of DeepLabV3+ for semantic segmentation (Source:ResearchGate)

This allows the model to capture multi-scale spatial context without reducing resolution — something extremely valuable in geospatial data where objects can vary significantly in size.

In practice I’ve found DeepLabV3+ particularly effective for land cover classification and urban mapping, where understanding both small and large structures is important. It also tends to produce cleaner segmentation boundaries compared to simpler models. The trade-off is that DeepLabV3+ is computationally heavier and requires larger labeled datasets to reach its full potential. Training can also be more sensitive to hyperparameters.

Most Applicable use case: Multi-scale segmentation in urban areas, mixed terrain, and large AOIs

Segmentation Ground Truth masking on geospatial images Land vs Water Mass for DeepLabV3+ model training

3. SegFormer

SegFormer represents a newer generation of models that leverage transformer-based architectures instead of purely convolutional networks.

Unlike CNNs, transformers capture global relationships across the image, making SegFormer particularly powerful for large-scale satellite imagery. Its lightweight decoder also keeps inference relatively efficient.

In real-world applications, SegFormer shines when working with heterogeneous landscapes, such as combining urban, rural, and natural terrain in a single model. It generalizes well across regions, especially when pretrained on large datasets while it requires substantial training data and GPU resources, and the training process is less straightforward compared to U-Net.

Most Applicable use case: Large-area segmentation with diverse terrain and strong generalization needs

Geospatial Object Detection Models

Object detection focuses on identifying and localizing discrete objects using bounding boxes — critical for ISR, monitoring, and asset tracking.

Image: Automating Aircraft object Detections with Geospatial AI (ArcGIS)

4. YOLOv8

YOLOv8 is a single-stage detector designed for speed and efficiency, making it ideal for real-time applications. It processes the image in a single pass, predicting bounding boxes and classes simultaneously. This makes it extremely fast and easy to deploy.

In geospatial applications, I’ve used YOLOv8 for vehicle and asset detection, especially in scenarios where near real-time inference is required (e.g., drone imagery or streaming data). The main limitation is its performance on very small or densely packed objects, which can be common in high-resolution satellite imagery.

Most Applicable use case: Real-time detection for vehicles, ships and dynamic monitoring.

Image: Yolov8 Model Validation Batch Prediction Labeling (Source: Author)

5. Faster R-CNN

Faster R-CNN is a two-stage detector, where the first stage proposes candidate regions and the second stage classifies them. This architecture delivers high accuracy, especially for small objects, which is crucial in satellite imagery (e.g., vehicles, aircraft).

Personally from my experimentation, I have found Faster R-CNN to be very reliable when accuracy is a priority in terms of performance, such as in aircraft detection in satellite imagery. However, it comes with a significant drawback: slow inference, making it unsuitable for real-time applications.

Most applicable use case: High-precision detection in small objects, critical assets.

6. RetinaNet

Although primarily an object detection model, RetinaNet can also be used in geospatial workflows where dense object localization behaves similarly to segmentation tasks.

Its core innovation is the Focal Loss, which addresses class imbalance which is actually a common issue in satellite imagery where background dominates. In my experience, RetinaNet performs well for dense object detection scenarios, such as identifying multiple small structures or trees. It bridges the gap between segmentation and detection when objects are numerous and tightly packed. However, it is still fundamentally a detection model, so it does not provide true pixel-level segmentation unless combined with additional techniques.

Most applicable use case: Dense object detection considering objects such as tree crowns andsmall infrastructure elements

7. EfficientDet

EfficientDet introduces the BiFPN (Bidirectional Feature Pyramid Network), which efficiently fuses features across multiple scales.This makes it particularly effective for multi-scale object detection, where objects vary in size — a common challenge in geospatial data.

From my experience, EfficientDet offers a strong balance between accuracy and computational efficiency, especially when compared to heavier models like Faster R-CNN. However, it requires careful tuning and is slightly more complex to set up, particularly when optimizing across different scales.

EfficientDet uses EfficientNet as the backbone network and a newly proposed BiFPN feature network.

EfficientDet architecture (source: GoogleReseach)

Most applicable use case: Multi-scale detection in complex environments e.g. urban + rural scenes.

8. DeepForest

I am adding also here one more interesting deep learning model which is mainly focused on deep forestation and vegetation pattern detection DeepForest is a specialized model built on RetinaNet, designed specifically for tree crown detection in aerial imagery.

What makes it unique is that it comes pretrained, allowing users to deploy it quickly without extensive training.

In practice, it works exceptionally well for environmental monitoring, forestry analysis, and vegetation mapping. However, its specialization also limits its use — it does not generalize well beyond tree detection tasks.

Most Applicable use case: Forestry, vegetation monitoring, canopy analysis

Tree-Crown Detection of the pre- and transfer-trained example derived from RGB imagery in the studied temperate, deciduous forest (Source: MDPI )

Conclusion

There is no “one-size-fits-all” model in geospatial intelligence. Each architecture brings different strengths depending on:

• Spatial resolution
• Object size and density
• Dataset size
• Operational constraints (Real-time vs Static)

From practical experience, the most effective approach is not choosing a single model, but combining them into a pipeline:

• Semantic segmentation for context understanding
• Detection for object-level insights
• Temporal models (if applicable) for change and prediction

As geospatial AI continues to evolve, we have also experiences the rise of foundation models and agentic AI systems, which will further transform how these models are orchestrated into end-to-end intelligence pipelines as we will explain in a following article.

Disclaimer: Part of the views expressed here are those of the article’s author and may or may not represent the views of IBM Corporation. Part of the content (including images) on the blog is copyright from IBM or 3rd party vendors and all rights are reserved — but, unless otherwise noted- under IBM Corporation.