Instance Segmentation with GeoAI¶

This notebook demonstrates how to use the new instance segmentation functionality in GeoAI for training models and running inference on geospatial data.

Overview¶

Instance segmentation combines object detection and semantic segmentation to identify and segment individual objects in images. This is particularly useful for:

• Building detection and segmentation
• Vehicle counting and tracking
• Infrastructure mapping
• Object delineation in satellite imagery

New Functions¶

GeoAI now provides clear wrapper functions for instance segmentation:

Training¶

• train_instance_segmentation_model() - Train a Mask R-CNN model

Inference¶

• instance_segmentation() - Run inference on a single GeoTIFF
• instance_segmentation_batch() - Run inference on multiple GeoTIFFs

Model Creation¶

• get_instance_segmentation_model() - Create a Mask R-CNN model with custom parameters

Install packages¶

To use the new functionality, ensure the required packages are installed.

In [ ]:

# %pip install geoai-py

# %pip install geoai-py

Import libraries¶

In [ ]:

import geoai
import os
from pathlib import Path

import geoai import os from pathlib import Path

Setup¶

First, let's check our environment and set up paths.

In [ ]:

# Check if CUDA is available
device = geoai.get_device()
print(f"Using device: {device}")

# Set up paths
out_folder = "instance_segmentation_buildings"
models_dir = Path(out_folder) / "models"
output_dir = Path(out_folder) / "output"

# Create directories if they don't exist
models_dir.mkdir(parents=True, exist_ok=True)
output_dir.mkdir(parents=True, exist_ok=True)

print(f"Working directory: {out_folder}")
print(f"Models will be saved to: {models_dir}")
print(f"Output will be saved to: {output_dir}")

# Check if CUDA is available device = geoai.get_device() print(f"Using device: {device}") # Set up paths out_folder = "instance_segmentation_buildings" models_dir = Path(out_folder) / "models" output_dir = Path(out_folder) / "output" # Create directories if they don't exist models_dir.mkdir(parents=True, exist_ok=True) output_dir.mkdir(parents=True, exist_ok=True) print(f"Working directory: {out_folder}") print(f"Models will be saved to: {models_dir}") print(f"Output will be saved to: {output_dir}")

Download sample data¶

We'll use the same dataset as the semantic segmentation example for consistency.

In [ ]:

train_raster_url = (
    "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip_rgb_train.tif"
)
train_vector_url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip_train_buildings.geojson"
test_raster_url = (
    "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip_test.tif"
)

train_raster_url = ( "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip_rgb_train.tif" ) train_vector_url = "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip_train_buildings.geojson" test_raster_url = ( "https://huggingface.co/datasets/giswqs/geospatial/resolve/main/naip_test.tif" )

In [ ]:

train_raster_path = geoai.download_file(train_raster_url)
train_vector_path = geoai.download_file(train_vector_url)
test_raster_path = geoai.download_file(test_raster_url)

print(f"Downloaded training raster: {train_raster_path}")
print(f"Downloaded training vector: {train_vector_path}")
print(f"Downloaded test raster: {test_raster_path}")

train_raster_path = geoai.download_file(train_raster_url) train_vector_path = geoai.download_file(train_vector_url) test_raster_path = geoai.download_file(test_raster_url) print(f"Downloaded training raster: {train_raster_path}") print(f"Downloaded training vector: {train_vector_path}") print(f"Downloaded test raster: {test_raster_path}")

Visualize sample data¶

In [ ]:

geoai.get_raster_info(train_raster_path)

geoai.get_raster_info(train_raster_path)

In [ ]:

geoai.view_vector_interactive(train_vector_path, tiles=train_raster_url)

geoai.view_vector_interactive(train_vector_path, tiles=train_raster_url)

In [ ]:

geoai.view_raster(test_raster_url)

geoai.view_raster(test_raster_url)

Create training data¶

We'll create training tiles for instance segmentation. Note that for instance segmentation, we need to ensure each building instance has a unique pixel value in the label.

In [ ]:

# Create training tiles
tiles = geoai.export_geotiff_tiles(
    in_raster=train_raster_path,
    out_folder=out_folder,
    in_class_data=train_vector_path,
    tile_size=512,
    stride=256,
    buffer_radius=0,
)

print(f"Created {len(tiles)} training tiles")
print(f"Images saved to: {out_folder}/images")
print(f"Labels saved to: {out_folder}/labels")

# Create training tiles tiles = geoai.export_geotiff_tiles( in_raster=train_raster_path, out_folder=out_folder, in_class_data=train_vector_path, tile_size=512, stride=256, buffer_radius=0, ) print(f"Created {len(tiles)} training tiles") print(f"Images saved to: {out_folder}/images") print(f"Labels saved to: {out_folder}/labels")

1. Model Creation¶

Let's create an instance segmentation model with custom parameters.

In [ ]:

# Create a model for binary segmentation (background + buildings)
model = geoai.get_instance_segmentation_model(
    num_classes=2,  # background + buildings
    num_channels=3,  # RGB channels
    pretrained=True,
)

print(f"Model created with {sum(p.numel() for p in model.parameters())} parameters")
print(f"Model device: {next(model.parameters()).device}")
print(f"Model type: {type(model)}")

# Create a model for binary segmentation (background + buildings) model = geoai.get_instance_segmentation_model( num_classes=2, # background + buildings num_channels=3, # RGB channels pretrained=True, ) print(f"Model created with {sum(p.numel() for p in model.parameters())} parameters") print(f"Model device: {next(model.parameters()).device}") print(f"Model type: {type(model)}")

2. Training Instance Segmentation Model¶

Now let's train the instance segmentation model using our prepared data.

In [ ]:

# Training configuration
training_config = {
    "images_dir": f"{out_folder}/images",
    "labels_dir": f"{out_folder}/labels",
    "output_dir": str(models_dir),
    "num_classes": 2,  # background + buildings
    "num_channels": 3,  # RGB
    "batch_size": 2,  # Small batch size for demo
    "num_epochs": 20,  # Few epochs for demo
    "learning_rate": 0.005,
    "val_split": 0.2,
    "visualize": True,
    "device": device,
    "verbose": True,
}

print("Training configuration:")
for key, value in training_config.items():
    print(f"  {key}: {value}")

# Training configuration training_config = { "images_dir": f"{out_folder}/images", "labels_dir": f"{out_folder}/labels", "output_dir": str(models_dir), "num_classes": 2, # background + buildings "num_channels": 3, # RGB "batch_size": 2, # Small batch size for demo "num_epochs": 20, # Few epochs for demo "learning_rate": 0.005, "val_split": 0.2, "visualize": True, "device": device, "verbose": True, } print("Training configuration:") for key, value in training_config.items(): print(f" {key}: {value}")

In [ ]:

# Train the model
print("Starting training...")
geoai.train_instance_segmentation_model(**training_config)
print("Training completed!")

# Train the model print("Starting training...") geoai.train_instance_segmentation_model(**training_config) print("Training completed!")

In [ ]:

geoai.plot_performance_metrics(
    history_path=str(models_dir / "training_history.pth"),
    figsize=(15, 5),
    verbose=True,
)

geoai.plot_performance_metrics( history_path=str(models_dir / "training_history.pth"), figsize=(15, 5), verbose=True, )

3. Running Inference¶

Once we have a trained model, we can run inference on new images.

In [ ]:

# Define paths
model_path = str(models_dir / "best_model.pth")
output_path = str(output_dir / "instance_segmentation_result.tif")

# Check if model exists
if os.path.exists(model_path):
    print(f"Model found at: {model_path}")
else:
    print(f"Model not found at: {model_path}")
    print("Please ensure training completed successfully")

# Define paths model_path = str(models_dir / "best_model.pth") output_path = str(output_dir / "instance_segmentation_result.tif") # Check if model exists if os.path.exists(model_path): print(f"Model found at: {model_path}") else: print(f"Model not found at: {model_path}") print("Please ensure training completed successfully")

In [ ]:

# Single image inference with improved parameters
inference_config = {
    "input_path": test_raster_path,
    "output_path": output_path,
    "model_path": model_path,
    "window_size": 512,
    "overlap": 128,  # Reduced overlap to minimize artifacts
    "confidence_threshold": 0.5,
    "batch_size": 2,
    "num_channels": 3,
    "num_classes": 2,
    "device": device,
}

print("Running inference with sliding window processing...")
result_path, inference_time = geoai.instance_segmentation(**inference_config)
print(f"Inference completed in {inference_time:.2f} seconds!")

# Single image inference with improved parameters inference_config = { "input_path": test_raster_path, "output_path": output_path, "model_path": model_path, "window_size": 512, "overlap": 128, # Reduced overlap to minimize artifacts "confidence_threshold": 0.5, "batch_size": 2, "num_channels": 3, "num_classes": 2, "device": device, } print("Running inference with sliding window processing...") result_path, inference_time = geoai.instance_segmentation(**inference_config) print(f"Inference completed in {inference_time:.2f} seconds!")

In [ ]:

geoai.view_raster(
    output_path, nodata=0, colormap="tab20", opacity=0.7, basemap=test_raster_url
)

geoai.view_raster( output_path, nodata=0, colormap="tab20", opacity=0.7, basemap=test_raster_url )

4. Vectorize and Visualize Results¶

Convert the predicted mask to vector format for better visualization and analysis.

In [ ]:

output_vector_path = "building_predictions.geojson"
gdf = geoai.orthogonalize(output_path, output_vector_path, epsilon=2)

output_vector_path = "building_predictions.geojson" gdf = geoai.orthogonalize(output_path, output_vector_path, epsilon=2)

In [ ]:

# Add geometric properties
gdf_props = geoai.add_geometric_properties(gdf, area_unit="m2", length_unit="m")

# Add geometric properties gdf_props = geoai.add_geometric_properties(gdf, area_unit="m2", length_unit="m")

In [ ]:

# Interactive visualization with area information
geoai.view_vector_interactive(gdf_props, column="area_m2", tiles=test_raster_url)

# Interactive visualization with area information geoai.view_vector_interactive(gdf_props, column="area_m2", tiles=test_raster_url)

In [ ]:

# Filter out small buildings and visualize
gdf_filtered = gdf_props[(gdf_props["area_m2"] > 50)]
print(f"Buildings after filtering (area > 50 m²): {len(gdf_filtered)}")

geoai.view_vector_interactive(gdf_filtered, column="area_m2", tiles=test_raster_url)

# Filter out small buildings and visualize gdf_filtered = gdf_props[(gdf_props["area_m2"] > 50)] print(f"Buildings after filtering (area > 50 m²): {len(gdf_filtered)}") geoai.view_vector_interactive(gdf_filtered, column="area_m2", tiles=test_raster_url)

In [ ]:

# Create a split map comparison
geoai.create_split_map(
    left_layer=gdf_filtered,
    right_layer=test_raster_url,
    left_args={"style": {"color": "red", "fillOpacity": 0.3}},
    basemap=test_raster_url,
)

# Create a split map comparison geoai.create_split_map( left_layer=gdf_filtered, right_layer=test_raster_url, left_args={"style": {"color": "red", "fillOpacity": 0.3}}, basemap=test_raster_url, )

8. Key Parameters Guide¶

Here are the key parameters you can adjust:

Summary¶

This notebook demonstrated the new instance segmentation functionality in GeoAI:

1. Model Creation: Created Mask R-CNN models with custom parameters
2. Training: Trained an instance segmentation model on building data
3. Inference: Ran inference on test images
4. Visualization: Converted results to vectors and visualized them
5. Analysis: Compared with semantic segmentation approaches

The new functions provide a cleaner API while maintaining backward compatibility with existing code. They're built on top of the robust MaskRCNN implementation already present in GeoAI.

Available Functions:¶

• geoai.train_instance_segmentation_model() - Train Mask R-CNN models
• geoai.instance_segmentation() - Single image inference
• geoai.instance_segmentation_batch() - Batch processing
• geoai.get_instance_segmentation_model() - Create custom models