QGIS Plugin for GeoAI

A QGIS plugin that brings the geoai models into dockable panels (Moondream VLM, segmentation training/inference, SamGeo) so you can keep QGIS as your main workspace while experimenting with GeoAI.

Quick Start

• Create a fresh conda env (conda create -n geo python=3.12) and install QGIS + deps (see below).
• Install the plugin (python install.py) from this repo.
• Restart QGIS → Plugins → Manage and Install Plugins... → enable GeoAI.
• Open a GeoAI toolbar panel and try the sample datasets below.

Video Tutorials

Check out this short video demo and full video tutorial on how to use the GeoAI plugin in QGIS.

Requirements

• QGIS 3.28 or later
• Python 3.10+ (conda recommended)
• PyTorch (CUDA if you want GPU acceleration)
• geoai and samgeo packages

Features

Each tool lives inside a dockable panel that can be attached to either side of the QGIS interface, so you can keep layers, maps, and models visible simultaneously.

Moondream Vision-Language Model Panel

• Caption: Generate descriptions of geospatial imagery (short, normal, or long)
• Query: Ask questions about images using natural language
• Detect: Detect and locate objects with bounding boxes
• Point: Locate specific objects with point markers

Segmentation Panel (Combined Training & Inference)

• Tab 1 - Create Training Data: Export GeoTIFF tiles from raster and vector data
• Tab 2 - Train Models: Train custom segmentation models (U-Net, DeepLabV3+, FPN, etc.)
• Tab 3 - Run Inference: Apply trained models to new imagery and vectorize results. Vector outputs can optionally be smoothed or simplified for immediate use in GIS workflows.

SamGeo Panel (Segment Anything Model)

• Model Tab: Load SAM models (SAM1, SAM2, or SAM3) with configurable backend and device settings
• Text Tab: Segment objects using text prompts (e.g., "tree", "building", "road")
• Interactive Tab: Segment using point prompts (foreground/background) or box prompts drawn on the map
• Batch Tab: Process multiple points interactively or from vector files/layers
• Output Tab: Save results as raster (GeoTIFF) or vector (GeoPackage, Shapefile) with optional regularization (orthogonalize polygons, filter by minimum area)

GPU Memory Management

• Clear GPU Memory: Release GPU memory and clear CUDA cache for all loaded models

Installation

1) Set up the environment

Use a clean conda env dedicated to QGIS—mixing with an existing QGIS install often breaks dependencies.

conda create -n geo python=3.12
conda activate geo

Install core geospatial deps first:

conda install -c conda-forge --strict-channel-priority gdal rasterio libnetcdf netcdf4
python -c "import rasterio; print('rasterio import successful')"

Install GeoAI:

conda install -c conda-forge geoai
python -c "import geoai; print('geoai import successful')"

Install QGIS:

conda install -c conda-forge qgis

Install SamGeo extras (PyPI is required for some parts):

pip install -U "segment-geospatial[samgeo3]" sam3
python -c "import samgeo; print('samgeo import successful')"

Windows + SAM3 often needs the PyTorch wheels from NVIDIA and PyPI:

conda create -n geo python=3.12
conda activate geo
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia
conda install -c conda-forge qgis segment-geospatial geoai
pip install "segment-geospatial[samgeo3]" triton-windows

2) Install the QGIS plugin

Option A — use the helper script (recommended):

git clone https://github.com/opengeos/geoai.git
cd geoai/qgis_plugin
python install.py

This links/copies the plugin into your active QGIS profile. Re-run after pulling updates. Remove with:

python install.py --remove

Option B — manual copy:

• Copy the qgis_plugin folder to your QGIS plugins directory:
  • Linux: ~/.local/share/QGIS/QGIS3/profiles/default/python/plugins/
  • Windows: C:\Users\<username>\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\
  • macOS: ~/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/

3) Enable in QGIS

Launch QGIS: conda run qgis

QGIS → Plugins → Manage and Install Plugins... → enable GeoAI. After updates, toggle the plugin off/on or restart QGIS to reload.

Usage

Moondream Vision-Language Model

Sample dataset: parking_lot.tif

Steps:

1. Click the Moondream button in the GeoAI toolbar (or GeoAI menu → Moondream VLM)
2. Load a Moondream model (default: vikhyatk/moondream2)
3. Select a raster layer or browse for an image file
4. Choose a mode:
   • Caption: Generate a description of the image
   • Query: Ask a question about the image
   • Detect: Detect objects by type (e.g., "building", "car")
   • Point: Locate specific objects
5. Click "Run"

6. Results are displayed and optionally added to the map. You can drag the panel to any side of QGIS to keep it out of the way while browsing results. Save the output table or vector layer if you want to reuse detections later.

   

Segmentation Panel (Create Data, Train, Inference)

Sample datasets:

• naip_rgb_train.tif
• naip_test.tif
• naip_train_buildings.geojson

Steps:

1. Download the sample datasets (links above) or prepare your own imagery/vector labels. Store them in a folder that is accessible to the conda environment.
2. Click the Segmentation button in the GeoAI toolbar (or GeoAI menu → Segmentation)

3. Use the tabs at the top of the panel to switch between:

   • Create Training Data: Select input raster and vector labels, configure tile size and stride, and export tiles to a directory.
   • Train Model: Select the images and labels directories, choose model architecture (U-Net, DeepLabV3+, etc.), configure training parameters, and start training.
   • Run Inference: Select input raster layer or file, specify the trained model path, configure inference parameters, run inference, and optionally vectorize the results.

   

   

   

SamGeo Panel (Segment Anything Model)

Sample dataset:

• uc_berkeley.tif
• wa_building_image.tif
• wa_building_centroids.geojson
• wa_building_bboxes.geojson

Steps:

1. Click the SamGeo button in the GeoAI toolbar (or GeoAI menu → SamGeo)

2. In the Model tab:

   • Select the SAM model version (SamGeo3/SAM3, SamGeo2/SAM2, or SamGeo/SAM1)
   • Configure backend (meta or transformers) and device (auto, cuda, cpu)
   • Click "Load Model" to initialize the model
   • Select a raster layer or browse for an image file and click "Set Image"

   

3. Choose a segmentation method:

   • Text Tab: Enter text prompts describing objects to segment (e.g., "tree, building")

     

   • Interactive Tab:

     • Click "Add Foreground Points" or "Add Background Points" and click on the map
     • Or click "Draw Box" and drag a rectangle on the map
     • Click "Segment by Points" or "Segment by Box"

     

   • Batch Tab: Add multiple points interactively or load from a vector file/layer

     

4. In the Output tab:

   • Select output format (Raster GeoTIFF, Vector GeoPackage, or Vector Shapefile)
   • For vector output, optionally enable regularization:
     • Check "Regularize polygons (orthogonalize)"
     • Set Epsilon (simplification tolerance) and Min Area (filter small polygons)
   • Click "Save Masks" to export results

   

Clear GPU Memory

Click the GPU button in the GeoAI toolbar to release GPU memory from all loaded models (Moondream, SamGeo, etc.) and clear CUDA cache. Use this frequently when switching between large models to prevent out-of-memory errors.

Supported Model Architectures (Segmentation)

The QGIS plugin supports any models supported by Pytorch Segmentation Models, including:

• U-Net
• U-Net++
• DeepLabV3
• DeepLabV3+
• FPN (Feature Pyramid Network)
• PSPNet
• LinkNet
• MANet
• PAN
• UperNet
• SegFormer
• DPT

Supported Encoders (Segmentation)

• ResNet (34, 50, 101, 152)
• EfficientNet (b0-b4)
• MobileNetV2
• VGG (16, 19)

Supported SAM Models (SamGeo)

• SamGeo3 (SAM3): Latest version with text prompts, point prompts, and box prompts
• SamGeo2 (SAM2): Improved version with better performance
• SamGeo (SAM1): Original Segment Anything Model

Troubleshooting

• Plugin missing after install: confirm the plugin folder exists in your QGIS profile path and that you restarted QGIS.
• GDAL/rasterio errors: verify you launched QGIS from the conda env (conda activate geo then qgis) so it picks up the same Python libs.
• CUDA OOM: use the GPU button to clear cache, lower batch sizes, or switch to CPU for smaller runs.
• Model download failures: check network/firewall, then retry loading models from the panel.

License

MIT License - see LICENSE for details.

Links

• GeoAI Documentation
• SamGeo Documentation
• GitHub Repository
• Report Issues