UAV-MapVQA

UAV-to-map registration

The frames come with an approximate GPS position but no camera pose. Dense matching against satellite imagery gives each one a homography to the ground, so every pixel has a latitude and longitude, and every map entity can be projected into the image.

One row per frame

The unified frame table, frames.csv, records each step. Read it left to right: what the drone gave us, how it was registered, the footprint that came out, what follows from the footprint, and what SAM3 found in the image.

aGNSS prior

The image and the position the drone logged. Altitude references differ by source.

  • image_path
  • width
  • height
  • datetime
  • gnss_lat
  • gnss_lon
  • alt_m
  • alt_ref

bDense matching

RoMa matches the frame to satellite imagery around the prior; RANSAC fits a homography.

  • reg_method
  • reg_score
  • reg_status

cFootprint

The four image corners on the ground: tl is pixel (0,0), then clockwise.

  • tl_lat
  • tl_lon
  • tr_…
  • br_…
  • bl_…
  • center_lat
  • center_lon

dDerived geometry

Heading of the image's up direction, edge lengths, area and ground resolution.

  • heading_deg
  • footprint_w_m
  • footprint_h_m
  • footprint_area_m2
  • gsd_m

eDetections

SAM3 buildings and cars, kept only if there are at least four buildings.

  • n_building
  • n_car
  • sam3_json
  • camera
  • fx_px

Drape a frame over the ground

Hand-checked registrations from each source, chosen to cover different scenes. The UAV image is warped onto its four registered corners: fade it against the satellite basemap to check the fit, switch to OpenStreetMap to see the map prior it produces, and compare the GNSS position with where the image landed.

How good is it?

Six annotators georeferenced a stratified sample of 1,440 benchmark frames by hand in QGIS with ground control points. Comparing the manual and automatic transformations on a 64 × 64 grid per image gives a median error of 1.9 m.

Grid points within each error threshold

automatic vs manual registration, 1,440 images

Median25th pct75th pct95th pct
1.9 m1.3 m2.6 m5.4 m

A few metres is enough to place the map window and to estimate the heading. It is not enough on its own for instance labels in dense neighbourhoods, where residual error, roof–footprint parallax or relief displacement can move a projected footprint onto the building next door. That is why the next stage never trusts a projected polygon by itself.

Before this check, three coauthors replayed every flight with its projected map overlay and discarded frames with visible drift, a wrong homography or a misaligned basemap tile, on top of the automatic checks: too few inliers, or a non-convex, degenerate, over-skewed or wrongly sized footprint.

The satellite imagery and the UAV frames were captured on different dates, and many USGS flights are post-hurricane surveys. Construction and storm damage show up as real change, not registration error.

Registration across the pool

Two registration pipelines produce the corners. USGS frames use heading, altitude and position priors from the survey metadata; AirLock, VETRA and UAV-VisLoc use a rotation sweep against satellite tiles. Inlier counts are comparable within a pipeline, not across them.

Match inliers

RANSAC inliers of the winning RoMa match, log scale

GNSS position to footprint centre

distance in metres between the logged GNSS position and the registered footprint centre, log scale

From candidate frames to the pool

all four sources