Provenance and QA

Provenance and QA

AbovePy provides built-in data quality validation and provenance metadata for survey/engineering deliverables.

Validation

Check for data quality issues before using or delivering data:

import abovepy

result = abovepy.search(county="Pike", product="dem_phase3")

warnings = result.validate()
for w in warnings:
    print(f"Warning: {w}")

What validate() checks:

• Mixed products in the result set
• Mixed STAC collections (different acquisition phases)
• Missing asset URLs (tiles that can't be downloaded)
• Coverage gaps (low tile density for the covered area)
• Missing acquisition date metadata

An empty list means no issues were detected.

Provenance Metadata

Generate source documentation for deliverables:

prov = result.provenance()

Returns a dict with:

Key	Example	Description
product	"dem_phase3"	Product key
display_name	"DEM Phase 3 (2ft)"	Human-readable name
source_program	"KyFromAbove"	Data source program
acquisition_period	"2022-2025"	When the data was collected
native_crs	"EPSG:3089"	Original coordinate reference system
resolution	"2ft"	Spatial resolution
format	"COG"	File format
tile_count	342	Number of tiles
estimated_size_mb	1710.0	Estimated download size
bbox	(-85.0, 38.0, ...)	Bounding box of results
query_params	{"county": "Pike"}	Original search parameters
asset_urls	["https://..."]	All tile download URLs
phases	["dem_phase3"]	Unique product phases in result

Export Provenance as JSON

import json

prov = result.provenance()
with open("provenance.json", "w") as f:
    json.dump(prov, f, indent=2, default=str)

CLI Provenance

abovepy search --county Franklin -p dem_phase3 --format provenance > provenance.json

Phase Comparison

Compare coverage between data phases:

phase2 = abovepy.search(county="Franklin", product="dem_phase2")
phase3 = abovepy.search(county="Franklin", product="dem_phase3")

overlap = phase3.compare(phase2)
print(f"Phase 2: {phase2.count} tiles")
print(f"Phase 3: {phase3.count} tiles")
print(f"Overlapping pairs: {len(overlap)}")