Code is now cheap. Good, reproducible tools still take effort—but when others can use them, they save far more time than they cost.

Code is becoming cheap to produce. A useful tool is not. It still takes care to turn an idea into something another person can install, understand, trust, restart after a failure, and adapt to their own work. That effort matters because a reproducible tool can save its users far more time than it took to build.

TerraVault is my attempt to do that for open satellite data. It is a lightweight Python toolkit for finding imagery through STAC catalogues, downloading the assets a project needs, and keeping a durable record of what has already been ingested. Run it again and it continues from its saved state instead of starting over.

The immediate use case is Sentinel imagery from the Copernicus Data Space Ecosystem, with Switzerland configured out of the box. TerraVault can filter scenes by area, time, and cloud cover; maintain rolling or historical ingests; organise imagery in predictable partitions; and index local raster pieces in DuckDB for spatial and temporal queries.

Getting the bytes onto disk is only the beginning. TerraVault can extract an area of interest into a stitched, multiband Cloud Optimized GeoTIFF without loading a country-scale dataset into memory. It also connects complete Sentinel-2 products to FORCE for atmospheric correction, quality information, tiling, and NDVI-oriented visual checks.

The less visible features are the ones I value most: bounded concurrency, retries with back-off, resumable transfers, checksums, explicit manifests, progress records, and atomic publication of completed outputs. They are not glamorous, but they are what make a long-running data workflow usable rather than merely demonstrable.

Reproducibility here does not mean promising that every machine will produce magic with one command. Credentials, data-provider limits, native runtimes, projections, storage, and input-product levels all matter. TerraVault tries to make those contracts explicit, record enough provenance to inspect a result, and fail honestly when a requirement has not been met.

Since the original introduction, TerraVault has grown into a monthly vegetation-processing toolkit. This update reflects the public repository through 5 October 2026. Separate routes create cloud-screened monthly medians from direct Copernicus Sentinel-2 L2A assets, completed FORCE surface-reflectance and quality products, or published swissEO orbit mosaics. Shared calculations produce NDVI, GNDVI, NDRE, and the project’s NIR/SWIR definition of NDWI from physical reflectance, while retaining each route’s source and masking contracts.

The completed July 2024 pilot compared Copernicus L2A and swissEO across the exact Swiss boundary on the same 20 m grid in the Swiss national coordinate system. It used 65 Copernicus scenes and 16 swissEO orbit mosaics. Both products show broadly consistent vegetation patterns, but their missing-data patterns differ. Clouds, terrain shadows, snow, acquisition coverage, and mask definitions can all remove pixels; a gap in one map is not evidence of vegetation change. The swissEO-derived products carry the attribution: © swisstopo, Contains modified Copernicus Sentinel data 2024.

The pilot measured about 2 hours 36 minutes for the Copernicus monthly command and 1 hour 11 minutes for swissEO. These are operational observations, not a controlled speed comparison: Copernicus started from already downloaded local assets, while swissEO read remote, already assembled orbit mosaics. The measurements predate the latest parallel-processing changes and should not be treated as benchmarks for the current version.

The newest changes reduce repeated work after a failed run. The Copernicus route stages inputs locally, prefers the original CDSE assets with an archive fallback, and checkpoints completed tile medians. The swissEO route retries downloads, processes aligned orbit mosaics and band medians in parallel, and retains checkpoints. The 10 m rerun workflow adds storage checks, local scratch space, comparison before publication, and copying of completed products to the network share. Support for this rerun is not itself a claim that the new 10 m results have finished.

swissEO v200 is now an explicit option alongside the default v100 route. It reads separate band assets, applies the v200 reflectance scale and offset, and uses its cloud and terrain quality layers. Distinct output names let the versions coexist. One scientific caveat matters: the current v200 terrain-mask policy excludes open water because that mask has no valid values over lakes and rivers. Comparisons need to account for that coverage difference.

Native FORCE processing has now been exercised at national scale. The July evaluation completed Level-2 atmospheric and quality processing for all 83 Sentinel-2 L1C tile-scenes. Its national monthly median did not finish: full-grid intermediates exhausted storage, with the workspace exceeding 434 GB near failure. FORCE was therefore excluded from the completed two-source comparison. The successful scene processing and the unfinished national aggregation are separate outcomes; the monthly route still needs a more economical aggregation strategy.

The latest FORCE pipeline processes several complete SAFE products concurrently while later downloads continue, with independent scene locks and interruption records. It batches output validation and skips unchanged completed scenes without starting Docker. Optional clipping to the region reduces output, but changes reflectance near the boundary because FORCE applies the clip before adjacency correction. That choice belongs in the processing record. This pipeline remains a bounded run over a selected region and period, separate from continuous rolling ingestion.

TerraVault is open source under the MIT license. If you work with satellite imagery, STAC, geospatial research infrastructure, or reproducible data pipelines, I would be glad for you to try it, inspect the design, open an issue, or contribute. The goal is not more code. It is less repeated effort for the next person facing the same problem.