Differentiable CREST on CAMELS:
benchmark & comparison experiments

2026-07-16

673 basins · daily · independent test decade same-harness model comparison optimizer verification (Adam vs SCE-UA) regionalization (dPL / PUB)
One line: a generic differentiable harness takes CREST into the classic conceptual-model band (NSE 0.61, KGE at HBV parity), the optimum is verified by two independent optimizers, and dPL adds the ungauged capability the classic models don't have.

Navigate: ← → or the corner buttons.

Benchmark design — fairness first

ItemSetting
Basins / scale673 CAMELS basins, lumped (basin-mean forcing), daily
ForcingMaurer (primary, Newman convention) + Daymet (sensitivity); Oudin PET
Splittrain 1999-10→2008-09 · test 1989-10→1999-09 (untouched decade)
CREST configSnow-17(9) + water balance(5) + reservoirs(5) + triangular UH(1) = 20 params/basin, direct sigmoid-bounded, Adam on (1−NSE), best-TRAIN-iterate selection
Comparison modelsHBV(14p) & EXP-HYDRO(6p) re-implemented differentiably, trained in the identical harness; SAC-SMA = Newman's published per-basin SCE runs (their own best; we only compute metrics)
Single metric codeevery row evaluated by one script (plain NSE/KGE, same window, ≥1000 valid days)
Auditindependent leakage audit; obs-parity gate vs Newman r=1.0000; two header-area bugs + a window off-by-one found and fixed before publishing numbers
Harness certification: our same-harness HBV lands at 0.639 — inside its published 0.63–0.68 range — so the harness neither cheats nor handicaps. Differences between rows are model differences.

Main results (test 1989–99, n=673)

ModelCalibmed NSEmed KGEβ
SAC-SMA + Snow-17 (published)per-basin 0.6620.7170.94
HBV (ours, same harness)per-basin 0.6390.6711.00
CREST+UH · β-loss · Daymetper-basin 0.6080.6640.946
CREST+UH · Maurerper-basin 0.6000.6360.891
CREST no-UH · Maurerper-basin 0.5190.5650.87
EXP-HYDRO (ours, same harness)per-basin 0.5230.6211.03
CREST-dPL (regional)one net 0.4620.4360.82
CREST-dPL (PUB, ungauged)zero-shot 0.44–0.460.430.81
context (Kratzert 2019): LSTM ens. 0.74 · mHM 0.66 · HBV-ub 0.68 · VIC 0.55 · FUSE 0.65–0.72

Forcing note: SAC-SMA, HBV and EXP-HYDRO rows use Maurer (Newman convention). Matched-forcing comparison: CREST+UH·Maurer 0.600 vs SAC-SMA 0.662. The Daymet row's edge over Maurer is only +0.002 once UH is present — its gain comes from the β-loss, not the forcing.

NSE CDF NSE map

Where the gap lives — a controlled water-balance comparison

Component-by-component, the SAC-SMA and CREST rows differ in one place:

ComponentSAC-SMA (Newman setup)CREST+UH (ours)Equivalent?
SnowSnow-17Snow-17✅ identical
Routingcalibrated Gamma UH (2 params: shape, scale) — SAC-SMA itself contains no routing; the UH is bolted on (CAMELS readme-verified) calibrated triangular UH (1 param: maxbas) ✅ same class (theirs slightly more flexible)
PET demandPriestley–Taylor, calibrated coefficient Oudin × calibrated ke multiplier ✅ same class
Water balance dual-zone, four stores: tension/free-water separation in both zones, two nonlinear baseflow reservoirs, demand-driven percolation single VIC bucket + one linear groundwater store ❌ the entire gap

Why daily scale makes routing a non-factor

CAMELS basins are small–medium (median ≈ 336 km²; concentration time mostly < 1 day). Daily aggregation hides wave dynamics — a calibrated UH captures all the timing freedom that is visible at this scale (our Stage-3 test: daily KW ≈ UH). So the benchmark is effectively a controlled experiment on the water-balance structure alone.

Reading

Snow identical, routing & PET equivalent, optimizer verified (Adam = SCE-UA), seeds agree ⇒ the 0.662 vs 0.608 gap has a single attributable source: SAC-SMA's five-decades-engineered vertical soil architecture (tension/free water, dual baseflow). That is exactly the structure the 3-layer CREST kernel targets — pending layered soil priors to make the extra storages identifiable.

Ablation experiments — where each point of skill comes from

ExperimentDesignResultReading
UH routingidentical config ± differentiable triangular UH (maxbas) 0.519 → 0.600 (+0.08) hydrograph-shape freedom; water balance still sets the volume
ForcingMaurer vs Daymet, same everything +0.06 without UH; ≈0 with UH UH absorbs timing differences between products
Convergence controlwarm-start + cosine LR, plain NSE loss (isolates optimization) 0.600 → 0.602 (+0.008 trainer-metric) baseline was ~converged; small honest gain
β (bias) lossNSE + 2(β−1)² penalty, selection metric matched β 0.87→0.946, KGE +0.03, mean NSE +0.04, median ≈ flat volumetric under-bias was a loss artifact (loss-tolerated leak), fixed for free
Multi-seed + merge3 seeds; per-basin best-TRAIN merge across runs seeds land 0.595–0.598; merge loses optimization surface effectively flat across seeds; train-argmax picks noise — rejected honestly
3-layer soil (A/B)new differentiable 3-layer kernel vs single-layer, same harness/loss 0.610 vs 0.608 (tie); KGE 0.671 = HBV parity; train +0.04 didn't transfer vertical storage split not identifiable from streamflow alone — kept for the gridded stage where layered soil priors constrain it

Optimizer comparison — Adam (backprop) vs SCE-UA

Design

OptimizerBudgetWall-clocktrain NSE test NSEtest KGEβ
Adam (backprop)400 iterations8–13 h ~0.670.600–0.602 0.636–0.6480.89–0.92
SCE-UA v1 (same wall-clock)2,542 evals11.8 h 0.6170.5640.5970.874
SCE-UA v2 (limit)10,000 evals72 h 0.6660.5970.6400.908
Conclusion: SCE-UA eventually converges to the same optimum (0.597 ≈ 0.600, train 0.666 ≈ 0.67) — at ~6× the compute. Two independent optimizers agreeing means 0.60 is the model-structure ceiling, not an optimizer artifact: the remaining 0.04 to HBV is structural, verified rather than assumed.

Regionalization (dPL) — the capability the classic rows don't have

Design

AlphaEarth 64-d ⊕ 12 attributes
→
shared MLP
→
20 CREST params / basin
  • Time-split: all basins seen, test decade unseen → cost of replacing 673 free calibrations with one network
  • PUB: 5-fold spatial hold-out (longitude-stratified) → true ungauged skill
  • Trained end-to-end through the full 15-year daily simulation graph

Results

Modemed NSE
per-basin (upper ref)0.600
dPL time-split (seen basins)0.462–0.478
dPL PUB (ungauged, 3 folds)0.44–0.46

Key reading: seen→unseen drop is tiny (0.475 → 0.44) — the attribute→parameter mapping generalizes spatially. Regional LSTM PUB (~0.53–0.56) stays above, as expected for 20-parameter physics.

Identifiability findings (streamflow-only calibration)

  • groundwater-partitioning parameter: 50.7% of basins within 1% of its upper bound (boundary collapse)
  • ET-scaling parameter: spread across the range, IQR 22.7%, no railing
  • real 1-D objective sweep: near-optimal plateau = 27.5% of the feasible range (cross-basin median); ~49% of basins have their optimum at the bound

Two signatures of under-constraint → motivates multi-source constraints (soil moisture / ET / storage) in the next phase.

Takeaways

1 · CREST enters the conceptual band. 0.608 NSE / 0.664 KGE vs HBV 0.639 / 0.671 and SAC-SMA 0.662 / 0.717 — on their home turf (lumped, daily), with a generic harness and no structure-specific tuning.
2 · Every point of skill is attributed. UH +0.08 · forcing +0.06 (absorbed by UH) · β-loss fixes bias for free · seeds/merge/3-layer honestly rejected. No unexplained gains.
3 · The optimum is optimizer-verified. Backprop and an exhaustive SCE-UA converge to the same score; backprop gets there ~6× faster. The 0.04 gap to HBV is structural.
4 · dPL is the differentiation. Ungauged (PUB) NSE 0.44 with a near-zero seen/unseen gap — per-basin-calibrated classics have no equivalent capability. This is the mechanism that scales CONUS calibration to a global parameter product.
5 · Throughput is the quiet result. 673 basins × 6 configs × multiple optimizers in days on one GPU — the experiment volume behind this deck is itself the case for differentiable calibration.
Next: soil-prior gate experiment (SoilGrids depth-layered WM), multi-source constraints against the identifiability findings, then the global 0.1° parameter prior.