Detect. Forecast. Outlook.

This is the part most comparisons leave out, and it decides the answer. Anvil is scored at p ≥ 0.096083 — the promoted model's operational detection threshold — the bar at which a user gets a detection. ProbSevere v3 is scored at p ≥ 0.30 and MESH at ≥ 25.4 mm, the points each publishes. An earlier version of this page graded Anvil at p ≥ 0.30, which is a previous generation’s bar; every catch-rate figure on the page moved when that was corrected, and the direction of several comparisons moved with it.

Anvil’s shipped bar is deliberately loose. It is a severity ranking over every storm, and the operating point is set low enough that fewer hailstorms are missed, knowingly paying for that in alarms — a missed hailstorm and a wasted notification are not the same cost to a person. The performance diagram on this page shows the whole dial, including the tighter settings that score better and are not what the app runs.

2026-05-01 → 2026-07-30: 2026-05-01..2026-07-30 (91 consecutive storm days, 6.88M object-scans, 26,186 report-matched). Hail reports are sourced from NOAA Storm Prediction Center storm reports and NWS Local Storm Reports, restricted to ≥ 19.05 mm (0.75″ severe threshold) and matched to a storm within 15 km and ±10 min. 7,016 storm tracks out of 1,803,465 carry a matched ground report; at scan grain that is 26,186 report-matched object-scans out of 6,878,167.

Every head-to-head figure on this page is measured on corroborated truth: a storm’s ground truth counts only when it can be attributed to a storm that drew two or more independent reports. Where it cannot, the row is removed from the evaluation entirely — counting as neither a hit, nor a miss, nor a false alarm, identically for all three systems. Storms nobody reported are never removed; only rows that already carry truth can be. Over this window that clusters 3,542 reports into 1,441 storms, of which 842 carry a single report and 599 are corroborated, and it removes 2,586 of the 7,016 verified storm tracks — 36.9% of the truth population.

The rule cannot be tuned in anyone’s favour, and the data shows it. It reads only the reports and the storm they attach to — never a score, never a threshold, never whether a system fired — so the removed rows are the same rows for every system by construction. The visible consequence is that not one false alarm moves: Anvil 21,344, ProbSevere 14,959, MESH 25,882 in both scopes. That identity is checked at build time.

It also makes every absolute precision and threat score lower, for everyone. Removing a storm removes the correct alarms raised on it while leaving every wrong alarm in place, so all three systems’ precision and threat scores sit mechanically below their all-truth values before any question of skill arises. That is why the comparisons above lead with the multiple between systems rather than the absolute, and why a corroborated precision is never quoted as a product number: a person still receives the alarms raised on the storms this scope sets aside. The precision figure that describes the product is in the published limits below, and it is measured on all truth.

Requiring two independent reports buys cleaner truth on a less representative sample. Two observers are likelier over populated ground and likelier on large, long-lived storms, so the corroborated scope is Anvil’s lead on well-observed hail — not on all hail. It discards 36.9% of the truth population to get there.

So the same comparison is published on all truth, nothing discarded, and it is the arm that shows the headline is not an artefact of the selection. Threat score across all 7,016 verified storms: Anvil 0.1632, ProbSevere 0.1258, MESH 0.1139 — 1.30× and 1.43×. The same order, the same winner, a smaller margin. Read the two arms as a floor and a ceiling on the same result, never as a change in it: the absolutes differ between them only by the mechanism in the note above, which moves all three systems the same way and says nothing about any of them.

Anvil’s margin widens when the truth is corroborated (1.30× to 1.34× against ProbSevere, 1.43× to 1.53× against the radar estimate). An earlier comparison at storm grain went the other way, with the baselines nearly closing the gap, and the reason is the grain rather than a disagreement between the measurements: at storm grain a storm counts as caught if any one of its many tracks ever fires, which hands a system with a larger alarm budget free credit. Judged track by track, that advantage disappears.

One measured cell moves against Anvil under this scope, and it is reported for that reason: day-by-day wins against ProbSevere on ranking area under an ROC curve go from 81 of 90 (90.0%) to 70 of 83 (84.3%). Every other cell moves the other way.

The headline scoreboard is per storm track, not per radar scan. A track is counted once, verified if any of its scans matched any ground report inside the tolerance above, and a system “fired” if its score ever crossed its own bar on any scan of that track. This removes per-scan timing bias — a system should not be penalised for firing early in a storm’s life, before the hail reaches the ground and an observer files anything. Scan-grain figures are reported separately and are not interchangeable with track-grain ones.

“Detection accuracy” on this page is the share of verified hail storms Anvil confirmed while the hail was falling — storms that really did drop hail an inch or bigger, counted once each, confirmed when any of their tracks cleared the shipped bar during the storm rather than afterwards. It is a catch rate, not a share of everything Anvil flagged: a storm Anvil never flagged counts against it, but an alarm on a storm with no ground report does not appear in this figure at all. How often an alarm turns out to be real is measured separately, at track grain, in the threat score and the matched alarm-budget sections — and that is also where the comparisons against ProbSevere v3 and MESH are made, because those are the quantities with a budget-matched rival row.

Every storm-grain figure on this page is measured on storms carrying 2 or more independent ground reports. Deduplicated ≥ 25.4 mm reports are clustered into storm events by transitive closure over a shared matched radar track, plus a 25 km / 60 min proximity merge that rejoins one storm split across two track IDs; the variant dropping the proximity half is a declared sensitivity, and under it the figure is 94.8% over the 580 corroborated storms it produces. Of the 1,343 storm events in this window, 779 carry exactly one report and are excluded: a lone report cannot be corroborated, and some are mis-measured, mis-timed, or logged well away from where the stone actually fell — a miss against one of those is not distinguishable from a report that was never right. What that exclusion costs is published rather than only what it buys. Anvil confirmed 73.7% of the excluded bucket, and 82.1% across all 1,343 storms with the tail left in. A lone report is often the only person outside, sometimes minutes off, sometimes a stone logged two towns from where it fell, and sometimes simply wrong — a miss against one of those is not distinguishable from a report that was never right. The filter is not neutral: requiring two observers selects for larger storms over more populated ground, so 93.6% is Anvil’s confirmation rate on well-witnessed hail, not on all hail.

The figure is built from 2,548 deduplicated ≥ 25.4 mm reports, 2026-05-01 to 2026-07-30. A storm counts as confirmed when any of its tracks cleared the shipped bar between an hour before its first report and half an hour after its last — that window is what “while the hail was falling” means, and a flag outside it does not count. Of the 564 storms counted, 4 had no radar object at all — nothing was there to score — and they are counted as misses rather than excused.

Both cohorts are Anvil-alone at this grain, and the reason is worth stating plainly. Requiring corroboration removes the sparse tail, and the sparse tail is where the systems actually differ: on these large, well-witnessed storms the two operational baselines also confirm the great majority, and the spread between all three is small. A comparison here would therefore be both weak and misleading about where the difference lies. The meaningful separation is in the quantities that price a catch against a false alarm — the critical success index and the matched alarm budget — and those are measured at track grain, which is also the only grain at which a budget-matched row was ever run. That is why every head-to-head on this page is made there and none is made here.

The critical success index is storms caught, divided by storms caught plus storms missed plus alarms that were wrong. Both failure modes sit in the denominator, which is why neither shouting constantly nor almost never firing can raise it. It is the number National Weather Service warning verification is scored on, and every system on this page is scored on it at the operating bar that system actually ships.

Truth on this page is what somebody wrote down, and most hail that falls is never reported by anyone. An alarm on a storm that really did hail but that nobody logged is counted as a false alarm — for every system, identically. That single fact drags all three threat scores down together, which is why the multiple between them is the readable quantity and the absolute is context. The size of that reporting gap is measured on this page, in the radar-versus-reported figure, and it is large. The corroborated scope adds a second deflation on top of it, described two notes above.

Up means more hail caught. Right means fewer wasted alarms. The top-right corner is perfect and nobody is near it. Anvil is drawn as a line because it is one model with a sensitivity dial — 19 settings of it are plotted and the ringed one is where the app runs. The other two are drawn where they operate, which is where a person actually meets them; neither lacks a dial of its own, and ProbSevere’s is simply set to its published bar. The dotted diagonals are alarm budgets: alarms raised per storm that really hailed. Anvil ships at 5.6× and the radar standard sits at 6.4× — more alarms, fewer catches.

Track grain, 1,800,879 storm tracks with 4,430 of them carrying corroborated truth inside 15 km and 10 minutes. 1 of the harness's rungs are so tight the model fires on nothing at all and have no success ratio to plot; they are not drawn. The best threat score anywhere on the dial is 0.210 at p ≥ 0.28; the app ships looser than that on purpose, because a missed hailstorm and a wasted notification are not the same cost to a person.

Both budget-matched rows are measured over every storm in the corroborated window, at 25.4 mm and up. The anchor’s fired-track count is taken as a budget and Anvil’s bar is derived as the score of the budget-th highest-ranked storm — never chosen. The threat-score multiple on each row is read over all 4,430 corroborated storms while the catch bars are read over the 4,284 of them that dropped damaging hail; the two denominators are different on purpose and are not interchangeable.

On the radar-estimate row Anvil’s scores tie at the cut, so it realises 29,717 fires against a 28,394 budget — 4.7% more alarms than the anchor raised. That overshoot travels with the margin everywhere the margin appears. A third cell of this measurement, pricing Anvil against the radar estimate on the ProbSevere-covered cohort alone, overshoots its budget by about 9% for the same reason and is withheld rather than published with a caveat.

The shipped-bar line under the figure is a different operating point and is printed with its own alarm counts so the two cannot be fused. It is measured on the storms ProbSevere covers, because Anvil’s shipped bar raises more alarms than ProbSevere’s budget across the whole window and “fewer alarms” is only a like-for-like statement on the cohort ProbSevere actually scores. There, Anvil reaches 79.2% from 12,647 alarms against 76.2% from 16,795 — 25% fewer alarms raised.

ProbSevere never scored 1,930 of these storms at all. Counting an unscored storm as a ProbSevere miss inflates the share of Anvil’s misses that look shared by everyone, which is why the figure prints the level cohort beside it. Restricted to the 2,354 storms all three systems actually looked at, 63.7% of Anvil’s misses are shared and 36.3% are storms a baseline did catch. The “shared blind spot” framing is materially weaker on the level cohort, and the number that says so renders rather than being left out. Neither view is the right one on its own: coverage is a real property of a product a person relies on, and so is skill on the storms everyone sees. A storm counts as caught when a system cleared its own bar on any scan of that track; 859 cleared no bar anywhere.

The size ladder is delivered catch rate: coverage included, counting every storm a tool never scored as a miss. ProbSevere scores only a minority of storms by design — it is built to flag the most organised cells rather than score every one — and on just the cells it does choose to score, its own catch rate is comparable to or better than Anvil’s. That trade-off is the point of being selective, not a flaw, and Anvil’s lead on the ladder comes from scoring nearly every storm rather than from beating ProbSevere cell for cell. The marginal band (19–25 mm reports) keeps only a third of its storms once truth is corroborated; it renders with its count and is deliberately given no lift figure, because that sample is too thin to carry one.

Every one of the 0.077 false warnings per quiet day came from a rainy day, where a storm really was overhead and really could have dropped hail. Nothing fired on a clear sky or in snow. It is measured at the gate that sends a notification, not at the detection score — whether a phone rings is a stricter bar than whether a storm is flagged — and it is a separate instrument from the regime figure beside it: different window, different population, and it grades one system rather than three. The release board checks the same question from four directions at once, and all four passed on 2026-08-05: clear days never push a false alarm; winter storms never push a false alarm; rain-no-hail days hold the false-alarm line; fewer than 5% of null days fire anything.

The clear-day and winter-storm counts are hard release gates, and that is literal: a build that fires a single false warning in either category does not ship.

That figure is a frozen cohort instrument with its own window and its own truth radius (truth: a ground-truth hail report >= 25.4mm (1 inch), matched to storm track), and it is measured on all truth rather than the corroborated scope the campaign comparisons use — there is no corroborated arm of it, so it is labelled rather than restated. Intervals are clustered by day, because storms on one day are not independent of each other. Its numbers are never quoted alongside the campaign figures above.

The west is a tie and it is published as one. East of the Rockies the margin is clear at every elevation. West of them the sample thins to 495 verified storms and the 95% interval spans zero (-2.27 to +6.88 pp), so the two systems are level there and the figure does not pretend otherwise. Mountain hail is the hardest case in this field for everyone: the reports are sparse, the storms are shallow, and the radar beam looks over the top of them.

On reports of two inches and up the raw radar estimate lands closer than our band (19.4 mm against 31.5 mm), and the reason is in the training data: hail reports pile up around the one-inch mark, so a model learning from them under-calls the rare monsters. That column carries the star on the baseline in the table above, because that is where the number puts it. The overall verdict is a pass on the pooled error, the day-by-day split and the share of stones landing inside the published band — not on winning every bracket.

Both methods are set to catch the same number of confirmed hail events, and the only question left is how many locations each had to warn to get there. The cut is rank-based on both sides, so no calibrator can move it. In the figure the dot is that ratio, the bar behind it is the 95% range across 20 storm days, and the tick is parity with the standard method.

At T+15 that range crosses parity — on the near horizon Anvil is not measurably better , and that is a horizon the free tier serves. The advantage is real at 30 and 60 minutes.

It is also concentrated rather than uniform: Anvil warns fewer locations on 12 of the 20 days. Measured over 20 storm days on a fixed grid of 3,489,685 point-forecasts — storm-blind, with report positions never used to choose where to score. The comparison is Lagrangian persistence: every storm keeps its current hail probability and drifts along its own measured motion. That is not a straw man — it is Anvil’s own previous shipped display, computed in the same call over the same candidates, and it is the standard every nowcast is judged against.

The comparison measures the probability forecast against persistence on an unbiased lattice, and a horizon there is how far ahead we score — not a promised warning time. No minutes-of-warning figure is publishable from it. The delivered-notice table beside it is a different artifact on a different window (2025-03-01..2026-06-10), graded at per-horizon (T+30/60/90) delivered notice on warned events and POD at production ml-threshold/confidence gates, plus per-horizon null-day false-alarm rate. Separate windows, separate counts, separate source lines: nothing on this page combines the two, and no number crosses from one to the other.

A horizon is how far ahead the model looks. Confidence decays with distance, and the two longer horizons never reach the bar a push requires. They are panel content — an early heads-up you go and look at, never something that wakes you up. That decay is deliberate: projecting one storm onto one address an hour and a half out is genuinely uncertain work, storms veer and collapse, and the honest response is to stop pushing rather than to push a guess. The confidence gate is what keeps the clear-day count at 0.

Anvil scored 100% of the storm objects in this window; ProbSevere v3 scored 55.71% of the same ones. It is selective by design, so its catch rate is reported unconditionally — a storm it never scored is a storm it did not catch. Of the 7,016 verified hail storms, 2,978 were never scored by it at all, and Anvil caught 1,782 of those.

Because a large share of severe hail never reaches the report database (Wendt & Jirak 2021; Blair 2020), we also verify against the polarimetric hail signature on the radar itself: composite reflectivity ≥ 55 dBZ, differential reflectivity ≤ 1 dB, correlation coefficient ≤ 0.95, and a 50 dBZ echo top ≥ 8 km. Recall is reported against each channel independently and never pooled into one number.

The record is incomplete because reports are filed by people: hail that falls over rural ground, overnight, or in the mountains rarely reaches the database at all, however large the stones were. That is the gap the radar channel exists to measure, and it is why an Anvil alarm with no report attached is usually still real hail rather than a mistake.

Precision measured only against incomplete reports systematically penalises every system for catching real hail nobody wrote down. The cleaner test is fire rate in environments where hail is meteorologically impossible — regimes gated on HRRR (MUCAPE, freezing level, wet-bulb zero, surface temperature). A fire in those regimes is a false positive regardless of reports. A regime is only shown when it can separate the systems: one with a sample this window and no fires from anyone is listed as omitted rather than drawn as a row of zeros.

Three blocks that this page has previously carried are withheld this refresh, and their absence is deliberate rather than an oversight. The early-warning / forecast figures: WITHHELD, and the reasoning is unchanged — only the wording is, because this string RENDERS on /research and the original named two internal model generations and an internal threshold value. It was graded on rows scored by an older detect generation (93% of the frozen registry window), at that generation's ML bar, whose candidate floor sits ABOVE the enter bar production now serves — a bar on a different probability scale, so no threshold-keyed number transfers between them — and over horizons {30,60,90} when production serves {15,30,60}. Three independent reasons, any one of which voids it. Keep it free of run ids and bar values if you rewrite it again. The cool-season comparison: the window holds 484 cold-regime object-scans out of 6,878,167, with zero cold-regime ground truth and zero cold-regime fires from any system — a “0% winter false alarms” headline off that sample would be vacuous, so no cool-season claim is made. And the quiet-day false-warning count: there is no curated null-day registry inside this window, and the regime-classified figures above are a different construction that may not be substituted for it.

The Outlook is evaluated separately from detection and shares no data path with it, on a backtest of 100 real events against matched controls, built from SPC outlook + HRRR convective fields + climatology. Skill is reported as AUC by lead day with a 95% confidence band. Day-of through day-3 clear the 0.70 floor this page uses to define a skillful lead; the leads beyond that fell below it at this measurement and are listed with their values below rather than plotted.

The reliability table in the underlying artifact is not published here, and it is not a product claim. It is measured over a separate window that is hardcoded frozen, and the configuration it grades is not the one that ships — two disqualifications the artifact states about itself. It used to render on this page carrying both of them, and no longer does. The reliability figure this page makes is the out-of-sample, bar-conditional rate in the Outlook lede above — the share of address-days that saw hail among those whose signal crossed the production bar — and that rate is not this table, is not a calibration curve, and does not say that a printed percentage verifies at the rate printed.

This refresh moved separation down at every lead. Only day-of through day-3 still clear the floor this page holds the outlook to. The leads that don’t are measured and listed rather than plotted, because a point on the curve is itself the claim: day-4 scores 0.696 (95% CI 0.634–0.757), day-5 scores 0.681 (95% CI 0.618–0.743), day-6 scores 0.670 (95% CI 0.607–0.733), against a floor of 0.70. The outlook is still produced that far out; its skill at those ranges is not something we publish at this refresh.

The reliability figure the pillar leads with is a separate, out-of-sample arm: 2026-07-01 to 2026-07-30, entirely after the model’s training cutoff, measured at address-day x ground-cell (0.05deg), MAX-over-object-centroids peak statistic within 8.05km, truth radius 15km · only address-days that crossed the production bar are counted, which is what makes it a reliability-given-alert rate rather than an unconditional hit rate.

It is a de-biasing layer: it makes the numbers mean what they say and fills the gaps where no number was published. Measured against a perfect forecast it scores below zero — what improves is calibration, not skill, and we would rather print that sentence than a chart that implies otherwise. Its four claims are coverage-and-honesty claims: it never overrides the official outlook where that outlook speaks, it puts a calibrated number where that outlook is silent, the odds it prints land closer to what happens, and on days with no hail anywhere it stays quiet.