Corrections and data checks
Numbers on this site can be wrong, and when one is, the fix is recorded here rather than made quietly. Three things live on this page: the automated checks that run after every nightly load, the events we have removed from the change feed and the rule that removed them, and a dated register of every correction that changed a published figure.
This morning's checks
Run of 2026-09-06 02:47 UTC: 45 passed, 0 warnings, 0 failures. A failure turns the nightly pipeline red and is looked at before the next load; a warning is printed and stored. Values are the number as of the run; the band is what the check allows; drift compares with the value stored the previous night.
| Check | Value | Band | Status | What it guards |
|---|---|---|---|---|
| duration_model_test_auc | 0.744 | ≥ 0.6, ≤ 0.95 | ok | |
| erosion_median_first_window_pct | 94.62 | ≥ 60, ≤ 105 | ok | |
| erosion_molecules_first_window | 195 | ≥ 100 | ok | |
| events_today_labeler_moves | 0 | ≤ 1,500 | ok | Normal days post a few hundred; the openFDA ingredient-order artifact posted thousands a day for weeks before slice A of the audit caught it. |
| events_today_shortage_new | 0 | ≤ 60 | ok | More than a few dozen new shortages in a day means the 'new' rule regressed to counting re-typed old rows. |
| events_today_total | 0 | ≤ 10,000 | ok | A flood of events is a diff gone wrong, not news. |
| facilities_with_country_pct | 91.7 | ≥ 85 | ok | |
| fragility_v1_score_age_days | 1 | ≤ 1 | ok | |
| fragility_v1_scored | 29,605 | ≥ 20,000, drift ≤ 10% | ok | Priced NDCs scored in the latest run. |
| fragility_v2_fit_age_days | 1 | ≤ 120 | ok | Refits are deliberate; past four months the coefficients describe a market that has moved on. |
| fragility_v2_score_age_days | 1 | ≤ 1 | ok | |
| fragility_v2_scored | 1,935 | ≥ 1,000, drift ≤ 15% | ok | |
| inspections_dqa_total | 26,390 | ≥ 20,000, drift ≤ 5% | ok | Drug-quality classifications drive every inspection figure; the count only grows. |
| inspections_latest_end_age_days | 19 | ≤ 200 | ok | Classifications post months after a visit; past 200 days the API pull is probably returning an old page. |
| km_episodes | 7,014 | ≥ 5,000 | ok | |
| km_median_injectable_days | 621 | ≥ 300, ≤ 1,200 | ok | |
| labeler_kinds_repackagers | 158 | ≥ 100, drift ≤ 25% | ok | Repackagers are excluded from maker counts; if this collapses, competition is silently overstated everywhere. |
| labelers_active_30d | 9,828 | ≥ 5,000, drift ≤ 10% | ok | Every maker count on the site starts from this roster. |
| learn_guides_oldest_review_days | 0 | calendar | ok | Each learn guide carries a reviewed date (web/src/app/learn/pages.ts); a guide unreviewed for a year has stale statutory figures. |
| medicaid_national_covers_states | 1 | ≥ 1 | ok | National rows include the cells CMS suppresses at state level, so they can never sum to less than the states. |
| medicaid_national_total_bn | 111.94 | ≥ 50, drift ≤ 10% | ok | Latest complete year, CMS national rows, gross. A restatement moves it a little; a broken load moves it a lot. |
| medicaid_survey_latest_quarter_rows | 92 | ≥ 40 | ok | |
| methodology_review_age_days | 0 | calendar | ok | The methodology page is re-read sentence by sentence at least twice a year (LAST_REVIEWED in web/src/app/methodology/page.tsx). |
| nadac_big_jump_share_pct | 0 | ≤ 10 | ok | Share of the last two weeks' price changes that moved 50% or more. A step change in this share is what a CMS methodology change looks like: the December 2024 moving average cut the monthly version of it thirty-fold. |
| nadac_latest_age_days | 4 | ≤ 14 | ok | CMS posts NADAC weekly; two weeks without a new effective date means the feed or the metastore discovery broke. |
| nadac_rows_last_14d | 171 | ≥ 50 | ok | Weekly updates carry a few hundred changed rows and the monthly refresh thousands; a handful means the file loaded empty. |
| ndc_products_marketed | 259,381 | ≥ 200,000, drift ≤ 5% | ok | |
| ndc_products_total | 259,990 | ≥ 200,000, drift ≤ 5% | ok | A 5% overnight swing in the NDC Directory is a broken load, not a market. |
| nhe_table16_year | 2,024 | calendar | ok | Homepage payer chart: six figures copied by hand from CMS NHE Table 16 each December (web/src/components/SpendByPayer.tsx). |
| oai_shortage_relative_risk | 2.64 | ≥ 1, ≤ 10 | ok | |
| orange_book_archive_fallback_age_days | 13 | ≤ 30 | ok | fda.gov blocks the pipeline's fetch; the loader serves the archived ZIP for at most 45 days. Past 30, plan a manual `ingest orange-book --file` load or the Orange Book goes dark. |
| partd_formulary_latest_quarter_age_days | 158 | ≤ 200 | ok | CMS posts the plan file about a quarter in arrears; past 200 days the tracker is two quarters stale. |
| partd_margin_proxy_median | 1.472 | ≥ 1.1, ≤ 2.5 | ok | NDC-level proxy for the /spread headline (which takes one NDC per molecule). Outside 1.1-2.5 the join or a unit changed, not the market. |
| partd_plans_in_latest_file | 5,496 | ≥ 3,000, ≤ 8,000 | ok | |
| rebound_fit_age_days | 1 | ≤ 120 | ok | |
| rebound_markets_with_amp_price_pct | 38 | ≥ 20 | ok | The FUL/AMP inputs were null for every market until H2 carried the manufacturer price forward across the missing month. |
| rebound_markets_with_open_shortage | 32 | ≥ 20 | ok | Zero here for seven weeks in 2026 was the archive-lag bug: every market scored as if no shortage were open. |
| rebound_scored_markets | 1,876 | ≥ 1,000, drift ≤ 15% | ok | |
| rebound_share_over_50pct | 0 | ≤ 10 | ok | Base rates are under 1%; a page of coin-flip probabilities means the calibration broke. |
| recalls_last_60d | 60 | ≥ 10 | ok | |
| shortage_archive_snapshot_age_days | 48 | ≤ 60 | ok | The models treat a listing as open only while it appears in the newest archived file (45-day grace). Past that lag every listing starts to look closed, which is how both live models scored seven weeks of zero open shortages in 2026. |
| shortages_current_listings | 1,179 | ≥ 200, drift ≤ 30% | ok | FDA's current list moves slowly; a 30% overnight swing is a load problem. |
| specialty_spend_share_pct | 62.3 | ≥ 45, ≤ 80 | ok | |
| specialty_threshold_years | 2,027 | calendar | ok | CMS Part D specialty-tier cost thresholds by contract year (web/src/lib/specialty.ts), from the April rate announcement. |
| spl_drug_plant_links | 59,392 | ≥ 50,000, drift ≤ 10% | ok | Label-to-plant links behind every origin figure. |
ⓘ How this was built
- derived.data_checks — one row per check per nightly run (drugdashboards.checks, migration 0059)
- The canonical and derived tables the site reads, queried after the day's loads
Each check is one SQL statement returning one number, evaluated against an absolute band, a maximum overnight change against the previous stored value, or both; four file checks read the hand-keyed constants (the NHE payer figures, the specialty thresholds, the reviewed dates on the methodology page and the learn guides) and warn when the calendar says a re-key is due. The bands were set from the values observed on September 5, 2026, wide enough that a normal night never fires and narrow enough that the failures found in the September 2026 audit would have fired the first night: every market scoring with zero open shortages, manufacturer-price inputs null for every market, thousands of labeler moves in a day, the Orange Book served from an archived copy about to age out. To check: the check names map to the queries in src/drugdashboards/checks.py; the methodology page lists the hand-keyed constants.
Pipeline and calculations built by Claude (Fable 5) from the public datasets above. Educational only — verify against primary sources before acting.
Quarantined events
When a rule that produced change-feed events turns out to have been wrong, the events it produced are moved to a quarantine table with the rule that condemned them. They are never deleted, so what the site once said stays auditable.
| Reason | Events | Event dates | Quarantined |
|---|---|---|---|
| roster re-key: labeler already listed this molecule/form before the event date | 20,674 | 2026-07-16 to 2026-09-03 | 2026-09-03 |
| roster re-key: labeler still listed this molecule/form on or after the event date | 19,354 | 2026-07-29 to 2026-09-03 | 2026-09-03 |
ⓘ How this was built
- derived.events_quarantine (migration 0050): the removed events with a reason and a timestamp
Grouped by the reason recorded at quarantine time. The September 2026 entries are labeler entry and exit events produced while a product's strength string sat in the roster key, so openFDA reordering an ingredient list re-keyed a listing and posted an exit and an entry that never happened (88% of all such events at the time). To check: the change feed no longer shows them; the rule that replaced them is described on the methodology page.
Pipeline and calculations built by Claude (Fable 5) from the public datasets above. Educational only — verify against primary sources before acting.
Corrections register
Every change since the September 2026 audit that altered a figure readers could already see, newest first. Each entry names the commit that made it; a correction to a correction is a new entry, and history is never edited.
2026-09-05 · Learn guides re-checked claim by claim
Learn · commit 31b5da1
Was: The specialty guide said "2% of prescriptions, half of all spend"; the market guide said prices fall 80–90% within a year or two of generic entry; the Medicaid top-drug list still named Stelara; the PBM guide had the FTC insulin case and the 2026 federal law half-told; the carve-out guide said eleven states never carved in.
Now: Each figure is dated and sourced: about 4% of Part D claims and 62% of gross spending at CMS's $950 line (2024); FDA's 2019 competition figures beside the site's own event study; the 2025 Medicaid top ten; the February and July 2026 FTC settlements and the Consolidated Appropriations Act, 2026; KFF's eight carve-out states as of July 2025. Every guide now ends with a last-reviewed date.
2026-09-05 · Methodology page rewritten against the code
Methodology · commit 8e294ac
Was: It said the pipeline checks every source daily (Medicaid utilization and DailyMed labels are monthly), that nothing is entered by hand, that the financials cover the "15 largest" drugmakers, that specialty-tier changes post to the change feed (none had), and that the v2 model had not been retrained on the repackager classification (it had).
Now: A four-column source table separating the publisher's cadence from ours, a list of what is typed in by hand, descriptions of the rebound, duration and erosion models, and a last-reviewed date.
2026-09-05 · Both live models scored seven weeks with zero open shortages
Price forecast · Fragility · commit 23f67c5
Was: From 2026-07-20 to 2026-09-05 the shortage-risk and price-rebound models treated every FDA listing as closed once the archived snapshot lagged the calendar; the manufacturer-price (FUL) inputs were zero for every market because the current month had no FUL row; rebound reasons said "sole-source market" on 860 multi-source markets.
Now: A listing counts as open while it remains in the newest archived file; FUL prices are carried forward two months and shown as missing after that; reasons are gated on the market's own values. Retrained: held-out AUC 0.73→0.71 (six-month target), 0.72→0.71 (three-month), 0.71→0.705 (v2), lower and honest. The nightly checks now fail if the open-shortage input is zero for every market.
2026-09-05 · OAI-to-shortage statistic counted planned discontinuations
Inspections · commit 23f67c5
Was: "To be discontinued" listings counted as shortages starting: 14.3% of molecules after an OAI vs 6.7% otherwise (2.11×).
Now: 3.2% vs 1.2% (2.64×), Drug Quality Assurance inspections only, recomputed nightly and dated on the page.
2026-09-05 · Texas and California WAC increases pooled as one percentage
Price increases · commit 01c9052
Was: A Texas filing reports the rise from the lowest WAC of the previous one or three calendar years; a California filing reports a single price change. Pooled, Galzin's +9.9% step on 2026-01-01 appeared as "+2,077%". California's 2019–2024 annual workbooks had never loaded; tiles counted 5,818 "filings" that were 5,030 package-dates; a "filed ahead of time" panel rested on a false premise (both states file after the effective date).
Now: Separate tables per state basis, the annual workbooks loaded, counts as distinct package-dates, a unit screen on 18 Texas rows, and the statutes, thresholds and deadlines stated on the page.
2026-09-01 · Price-spike model retired
What changed · commit 18551c4
Was: A +20%-in-90-days model trained across CMS's December 2024 change to a three-month moving average for generic NADAC, without harmonizing the series, so it scored smoothed prices its training never saw.
Now: Retired from the daily run; the price forecast (harmonized series, two targets) supersedes it. Its tables are kept so past numbers remain explainable; nothing displays it.
2026-09-04 · Generic price-erosion event study rebuilt
Big Picture · Patent cliff · commit 3058780
Was: Molecules were keyed on raw name strings (Atripla split over a comma), repackagers and bulk listings counted as makers (a relabeler was amlodipine/valsartan's "sole marketer"), and the entry month sat inside its own baseline.
Now: Normalized ingredient keys, makers only, baseline months −6 to −1, two cohorts published with the molecule count behind every bar: median 71.6% of the pre-entry price at 25–30 months for all entries, 55.6% for the brand cohort.
2026-09-04 · Patent-cliff population, biosimilar counts and the specialty cost test
Patent cliff · Biosimilars · Specialty · commit 3058780
Was: ANDAs with trade names and discontinued products counted as brands facing a cliff; Purple Book presentations with a blank reference brand counted as extra biosimilars (123 "marketed" products, 30 groups); the specialty trend applied a $780 CY2021 threshold that CMS proposed but never finalized, and the lead claimed "half of American drug spending".
Now: NDAs with a marketed product and the last listed protection; one biosimilar product per brand per reference product (96 marketed, 22 groups); a fixed $950 line with each year's official threshold ($670 through 2021) printed beside it; the lead states 62% of 2024 Part D gross spending.
2026-09-04 · Company revenue took the narrowest EDGAR tag
Big Picture · Companies · commit 3058780
Was: Pfizer's 2022 revenue read $58B from its contract-revenue tag.
Now: The broadest revenue tag per year ($101.2B, restated 2025 10-K); buybacks and dividends are summed over the filers that tag them, and an untagged value is shown as unknown, not zero.
2026-09-04 · Inspection country ranking counted the wrong program areas
Inspections · Facilities · commit 26420a6
Was: Bioresearch Monitoring, blood, tissue and vaccine inspections (62% of the "plants", mostly US) were in the denominators: US 6.7% OAI vs India 10.3%.
Now: Drug Quality Assurance inspections only, counted per plant: US 21.4% vs India 14.7%. The drugs-at-risk panel matches shortages by normalized ingredient (exact matching had found 0 of 44) and counts makers per dosage form (lidocaine injection: 2, not 161).
2026-09-04 · Recall counts are reports, not events
Recalls · commit 26420a6
Was: openFDA enforcement rows (one per lot or presentation) were presented as recalls.
Now: Counts say "reports" and show FDA's event count beside them (twelve-month Class II: 611 reports, 229 events).
2026-09-04 · The Part D tracker was not measuring PBM spread
Part D prices vs. pharmacy cost · Big Picture · commit d528a2e
Was: Part D negotiated price ÷ NADAC was labelled "PBM spread pricing"; each molecule's highest-markup NDC led the table, so a 22-plan listing with a typo'd name sat at 62×; an "estimated spread per year" exceeded the molecule's entire Part D spend for 29 of 122 rows.
Now: By regulation the negotiated price is the pharmacy's reimbursement, dispensing fee included, so the ratio is the pharmacy's gross margin. One representative NDC per molecule (the one most plans price, at least 100 plans): median 1.68×→1.60×, 38%→34% at 2× or more. The annual estimate is removed.
2026-09-04 · Part D overview tiles and the negotiated-price join
Part D · Negotiated prices · commit d528a2e
Was: The price-vs-cost tile compared today's NADAC with an old plan quarter (1.44×); the restriction tile counted any one plan; Stelara had no before-price because CMS's trailing asterisk broke the join; NovoLog's six products were unweighted.
Now: Quarter-aligned with a 5¢ floor (1.28×); majority-of-plans 33% beside any-plan 58%; asterisks stripped and rows claims-weighted ($325 for NovoLog).
2026-09-04 · Medicaid units priced as doses swung whole states
Medicaid · State reports · commit 76dc65d
Was: State claims count grams of PEG 3350, millilitres of lactulose or one spray bottle where NADAC prices a unit, so a few cells were off 10–600×: Texas managed care read $1.67 per prescription ("fee missing") and a "$104M below-cost shortfall" was $0.4M. National totals summed publishable state cells, missing what CMS suppresses.
Now: A unit screen drops cells paid under 0.2× NADAC or, with fewer than three units per prescription, over 20×; the excluded share is printed. Seven state verdicts changed. National figures come from CMS's national rows (2025: $111.9B vs $104.1B of publishable cells).
2026-09-04 · State top-drug lists kept too little
State reports · commit 76dc65d
Was: Forty NDCs per state; Oklahoma's Humira total had lost $10.6M of $73.4M and Keytruda was split across names.
Now: Four hundred NDCs per state grouped by a drug key (brand name for branded products, normalized ingredient for generics), with the covered share of spend printed.
2026-09-04 · Interchangeability keyed on ingredient strings
Interchangeability · Insulin glargine · commit d6041ea
Was: Lantus and Basaglar returned "same licensed biologic".
Now: Products are compared by BLA number and per presentation; the same NDA/ANDA is tested before ingredient strings; doses compared numerically.
2026-09-03 · Maker counts included repackagers
Fragility · Drug pages · commit f64442d
Was: NDC Directory labelers such as Bryant Ranch, A-S Medication and Proficient Rx counted as manufacturers, so a market with one maker and five repackagers scored zero sole-source points.
Now: Labelers are classified from their DailyMed labels; repackagers are excluded from maker counts. Sole-source priced markets 698→852; the fragility high band 62→102 NDCs. The v2 model's reasons were also captioned from the wrong sign and said "sole-source" on 1,657 of 1,935 markets; now gated on the feature's value.
2026-09-03 · 88% of labeler entry and exit events were artifacts
What changed · commit 4f8034f
Was: openFDA reorders a product's ingredient list from one day to the next; with the strength string in the roster key, 40,028 "entered" and "exited" events were re-keys, not market moves.
Now: Moved to derived.events_quarantine with the rule that condemned them; entry and exit are decided per company, molecule and dosage form with a check that a product actually appeared or disappeared.
2026-09-03 · "New shortage" fired on any never-seen FDA row
Shortages · What changed · commit 4f8034f
Was: Re-typed 2017 postings and planned discontinuations posted as new shortages.
Now: Only a Current listing posted within 30 days for a company with no Current listing is "new"; the rest are recorded as listings.
2026-09-03 · Homepage counted first-ever NADAC listings as price moves
Home · commit 4f8034f
Was: "Prices moved this week" read 94; one was a real move. The spending-by-payer chart carried unsourced figures.
Now: Drugs and listings counted separately; recalls labelled reports; the payer chart transcribed from CMS NHE Table 16 (2024) and marked as hand-keyed.
How the site is built, what is typed in by hand, and what the data cannot tell you are on the methodology page.