Statistical Model Validation Under A-11 §185
OMB A-11 §185 requires statistical models to be predictive of defaults and other deviations and to consider general market conditions, population shifts, and climate change, but RD's program models — covering RHS, RBCS, RUS portfolios across 750+ cohorts — face a model validation backlog that grows as climate and macroeconomic conditions shift faster than RD's annual model refresh cycle.
Why It Matters
Out-of-tolerance models drive subsidy rate misstatements that flow directly into the President's Budget, OMB Credit Supplement, AFR, and Treasury reporting envelope. GAO and OIG reports (GAO-16-41, GAO-AIMD-97-145) have repeatedly criticized federal credit subsidy methodology rigor.
HSG's Approach
- 1Stand up an A-11 §185-aligned model validation framework documenting every model's intended use, theoretical underpinning, key assumptions, sensitivities, performance over time, and limitations.
- 2Apply SR 11-7 / OCC 2011-12 model-risk-management discipline (development, validation, back-testing, sensitivity analysis, peer review, monitoring).
- 3Build climate, population, and macroeconomic scenario layers Section 185 explicitly contemplates — drawing on HSG's quantitative bench and Ginnie Mae / FHA prepayment / default curve experience.
- 4Use AI tooling to flag model performance drift against acceptable tolerance bands and queue models for priority validation review.
- 5Maintain a model inventory with risk-tiering (high / medium / low) so validation effort matches model materiality.
Expected Deliverables
- Model validation framework document keyed to A-11 §185
- Model inventory with risk-tiering and validation cadence
- Back-testing and sensitivity-analysis workbooks per program
- Climate / population / macro scenario library
- Model-drift surveillance dashboard
Expected Outcome
Bring 100% of high-materiality cohort models into a documented A-11 §185-compliant validation cadence within 12 months, eliminating the OMB / OIG / GAO model-rigor critique vector.