Following up on my earlier posts, here is a high level summary of having an AI agent run through a complicated assignment autonomously while using encoded human judgment.
It was an interesting exercise, and I found more support for my conviction that human expertise and judgment isn't going anywhere. I used a expensive frontier model, yet left to its own devices it made basic errors that any health actuary would catch, and also made brilliant suggestions. Only someone with sufficient expertise would be able to know what is a great insight and what is garbage.
And all that distillation goes back into the encoding of human expertise - so that we have guided and governed autonomous agent execution for future runs with predictable results.
Of course, this isn't about actuarial risk assessment. This is a blueprint demonstrating how any complex knowledge work can be handled with high determinism, judgment, transparency, and traceability.
Next week I'll write a detailed article that shows step by step how to do this, including the exact decision logic.
The pipeline ran an autonomous data quality gate before scoring proceeded: medical and pharmacy claims without eligibility, members with valid age, and members with recorded gender all passed against pre-set thresholds. It then ran a prospective statistical validation, predicting year-N+1 spend from year-N diagnoses, a harder forward-looking stress test than the concurrent design the ACA HHS-HCC model itself uses.
During recalibration, 82 of 151 prospective HCC/RXC coefficients had to be floored to zero rather than allowed to go negative, since a diagnosed condition should never lower a risk score. Refitting with that constraint held fit quality unchanged. Condition prevalence across the panel: 42.9% of member-years carried at least one qualifying HCC.
Both the concurrent and prospective risk scores were fit PERWT-weighted and checked against their exact targets: a concurrent ratio of 1.0000 against same-year cost across the full population, and a prospective ratio of 1.0000 against next-year cost across the longitudinal pairs the model was trained on. Passed every gate. Scores normalized and audit-logged.