The science (or art) of healthcare claim-based risk adjustment is about three decades old. For the most part, the information used in these models has remained the same: diagnoses and pharmaceutical usage from administrative claim data.
I've noticed that recently there has been a lot more talk about including social determinants of health (SDOH) in some way into risk adjustment methodologies. Back in 2012, I wasn't cool enough (not that I am now!) to call my paper "Risk Adjustment And SDOH!". Instead, it was titled in the rather unimaginative way "Non-Traditional Variables in Healthcare Risk Adjustment".
Research on Non-Traditional Variables
The paper investigated a smorgasbord of new variables and their impact on healthcare risk adjustment. These include variables that we might think of as social determinants of health (e.g. income, education, etc.). I'll highlight some of the conclusions of the (rather lengthy) paper in future byte-sized posts.
In this one, I want to talk about how to think about the impact of such variables on the performance of a risk adjustment model.
The Initial Challenge: Predictive Accuracy
Often times the impetus of including new information into a predictive model is the increase in predictive accuracy. I spent the first three months of my investigation hitting a brick wall.
The whole premise of this research effort was that variables like income and education will in some way improve the predictive model that is healthcare risk assessment. The result of adding these variables was a near-zero lift in predictive accuracy measures such as R-square. There was essentially no improvement!
Or so I thought. I felt bad as the principal investigator who the Society of Actuaries had entrusted with this research, and now my convictions were shaken about why these variables were important to study at all.
A Turning Point: Bias Mitigation
But it often happens in research that some failures turn out to be important and open a window into something else.
It turns out that including such variables is incredibly important to mitigating bias in risk assessment models.
For example, including age-sex in a concurrent risk assessment model offers almost no improvement in the predictive accuracy of the model, but we know that excluding these variables would yield highly biased risk adjustment payments offering health plans a veritable blueprint on how to select members in a risk adjusted environment. This would completely undermine the intent of risk adjustment.
Similarly, these non-traditional variables help mitigate bias in the risk adjustment models in important ways - by income, education, etc. - furthering the policy intents of a risk adjustment program.
Bias vs. Variance: The Key Insight
When it comes to risk adjustment and neutralizing the incentives for risk-selection, mitigating bias is more important than the variance.
New variables can help, but there are operational, privacy, and other challenges that will need a careful think-through when integrating social determinants of health into risk adjustment models.
Additional Reading
If interested in a primer on risk adjustment (including concurrent/prospective), please refer to: https://www.soa.org/research-reports/2016/2016-risk-scoring-primer/
Note: The opinions expressed do not necessarily reflect those of my colleagues or employer.