Selection Factors in Analyzing Health Research Outcomes

Scientific research of all types is extremely important for developing both new knowledge and new theories about how things work.  Assessing the validity of research results is obviously important, but has become more important as scientific fraud or misinformation spreads.   A paper at the National Bureau of Economic Research examines how selection into certain behaviors may affect the outcomes reported for studies and how changes in selection confound understanding research results even more.   (NBER Paper)  Especially in regard to research on health behaviors, selection bias is a significant issue, particularly in non-randomized trials or studies.  Healthy people tend to be more likely to adopt more healthy behaviors.  And people with higher incomes and greater educational attainment also tend to engage in healthy behaviors.  (I actually think this goes both ways, engaging in healthier behaviors probably helps people to do better in school and be more successful at work.  I suspect there are clusters of good and not-so-good behaviors across a wide variety of life activities that are strongly related to outcomes in those activities.)  So observational and other study designs can easily fail to account for this and appear to have better results for the intervention than would actually exist in real life in an entire population.

The researchers were also concerned that as healthier people chose additional recommended healthy behaviors, the selection bias could increase over time and health research needs to account for its own potential impact on study outcomes.  They used the example of vitamin and diet recommendations to test their theory.  The used national health survey data and Nielsen purchasing data to track behavioral changes in both vitamin use and diet following changes in recommended practices.  In both cases there was a stronger uptake after a positive change in recommendation among better educated and higher income persons, and that uptake lessened when the recommendation was weakened.  In addition as the behaviors had more total uptake, they were more correlated with various outcome measures, but remember that this is at least partly due to the selection factor of who is engaging in the behavior.  For both Vitamin D and E, the authors show that once there was research available suggesting they may be preventive of cancer, there was greater uptake and apparent greater association with a reduction in cancer occurrence.  This is attributed by the authors to selection dynamics.  Finally, the authors suggest some approaches to more comprehensively address these selection issues and adjust research results for them.  Some of the most important research is that which attempts to strengthen the quality of gathering knowledge.