Q-learning: A data analysis method for constructing adaptive interventions.
Inbal Billie Nahum-Shani, Min Qian, Daniel Almirall et al. · Psychological Methods · 2012
Increasing interest in individualizing and adapting intervention services over time has led to the development of adaptive interventions. Adaptive interventions operationalize the individualization of a sequence of intervention options over time via the use of decision rules that input participant information and output intervention recommendations. We introduce Q-learning, which is a generalization of regression analysis to settings in which a sequence of decisions regarding intervention options or services is made. The use of Q is to indicate that this method is used to assess the relative quality of the intervention options. In particular, we use Q-learning with linear regression to estimate the optimal (i.e., most effective) sequence of decision rules. We illustrate how Q-learning can be used with data from sequential multiple assignment randomized trials (SMARTs; Murphy, 2005) to inform the construction of a more deeply tailored sequence of decision rules than those embedded in the SMART design. We also discuss the advantages of Q-learning compared to other data analysis approaches. Finally, we use the Adaptive Interventions for Children With ADHD SMART study (Center for Children and Families, University at Buffalo, State University of New York, William E. Pelham as principal investigator) for illustration.
Cited by 172 works. doi:10.1037/a0029373
Authors
- Inbal Billie Nahum-Shani
- Min Qian
- Daniel Almirall
- William E. Pelham
- Beth Gnagy
- Gregory A. Fabiano
- James G. Waxmonsky
- Jihnhee Yu
- Susan A. Murphy
References
- The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations.
- Reinforcement Learning: An Introduction
- Statistical Analysis With Missing Data
- Multiple Imputation for Nonresponse in Surveys
- Reinforcement Learning: An Introduction
- Applied Dynamic Programming.
- Social Foundations of Thought and Action : A Social Cognitive Theory
- Applied Dynamic Programming
- Bayesian Inference for Causal Effects: The Role of Randomization
- A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect