Stepwise Multiple Testing as Formalized Data Snooping
Joseph P. Romano, Michael Wolf · Econometrica · 2005
In econometric applications, often several hypothesis tests are carried out at once. The problem then becomes how to decide which hypotheses to reject, accounting for the multitude of tests. This paper suggests a stepwise multiple testing procedure that asymptotically controls the familywise error rate. Compared to related single-step methods, the procedure is more powerful and often will reject more false hypotheses. In addition, we advocate the use of studentization when feasible. Unlike some stepwise methods, the method implicitly captures the joint dependence structure of the test statistics, which results in increased ability to detect false hypotheses. The methodology is presented in the context of comparing several strategies to a common benchmark. However, our ideas can easily be extended to other contexts where multiple tests occur. Some simulation studies show the improvements of our methods over previous proposals. We also provide an application to a set of real data.
Cited by 1,006 works. doi:10.1111/j.1468-0262.2005.00615.x Open-access PDF
Authors
References
- Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
- Bootstrap Methods: Another Look at the Jackknife
- The control of the false discovery rate in multiple testing under dependency
- Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimation
- The Jackknife and Bootstrap
- Data-Snooping Biases in Tests of Financial Asset Pricing Models
- The Jackknife and the Bootstrap for General Stationary Observations
- The Bootstrap and Edgeworth Expansion
- An Improved Heteroskedasticity and Autocorrelation Consistent Covariance Matrix Estimator
- The Stationary Bootstrap