A prediction-based resampling method for estimating the number of clusters in a dataset
Sandrine Dudoit, Jane Fridlyand · Genome biology · 2002
BACKGROUND: Microarray technology is increasingly being applied in biological and medical research to address a wide range of problems, such as the classification of tumors. An important statistical problem associated with tumor classification is the identification of new tumor classes using gene-expression profiles. Two essential aspects of this clustering problem are: to estimate the number of clusters, if any, in a dataset; and to allocate tumor samples to these clusters, and assess the confidence of cluster assignments for individual samples. Here we address the first of these problems. RESULTS: We have developed a new prediction-based resampling method, Clest, to estimate the number of clusters in a dataset. The performance of the new and existing methods were compared using simulated data and gene-expression data from four recently published cancer microarray studies. Clest was generally found to be more accurate and robust than the six existing methods considered in the study. CONCLUSIONS: Focusing on prediction accuracy in conjunction with resampling produces accurate and robust estimates of the number of clusters.
Cited by 702 works. doi:10.1186/gb-2002-3-7-research0036 Open-access PDF
Authors
References
- Bagging Predictors
- A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
- Cluster analysis and display of genome-wide expression patterns
- Molecular Classification of Cancer: Class Discovery and Class Prediction by Gene Expression Monitoring
- Algorithms for Clustering Data
- Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling
- Finding groups in data: an introduction to cluster analysis
- Arcing classifier (with discussion and a rejoinder by the author)
- Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays
- Objective Criteria for the Evaluation of Clustering Methods