Show the data, don't conceal them
Gordon B. Drummond, Sarah L. Vowler · The Journal of Physiology · 2011
This short editorial is the first of a planned series to guide the handling of data from laboratory experiments. We hope that authors (and editors) may find this helpful in writing and assessing future journal articles. It's clear that data handling in science needs to be improved, if only from the number of attempts that have been made to rectify the problem. Books, articles and websites give copious words of advice, too numerous to cite. Many are indeed helpful, but others are often too dense and discouraging for the non-specialist. Often the advice is ‘too general in nature, too limited in scope, and too specialized in vocabulary to be useful to most authors and editors’ (Lang & Secic, 2010). On one occasion the American Physiological Society gave specific advice (Curran-Everett & Benos, 2004). When the guidelines were followed up, the results were found to be mixed, with some praise but also criticism and controversy, with little overall effect on the quality of publication (Curran-Everett & Benos, 2007). In general, encouraged by the slow but substantial change that has occurred in medical journals, there seems to be a mood to improve, although changes in basic science journals have been slow, if they have occurred at all. Comparison of basic science reports with clinical studies is not flattering (Watters & Goodman, 1999). We are well aware that advice alone has been ineffective. We could easily list a comprehensive set of guidelines, which would probably suffer a fate similar to those that have gone before. In contrast we hope that by keeping the articles in this series short and focused, they will offer advice that is digestible and palatable. They may even attract readers who find such topics difficult and unappealing when they are presented in larger portions. We shall try to avoid technical terms and complicated maths. Much advice starts with a suggestion that one should consult a statistician. Considering the time and cost of some experiments, this would often be prudent. Admittedly statisticians are a rare breed and can be hard to find. Often when the data are given to a statistician there is a problem with them that might have been avoided if the consultation had occurred at the start. Fisher once said: ‘To consult a statistician after an experiment is finished is often merely to ask him to conduct a post-mortem examination. He can perhaps say what the experiment died of’ (Fisher, 1938). In practice, it's likely that basic scientists do not know that they should seek advice beforehand, as they may not be sure what they will find, or they consider that what they will find will be easily described and analysed with a simple toolbox of tests. On occasion, physiological studies resemble a random walk through a series of ‘what ifs?’ and ‘how can we prove this mechanism?’ stages that may leave a statistician searching for a single testable hypothesis. When basic scientists do have data to analyse, they may think the exciting part of the study is done. They look around for an easy way to deal with their data, and often ask someone with possibly more experience but probably no more training what to do. The advice is often ‘well, this is what I did …’, almost designed to sustain the status quo and perhaps perpetuate error. A recent article repeated the often heard advice: ‘The choice of how to express the data is very important and should not be made solely on the basis of habit or convention. Always inspect the data in its raw form’ (Lew, 2007). Display data in their raw form. A dot plot is a good start. ‘Dynamite plunger plots’ conceal data. Check the pattern of distribution of the values. Take care when samples are small. Choose appropriate measures of precision or dispersion. In this editorial, we argue that we should use methods of data presentation that allow inspection, not concealment, of the nature of the distribution of the data. This is a first important step in good statistical practice. The next step is entirely dependent on the distribution of the data, so it is important that this is verified rather than just assumed. In many journals, including The Journal of Physiology, the current convention used to illustrate a set of data is to use a solid vertical bar, with a T at the top to indicate the precision of the estimate: this is often called the ‘dynamite plunger plot’. Most often, the bar indicates a mean value and the plunger the standard error of the mean (SEM). In most circumstances this convention is unnecessary, and can conceal important features of the data. Is there a better way? To some extent this depends on what the figure intends to show. However, in many cases this concept of ‘what do we wish to show?’ may not even have been considered: the author's aim is often just to present ‘a result’ without first considering the underlying implication or characteristics of this ‘result’. Time spent inspecting the raw data allows a visual impression, which can indicate their nature. Figure 1 shows two sets of data, each with 50 observations, randomly sampled, with very similar mean values, but drawn from two populations with differing distributions. The dot plots of the raw data clearly demonstrate that the distribution of the points is different between the two sets. The first set has a Gaussian (or normal) distribution, so the values are symmetrically distributed. The second set is right skewed (lognormal), with a limited number of smaller values and a few large values. This type of distribution of values is quite common in biology, for example plasma concentrations of immune or inflammatory mediators. Imagine presenting the plunger plots only: who would know that the values were skewed – and that the common statistical tests would be inappropriate? Comparison of a dot plot with a plunger plot A, two datasets, with the same mean value, displayed as either dot plots or plunger plots. The bars are mean values: error bars indicate SEM. B, subsamples of 6 data points fr
Cited by 44 works. doi:10.1113/jphysiol.2011.205062
Authors
References
- How to report statistics in medicine: annotated guidelines for authors, editors, and reviewers
- Guidelines for reporting statistics in journals published by the American Physiological Society
- Guidelines for reporting statistics in journals published by the American Physiological Society: the sequel
- Good statistical practice in pharmacology Problem 1
- Guidelines for reporting statistics in journals published by the American Physiological Society
- Comparison of basic methods in clinical studies and in vitro tissue and cell culture studies reported in three anaesthesia journals