Now suppose you collect the same data over the next six weeks. This time the average age for those with an IV is 9. Why did your different samples yield different results? Is one sample more correct than the other? Remember that there is variability in your outcomes and statistics. The more individual variation you see in your outcome, the less confidence you have in your statistics.
In addition, the smaller your sample size, the less comfortable you can be asserting that the statistics you calculate are representative of your population. A confidence interval provides a range of values that will capture the true population value a certain percentage of the time.
Confidence intervals use the variability of your data to assess the precision or accuracy of your estimated statistics. You can use confidence intervals to describe a single group or to compare two groups. We will not cover the statistical equations for a confidence interval here, but we will discuss several examples.
A confidence interval for a population mean with a known population standard deviation is based on the conclusion of the Central Limit Theorem that the sampling distribution of the sample means follow an approximately normal distribution. Consider the standardizing formula for the sampling distribution developed in the discussion of the Central Limit Theorem:.
In this formula we know , and n, the sample size. In actuality we do not know the population standard deviation, but we do have a point estimate for it, s, from the sample we took. More on this later. We can solve for either one of these in terms of the other.
This is where a choice must be made by the statistician. The analyst must decide the level of confidence they wish to impose on the confidence interval. These numbers can be verified by consulting the Standard Normal table. Divide either 0. Then read on the top and left margins the number of standard deviations it takes to get this level of probability.
Common convention in Economics and most social sciences sets confidence intervals at either 90, 95, or 99 percent levels. A good way to see the development of a confidence interval is to graphically depict the solution to a problem requesting a confidence interval. This is presented in Figure for the example in the introduction concerning the number of downloads from iTunes.
However, the level of confidence MUST be pre-set and not subject to revision as a result of the calculations. There is absolutely nothing to guarantee that this will happen. Further, if the true mean falls outside of the interval we will never know it.
We must always remember that we will never ever know the true mean. Statistics simply allows us, with a given level of probability confidence , to say that the true mean is within the range calculated.
Here again is the formula for a confidence interval for an unknown population mean assuming we know the population standard deviation:. It is clear that the confidence interval is driven by two things, the chosen level of confidence, , and the standard deviation of the sampling distribution. The Standard deviation of the sampling distribution is further affected by two things, the standard deviation of the population and the sample size we chose for our data. Here we wish to examine the effects of each of the choices we have made on the calculated confidence interval, the confidence level and the sample size.
For a moment we should ask just what we desire in a confidence interval. Our goal was to estimate the population mean from a sample. We have forsaken the hope that we will ever find the true population mean, and population standard deviation for that matter, for any case except where we have an extremely small population and the cost of gathering the data of interest is very small.
In all other cases we must rely on samples. With the Central Limit Theorem we have the tools to provide a meaningful confidence interval with a given level of confidence, meaning a known probability of being wrong.
By meaningful confidence interval we mean one that is useful. Imagine that you are asked for a confidence interval for the ages of your classmates. You have taken a sample and find a mean of You wish to be very confident so you report an interval between 9. This interval would certainly contain the true population mean and have a very high confidence level. However, it hardly qualifies as meaningful. The very best confidence interval is narrow while having high confidence. There is a natural tension between these two goals.
We can see this tension in the equation for the confidence interval. The confidence interval will increase in width as increases, increases as the level of confidence increases. There is a tradeoff between the level of confidence and the width of the interval. The sample sized, , shows up in the denominator of the standard deviation of the sampling distribution. As the sample size increases, the standard deviation of the sampling distribution decreases and thus the width of the confidence interval, while holding constant the level of confidence.
This relationship was demonstrated in Figure. If the repeated sampling properties of the W -optimal interval are confirmed to be satisfactory in future simulation studies and analytical work then this would become our recommended approach, but at this stage we wish to remain cautious in this regard.
DerSimonian R, Laird N. Meta-Analysis in clinical trials. Control Clin Trials. Hardy R, Thompson SG. A likelihood approach to meta-analysis with random effects. CAS Google Scholar. A re-evaluation of random-effects meta-analysis. Article Google Scholar. Thompson S, Sharp S. Explaining Heterogeneity in Meta-analysis: a comparison of methods. Stat Med. Paule RC, Mandel J. Concensus values and weighting factors. J Res Natl Bureau Stand. Approximate confidence intervals for moment-based estimators of the between-study variance in random effects meta-analysis.
Res Synth Methods. Biggerstaff BJ, Jackson D. Article PubMed Google Scholar. Jackson D. Confidence intervals for the between—study variance in random effect meta-analysis using generalised Cochran heterogeneity statistics.
Hoes does the DerSimonian and Laird procedure for random effects meta-analysis compare with its more efficient but harder to compute counterparts? J Stat Plan Infer. Assessing the amount of heterogeneity in random-effects meta-analysis. Biom J. Viechtbauer W. Confidence intervals for the amount of heterogeneity in a meta-analysis.
Casella G, Berger RL. Statistical Inference. Google Scholar. Predictive distributions were developed for the extent of heterogeneity in meta-analyses of continuous outcome data. J Clin Epidemiol. Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis.
Hartung J, Knapp G. Sidik K, Jonkman JN. On tests of the overall treatment effect in meta-analysis with normally distributed responses. Jackson D, Baker R. Meta-analysis inside and outside particle physics: convergence using the path of least resistance?
Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis. Cochran WG. The combination of estimates from different experiments. Hoaglin DC. DerSimonian R, Kacker R. Random effect models for meta-analysis of clinical trials: an update. Contemp Clinical Trials.
A random-effects regression model for meta-analysis. Morris CN. Parametric empirical Bayes inference: Theory and applications. J Am Stat Assoc.
Psychol Methods. Quantifying, displaying and accounting for heterogeneity in the meta-analysis of RCTs using standard and generalised Q statistics. BMC Res Method. Methods for calculating confidence and credible intervals for the residual between-study variance in random effects meta-regression models.
Quantifying heterogeneity in a meta-analyses. Biggerstaff BJ, Tweedie R. Incorporating variability in estimates of heterogeneity in the random effects model in meta-analysis.
Conducting Meta-Analyses in R with the metafor package. J Stat Softw. The power of the standard test for the presence of heterogeneity in meta-analysis. Jackson D, Bowden J. Empirical Bayes meta-analysis. J Ed Stat. Testing for homogeneity in meta-analysis I. The one-parameter case: Standardized mean difference. Methods to estimate the between-study variance and its uncertainty in meta-analysis. Extreme between-study homogeneity in meta-analyses could offer useful insights. Download references.
DJ wrote the majority of the paper, performed the analytical investigation and produced the corresponding Additional file 1. JB originally conceived and investigated the idea of using unequal tail probabilities in conjunction with the Q profile method. JB also performed the all the empirical investigation and the entire simulation study, and prepared the other Additional file 2.
Both authors contributed to the writing of the paper. Both authors read and approved the final manuscript. This is a paper about statistical methods. All data are from published meta-analyses, and so research in this paper has not involved the undertaking of any new trials. Hence this section is not applicable.
0コメント