No σ for you!

[carey]

(11-22-2024 02:54 PM)KeithB Wrote:  Also, one of my favorite formulas is one I learned in an excellent DOE class 35 or so years ago, taught by David Doehlert.

The 95% confident limits for a mean of a sample of a population is:

Mean +/- t * smplstddev / sqrt(r)

r is the number of replicants for the mean calculation
t is a factor based on the degrees of freedom for the sample stddev calculation. It is the t statistic
It ranges from 12.7 with 1 degree of freedom to 1.96 with infinite degrees or freedom.

In case it's of interest to anyone, here's some context to the formula in Keith's post and some comments relating topics in this thread to their use in physics.

The term "smplstddev / sqrt(r)" in the formula is the standard error (SE), i.e., standard deviation (SD) of the mean and is the SD divided by the square root of the number of trials. It's of great importance to experimenters because, unlike the SD (which experimenters can't control as it's a measure of the intrinsic variation in whatever is being studied), the SD of the mean (i.e., the SE) can be made arbitrarily small just by increasing the number of trials. This makes sense because, as the number of trials increases, our uncertainty in the value of the mean should decrease.

Now consider the "t factor" in the formula. Without it, we have: mean ± SE. This encompasses around 68% of the data because the area under the normal curve between limits of the mean ± 1 SD is around 68% of the total area. To encompass 95% of the data (corresponding to the "95% confidence limit" mentioned in the post) it's necessary to integrate the normal curve between limits of the mean ± 2 standard deviations (or more precisely 1.96 standard deviations as mentioned in the post). So the "t factor" in the formula is just the number of standard errors (SD of the mean) needed to encompass a particular % of the data. Since the value 1.96 is mentioned with "infinite degrees of freedom" the population standard deviation is appropriate.

Notes re: topics in this thread as applied to physics.

1) While σ often denotes SD in stat books and calculator manuals, in physics σ represents SE (not SD) since measurements are usually mean values and SE is the SD of the mean.

2) If the final output of a series of measurements is a mean ± SD, sure, use the sample SD for small data sets. However, if the goal is model testing, as in physics experiments using chi-squared minimization, SDs are used only to find SEs, and population SD is often used. While sample SD has less bias than population SD, dividing by N-1 vs N markedly increases variability at very low sample sizes (e.g., N = 2) where measurement uncertainties aren’t reliable. Note that this preference for population SD has nothing to do with it being easier to calculate as that hasn't been an issue for many decades.

3) Using the population SE and obtaining the "t-factor" just by direct integration of the normal curve makes the t-statistic unnecessary.