�ݺ�ߣ

© TNS
Statistical Significance and Margin of Error
in TRI*M Studies

© TNS
in TRI*M Studies
2
Generally, all standard rules for statistical significance also apply to TRI*M studies. In order to
determine the level of statistical margin the following parameters have to be taken into
account:
 Sample size
 Customized sample designs (e.g. stratified sampling of different customer segments)
 Size of universe
 Standard deviation
 Significance level
The mathematical calculation of the statistical margin of error for the TRI*M Index for various
sample sizes is based on:
 Normal distribution of TRI*M Index
 Infinite universe
 Significance level of 95%, 90% and 80%
 The challenge is to determine the effect of high quality sample designs. We therefore
calculate the standard deviation from all TRI*M Indices recorded in the TRI*M database. All
these scores are based on highly customized sample designs of our customer research
programs

© TNS
in TRI*M Studies
3
The standard deviations for the following error rates for the TRI*M Index are calculated from
the TRI*M Database that includes more than 17,000 TRI*M Customer Retention studies with
more than 15 Mio. TRI*M Customer Retention interviews for 1,600+ companies.
 Most importantly, in order to apply these error rates to TRI*M Index comparisons over time
it is absolutely essential to keep the sample design stable and robust from wave to wave
 Any changes in the design impact the index and error rates!
Sample Size Confidence Coefficient
95% 90% 80%
n = 30 +/- 10.7 +/- 9.0 +/- 7.0
n = 50 +/- 8.3 +/- 7.0 +/- 5.4
n = 75 +/- 6.8 +/- 5.7 +/- 4.4
n = 100 +/- 5.9 +/- 4.9 +/- 3.8
n = 200 +/- 4.2 +/- 3.5 +/- 2.7
n = 300 +/- 3.4 +/- 2.8 +/- 2.2
n = 500 +/- 2.6 +/- 2.2 +/- 1.7
Example:
In case of a sample size of
n=200, the „true“ TRI*M
Index lies with a probability
of 90% within a range of
+/- 3.5 points

© TNS
Significant vs. meaningful differences
4
Statistical significance:
In statistics, a result is called statistically significant if it is unlikely to have occurred by chance.
Statistical significance heavily depends on the number of respondents in a survey.
Example:
 In a survey with n=8,000 respondents, a difference in the TRI*M Index (e.g. between two waves of the
survey) of 0.5 points can be considered as significant on a 90% significance level.
 This means that with a probability of 90% (or more), the TRI*M Index in the two groups (e.g. waves)
really is different (= not equal).
 But a significance does not give any information about the extent of this difference.
Meaningful differences:
A difference in results is called meaningful if it is of (business) relevance for the company. The
question, whether a difference in the TRI*M Index is meaningful or not, is completely independent of the
number
of respondents.
In the example, the difference in the TRI*M Index of 0.5 would be statistically significant (= not occurred by
chance), but the difference is small enough to be utterly unimportant. The company will not perceive any
difference in customer behaviour.
For the TRI*M Index, a difference of 3 points can be considered as meaningful
The use of the word significance in statistics is different from the standard one, which
suggests that something is important or meaningful

�ݺ�ߣ

Trim statistical significance and margin of error 2013

Recommended

More Related Content

What's hot (20)

Similar to Trim statistical significance and margin of error 2013 (20)

Trim statistical significance and margin of error 2013