Understanding the p-value
A short explanation of the meaning, usage (SPSS) and interpretation of the p-value
If you have read articles for your thesis, chances are that you have seen the P-value. The p-value is often used for hypothesis testing in statistics. Therefore, it’s helpful to be able to interpret and report p-values in your thesis.
A hypothetical example of hypothesis testing
To understand P-values properly, you have to grasp what is meant by hypothesis testing. Every scientific research starts with a hypothesis. A simple hypothetical example. Suppose that a person claims that men are, on average, taller than women. Another person plays the devil’s advocate and says that there is no difference between the average height of men and women. In science, we must be just as critical as the devil’s advocate and assume (albeit very pessimistically) that there isn’t a difference in average height. We will call the proposition of the devil’s advocate the null hypothesis. We want to test the first proposition (namely that men are taller than women on average) and we call this the alternative hypothesis. To test this, you measure the height of 50 men and 50 women. You calculate the average length of both groups. You find that the men are on average 14 cm taller than the women. But is this statistically significant? You compare the two averages with a statistical test (t-test) and you find a one-sided p-value 0.001 (1 in 1000) what does this mean? The p-value indicates the probability that we find this difference in length if we assume that the null hypothesis is true. We said at the beginning that according to the null hypothesis there is no difference in length (0 cm). The probability that we will find such a big difference (14 cm), assuming that there is no difference in length, is the found p-value. In other words, in 1 in a 1000 studies, we could find this difference or greater purely by chance if the null hypothesis were true. If we previously used a statistical significance of 0.05, we can reject the null hypothesis.
Pitfalls
In science, it is common that p values <0.05 are statistically significant. A lot of value is attributed to statistical significance. Studies that do not show a significant difference are published less often for example. Even so, try to avoid looking for significant results. On the basis of chance (5%) you already find 1 significant result for 20 statistical tests. Think of your statistical plan in advance and avoid p-hacking.
How do I report the p-values in my thesis?
If you write your thesis in English, use a period instead of a comma (for example p <.01). As you can see we don’t put a 0 before the dot in this case either. This is because a p-value can never be greater than 1 by definition! In Dutch, you use a comma (for example p <0,01) SPSS sometimes gives p <.000, for p values less than 0.001, in that case write p <0,001.
Statistical significance vs relevance
Moreover, it is important to not only report the p values found, but the differences found as well. With large sample sizes, you will find significant differences more often, even though the actual differences may not be all that impressive. In this case its far more informative to mention the confidence intervals. This will be the subject of a following blog.