For significance tests including tests of fit there is a hypothesized condition (called null hypothesis or H0) that one is testing to see if it is true. For a test of fit the hypothesized condition is that the selected distribution generated the data. For a test that the means are equal, the hypothesized condition is equal means. The p-value is then probability that the data or one more extreme than it would have been generated under the hypothesized condition. A p-value of 0.05 would indicate that the chance of the observed data is low, 1 in 20, due to variation alone. This is good evidence that the data was not generated under the hypothesized condition. The hypothesized condition is rejected if the p-value is 0.05 or below. This provides 95% confidence the hypothesized condition is not true, i.e., the data does not fit the selected distribution or the means are not the equal.
The smaller the p-value, the greater the evidence that the data did not come from the selected distribution. For tests of fit and other tests, the confidence level is calculated from the p-value as 100*(1 - p-value). Therefore:
Confidence Level p-value
The p-value is also know as alpha level or significance level.