Koo wrote: "Bear this P value ( p<0.05) throws me for a loop each time. Does it mean that there is a 95% chance that the results of BetonMace would be replicated if the study were repeated? What would the P value need to be to on the negative side of things to be a failure so to speak?"

Koo, great question. I deal with p-values all the time so I throw that term around casually and forget it might not be so intuitive to others. The p-value is the probability that the observed result has nothing to do with what one is actually testing for. In other words, you are testing whether there is no effect of your drug, or no difference between the groups. The lack of a difference is called the null hypothesis. In the biology/medical field, there is a kind of arbitrary P-value cut off at 0.05. If you achieve 0.05 or less, then it is deemed significant and you can reject the null hypothesis. However, if you achieve a P-value of greater than 0.05 (for example 0.051), then you are unable to reject the null hypothesis and cannot say that your treatment is significantly different than random chance alone.

So back to your questions "Does it mean that there is a 95% chance that the results of BetonMace would be replicated if the study were repeated?" In the case of a P-value of 0.05, you would observe no real difference between your groups (no effect of the drug) in 5% of studies due to random sampling error. Thus, the null hypothesis would still be true 5% of the time. To put it another way, a P-value of 0.05 in this context would mean that you can reject the null hypothesis 95% of the time, but that you could not reject the null hypothesis 5% of the time if this study was replicated.

"What would the P value need to be to on the negative side of things to be a failure so to speak?" If the cut-off for significance is P<0.05, then it would only need to be 0.051 to be statistically deemed a failure.

That's why it sucks to be working at the margins of 0.05. In reality, there is not much real difference between 0.048, 0.049, 0.05, 0.051 and 0.052, etc. However, only those values of 0.05 and less would meet the arbitrary P<0.05 cut-off that the biology/medical field has deemed to be the magical relevant significance cut-off.

BearDownAZ