p value demonstration
We’ve got two short pieces of code to work with today.
https://www.dropbox.com/s/loawliymkl2tex7/p%20value%20demonstration.R?dl=1
The first is the demonstration
######################################### # PLAY WITH THE VALUES YOURSELF ######################################### # set the probabilities for your set-up alpha <- .05 # the probability of a sig result given that the null is true (confidence level) beta <- .8 # the probability of a sig result given that the null is false (power) p_null <- .5 # how likely is the null hypothesis to be true? p_sig <- alpha*p_null + (1-beta)*(1-p_null) # the probability of a significant result (regardless of whether the null is true or not) p_null_given_sig <- (alpha*p_null)/p_sig # the probability of the null hypothesis being true, given that you've got a sig result
And the second is the plot
# see how your inferences about the null hypothesis following a sig result should change based on how probable the null was
betas_to_try <- c(.1, .3, .5, .7, .9) # the shape of that curve depends on the power of your experiment
plot.data <- data.frame(alpha=.05,
power_level=gl(k=15, n=length(betas_to_try), labels=c(betas_to_try)),
p_null=seq(from=0, to=1, length.out=15))
plot.data$beta <- as.numeric(as.character(plot.data$power_level)) # making a numeric version of the power factor, to use in the calculations below
plot.data$p_sig <- with(plot.data, alpha*p_null + (1-beta)*(1-p_null))
plot.data$p_null_given_sig <- with(plot.data, (alpha*p_null)/p_sig)
library(ggplot2)
ggplot(plot.data, aes(x=p_null, y=(1-p_null_given_sig), color=power_level)) +
geom_line() +
xlab("Probability of the null") +
ylab(NULL) +
ggtitle("What's the probabilty the null is false,\ngiven a significant result?")
