It’s strange to think, but the next meeting of Bayes Club will be our last for the term. That’s right: the end of the term is approaching, and our next meeting will be during 10th week.

For our final meeting of the term, we’d like to do two things:

1. Consolidate what we’ve learned by reading through some real journal articles or chapters, working together to capture some Bayesian analyses in the wild.
2. Focus on any questions that you have — things that we’ve discussed that still don’t seem to make sense on even a conceptual level.

To those ends, Rose and I would like submissions for papers and/or topics and questions of discussion. If you have a Bayesian paper that you’ve been trying to figure out all term, send it in! And if any questions are burning in your mind just as much as an MCMC sampler*, send those in, as well!

If you have something in mind, please send it to Rose (rosem@uoregon.edu) and me (jleverni@uoregon.edu) by the end of this Tuesday, 2014-03-04. If you submit an idea, the three of us can meet sometime next week to try to figure it out a bit together in advance of the bigger meeting, and then you can talk in the bigger meeting for a few minutes about what headway you have made, and about what questions remain.

* (Get it? Like, an MCMC sampler “burns in”? lol.)

We’ll be looking at methods for comparing models using Bayesian statistics. Pretty nifty!

To get the code for an example we’ll go through, download Kruschke’s code here, and then open the file called “BernBetaModelCompJags.R”.

And check out this book chapter by Lee & Wagenmakers for more ideas.

(Here’s the site for their book by the way: http://bayesmodels.com/ Tons of good stuff.)

If you’ve thought about it, it’s on the internet. For example, a Bayesian remix of As Long As You Love Me by Justin Bieber: http://www.youtube.com/watch?v=JmSKam4BCvU

(Be sure to scroll down so you can see the lyrics.)

Posted on behalf of Rose:

Robert is out of town this week, so Jacob and I are doing a guest lecture on Bayesian inference in PSY612: Data Analysis II. It will have a conceptual focus (vs. the more practical nuts-and-bolts focus of BayesClub), including a fair amount of discussion about the differences and similarities between Bayes and NHST. We’re building the slides in a web-ready format rather than powerpoint, so  we’ll post everything publicly and you’ll be able to access the materials quite easily if you’re curious.

You’re also welcome to come sit in on the lecture, if you like! It’s Thursday 10:00am-11:20am in PLC 189. Please keep in mind that the room is not super big, and students who are registered for PSY612 need to get first priority for seats. We’re asking the 612 students to read this article before class: http://www.nature.com/news/scientific-method-statistical-errors-1.14700

During tomorrow’s Bayes Club, we want to 1) get some more practice reading and interpreting JAGS models, and 2) get a feel for how your priors influence your results.

So, having already gone through the model for estimating the difference between two groups (i.e. the “BEST” model), we’ll check out the code for exploring the relationship between two continuous predictors (conceptually similar to a simple regression). For the sake of consistency in style, etc., we’re using another of Kruschke’s models for this. Download it here: http://www.indiana.edu/~kruschke/DoingBayesianDataAnalysis/Programs/ProgramsDoingBayesianDataAnalysis.zip (The model we’ll be looking at is called “SimpleLinearRegressionJags-Original.r”)

The mechanics for running this model are almost exactly the same as what we saw in the BEST example, it’s mostly just the JAGS model itself that’s different. We’ll walk through it, and link it up to a graphical representation of the model, like we did for BEST.

Then, we’ll ask you to play around changing the priors on this model (and/or the BEST model, if you like). We’re hoping that you’ll come away with 1) a better understanding of what distributions are available in JAGS and what they look like, and 2) a better sense of how changes to your priors affect your posterior. To encourage maximum self-directed hands-on-ness, we’d like you all to play around with this yourselves on your own machine for the second half of the meeting. Here are some ideas to get you started:

• To begin with, the prior for the residuals is a normal distribution. Change it to a t distribution, and, for extra points, infer the degree of normalness (i.e. the nu parameter, that operates as df in the t dist to determine how chunky the tails are).
• Strongly assume that the null hypothesis is true (i.e. that these two variables do NOT have a relationship, so the slope should be 0). Try this one several ways.
• Standardize your variables, and then set a uniform prior on the slope, such that it ranges from -1 to 1, with all possible values equally likely. 
• Assume that the slope can only be positive (like a one-directional test).
• Tweak the values for the parameters on the priors Kruschke provides (for example, play around with the parameters for the gamma distribution that defines the prior for precision).
• Choose your own adventure!

Each time you change a prior, try plotting it first so you can see what it looks like. Typically, the easiest way to do this is plot(function(x)dfunc(x, parameters)) where “dfunc” is the name of the distribution and “parameters” is whatever parameters you need to set for that distribution. For example, you can plot a normal curve with mean=0 and SD=1 with plot(function(x)dnorm(x, mean=0, sd=1)).

Spoiler alert: Most changes you make to your priors won’t have that much of an effect on the posterior if you have enough data. When you have a smaller N and/or messier data (weaker relationships), the priors have more sway. Try changing your sample size as well as changing the priors, and see where the balance is. 

(Sorry  – I wrote this post after our last BayesClub meeting, but I forgot to actually post it! Better late than never, I hope.)

Let’s say you’re collecting data and you check on your results every once in a while as the data are coming in. You decide to stop collecting data once you see that the results look “good”, and you report your findings using a standard NHST test.  Then the Stats Police break down your door, handcuff you, and throw you in the back of an armored van that takes you to Stats Prison. A common tale, but true.

But what happens if you’re analyzing your data using Bayesian inference instead of NHST? We know that multiple comparisons aren’t a problem when you’re dealing with a posterior distribution instead of running NHST tests, so maybe it’s also okay to run your analyses multiple times and then stop collecting data whenever you’re satisfied. 

Turns out, no. If your stopping rule is based on your results, your results will be biased, no matter what flavor stats you’re running. 

http://datacolada.org/2014/01/13/13-posterior-hacking/

One thing to note is that Simonsohn is using the Bayes Factor to get a result analogous to a significance test in his Bayesian analyses, which operates differently from the HDI and ROPE method we’ve been reading about in Kruschke’s work. Kruschke actually has a blog post on this as well, in which he discusses stopping rules with NHST, Bayes Factors, and HDIs in more detail:

http://doingbayesiandataanalysis.blogspot.com/2013/11/optional-stopping-in-data-collection-p.html 

Hello All,

Bayes Club will meet for the first time on Tuesday, 2014-01-14, and on even weeks after that (alternating with R Club). How exciting!

In our first meeting, before diving into “How Bayes?”, we will be discussing “Why Bayes?” If possible, by Tuesday, please take a look at the following reading(s), which can be found on the Syllabus page:

• Kruschke, J. K. (2010). Bayesian data analysis. Wiley Interdisciplinary Reviews: Cognitive Science, 1(5), 658Ð676. doi:10.1002/wcs.72 [pdf]
• If you have time: A practical and pithy post by Andrew Gelman on the practice of Bayesian stats in psychology, with links to several relevant articles and discussion.

Rose wrote up an excellent introduction to the logistics of the term in an R Club post here. Everything said in that post applies to Bayes Club, so take a look, especially if you’ve signed up for credit for the course.

As mentioned in Rose’s post above, we will unfortunately not be meeting in a computer lab this term. Thus, if possible, please bring a laptop with you to our meetings.

The code examples that we’ll be discussing this term require the use of three pieces of software: R, JAGS (“Just Another Gibbs Sampler”), and rjags, an R package that allows R and JAGS to speak to one another. Please try to install these by Tuesday the 14th, or at least by our following meeting. I’ve written up a post on the installation process here for you; please feel free to comment on it to bring up any questions or difficulties that you experience, so that we can all try to figure out answers together.

Looking forward to seeing you! I have good reason to believe that we’ll have a credibly good time*!

* If you don’t understand that weak joke yet, then all learning objectives for the course will be met if you do by Finals Week.