Bayes, bias, p-hacking and the Monty Hall problem

TL;DR: Bias is a fundamental concept in the science of agents; bias of a source is fundamentally related to cognitive bias.

---

You open the news in the morning, and you see the following headline from The Pan-Pan times: 80% of Chimpanzee Party lawmakers have criminal cases filed against them.

"Outrageous!" You cry-- "The Chimpanzee Party has no respect for the law! I certainly shall not be voting for them!" 

But upon subsequent investigation, you find that while this statement is true, in fact, 80% of Bonobo Party lawmakers also have criminal cases filed against them. Or perhaps you find that they have criminal cases filed against them -- in an unrecognized country created by some crackpot on Reddit. And the person responsible for generating these headlines was aware of this fact.

You feel cheated, even though everything you heard was the truth. Even though The Pan-Pan times only gave you information, and you acted as a rational agent when exposed to this information (well, actually you didn't, but let's ignore that for now), you feel like you've been "exploited" somehow, "tricked" even. 

Is this truly possible? Can a Bayes-rational agent truly be fooled by cleverly selecting information to provide?

Let's think about the problem more carefully. 

Our ultimate decision might be governed by the following rule: vote for whichever party you expect to have fewer criminal cases filed against its lawmakers. We may have some prior distribution on the fraction of criminal-accused lawmakers of each party, say $\mathrm{B}(2,3)$ and $\mathrm{B}(3,2)$ for the Chimpanzee and Bonobo parties respectively. Under the prior, the expected fraction of criminal-accused lawmakers are 0.4 and 0.6 respectively, and one would vote for the Chimpanzee Party; under the posterior, the expected fraction of criminal-accused lawmakers are 0.8 and 0.6 respectively, and one would vote for the Bonobo Party. 

Or.

Our ultimate decision might be governed by the following rule: vote for whichever party minimizes the expected sum $X_1+\dots+X_{100}$, where say: $X_1$ is the regulatory burden the party will place upon coming to power, $X_2$ is the tax rate the party will implement upon coming to power, $X_3$ is the number of criminal cases against the party lamakers, $X_4$ is the number of bad words the party's candidates use on TV, $X_5$ is the number of lies the party's candidates say on TV, $X_6$ is the number of dissidents the party will throw in prison upon coming to power, etc.

Let's say, for simplicity, that these are all Bernoulli distributed variables -- further that each $X_i$ (Chimpanzee Party) and $Y_i$ (Bonobo Party) are distributed as $\mathrm{Bernoulli}(0.5)$. Then in this prior distribution, we would be uncertain as to whom to vote for, as both parties have $\mathrm{E}[\sum X_i]=50$ is lower. 

And suppose the Pan-Pan Times tells us: "We looked at $X_{3}$ and it turned out to be 1 for the Chimpanzee Party!" Now, $\mathrm{E}[\sum X_i]=50.5$, while $\mathrm{E}[\sum Y_i]=50$, so we vote for the Bonobo Party. 

But here's the thing though: the precise information you receive isn't $X_3$ is equal to 1 -- it is the Pan-Pan Times's report is "$X_3$ is equal to 1". And that's what you should be conditioning on.

If you were to condition on the Pan-Pan Times's report is "$X_3$ is equal to 1" (call this variable $\Pi$), well, what would your inference look like? You could apply Bayes's theorem, etc. but simply put -- let's say the Pan-Pan Times's report is some generative process that looks at all the $X_i$, and reports one that is equal to 1 (i.e. tosses a hundred coins and reports a heads) -- it can do so in all but $2^{-100}$ of outcomes, so the only information we're given is that that particular outcome (where all $X_i$ are 0) is not the case -- the only information we're given is that the Chimpanzee Party isn't literally perfect -- so that $\mathrm{E}[\sum X_i] = 50/(1-2^{-100})$.

(OK, in this case, the decision is the same -- but for example, suppose our decision was instead "donate some sum of money proportional to the difference in $\mathrm{E}[\sum X_i]-\mathrm{E}[\sum Y_i]$" then the decision would be different.)

To instead condition on just $X_3=1$ -- rather than the full information provided -- is a cognitive bias. As actually implementing Bayes's theorem everywhere is expensive, the mind processes information using heuristics -- one such heuristic is that only some information is selected to be conditioned on, leading to selection bias. Indeed, all such "source biases" are fundamentally manifested as some form of cognitive bias -- the use of negative terms to describe the Chimpanzee Party, for example, is the exploitation of some sort of the association fallacy, repeating the word "Bonobo Party" for hours on screentime is an exploitation of the availability heuristic, etc. 

A perfectly Bayes-rational agent -- that takes into account all information it is exposed to -- is immune to being tricked in this way. But a real agent, which uses heuristics, can be exploited. The idea is that if such biases are systematic, then it can be predicted and avoided cheaply. 

You can't hack Bayes's theorem

Aka the difference between rationalization and rationality.

Eliezer Yudkowsky calls this the dilemma of the clever arguer: A propagandist (clever arguer) is hired to sell you a box that may or may not contain a diamond -- he tells you that the box has a blue stamp on it (which you know occurs more on boxes containing diamonds). If you could handle the box yourself, you could rationally evaluate all the characteristics of the box and compile their influences on your probability estimate Are you then forced, for each argument the clever arguer provides, to helplessly update your probabilities as the clever arguer wishes, even though you know the propagandist has omitted the evidence he doesn't want you to know of?

There are various equivalent formulations of the problem including:

• p-hacking
• Filtered evidence: an experimenter flips a coin 10 times and tells you that the 4th, 5th and 7th tosses came up heads, without telling you anything about the other tosses. 

The key matter to realize is that the information you get is not simply the 4th, 5th and 7th tosses came up heads, but instead: the propagandist tells me that the 4th, 5th and 7th tosses came up heads. The statistical process we are studying is no longer the "natural" (IID Bernoulli) process we're used to, but a different process, which depends on the inner mechanism used by the propagandist. For example:

• If the propagandist always tells you only the 4th, 5th and 7th tosses, you update your beliefs from this evidence as normal.
• If the propagandist only tells you the coin tosses that came up heads, then you now know that the other seven tosses come up tails, and you update your beliefs accordingly.
• If the propagandist chooses any three heads that came up to tell you about, then the probability of 4, 5 and 7 specifically being chosen is only slightly greater with a biased coin (you can calculate this, but the key point is that the process we're observing is not about which coins come up heads, but which coins are chosen by the propagandist). 

We could have a probability distribution on the various possible mechanisms underlying the propagandist, and then the information we get from our observations is actually split between information on the propagandist's mechanism and information on the coin's mechanism.

Here's a model that makes it easier to perform inference on: we have $n$ features $X_i$ distributed as $N(\mu, 1)$ with prior $N(0,1)$ on $\mu$, and the propagandist chooses to tell us the value $x$ of $X_j$ that has the greatest value among all $X_i$s. The posterior density on $\mu$ can then be calculated:

\[\frac{{{\Phi _\mu }{{(x)}^{n - 1}}{\phi _\mu }(x){\phi _0}(\mu )}}{{\int_\mu  {{\Phi _\mu }{{(x)}^{n - 1}}{\phi _\mu }(x){\phi _0}(\mu )\,d\mu } }}\]

Where $\phi_\mu$, $\Phi_\mu$ are the normal PDF and CDF respectively. Here's an interactive visualization of this density:

Link to interactive version.(Note that in this visualization, $\mu$ is $x$ and $x$ is $z$).

For example for $n=10$, $x=1$, the distribution actually shifts leftwards, because surely if $\mu$ were 0, the propagandist could have found a feature with a better value than 1.

The moral of the story is that you can't hack Bayes's theorem; you can't fool a rational agent. If you have a parameter whose value you know, you can't systematically produce misinformation about the parameter. This is a result of the conservation of expected evidence: the expectation of the posterior probability of each value is its prior probability.

So when we talk about scientific standards -- about scientists revealing all relevant information and not p-hacking, etc. -- these are not requirements for Bayesian inference, but they're simply a way to ensure that scientific research is maximally informative. If you don't know the underlying process that scientists use to report their data, or if you know that they use a "biased" process, then your estimator of the relevant parameter will be less informative than it could have otherwise been.

The method they teach you in primary school to combat filtered evidence -- listing "arguments and counter-arguments", "pros and cons", etc. -- is far inferior to the standard of scientific ethics. Listing arguments and counter-arguments replaces a one-sided rationalization with a two-sided rationalization, but it doesn't truly approach rationality -- you just have two propagandists instead of one. 

I don't believe p-hacking is a problem.

Or more precisely: I don't believe p-hacking is a fundamental mathematical or statistical issue, but rather an issue with the methods adopted by experimental researchers.

If you haven't heard of p-hacking, it's as follows: suppose you want to find predictors for cancer. You test 100 possible predictors each at p-value 5%. Now although the chance of a false positive on any given test is 5%, you're expected to get 5 false positives in the test. So you can "always" find a (fake) predictor for cancer just by surveying enough things.

Another way of putting it: even if your factors don't actually predict cancer, the probability distribution for the observed correlation for any one factor may look something like this.

So when you test for these factors, when the correlations you observe actually sample the above curve, you're faced with a question: do you believe the points that lie in the shaded region (beyond your significance level) actually reject the null hypothesis?

On one hand: they lie beyond your significance level. Individually, you need to reject those null hypotheses.

On the other hand, one can also think of a "mega-" null hypothesis as implying the above curve: since your points sample the curve, you need to accept the null hypothesis.

---

I believe the answer is to reject those null hypotheses, i.e. to not make any "corrections" for having tested multiple parameters.

Here are some explanations:

• An individual researcher "p-hacking" is fundamentally/mathematically no different from a large number of researchers investigating of various different parameters. Surely it makes no sense to argue that all positive results in the literature should be ignored, or that they should be evaluated at much stronger significance levels. 
• When you investigate a large number of parameters, the probability of a true positive is also higher (in a Bayesian sense). If your positives are more likely to be false than true when you're testing a hundred parameters, they were more likely to be false than true when testing one parameter too. Of course, you are much more likely to have false positives when testing more parameters, but that doesn't increase the chance that any given deduction is false, because there are more true positives, too. 
• Or in other words, the "mega-null hypothesis" probably isn't true. If the $\theta$ parameters are independent, then you'll probably have a large number of false null hypotheses. The "mega-null hypothesis" argument actually seems to assume zero probability of a true positive. 
• Equivalently: just apply Bayes's theorem/the fact that probability is commutative. (equivalent to the first point)

Also note how the Bonferroni correction has nothing to do with p-hacking: it applies to looking at probabilities/confidence levels of several hypotheses being true, not about one.

---

So, then, why do the consequences of p-hacking all seem to bizarre? Stuff like this:


Or this:

Full comic for context: xkcd 882

Surely we don't actually believe that margarine causes divorces in Maine, or that only green jelly beans cause acne?

And no, we don't.

Why not?

Because there is no a priori reason to suspect that margarine causes divorces in Maine. Because a priori, we know that it's highly likely that whether green jelly beans cause acne is correlated with whether all the other color of jelly beans cause acne (because it's very unlikely that colour itself causes acne). These correlations should be embedded in our prior.

But these aren't fundamental issues with the nature of statistics. These are issues with how frequentist researchers may often decide which hypotheses to test. One should have some theoretical justification to formulate a hypothesis: that's how you know the prior probability is significant. Unless you have a good theoretical model for why a certain correlation/etc. should hold, the hypothesis should not be tested.

This problem is particularly prevalent in the social sciences, where a "general mathematical theory" of social science does not exist. Even in economics, you often end up with pseudo-science like this: Want to expand the economy? Tax the rich! (this particular study was terrible on several levels: (1) the conflation of correlation and causation -- this is always a problem when you have temporal trends, because time is a hidden parameter; that's why you should do cross-sectional studies (2) the correlation was statistically insignificant by any standard (3) there was no theoretical justification for why progressive taxation would expand the economy, leading to the problems discussed in this post.)

This point is essentially the point made by several papers (links: [1][2]) discussing "multiple comparisons in a Bayesian setting" -- it is what is meant by claims like "the multiple comparisons problem disappears when you use a hierarchial Bayesian model with correlations between your parameters".