Barnard Test

Barnard exact model

• $n_t\in \mathbb{N}$,
  • the number of things in treatment group
• $N\in \mathbb{N}$,
  • the number of things in treatment group and control group
• $X_i^c$,
  • i.i.d. of bernoulli r.v with probability $p_c$.
  • i.e. $\{X_i^t\}$ follows binomial distribution.
• $x_i^c:=X_i^c(\omega )(i=1,\dots ,n_c)$,
• $X_i^t$,
  • i.i.d. of bernoulli r.v.with probability $p_t$
  • i.e. $\{X_i^t\}$ follows binomial distribution.
• $x_i^t:=X_i^t(\omega )(i=1,\dots ,n_t)$,
• $\mathrm{\Theta }:=[0,1]\times [0,1]$,

the number of YES condition in treatment group and the number of YES condition in control group are

$$\begin{array}{rcl}{x}^t& :=& \sum _{i=1}^{n_t}{x}_i^t\\ x^c& :=& \sum _{i=1}^{n_c}{x}_i^c.\end{array}$$

Condition	treatment group	control group	total
YES	$x^t$	$x^c$	$x^t+x^c$
NO	$n-x^t$	$N-n-x^c$	$N-x^t-x^c$
total	$n=c_t$	$N-n=c_c$	$N$

We assume $\{X_i^c\}$ and $\{X_i^t\}$ are independent. The probability of r.v.s are given by

$$\begin{array}{rcl}f(x^c,x^t;p_c,p_t)& :=& f(x_1^c,\dots ,x_{n_c}^c,x_1^t,\dots ,x_{n_t}^t;p_c,p_t)\\ & :=& P((\bigcap _i\{X_i^c=x_i^c\})\bigcap (\bigcap _i\{X_i^t=x_i^t\}))\\ & =& \mathrm{Bi}(x_1^c,\dots ,x_{n_c}^c;p_c)\mathrm{Bi}(x_1^t,\dots ,x_{n_t}^t;p_t)\\ \text{(1)}& & =& \left(\begin{array}{c}{n}_c\\ x_c\end{array}\right)p_c^{x_c}(1-p_c{)}^{n_c-x_c}\left(\begin{array}{c}{n}_t\\ x_t\end{array}\right)p_t^{x_t}(1-p_t{)}^{n_t-x_t}\end{array}$$

In Barnard’s exact model, the null hypothesis is $H_0:=\{(p,p)\in \mathrm{\Theta }\mid p\in [0,1]\}$. Under the null hypothesis, the p.d.f is given by

$$\begin{array}{}\text{(2)}& f(x^c,x^t;p,p)& =& \left(\begin{array}{c}{n}_c\\ x_c\end{array}\right)\left(\begin{array}{c}{n}_t\\ x_t\end{array}\right)p^{x_c+x_t}(1-p{)}^{N-x_c-x_t}\end{array}$$

If we give the value of $p$, we can calculate the value of p.d.f. Let $Y$ be some measurable function to determine decision area.

$$\hat{p}(p):=\sum _{({\hat{x}}^c,{\hat{x}}^t):Y({\hat{x}}^c,{\hat{x}}^t)\ge Y(x^c,x^t)}f({\hat{x}}^c,{\hat{x}}^t)$$

p-values is defined by

$$p_{\mathrm{Barnard}}:=sup\{\hat{p}(p)\mid p\in (0,1)\}.$$

Possible $Y$s are

• Wald statistics,
• Score statistics,
  • aka Z-pooled

$$\begin{array}{rcl}\overline{\pi }& :=& \frac{x^c+x^t}{N}\\ {\overline{\pi }}_t& :=& \frac{x^t}{n}\\ \text{(3)}& {\overline{\pi }}_c& :=& \frac{x^c}{N-n}\text{(4)}& c_t& :=& n{c}_c& :=& N-n\end{array}$$

Wald statiscs

$$\begin{array}{rcl}Y(x^c,x^t)& :=& \frac{{\overline{\pi }}_c-{\overline{\pi }}_t}{\sqrt{\frac{{\overline{\pi }}_t(1-{\overline{\pi }}_t)}{c_t}+\frac{{\overline{\pi }}_c(1-{\overline{\pi }}_c)}{c_c}}}\end{array}$$

Score statistics

$$\begin{array}{rcl}Y(x^c,x^t)& :=& \frac{{\overline{\pi }}_c-{\overline{\pi }}_t}{\sqrt{\overline{\pi }(1-\overline{\pi })(\frac{1}{c_t}+\frac{1}{c_c})}}\end{array}$$

Let $\alpha \in [0,1]$ be significance level. If

$$\underset{p\in (0,1)}{sup}\hat{p}(p)\le \alpha ,$$

the null hypothesis is rejected.

Difference between Fisher’s exact test and Barnard test

See Fisher’s exact test.

Fisher’s exact test considers a set of experiments is single random variable. For instance, the number of Treatment Groups with YES condition is a random varaible with hypergeometric distribution. The each experiment in Treatment Groups is not considered as a random variable. In this model, adding another result of experiment is a bit off because the each experiment is not modeled as a random variable. Intuitively, the set of experiments should be conducted at the (almost) same time.

On the other hand, in Barnard test, each experiment is a random variable with bernoulli distribution and the experiments are independent. Each experiment could be conducted at the same situation to ensure independence of each experiment. Note that this does not mean we can add another result of experiment after observation of experiments. Modyfing the results of experiments are not allowed.

Intuitively, the Fisher’s exact test seems less flexible than Barnard test.

Example

• $N=15$,
• $x_i^c\in \{0,1\}$,
  • 1 means that $i$-th person became infected with influenza
  • people innoculated with a recombinant DNA influenza vaccine
  • control group
• $x_i^t\in \{0,1\}$,
  • 1 means that $i$-th person became infected with influenza
  • people innoculated with a placebo
  • treatment group
• $x^c:=\sum _{i=1}^N{x}_i^c$,
  • the number of infected people in the control group
• $x^t:=\sum _{i=1}^N{x}_i^t$,
  • the number of infected people in the treatment group

Infection status	Vacctine	Placebo	Total
Yes = 1	7 = $x^t$ (47%)	12 = $x^c$ (80%)	19
No = 0	8 (53%)	3 (20%)	11
Totals	15	15	30

Barnard exact model

• $X_i^c$,
  • i.i.d. of bernoulli r.v with probability $p_c$.
  • i.e. $\{X_i^t\}$ follows binomial distribution.
• $X_i^c(\omega )=x_i^c(i=1,\dots ,N)$,
• $X_i^t$,
  • i.i.d. of bernoulli r.v.with probability $p_t$
  • i.e. $\{X_i^t\}$ follows binomial distribution.
• $X_i^t(\omega )=x_i^t(i=1,\dots ,N)$,

Score statistics of this contingency table is

$$\begin{array}{}\text{(5)}& Y(x^c,x^t)& \approx & 1.8943380760602064\end{array}$$

One-side test

$$\begin{array}{}\text{(6)}& \hat{p}(p)& =& \sum _{({\hat{x}}^c,{\hat{x}}^t):Y({\hat{x}}^c,{\hat{x}}^t)\ge Y(x^c,x^t))}f({\hat{x}}^c,{\hat{x}}^t)\end{array}$$

Let $\alpha \in [0,1]$ be significance level. If

$$\underset{p\in (0,1)}{sup}\hat{p}(p)\le \alpha ,$$

the null hypothesis is rejected.

Reference

• Lecture 14: Statistical Significance of 2x2 Contingency Tables, Part 2
• chi squared - Which test for cross table analysis: Boschloo or Barnard? - Cross Validated
• breakfast-club-python/barnard.py at master · roemera/breakfast-club-python
• Peter, C. (2016). Package ‘ Exact .’ Journal of the American Statistical Association, 89, 1012–1016.
• Mehta, C. R., & Senchaudhuri, P. (2003). Conditional versus Unconditional Exact Tests for Comparing Two Binomials.