Credible interval | Jason Siu

TOC

What When do we use credible Interval?Why Why and How is Confidence interval and credible Interval different ?How Example : The interval ENIGMA (Lambert p207)Reference Supplementary questions :

What

Aka. highest posterior density interval (HPDI)

What is the difference between Credible interval (Bayesian) & Conf. interval (Frequentist)?

The properties of upper and lower bounds:

The upper and lower bounds of Credible interval are fixed.

For the frequentist confidence intervals' upper bounds are random variables

The properties of the estimated parameter:

For Credible interval, the estimated parameter is a random variable

For confidence intervals, the parameter as a fixed value.

The properties of Prior knowledge :

Credible intervals entails prior knowledge of the prior distribution, while the frequentist confidence intervals do not entail so.

When do we use credible Interval?

A P% interval estimate for a random variable Θ is an interval (a,b) where P(a < Θ < b) = P%.

A P% interval estimate for a parameter 𝜃 is called a credible interval. (BW)

Once you have computed a posterior distribution, it is often useful to summarize the results with a single point estimate or an interval. For point estimates it is common to use the mean, median, or the value with maximum likelihood. (TB p.354)

For intervals we usually report two values computed so that there is a 90% chance that the unknown value falls between them (or any other probability). These values dene a credible interval. (TB)

Besides point estimates, we often want a measure of confidence. A standard measure of confidence in some (scalar) quantity θ is the “width” of its posterior distribution. This can be measured using a 100(1 − α)% credible interval. (BR-ML)

The interval in the domain of a posterior probability that contains 95% of the distribution (BM4Hacker)

Calculated from the posterior density. In particular, a 95% credible region satisfies the condition that 95% of the posterior probability lies in this parameter range.

Why

Rather than provide a single point estimate, we can provide a range of credible values, like a confidence interval. (ETC2420)

To describe our uncertainty in the location of the parameter values. (Lambert)

Why and How is Confidence interval and credible Interval different ?

(The definition of uncertainty is different.)

The uncertainty here refers to our inherent uncertainty in the value of the parameter, estimated using the current sample, rather than an infinite number of counterfactual samples like Frequentist confidence intervals (Lambert p205)

Frequentists view data sets as one of an infinite number of exactly repeated experiments, and hence design an interval which contains the true value X% of the time across all these repetitions.

The Frequentist confidence interval represents uncertainty in terms of the interval itself. By contrast, Bayesians view the observed data sample as fixed and assume the parameter varies, and hence calculate an interval where X% of the parameter’s estimated probability mass lies. (Lambert p206)

(The params properties are different.)

In short from (BW) , the difference is that 𝜎 is fully known to the frequentist confidence intervals but fully unknown to Bayesian credible intervals (with neutral prior) .

(ETC2420 DanSimp Week 10 lecture )

◮ In the classical framework, the data y is random and the parameters are ﬁxed (but unknown)
◮ In the Bayesian framework, the parameters are random, while the data is held ﬁxed

How

A simple way to compute a credible interval is to add up the probabilities in the posterior distribution and record the values that correspond to probabilities 5% and 95%.

In other words, the 5th and 95th percentiles. (TB)

To find the credible intervals, we first need to find 𝛼.

For the one- sided intervals, that means solving the equation (1−𝛼) 100% = 95%
For the symmetric ones, solving(1−2𝛼)100% = 90%.
In both cases, 𝛼 = 0.05.
The intervals are then

Example : The interval ENIGMA (Lambert p207)

Reference

(ETC2420) Di cook Statistical Thinking

(BR-ML) Barber, D. (2012). Bayesian Reasoning and Machine Learning (1st ed.). Cambridge University Press. (Here)

(BM4Hacker) Bayesian Methods for Hackers: Probabilistic Programming and Bayesian Inference (amazonaws.com)

(BW) — The best source to get the gist. The Bayesian Way (amazonaws.com)

(Lambert) — The best source to get the gist. A Student’s Guide to Bayesian Statistics (amazonaws.com)

Supplementary questions :

There is a term called central posterior interval in (Lambert). You can have a look at the example on p. 204 of how to choose central posterior interval and e highest density interval (i.e Credible interval)