Statistics How To

Bayesian Statistics > The maximum entropy principle is a rule which allows us to choose a ‘best’ from a number of different probability distributions that all express the current state of knowledge. It tells us that the best choice is the one with maximum entropy. This will be the system with the largest remaining uncertainty, … Read more

Statistics Definitions > An admissible decision rule is a rule for making a statistical decision; There isn’t any other rule which is, generally speaking, better. If it’s not admissible, then it’s inadmissible. An inadmissible decision rule is never worth using, since by definition there will always be another rule that is better than it. But … Read more

Hypothesis Testing > Decision rule This article is about the decision rule used in Hypothesis Testing. For the decision rule used in clinical trials, see: Adaptive Design Clinical Trials. A decision rule spells out the circumstances under which you would reject the null hypothesis. The null hypothesis is the backup ‘default hypothesis’, typically the commonly … Read more

Bayesian Statistics > A credible interval is the interval in which an (unobserved) parameter has a given probability. It’s the Bayesian equivalent of the confidence interval you’ve probably encountered before. However, unlike a confidence interval, it is dependent on the prior distribution (specific to the situation). In confidence intervals we also treat the parameter as … Read more

Dirichlet process > The Chinese Restaurant Process is a metaphorical way for how a Dirichlet process generates data. The Dirichlet process models randomness of a probability mass function (PMF) with unlimited options (e.g. an unlimited amount of dice in a bag). It’s specifically called the Chinese Restaurant Process because the algorithm’s creators, Jim Pitman and … Read more

Statistics Definitions > Bell’s numbers Bell’s Numbers and the Bell Triangle (sometimes called the Pierce triangle or Aitken’s array) are a sequence of numbers which count the possible partitions of a set, and the triangle which makes derivation of them easy. Bell’s Numbers: What they Are and What they Mean The first Bell numbers are: … Read more

Statistics Definitions > Jeffrey’s prior (also called Jeffreys-Rule Prior), named after English mathematician Sir Harold Jeffreys, is used in Bayesian parameter estimation. It is an uninformative prior, which means that it gives you vague information about probabilities. It’s usually used when you don’t have a suitable prior distribution available. However, you could choose to use … Read more

Statistics Definitions > A hierarchical model is a model in which lower levels are sorted under a hierarchy of successively higher-level units. Data is grouped into clusters at one or more levels, and the influence of the clusters on the data points contained in them is taken account in any statistical analysis. For example, in … Read more

Descriptive Statistics > A radar chart is a 2D chart presenting multivariate data by giving each variable an axis and plotting the data as a polygonal shape over all axes. All axes have the same origin, and the relative position and angle of the axes are usually not informative. The equi-angular spokes, from the origin … Read more

Statistics Definitions > A free parameter is one which is not pre-defined by the model, but which can be chosen or estimated based on theoretical ideas or experimental data. Other types of parameters include fixed and constrained. Fixed parameters are completely defined by the model; for example, if your model defines a parameter j = … Read more