Probability Distributions > Rademacher distribution
What is a Rademacher Distribution?
The Rademacher distribution is a recoding of the Bernoulli distribution with two possible values {-1, 1}. It’s second moment (the variance) equals 1; all other moments equal 0 [1]. It is named after German-American mathematician Hans Rademacher and denoted Rad½. Like the Bernoulli, a random variable has a 50% chance of a success and 50% chance of failure.
- Bernoulli: 0 (failure) and 1 (success),
- Rademacher: -1 (failure) and 1 (success).
The distribution is used for formulating statistical proofs, random sampling [1], and bootstrapping , where weights dg = {−1, 1} are called Rademacher weights [2]. In machine learning, it can be used to generate binary data, such as whether or not a customer clicks on an ad. It can also be used to create synthetic data sets for training and testing machine learning models. A sequence of successive sums of Rademacher random variables is a simple symmetrical random walk when the step size equals 1.
The analysis of the sum of independent and identically distribution (i.i.d.) Rademacher variables has resulted in various findings including concentration inequalities, such as the Bernstein inequalities [3], and anti-concentration inequalities, such as Tomaszewski’s conjecture [4]. In probability theory, the Rademacher distribution can be used to show that normally distributed and uncorrelated does not imply independent [5].
Rademacher distribution properties
The Rademacher distribution is a discrete probability distribution and can be described by a probability mass function (pmf):
The distribution can also be written in terms of the Dirac Delta function: f(k) = ½ (δ(k – 1) + δ(k + 1).
The cumulative distribution function (CDF) is given by:
Rademacher random variables can be defined in terms of Bernoulli random variables. If Y is a Bernoulli random variable, then X = 2Y −1 is a Rademacher random variable [6]. Conversely, if X is a Rademacher random variable, then (X + 1) /2 is a Bernoulli random variable.
These variables can also be defined in terms of the Laplace distribution. Given a Rademacher random variable X, if Y ~ Exp(λ) is independent from X, then XY ~ Laplace (0, 1/λ).
References
- Contreras, D. (2021). Estimation of Flexibility Potentials in Active Distribution Networks. Books on Demand.
- Miller, D. & Cameron, C. A Practitioner’s Guide to Cluster-Robust Inference.
- 4 Concentration Inequalities, Scalar and Matrix Versions. Retrieved May 21, 2023 from: https://ocw.mit.edu/courses/18-s096-topics-in-mathematics-of-data-science-fall-2015/87db5678b0405c1087b0e85181b64c1a_MIT18_S096F15_Ses12_14.pdf
- Proof of Tomaszewski’s Conjecture on Randomly Signed Sums.
- Miller, D. & Cameron, C. A Practitioner’s Guide to Cluster-Robust Inference.
- Border, C. Supplement 2: Review Your Distributions. Retrieved January 1, 2022 from: http://www.math.caltech.edu/~2016-17/2term/ma003/Notes/DistributionReview.pdf.