Statistics How To

Confidence Level: What is it?

Statistics Definitions > Confidence Level

confidence level

When a poll is reported in the media, a confidence level is often included in the results. For example, a survey might report a 95 percent confidence level. But what exactly does this mean? At first glance you might think that it means it’s 95 percent accurate. That’s close to the truth, but like many things in statistics, it’s actually a little more defined.

Real Life Example

Example: A recent article on Rasmussen Reports states that “38% of Likely U.S. Voters now say their health insurance coverage has changed because of Obamacare”. If you scroll down to the bottom of the article, you’ll see this line: “The margin of sampling error is +/- 3 percentage points with a 95% level of confidence.”

It’s impractical to survey all 300 million+ U.S. residents, so it’s impossible to know exactly how many people would actually respond “yes my health insurance has changed.” We take a sample (say, 2,000 people) and, using good statistical techniques like simple random sampling, take our “best guess” at what that actual figure is (we call that unknown figure a population parameter). What a 95 percent confidence level is saying is that if the poll or survey were repeated over and over again, the results would match the results from the actual population 95 percent of the time.

What about “+/- 3 percentage points”?

The width of the confidence interval tells us more about how certain (or uncertain) we are about the true figure in the population. This width is stated as a plus or minus (in this case,+/- 3) and is called the confidence interval. When the interval and confidence level are put together, you get a spread of percentage. In this case, you would expect the results to be 35 (38-3) to 41 (35+3) percent, 95% of the time.

Factors that Affect Confidence Intervals (CI)

  • Population size: this does not usually affect the CI but can be a factor if you are working with small and known groups of people.
  • Sample Size: the smaller your sample, the less likely it is you can be confident the results reflect the true population parameter.
  • Percentage: Extreme answers come with better accuracy. For example, if 99 percent of voters are for gay marriage, the chances of error are small. However, if 49.9 percent of voters are “for” and 50.1 percent are “against” then the chances of error are bigger.

0% and 100% Confidence Level

A 0% confidence level means you have no faith at all that if you repeated the survey that you would get the same results. A 100% confidence level means there is no doubt at all that if you repeated the survey you would get the same results. In reality, you would never publish the results from a survey where you had no confidence at all that your statistics were accurate (you would probably repeat the survey with better techniques). A 100% confidence level doesn’t exist in statistics, unless you surveyed an entire population — and even then you probably couldn’t be 100 percent sure that your survey wasn’t open to some kind or error or bias.

Confidence Coefficient

The confidence coefficient is the confidence level stated as a proportion, rather than as a percentage. For example, if you had a confidence level of 99%, the confidence coefficient would be .99.

In general, the higher the coefficient, the more certain you are that your results are accurate. For example, a .99 coefficient is more accurate than a coefficient of .89. It’s extremely rare to see a coefficient of 1 (meaning that you are positive without a doubt that your results are completely, 100% accurate). A coefficient of zero means that you have no faith that your results are accurate at all.

The following table lists confidence coefficients and the equivalent confidence levels.

Confidence coefficient (1 – α) Confidence level (1 – α * 100%)
0.90 90 %
0.95 95 %
0.99 99 %
------------------------------------------------------------------------------

If you prefer an online interactive environment to learn R and statistics, this free R Tutorial by Datacamp is a great way to get started. If you're are somewhat comfortable with R and are interested in going deeper into Statistics, try this Statistics with R track.

Comments are now closed for this post. Need help or want to post a correction? Please post a comment on our Facebook page and I'll do my best to help!
Confidence Level: What is it? was last modified: October 31st, 2017 by Stephanie Glen

4 thoughts on “Confidence Level: What is it?

  1. Bryan W

    The statement about the +/- 3 percentage points sounds incorrect. Shouldn’t it be that if the survey were conducted over and over, that the results would fall between 38-3=35% and 38+3=41% in 95% of the survey instances?

  2. Andale Post author

    Yes, you are completely right. I updated it. Thanks for spotting that error.

  3. Maria

    Shouldn´t it be:
    However, if 49.9 percent of voters are “for” and 50.1 percent are “against” then the chances of error are BIGGER.

    thanks,
    Maria