Design of Experiments > Research Methods
Quantitative Research vs. Qualitative.
Quantitative research is statistical: it has numbers attached to it, like averages, percentages or quotas. Qualitative research uses non-statistical methods. For example, you might perform a study and find that 50% of a district’s students dislike their teachers. The quantity (50%) makes it quantitative research. A follow up qualitative study could interview a small percentage of those students to find out why. The answers are free-form and don’t have numbers associated with them, so that makes them qualitative.
- What is Qualitative Research?
- Quantitative Research Methods.
- Basic Research Methods.
Qualitative research(QR) is way to gain a deeper understanding of an event, organization or culture. Depending on what type of phenomenon you are studying, QR can give you a broad understanding of events, data about human groups, and broad patterns behind events and people. While traditional lab-based research looks for a specific “something” in the testing environment, qualitative research allows the meaning, themes, or data to emerge from the study.
Qualitative research uses non-statistical methods to gain understanding about a population. In other words, you’re not dealing with the numbers you’d find in quantitative research. For example, let’s say your research project was to answer the question “Why do people buy fast food?”. Instead of a survey (which can usually be analyzed with math), you might use in-depth interviews to gain a deeper understanding of people’s motives. Another major difference between qualitative and quantitative research is that QR is usually performed in a natural setting (As opposed to a lab).
All of the different qualitative research methods have several characteristics:
- Findings are judged by whether they make sense and are consistent with the collected data.
- Results are validated externally by how well they might be applicable to other situations. This is tough to do; rich, detailed descriptions can help to bolster external validity.
- Data is usually collected from small, specific and non-random samples.
Although qualitative research doesn’t have the same structure as a formal lab-testing environment, there are certain requirements you must meet in order for your qualitative study to be called “research.” Your study must:
- Have a formulated research purpose. For example “Examine the lifestyles of Chinese immigrants.”
- Be related to existing theories, published or unpublished. You can’t just make up an idea that has no basic in current thinking. For example, a study to see how immigrants cope in the workplace would build on previous, similar studies. However, there would be no previous theories for something out-of-left field like “Why do Italian immigrants prefer Pepsi?”
- Be well-planned. You can’t go into the jungle with no plan and no idea of how you’re going accomplish your goals.
- Be recorded carefully with notes and other media like film or voice recordings. If you don’t take careful notes, you could miss something of vital importance.
Other rules you must follow include selecting the people or events you want to observe, having a plan on how you’re going to get into the “world” you want to observe, and deciding ahead of time what types of data you’re going to gather.
A semistructured interview is a way of research which allows room for new information and insights to be incorporated as the interview is run.
While structured interviews are based on a well-defined set of questions that are settled beforehand, semistructured interviews work from only a general framework of themes. A number of questions may be determined beforehand, especially initial questions, but freedom is left for deviation—within limits. The interviewer is expected to formulate most questions on the fly, based on the interviewees responses, and has the freedom to follow relevant tangents and ideas as they come up throughout the interview.
A semi-structured interview is more structured than an open interview, in which the interviewee is expected to chat about whichever topic interests him, but it still gives a fair degree of freedom and allows the interviewee to take the initiative to a degree.
Advantages and Disadvantages of a Semistructured Interview
A semi structured interview format is often chosen because the open format mitigates bias that might be imposed by the leading questions which often form a part of structured interviews. It also allows an interviewer to incorporate new information and follow new ideas as they come up in the interview, without being bound by a preconceived set of ideas.
The main negative of using a semi structured interview format is that it requires some skill on the part of the interviewer, who must be able to establish rapport with the interviewee and allow enough room to explore related ideas while still keeping focus. Data from semi structured interviews is also typically harder to organize and analyze than data from a structured interview might be.
Planning and Conducting a Semi Structured Interview
While you don’t need to write down a complete set of questions for a semistructured interview, you will want to plan extensively:
- Prepare a thorough list of topics and write down some tentative questions.
- Think about different ways of approaching the ideas you want to discuss.
- Make sure the interview site is comfortable and conducive to discussion, as the success of your interview depends on your interviewee feeling free to talk.
- Begin by introducing your topic and research, then ask a few general background questions.
Go on to your topic, asking questions that instigate discussion (as opposed to yes/no replies). Allow the interviewee freedom to expand on side issues that particularly interest him, but be prepared to bring the topic back to the main topic as needed. At the end of the interview, give the interviewee a chance to share any additional points he or she feels is important.
Zorn, Ted. Designing and Conducting SemiStructured Interviews for Research. Retrieved from http://home.utah.edu/~u0326119/Comm4170-01/resources/Interviewguidelines.pdf on June 23, 2018.
Jamshed, Shazia. Qualitative research method-interviewing and observation. Journal of Basic and Clinical Pharmacy. September 2014-November 2014; 5(4): 87–88. doi: 10.4103/0976-0105.141942. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4194943/ on June 23, 2018.
Oxfam. Conducting Semi-structured Interviews. Retrieved from https://itp.nyu.edu/classes/fungus/interview_technique/conductingInterviews.pdf on June 24, 2018
Anthropological researchers study people in their natural environment, sometimes immersing themselves in foreign cultures for years. The focus is on meaning, transmitted through language and action. For example, a raised finger might mean something trivial in one society while in another it’s a gross insult. This type of research is invaluable when it would be inappropriate or impossible to put people in a laboratory setting or even conduct a simple interview. For example, members of a street gang would likely refuse interviews, and it’s both highly expensive and inappropriate to place people from non-Western cultures (i.e. pygmies) in a lab. People from different cultures might react in ways that aren’t expected. For example, they might be fearful of a laboratory setting. Or they might react in ways that you don’t understand. For example, it might be culturally acceptable to lie when being interviewed by a “reporter.” Observing people in their natural setting helps to eliminate these factors.
In this research method, you use your own experiences to address a cultural, political, or social issue. It is considered by many to be a non-traditional ethnographic method. This type of research can involve several people. For example, a group of immigrant women researchers conducted a study on how they navigated the US academy as immigrant women faculty (Ngunjiri et. al 2010).
Critical Social Research.
Critical social research studies specific oppressive social structures (Harvey, 1990). This type of research attempts to expose problems, evaluate the problems and find their root causes. For example, critical social research could attempt to uncover cases of juvenile crime, racism, or suicide. The main difference between this type of research and other qualitative types is that there is always “a problem” that needs “fixing” going into the study. The research question revolves around an existing, known, problem. Traditional research uncovers problems or issues with interviews, data collection and other QR methods.
Ethnography is the study of people in their own environment through methods like participant observation and face-to-face interviewing. According to ethnographer David Fetterman, this method gives a voice to people in their own local context, giving a complete description of events as they happen.
[Ethnography] is about telling a credible, rigorous, and authentic story.
There is a small, but important difference between ethnographic research and anthropological research:
- Anthropological research involves all of the cultures on the planet,
- Classic ethnographic research provides a detailed description of an complete culture outside of the researcher’s country of origin(Ingold, 2008).
Traditionally, ethnographers spend at least one year inside the culture they are studying, relying heavily on participant observation as the key data collection method. Other methods used include field notes, focus groups, interviews, and surveys.
Field research is research outside of a lab, in a natural setting. This type of research usually involves first hand note-taking. It may also include video footage, interviews with experts in the area being studied, conducting surveys or attending public discussion forums.
Grounded Theory Research.
Grounded theory is often categorized as a qualitative research method, but technically it can be applied to either quantitative research or qualitative research; it’s a general research method involving a set of rigorous procedures. The result is (hopefully) a set of conceptual data categories. The “Theory” in grounded theory refers to the theory of what you are studying. For example, you might have a theory about the eating habits of the Nez Perce tribe. The “Grounded” part refers to the fact that your theory needs to be grounded in research. That’s the simple definition. In reality, it’s a type of QR that’s poorly understood, with many researchers claiming they used it, when in fact they did not. It’s actually a fairly complicated process that builds upon itself. You start by generating questions to guide your research. These questions identify core concepts, which lead you to identifying links between your questions and your data. This part of the process can take months.
Naturalistic research is research that doesn’t manipulate anything in the environment. In other words, it’s the opposite of a lab environment where variables are manipulated on purpose. Care should be taken with naturalistic research, as even your presence can alter the environment–taking away the “naturalistic” component. Bias can easily creep in to these types of studies; two people can have different viewpoints of the same thing. It’s a common safeguard to have two or more researchers observing the same thing so that any differences in viewpoint can be addressed.
Participant Observer Research.
In this type of research, you participate in the activity and record observations. It differs from naturalistic research because you actually participate in the activity you are researching. For example, you might become a member of a cult, enroll in spelunking courses, or go undercover as a dishwasher at a restaurant. Although this is a great way to get an insider viewpoint, it carries risks. For example, bias and reactivity are magnified in participant observer research. Bias seeps in because you are looking through one lens (yours). As hard as you try to make sure your notes are accurate, you might have biases that you aren’t aware of. Reactivity is where your actions change what is happening. As you are more than just an observer, it’s difficult not to have some influence in what’s going on around you. A fairly famous example of this type of research was undertaken by Leon Festinger, Henry Riecken and Stanley Schachter, who infiltrated a UFO religion called “The Seekers.”
Phenomenology studies someone’s perception of an event. The focus of this type of study is what people’s experiences are for a particular event and how they interpret their experiences. For example: a study of Hurricane Katrina survivors perceptions, understandings, and perspectives of the hurricane.
Qualitative research is not part of statistical analysis. That’s because the results can’t be tested to see if they are statistically significant (i.e. to see if the results could have occurred by chance). As a result, findings can’t be extended to a wider population. That doesn’t mean this type of research is useless: in many studies, getting hard numbers is inappropriate or just impossible.
If qualitative research can’t be used to estimate statistics for a population, why use it at all? One reason is that while statistics concentrates on specific, narrow areas (for example, population means, medians or standard deviations), qualitative analysis paints a wider, complete picture. In addition, phenomena that’s rare receives the same level of attention as more common phenomena. Other advantages include:
- It’s useful for finding out more about complex situations.
- Allows the use of an “insider viewpoint.”
- Data is based on the participant’s views of the world, rather than a world created by a researcher.
- Can be used to figure out how people interpret constructs like IQ or fear, which can be hard to quantify.
- The study focus can be shifted in the middle of research, if necessary. In a traditional lab setting, this would usually null-and-void the experiment.
- An important case can be used to vividly paint a picture in a report.
One of the main disadvantages to qualitative research is that your data usually can’t be generalized outside of your research. For example, if you find that an Asian street gang has a certain hierarchy, then that hierarchy likely exists only within Asian street gangs, and perhaps only in the particular gang you studied.
- Predictions of future events are usually impossible to quantify (i.e. you wouldn’t be able to say “there is a 95% chance of this event happening in the future.”).
- Qualitative research, in general, has lower credibility than quantitative research. This may make it more difficult to get your results published.
- Data collection takes a lot longer than in a traditional lab setting.
- Your own personal biases and other idiosyncrasies are more likely to affect the research.
Both qualitative and quantitative research methods have their limitations. There is a recent trend towards a multi-method research approach which uses both types to:
- Quantify phenomena and make sure it’s statistically sound.
- Paint a broader picture of the phenomena.
What is Quantitative Research?
Quantitative research is about collecting and analyzing data to explain phenomena. Information from a sample is used to make generalizations or predictions about a population. Some questions that are easily answered using information from samples include:
- What percentage of high school teachers belong to minority groups?
- How many females in college study mathematics compared to males?
- Has the high school graduation rate in our district increased over time?
However, data doesn’t always naturally happen in a numerical way. You may want to answer questions like:
- What do high school students think of their teachers?
- What is the general public opinion of health care reform?
- What do customers at a particular business think of customer service?
These questions aren’t immediately quantifiable, but you can turn them into quantifiable questions by assigning numbers to them. For example, you could make a survey with the following question and responses:
“I think that customer service at this business is excellent.”
- Strongly Agree.
- No opinion.
- Strongly disagree.
Elements of Quantitative Research Methods.
The crucial elements of quantitative research design are:
- Research Design. Quantitative research designs are either descriptive or experimental. Descriptive designs are where you measure an association between two variables (independent and dependent variables). Sample sizes are typically large. For example, you might survey thousands of local students. Subjects are usually only measured once. In an experimental design, subjects are usually measured both before and after a treatment and you’re looking for causality. Sample sizes tend to be small. For example, you might be analyzing a treatment for a small number of cancer patients.
- Choice of data collection instrument. Typical methods for data collection are surveys or questionnaires with closed-ended questions, using data from another source (for example, a government database) or an experiment with a control group and an experimental group.
- Choice of analysis tool (i.e. statistics). For example, you might choose to report your results using confidence intervals and test statistics from t tests or f tests with significance levels (alpha levels) and p values.
- Archival research. Researching through archives: rare books, historical records and other historical data like school yearbooks.
- Content analysis. Content analysis looks at whether certain words or phrases are present in texts. Inferences can then be made about the writer, the audience and the culture surrounding those texts.
- Case study. Case studies are usually performed on an individual or small group of individuals. A detailed history is taken of some aspect of the participant’s life. For example, you might make a case study of a group of 12 alcoholics on their drinking habits, personal history and home life.
- Computer simulation. Computer modeling is one of the research methods gradually becoming more popular especially, where ethical constraints prevent actual experiments or observation. For example, instead of watching the effects of a poisonous gas on mice, a computer simulation can model the effect without the need for live animals.
- Experiment. An experiment is where you deliberately manipulate one variable (the independent variable) to see what the outcome is on another variable (the dependent variable). Experiments are typically performed in a closed setting, like a laboratory. Field experiments are where the experiment takes place outside of the controlled setting, in the real world.
- Interview. In an interview, you meet face to face with participants. Taping an interview is usually preferable, as note-taking can be distracting. Interview questions can be closed–where participants are given a choice of answers–or open questions, which allow for thoughtful, in-depth responses. An interview can also be a mixture of both.
- Meta-analysis. A meta-analysis combines the findings from existing research into one, comprehensive thesis. A meta-analysis can uncover trends or themes that weren’t apparent in individual pieces of research.
- Observational Study. A type of study where the researcher observes participants without any kind of interference. Participants are placed into two groups with one control group and one experimental group (i.e. smokers and non-smokers).
- Naturalistic Observation. In naturalistic observation, you observe participants in their natural setting. You should be inconspicuous and not change anything in the setting.
- Unobtrusively observing wild chimpanzees
- Sitting in a coffee shop and observing interactions between patrons
- Survey. Surveys usually involve a representative sample of the population, using a technique like random sampling. A questionnaire is given to each member of the sample and used to infer characteristics of the whole population. Surveys are easy in theory but can be difficult to put into practice, mainly because of a typically low response rate.
Asad, T. (1994). Ethnographic Representation, Statistics and Modern Power. Social Research Vol. 61, No. 1. Retrieved January 6, 2021 from: https://www.jstor.org/stable/40971022?seq=1
Atkinson, P. et al. (Eds.). Handbook of Ethnography 1st Edition. SAGE Publications.
Beyer, W. H. CRC Standard Mathematical Tables, 31st ed. Boca Raton, FL: CRC Press, pp. 536 and 571, 2002.
Dodge, Y. (2008). The Concise Encyclopedia of Statistics. Springer.
Fetterman, D. (2009). Chapter 17: Ethnography. In The SAGE Handbook of Applied Social Research Methods.
Ingold, T. (2008). Anthropology is not ethnography. Proceedings of the British Academy, 154, 69-92.
Kotz, S.; et al., eds. (2006), Encyclopedia of Statistical Sciences, Wiley.
Vogt, W.P. (2005). Dictionary of Statistics & Methodology: A Nontechnical Guide for the Social Sciences. SAGE.