The Scientific Research Process

The system of science empowers you to make your own judgements about the evidence. You don't have to believe everything you read in the newspapers or even what a scientist tells you.

hero image
Tip - If you have a blog and want your featured images to look great, it helps to use the same color palette and font style across all of them.

What is the research process?

The research process is a system for trying to find out the true answer to questions.

  1. Scientists begin with an observation about the world. Something they notice, which they then use to generate a research question.
  2. A research question is a specific inquiry about a problem or a concern, which is used to try and help you establish a theory. An example of a research question might be: “Do free gifts help to sell t-shirts?”
  3. A theory is a principle, or a set of principles (theory) which explain why something is likely to happen or not in a specific context. An example of a theory for our research question (based on an existing consumer research theory) might be: “People buy things when they believe they are getting value for money.” Scientists are generally interested in theories that apply very generally (across all entities and situations). An entire set of entities is known as a population which can be diverse (every band on the planet), or very specific (bands that play a specific style of music).
  4. You can use existing theory to generate a hypothesis, which is a proposed explanation for the specific observation that interests you (we did this above): “T-shirt sales increase because a free gift improves the value for money.”
  5. We can test a hypothesis by operationalising it as a prediction about what will happen. We might predict from our hypothesis that “If a band offers a free gift, they will sell more T-shirts”. A prediction should be a scientific statement that can be verified (or not) using data. This means that you can break the statement down into things that can be measured (variables). E.g. Number of T-shirts sold, and whether a free gift was given or not.
  6. The next step in the process is to collect data. A constraint is that we usually don’t have access to the entire population to collect data from, so instead we collect data from a sample (smaller set of entities from our chosen population). We choose our set of entities randomly to ensure they are representive of the wider population. We can use the data to compute statistics, which are values that describe our sample. E.g. “The average number of T-shirts sold in our sample” in our sample is a statistic. We can then use this value to estimate what the value would have been if we had collected data from the entire population (parameter).
  7. Once we have our data, we use it to decribe what happened in the sample that we collected by conducting data analysis. We might draw a graph of the data, or calculate some summary information such as the average T-shirt sales. This is known as descriptive statistics.
  8. Scientists usually want to generalise results beyond the data they collected to the entire population. They use their sample data to estimate what the likely values are in the population. This is known as inferential statistics.

How is science a life skill?

The system of science empowers you to make your own judgements about the evidence. You don’t have to believe everything you read in the newspaper or what a scientist tells you. You can look at the science for yourself. You won’t be subject to the goals of journalists who might want to spin the data to write a good story, or politicians who might spin the data to make themselves look better, or a salesman who it might suit to play down the risks.

What are research methods?

There are different ways to collect data and different types of data to collect.

We can test a hypothesis in one of two ways: by observing what naturally happens (correlational research), or by manipulating some aspect of the environment and observing the effect it has on the variable that interests us (experimental).

What are correlational research methods?

Correlational research is where you observe what naturally happens in the real world without interfering with it - the measures of the variables should not be biased by the researcher being there. This makes it more likely that the results of the study will have ecological validity, which means that the results can be applied to real-life situations.

A problem with correlational research is that it tells us nothing about whether one variable causes another.

  • Cross-sectional study: Is where we take a snapshot of many variables at a single point in time. The problem with this is that we can’t infer a causal relationship between the two variables.
  • Longitudinal study: Is where we measure variables repeatedly at different time points. We are better able to infer a causal relationship between variables here, but we can’t know whether other variables that we can’t control (confounding variables) have influenced our results or not. When a third variable explains the relationship between two variables, that is known as a tertium quid.

Most scientific questions break down into a proposed cause and a proposed outcome. The cause and outcome are both variables (because they vary). We answer the research question by uncovering the relationship between the proposed cause and the proposed outcome.

Cause and Effect Rules

  1. Cause and effect must happen in a similar time frame (called contiguity).
  2. The cause should happen before the effect.
  3. The cause and effect should always co-occur. (Hume)
  4. All other explanations of the cause-effect relationship must be ruled out (Mill). To do this, an effect should be present when the cause is present and absent when the cause is absent. The only way to show causality is to compare two controlled situations, one in which the cause is present and one in which the cause isn’t present.

What are experimental research methods?

Experimental methods is where you compare two conditions in a controlled way: Where the proposed cause is present or absent, while controlling for all other variables that might influence the effect in which we’re interested. This is known as an experiment.

If we give a way a free gift along with T-shirts we are selling in a band, the free gift is known as the independent variable because it is not affected by the other variables in the experiment. It is also known as the predictor variable, because it can be used to predict the scores of another variables (whether more T-shirts are sold as a result of the free gift or not).

While the T-shirt sales variable is known as the dependent variable, because we assume that its value will depend on whether or not a free gift was provided or not. It is also known as the outcome variable, because it is the variable that we’re trying to predict the values of (we want to know how many sales were made).

The no-gift group is critical because this is a group in which our proposed cause (incentive to buy) is absent, and we can compare the outcome in this group against the situation in which the proposed cause is present. If the T-shirt sales are different when the free gift is offered (cause is present) copared to when it is not (cause is absent) then this difference can be attributed to the free gift.

Two methods of data collection:

We can manipulate variables in experiments in two ways:

  • Between-groups, between-subjects or independent design: Manipulate the independent variables using different entities - we allocate different bands (entities) to two different groups.
  • Within-subject, related or repeated measures design: Where we manipulate the independent variable using the same entities - we ask a band to give a free gift with each T-shirt sale in one concert, and no free gift in another concert of theirs.

Two types of variation

  • Unsystematic variation: Is when there are small differences caused by unknown or unmeasured factors. A band is likely to make similar T-shirt sales across performances, with little variation (unsystematic)
  • Systematic variation: is when differences are caused by specific experimental manipulation. If there is a difference between T-shirt sales made with gifts and T-shirt sales made without gifts is likely to be because of the manipulated variable (systematic).

If you use two different bands, then you would expose both bands to both conditions (free gift, and not free gift). Unsystematic variation will be bigger for an independent design than for a repeated measures design, because different groups are likely to have more tiny differences that will contribute to score variation (different music, t-shirt designs, prices etc).

What are examples of practice, order and randomization?

Practice effects and bordom effects on systematic variation

  • A band might sell less T-shirts in the second concert if people already bought T-shirts in the first concert (Practice).
  • A puzzle solver may do better in their second puzzle after having had practice with the first puzzle (Practice).
  • Completing similar quizzes multiple times but imagining yourself as a different person each time. You might be pretty bored or fatiqued by the fifth quizz which could influence the results (Bordom).

Order:

Counterbalancing is a technique that can be used to eliminate sources of the above sources of unsystematic variation, by changing the order in which groups are exposed to conditions (with the free gift first, or without the free gift first).

  • Latin Square Design: An experiment with three conditions and three different groups. Group A are exposed to the conditions in 123 order, group B are exposed to the conditions in 312 order and group C are exposed to the conditions in 231 order.

Randomization

Randomization is where you randomly allocate participant to each different control group, to try and minizise the differences between the two groups.

Sometimes, you can’t randomise people into two groups. E.g. It would be unethical to randomly select children to watch horror movies when some might be disturbed by them. So instead you’d wait for the children who choose to watch horror movies themselves. This is known as quasi-experimental design.

Why do we need science?

We need science to see through all of the bullshit and make our own informed decisions.