A Live Developer Journal

The Scientific Research Process

What is the research process?

The research process is a system for trying to find out the true answer to questions.

  1. Scientists begin with an observation about the world. Something they notice, which they then use to generate a research question.
  2. A research question is a specific inquiry about a problem or a concern, which is used to try and help you establish a theory. An example of a research question might be: “Do free gifts help to sell t-shirts?”
  3. A theory is a principle, or a set of principles (theory) which explain why something is likely to happen or not in a specific context. An example of a theory for our research question (based on an existing consumer research theory) might be: “People buy things when they believe they are getting value for money.” Scientists are generally interested in theories that apply very generally (across all entities and situations). An entire set of entities is known as a population which can be diverse (every band on the planet), or very specific (bands that play a specific style of music).
  4. You can use existing theory to generate a hypothesis, which is a proposed explanation for the specific observation that interests you (we did this above): “T-shirt sales increase because a free gift improves the value for money.”
  5. We can test a hypothesis by operationalising it as a prediction about what will happen. We might predict from our hypothesis that “If a band offers a free gift, they will sell more T-shirts”. A prediction should be a scientific statement that can be verified (or not) using data. This means that you can break the statement down into things that can be measured (variables). E.g. Number of T-shirts sold, and whether a free gift was given or not.
  6. The next step in the process is to collect data. A constraint is that we usually don’t have access to the entire population to collect data from, so instead we collect data from a sample (smaller set of entities from our chosen population). We choose our set of entities randomly to ensure they are representive of the wider population. We can use the data to compute statistics, which are values that describe our sample. E.g. “The average number of T-shirts sold in our sample” in our sample is a statistic. We can then use this value to estimate what the value would have been if we had collected data from the entire population (parameter).
  7. Once we have our data, we use it to decribe what happened in the sample that we collected by conducting data analysis. We might draw a graph of the data, or calculate some summary information such as the average T-shirt sales. This is known as descriptive statistics.
  8. Scientists usually want to generalise results beyond the data they collected to the entire population. They use their sample data to estimate what the likely values are in the population. This is known as inferential statistics.

How is science a life skill?

The system of science empowers you to make your own judgements about the evidence. You don’t have to believe everything you read in the newspaper or what a scientist tells you. You can look at the science for yourself. You won’t be subject to the goals of journalists who might want to spin the data to write a good story, or politicians who might spin the data to make themselves look better, or a salesman who it might suit to play down the risks.

What are research methods?

There are different ways to collect data and different types of data to collect.

We can test a hypothesis in one of two ways: by observing what naturally happens (correlational research), or by manipulating some aspect of the environment and observing the effect it has on the variable that interests us (experimental).

What are correlational research methods?

Correlational research is where you observe what naturally happens in the real world without interfering with it - the measures of the variables should not be biased by the researcher being there. This makes it more likely that the results of the study will have ecological validity, which means that the results can be applied to real-life situations.

A problem with correlational research is that it tells us nothing about whether one variable causes another.

Most scientific questions break down into a proposed cause and a proposed outcome. The cause and outcome are both variables (because they vary). We answer the research question by uncovering the relationship between the proposed cause and the proposed outcome.

Cause and Effect Rules

  1. Cause and effect must happen in a similar time frame (called contiguity).
  2. The cause should happen before the effect.
  3. The cause and effect should always co-occur. (Hume)
  4. All other explanations of the cause-effect relationship must be ruled out (Mill). To do this, an effect should be present when the cause is present and absent when the cause is absent. The only way to show causality is to compare two controlled situations, one in which the cause is present and one in which the cause isn’t present.

What are experimental research methods?

Experimental methods is where you compare two conditions in a controlled way: Where the proposed cause is present or absent, while controlling for all other variables that might influence the effect in which we’re interested. This is known as an experiment.

If we give a way a free gift along with T-shirts we are selling in a band, the free gift is known as the independent variable because it is not affected by the other variables in the experiment. It is also known as the predictor variable, because it can be used to predict the scores of another variables (whether more T-shirts are sold as a result of the free gift or not).

While the T-shirt sales variable is known as the dependent variable, because we assume that its value will depend on whether or not a free gift was provided or not. It is also known as the outcome variable, because it is the variable that we’re trying to predict the values of (we want to know how many sales were made).

The no-gift group is critical because this is a group in which our proposed cause (incentive to buy) is absent, and we can compare the outcome in this group against the situation in which the proposed cause is present. If the T-shirt sales are different when the free gift is offered (cause is present) copared to when it is not (cause is absent) then this difference can be attributed to the free gift.

Two methods of data collection:

We can manipulate variables in experiments in two ways:

Two types of variation

If you use two different bands, then you would expose both bands to both conditions (free gift, and not free gift). Unsystematic variation will be bigger for an independent design than for a repeated measures design, because different groups are likely to have more tiny differences that will contribute to score variation (different music, t-shirt designs, prices etc).

What are examples of practice, order and randomization?

Practice effects and bordom effects on systematic variation

Order:

Counterbalancing is a technique that can be used to eliminate sources of the above sources of unsystematic variation, by changing the order in which groups are exposed to conditions (with the free gift first, or without the free gift first).

Randomization

Randomization is where you randomly allocate participant to each different control group, to try and minizise the differences between the two groups.

Sometimes, you can’t randomise people into two groups. E.g. It would be unethical to randomly select children to watch horror movies when some might be disturbed by them. So instead you’d wait for the children who choose to watch horror movies themselves. This is known as quasi-experimental design.

Why do we need science?

We need science to see through all of the bullshit and make our own informed decisions.