Hypothesis testing

Think of Hypothesis Testing like being a judge in a court case. Someone makes a claim (the 'defendant's' statement, or your hunch), and you, the judge, need to decide if there's enough evidence to say the claim is true, or if it's probably false.

In maths, we start with two opposing statements, like two sides of an argument:

• Null Hypothesis (H₀): This is the 'boring' or 'default' statement. It usually says there's no change, no difference, or no effect. For example, 'The new medicine has no effect on headaches.' Or, 'Exactly half of my friends prefer pizza.' It's like assuming the defendant is innocent until proven guilty.
• Alternative Hypothesis (H₁): This is the exciting one! It's your hunch, the claim you're trying to find evidence for. It usually says there is a change, there is a difference, or there is an effect. For example, 'The new medicine does reduce headaches.' Or, 'More than half of my friends prefer pizza.' This is what you're hoping to prove.

You then collect some data (your evidence) and use statistical tests to see if your data is so unusual that it makes you doubt the Null Hypothesis. If the data is really unusual under the assumption that H₀ is true, then you might 'reject' H₀ in favour of H₁.

Let's say a company makes a new super-duper long-lasting battery for phones. They claim their new battery lasts longer than their old one, which used to last 10 hours on average.

Here's how hypothesis testing would work:

1. The Claim (Alternative Hypothesis, H₁): The company believes the new battery lasts more than 10 hours.
2. The Default (Null Hypothesis, H₀): If the new battery isn't better, then it lasts 10 hours or less (or exactly 10 hours, depending on how it's set up).
3. Collecting Evidence: The company takes a sample of, say, 50 new batteries and tests how long each one lasts. They find the average (mean) life of these 50 batteries is 10.5 hours.
4. The Big Question: Is 10.5 hours enough better than 10 hours to confidently say the new battery is truly better? Or could this 0.5-hour improvement just be a fluke (a random chance occurrence) because they only tested 50 batteries and not all of them?
5. The Decision: Using special statistical calculations, they figure out the probability of getting an average of 10.5 hours (or even higher) if the battery actually still only lasted 10 hours. If this probability is very, very small (like less than 5%), it means that getting 10.5 hours by chance is super unlikely. So, they would conclude: "Wow, that's such a big improvement, it's highly unlikely to be random chance. We can confidently say the new battery is better!" They reject H₀.

If the probability was high (e.g., 20%), it means getting 10.5 hours could easily happen by chance even if the battery wasn't truly better. In that case, they'd say: "Hmm, not enough evidence yet. We can't confidently say the new battery is better." They would 'fail to reject' H₀.

Here's the recipe for conducting a hypothesis test:

1. State your Hypotheses (H₀ and H₁): Clearly write down the Null and Alternative Hypotheses for your problem.
2. Choose a Significance Level (α): This is like setting your 'doubt threshold' – how unlikely does your evidence need to be before you reject H₀? Common values are 5% (0.05) or 1% (0.01).
3. Collect your Sample Data: Gather the information you need to test your hypothesis.
4. Calculate the Test Statistic: This is a special number calculated from your sample data that helps you measure how far your sample result is from what H₀ would expect.
5. Find the p-value (or Critical Value): The p-value is the probability of getting your sample result (or something even more extreme) if H₀ were actually true.
6. Make a Decision: Compare your p-value to your significance level (α). If p-value < α, you reject H₀. If p-value ≥ α, you fail to reject H₀.
7. Write a Conclusion in Context: Explain what your decision means in simple terms related to the original problem.

Imagine you're checking if a new fertiliser makes plants grow taller. You're only interested if they grow taller, not shorter. This is a one-tailed test because your Alternative Hypothesis (H₁) only looks for a change in one direction (e.g., 'mean height is greater than X').

But what if you're checking if a new manufacturing process changes the weight of a chocolate bar? You don't care if it's heavier or lighter, just that it's different from the old weight. This is a two-tailed test because your Alternative Hypothesis (H₁) looks for a change in either direction (e.g., 'mean weight is not equal to Y').

• One-tailed test: H₁ uses '<' or '>'. You're looking for a specific direction of change.
• Two-tailed test: H₁ uses '≠' (not equal to). You're looking for any change, either up or down.

1. ❌ Confusing 'Fail to Reject H₀' with 'Accept H₀': Just because you don't have enough evidence to prove someone guilty doesn't mean they are innocent. It just means there wasn't enough proof. ✅ How to avoid: Always say 'fail to reject H₀'. This means your data doesn't provide enough evidence to say H₀ is false. It doesn't mean H₀ is definitely true.

2. ❌ Mixing up Null and Alternative Hypotheses: Putting your claim (what you want to prove) in H₀. ✅ How to avoid: Remember, H₀ is always the 'status quo' or 'no effect' statement. H₁ is what you're trying to find evidence for. H₀ usually includes an '=' sign (e.g., μ=10), while H₁ uses '<', '>', or '≠'.

3. ❌ Forgetting to state your conclusion in context: Just saying 'reject H₀' isn't enough. ✅ How to avoid: Always link your statistical decision back to the original problem. For example, instead of just 'Reject H₀', say 'There is sufficient evidence to suggest that the new battery lasts longer than 10 hours.'

4. ❌ Not understanding the Significance Level (α): Thinking a 5% significance level means there's a 5% chance your conclusion is wrong. ✅ How to avoid: The significance level (α) is the probability of incorrectly rejecting H₀ when it was actually true (this is called a Type I error). It's your 'risk tolerance' for making that specific mistake.