What is Multicollinearity?

S3-SA3-0392

Grade Level:

Class 9

AI/ML, Data Science, Physics, Economics, Cryptography, Computer Science, Engineering

Definition

What is it?

Multicollinearity is a problem in data analysis where two or more 'input' variables in a model are highly related to each other. Imagine trying to understand what affects your exam score, but two of your 'factors' are 'hours spent studying' and 'number of chapters revised' – these two are probably very similar.

Simple Example

Quick Example

Think about predicting the price of a mobile phone. If you use 'screen size in inches' and 'screen area in cm^2' as separate factors, they are basically measuring the same thing. This is multicollinearity because these two factors are almost identical and give redundant information.

Worked Example

Step-by-Step

Let's say you want to predict how much ice cream a shop sells (Y) based on 'daily temperature in Celsius' (X1) and 'daily temperature in Fahrenheit' (X2).

Step 1: Understand the variables. Y is ice cream sales. X1 is temperature in Celsius. X2 is temperature in Fahrenheit.
---
Step 2: Notice the relationship. You know that Celsius and Fahrenheit temperatures are directly related by a mathematical formula (F = C * 9/5 + 32). They are not independent.
---
Step 3: Identify the problem. If you try to use both X1 and X2 to predict Y, your model will struggle to figure out which temperature scale is truly affecting sales, because they are essentially the same information presented differently.
---
Step 4: Conclude. This situation shows multicollinearity because X1 and X2 carry almost identical information about the temperature.
---
Answer: Using both Celsius and Fahrenheit temperatures as separate predictors for ice cream sales demonstrates multicollinearity.

Why It Matters

Understanding multicollinearity is crucial in fields like AI/ML and Data Science because it can make your predictive models unreliable and hard to interpret. Engineers and Economists use this concept to build more accurate models for things like predicting stock prices or designing efficient systems. It helps data scientists make better decisions and build stronger AI.

Common Mistakes

MISTAKE: Thinking multicollinearity means input variables are completely unrelated to the 'output' variable. | CORRECTION: Multicollinearity is about the strong relationship BETWEEN input variables, not their relationship with the output.

MISTAKE: Believing multicollinearity is always a fatal flaw that requires removing all related variables. | CORRECTION: While it can be problematic, sometimes a moderate level is acceptable, or you can combine related variables instead of just removing them.

MISTAKE: Confusing multicollinearity with simply having many input variables. | CORRECTION: Multicollinearity isn't about the number of variables, but specifically about how strongly some of those input variables are correlated with each other.

Practice Questions

Try It Yourself

QUESTION: If you are trying to predict a student's exam score using 'hours spent studying' and 'number of pages read in the textbook', would this likely lead to multicollinearity? | ANSWER: Yes, because hours spent studying and pages read are often highly related.

QUESTION: You want to predict the price of a house. You use 'area of the house in square feet' and 'number of rooms'. Is this a clear case of multicollinearity? Explain why or why not. | ANSWER: Not a clear case. While larger houses often have more rooms, they aren't directly mathematically linked like Celsius and Fahrenheit. There might be some correlation, but not necessarily strong multicollinearity.

QUESTION: A data scientist is building a model to predict electricity consumption in a city. They include 'average daily temperature' and 'number of air conditioners running'. Is this multicollinearity? What if they also included 'average daily humidity'? | ANSWER: 'Average daily temperature' and 'number of air conditioners running' might show some correlation (more ACs run on hotter days), but it's not strong multicollinearity because they are distinct factors. If they also included 'average daily humidity', this would likely not cause multicollinearity with temperature, as humidity and temperature, while sometimes related, are different atmospheric measures.

MCQ

Quick Quiz

Which of the following scenarios is most likely to exhibit multicollinearity?

Predicting a person's height using their age and gender.

Predicting a car's fuel efficiency using its engine size and weight.

Predicting a student's score using 'number of hours slept' and 'number of hours awake'.

Predicting vegetable prices using rainfall and fertilizer cost.

The Correct Answer Is:

Option C is correct because 'number of hours slept' and 'number of hours awake' in a day are directly and inversely related (they sum up to 24 hours), making them highly correlated. The other options involve factors that are related but not to the extent of multicollinearity.

Real World Connection

In the Real World

Imagine a meteorologist at the India Meteorological Department (IMD) trying to predict monsoon rainfall. If their model includes both 'atmospheric pressure in millibars' and 'atmospheric pressure in hectopascals', it's a multicollinearity problem. They would simplify by using only one unit of pressure to make their rainfall predictions more accurate and reliable for farmers and disaster management.

Key Vocabulary

Key Terms

VARIABLE: A factor or characteristic that can change or be measured. | CORRELATION: A statistical measure that describes the extent to which two variables tend to change together. | PREDICTIVE MODEL: A mathematical tool that uses data to forecast future outcomes or behaviors. | REDUNDANT: Information that is unnecessary because it repeats what has already been said or given.

What's Next

What to Learn Next

Great job understanding multicollinearity! Next, you should explore 'Correlation vs. Causation' to understand the difference between two things happening together and one actually causing the other. This will further strengthen your data analysis skills and help you build even better models!