What is Regression Analysis?

Regression analysis is a statistical method in which it estimates the relationship between a dependent variable and one or more independent variable. In simple terms dependent variable is called as outcome variable and independent variable is called as predictors. Regression analysis is one of the methods to find the trends in data. The independent variable used in Regression analysis is named Predictor variable. It offers data of an associated dependent variable regarding a particular outcome.

“Regression analysis”

Types of Regression Analysis

There are three major types of regression analysis

  • Linear regression
  • Multiple linear regression
  • Non-linear regression
“Types of Regression Analysis”

How Does Regression Analysis Work?

Regression analysis does this by estimating the change in one independent variable has on the other dependent variable where all other independent variable will be constant. By this we can learn the nature of each independent variable.

Formula

Y = mX + b

where,

  • Y is the dependent variable
  • m is the slope
  • X is the independent variable
  • B is constant

Linear Regression

Linear regression is a model that finds the relationship between two variables by fitting a linear regression line equation .one variable is independent, and another is dependent.

For example,

 For every year, five randomly selected students will take a math's test on aptitude before they selected for their statistics course.

  • find the regression equation?
  • If a student takes 80 marks in aptitude, then how much percentage a student will score in statistics exams?
  • Is the regression equation being a good fit?

Solution

Step 1:

To find a regression equation by the below data.

STUDENT ROLL NOTEST SCORE (X)STATISTICS PERCENTAGE (Y)DEVIATION    OF XDEVIATION OF Y
19282156
28292516
379702-6
47065-7-11
56070-17-6
SUM383379  
MEAN7776  

From the above table we have calculated the deviation of x and y by the formula of ( X ¯ - x ¯ ) and ( Y ¯ - y ¯ ).

Step 2:

 Now we want to find the square for the deviation for finding regression analysis.

Step 3:

Now we need to find the product of the deviation X and Y.

Now the linear regression equation is of the form y ¯ =b 0 +b 1 x ¯ 1 . To do a regression analysis we need the value for b0 and b1. So now we want to solve this is two,

b 1 = X- x ¯ Y- y ¯ X- x ¯ 2

b 1 337 592

b 1 = 0.569now we know b1 so we substitute in equation (1), we get b 0 = y ¯ b 1 × x ¯ b 0 =760.569×77 b 0 =32.187

the regression equation is y ¯ =32.187+0.569x.

If a student score 80 marks, then his/her statistics percentage will be y ¯ = b 0 + b 1 x y ¯ =32.187+0.569×80 y ¯ =77.707

To check whether the regression is good fit or not we want to find the coefficient of determination.

R 2 ={ 1/N × X i - X ¯ × Y i - Y ¯ / σ x * σ y }

Now we want to calculate the standard deviation of x and y

σ x  =  X I X ¯ 2 /N

σ x  =  592/5

σ x = 10.88

σ y = Y y ¯ 2 /N

σ y = 485/5

σ y = 9.84

R 2 1/5 × 337/ 10.88×9.84 2 R 2 67.4/107.059 2 R 2 = 0.39

The coefficient of determination is equal to 0.39 it is about 39 % of the variation in statistics percentage can be predicted the relation to math aptitude scores. This could be considered as the good fit of data in the way that it would improve a tutor ability to predict student performance in statistics class.

Multiple Regression

Multiple regression is a way like linear regression but here we deal with two or more other variables. Here it requires two or more predictors. For example, a student IQ level can be test with aptitude test, exam marks, general knowledge.

Formula

Y=b 0 + b 1 x 1 +b 2 x 2 +……+b n x n +c  

Non-Linear Regression

Non-linear regression relates two variables as of simple linear regression but in a curved (non-linear) form. This curve function is of X variable where we use to predict the value of Y. non-linear regression are so complicated because here, we use the function which is created through iterations. Some examples are gauss newton method, iteration method etc. non-linear regression can predict a population growth over time.

Formula

Y=a 0 + b 1 x 1 2

How to select the best Regression Model?

Testing regression model by manual may be difficult so we use software to find the best fit and regression for the data some statistical software to use are SPSS, MINITAB, STATISTICA, R, PYTHON.

There are two steps to predict the best regression model for the data.

  • Adjusted R2 and predicted R2 value

If the R2 value is high (nearly 60%) then the regression model is a good fit for the data. For some human related problems low R2 value may be a good fit it relates upon to the data.

  • P – value

In regression the lower the P value more the data is statistically significant.

Advantages of Regression Analysis

Usually, regression analysis is used to know the impact of one variable among the another. This is mainly useful in small scale industries to predict their future share and which factor influencing the shares of the company. The important application of regression analysis is predictive analysis, operation efficiency, supporting decisions.

“Advantages of Regression Analysis”

Disadvantages of Regression Analysis

The data will be easily affected by the outlier and it is limited to the linear relationship. There is a lengthy and complicated step of calculation and analysis. If there is a case of qualitative phenomenon regression analysis cannot be used.

Application of Regression Analysis

 By this application we can improve the efficiency of small-scale industries

  1. Predictive analysis
  2. Operation efficiency
  3. Supporting decisions
  4. Correcting errors
  5. New insights

Difference Between Correlation and Regression

  • If there is a measure of degree of relationship between the variables is called correlation. if there is a nature of relationship between the variables to predict one by the another is called regression
  • In correlation there is no cause and effect between the variables .in regression there is a cause and effect between the variables.
  • By using correlation, we cannot predict anything, but regression is a predictive tool.
  • Coefficients are symmetrical in correlation but in regression they are asymmetrical.
  • In correlation the change in origin and scale are of independent but in regression it is dependent of change in scale.                  

Context and Applications:

This topic is significant in the professional exams for both undergraduate and graduate courses, especially for      

  • BBA
  • B.COM
  • M.COM

Want more help with your statistics homework?

We've got you covered with step-by-step solutions to millions of textbook problems, subject matter experts on standby 24/7 when you're stumped, and more.
Check out a sample statistics Q&A solution here!

*Response times may vary by subject and question complexity. Median response time is 34 minutes for paid subscribers and may be longer for promotional offers.

Search. Solve. Succeed!

Study smarter access to millions of step-by step textbook solutions, our Q&A library, and AI powered Math Solver. Plus, you get 30 questions to ask an expert each month.

Tagged in
MathStatistics

Inferential Statistics

Correlation, Regression, and Association

Search. Solve. Succeed!

Study smarter access to millions of step-by step textbook solutions, our Q&A library, and AI powered Math Solver. Plus, you get 30 questions to ask an expert each month.

Tagged in
MathStatistics

Inferential Statistics

Correlation, Regression, and Association