Statistical Analysis Lecture 7: Linear Regression

Introduction to Linear Regression

• Regression Analysis: A statistical technique to evaluate the relationship between a continuous dependent variable (outcome) and one or more independent variables (predictors).
• Main Objectives:
  1. Evaluate the overall model fit (like the F test in ANOVA).
  2. Measure the unique effect of each predictor on the outcome (beta coefficient).

Types of Statistical Methods in Regression

• General Linear Model:
  • Assumes normality of residuals.
  • Requires a continuous dependent variable.
  • Includes analyses like ANOVAs, ANCOVAs, MANOVAs, MANCOVAs.
• Generalized Linear Model:
  • For non-continuous data (nominal and ordinal).
  • Includes binary and multinomial logistic regression.
  • Uses a link function to transform the dependent variable for linear modeling.

Choosing a Regression Model

• Dependent Variable Type:
  • Scale variable: Use Linear Regression.
  • Non-scale variable: Use Generalized Linear Model approaches (e.g., logistic regression).

Linear Regression Analytical Definition

• Used to determine if a set of predictor variables effectively predict a continuous outcome variable.
• Example: Predicting systolic blood pressure using age and weight.

Assumptions of Linear Regression

1. Multicollinearity:
   • Predictor variables should not be highly correlated.
   • Assessed using Variance Inflation Factors (VIFs); values >10 indicate high multicollinearity.
2. Homoscedasticity:
   • Consistent variance of residuals across predictor values.
   • Tested using Levene’s test or scatter plots.
3. Independent Observations:
   • Each observation should be independent of others.
4. Normality:
   • Residuals should follow a normal distribution.
5. No Significant Outliers:
   • Outliers should be identified visually or analytically.

Formulating a Research Question for Linear Regression

• Research Question: Do the independent variables predict the continuous dependent variable?
• Hypotheses:
  • Null: Independent variables do not predict the dependent variable.
  • Alternative: Independent variables do predict the dependent variable.

Interpreting Linear Regression Output

• Beta Coefficients (B):
  • Effect of predictor on the outcome variable.
• Standardized Beta Coefficients (β):
  • Effect size in standard deviations.
• Standard Error:
  • Uncertainty of beta coefficients.
• Confidence Interval:
  • Range where the true beta value likely falls.
• T-statistic & p-value:
  • Significance of predictor influence on outcome.

Regression Equation

• Allows prediction of the dependent variable based on independent variables.
• Example equation:
  • Ŷ = 13.24 + 0.50 * (self-acceptance) where Ŷ is the predicted outcome.

Writing up Regression Results

• Describe the overall model fit (F statistic, p-value, R-squared).
• Interpret individual predictors (beta value, t-statistic, p-value).

Summary

• Regression analysis is used to assess predictor variables' effect on a continuous outcome variable.
• Key statistics include beta (effect size), standard error, confidence intervals, and p-values.
• Linear regression requires assumptions to be met for valid results.
• Practice applying linear regression in analyses for quizzes and assignments.