Regression Analysis Lecture Notes

Introduction

• Instructor: Hannah
• Topic: Regression Analysis
• Goals:
  • Understand regression analysis
  • Differentiate between simple and multiple linear regression
  • Interpret results and understand assumptions
  • Use of dummy variables
  • Basics of logistic regression
  • Calculate regression online

What is Regression Analysis?

• Purpose: Infer/predict a variable based on others
• Example: Influences on salary (education, working hours, age)
  • Dependent Variable: Criterion (e.g., salary)
  • Independent Variables: Predictors (e.g., education level)
• Goals:
  • Measure influence of variables
  • Predict a variable

Types of Regression

Simple Linear Regression

• One Independent Variable
• Example: Impact of working hours on hourly wage

Multiple Linear Regression

• Several Independent Variables
• Example: Influence of working hours and age on hourly wage

Logistic Regression

• Categorical Dependent Variable
• Example: Probability of burnout (Yes/No)

Key Concepts

• Linear Regression: Dependent variable is metric
• Logistic Regression: Dependent variable is categorical
• Dummy Variables: Used for categorical variables with more than two characteristics

Simple Linear Regression

• Goal: Predict dependent variable based on one independent variable
• Method: Least squares to determine the best fit line
• Example: Hospital stay prediction based on age
• Calculation:
  • Slope (b) and intercept (a)
  • Error (epsilon) represents the difference between estimated and true values

Multiple Linear Regression

• Allows Multiple Variables: Controls influence of multiple factors
• Equation: Coefficients (b) indicate change in dependent variable per unit change in independent variable
• Use Cases: Empirical social research, market research

Interpretation of Results

• Model Summary: Correlation coefficient (R), coefficient of determination (R^2)
• F-test: Tests null hypothesis that all slopes are zero
• Significance Levels: P-values for coefficients

Assumptions for Linear Regression

1. Linearity: Relationship between variables should be linear
2. Normal Distribution of Error: Error term epsilon should be normally distributed
3. No Multicollinearity: Independent variables should not be highly correlated
4. Homoscedasticity: Constant variance of residuals

Dummy Variables

• Use: For categorical predictors with more than two characteristics
• Method: Create binary variables for each category
• Example: Vehicle type (sedan, sports car, family van)

Logistic Regression

• Dependent Variable: Categorical (e.g., disease risk)
• Uses: Predict outcome probabilities
• Equation: Logistic function to ensure probabilities between 0 and 1
• Calculation: Maximum likelihood method

Online Regression Calculation

• Tool: DataTab
• Process: Input data, select variables, and interpret results

Conclusion

• Regression analysis is a powerful tool for prediction and understanding variable influences.
• Different types of regression are suited for different kinds of data.
• Statistical software facilitates complex calculations easily.