Logistic Regression Lecture Notes

Overview

• Logistic Regression: Widely used for classification tasks.
• Example: Classifying tumors as malignant or benign.
  • Label 1 (Yes) for malignant.
  • Label 0 (No) for benign.

Key Concepts

Graphical Representation

• Horizontal Axis: Tumor size.
• Vertical Axis: Classification labels (0 or 1).
• Linear regression is ineffective for this task.
• Logistic regression fits an S-shaped curve (sigmoid function).

Sigmoid (Logistic) Function

• Outputs values between 0 and 1.
• Formula: ( G(Z) = \frac{1}{1 + e^{-Z}} )
  • (e): Mathematical constant (~2.7).
  • (Z): Can take negative and positive values.
• Behavior:
  • As (Z) increases, ( G(Z) ) approaches 1.
  • As (Z) decreases, ( G(Z) ) approaches 0.
  • At (Z = 0), ( G(Z) = 0.5 ).

Logistic Regression Model

• Two steps:
  1. Compute (Z = w \cdot X + b).
  2. Pass (Z) through the sigmoid function to get a probability (0 to 1).
• Formula: ( f(X) = G(w \cdot X + b) = \frac{1}{1 + e^{-(w \cdot X + b)}} )

Interpretation

• Probability output of being class 1 (malignant in tumor example).
• Example: Output 0.7 means a 70% chance of malignancy.
• Complementary probability (1 - output) represents probability of the opposite class.

Mathematical Notation

• ( f(X) = P(Y = 1 | X; w, b) )
  • ( ; ): Parameters (w) and (b) affecting probability.
  • Not crucial for basic understanding but may appear in literature.

Practical Application

• Logistic regression used in internet advertising.
• Determines ad display decisions on large websites.

Further Learning

• Next video to cover:
  • Details and visualizations of logistic regression.
  • Concept of decision boundary.
  • Mapping model outputs to binary predictions.

Optional Lab

• Implement and visualize the sigmoid function in code.
• Provided code for practice.

Conclusion

• Understanding logistic regression is key for classification tasks.
• Explored basic concept and formula.
• Upcoming content to deepen understanding of logistic regression.