The perceptron learning algorithm is a method for adjusting weights and thresholds to classify data.
The discussion begins with an example of movie ratings, distinguishing between movies we liked (label 1) and those we didn't (label 0).
Problem Statement
Given M movies with associated liking labels (1 for like, 0 for dislike).
Movies have n different variables (features) influencing our decisions, with ratings as real numbers (0 to 1).
Goal: Train the perceptron to correctly predict the labels for movies based on the features.
Learning Process
Initialization: Start with random weights (small values).
Convergence: The algorithm runs until predictions on training data no longer change (i.e., no more errors).
Algorithm Steps
Randomly Select Data Point: Choose a point from the dataset (can be positive or negative).
Evaluate Conditions:
If the point is positive (label 1) and the output is less than or equal to 0:
Error: Adjust weights by adding the feature vector to weights (W = W + X).
If the point is negative (label 0) and the output is greater than 0:
Error: Adjust weights by subtracting the feature vector (W = W - X).
Mathematical Interpretation
The algorithm can be described in terms of vector mathematics:
Dot Product: The formula W^T * X = 0 defines a decision boundary.
The angle between vector W and point X determines classification:
Angle < 90°: point classified as positive (output ≥ 0).
Angle > 90°: point classified as negative (output < 0).*
Geometric Interpretation
The vector W is orthogonal to the decision boundary (line defined by W^T * X = 0).
Positive points yield angles less than 90°, while negative points yield angles greater than 90° relative to W.*
Algorithm in Action
Initially, the weights are set randomly; adjustments are made based on classification errors.
Continue picking random points and adjusting weights until convergence is observed (i.e., no corrections needed).
Challenges
The algorithm assumes data is linearly separable; if not, it may cycle through points without converging.
The concern is that adjustments may continually revert to previous states, preventing convergence.
Conclusion
The perceptron learning algorithm effectively adjusts weights to classify data points based on features, converging on a solution given linearly separable data.