Advanced Clustering Algorithms

Introduction

• Topic: Advanced clustering algorithms beyond K-means.
• Focus on Gaussian Mixture Models (GMM), Spectral Clustering, Hierarchical Clustering, and others.

Gaussian Mixture Models (GMM)

• Often referred to as EM-based clustering.
• Distinct because of the type of clusters it fits.
• Key Features:
  • Each cluster has a centroid and a radius.
  • Can handle overlapping clusters and outliers.
  • GMM assigns every point to a cluster but considers thresholds for actual inclusion.
• Advantages:
  • Can assign points to multiple clusters.
  • Handles outliers explicitly.
• Disadvantages:
  • Slower to create than K-means.
  • Can be overkill for some problems.

Assessment of GMM

• Assessed using distortion, BIC, AIC, and log-likelihood.
• Evaluates the probability of data given the model configuration.

Spectral Clustering

• Involves dimensionality reduction before clustering.
• Analogous to support vector machines.
• Clusters in a reduced-dimensional space.
• Benefits:
  • Useful for finding clusters that aren't spherical in high-dimensional data.

Hierarchical Clustering

• Clusters can contain sub-clusters.
• Types:
  • Hierarchical Agglomerative Clustering (HAC or Hack).
• Starts with each point as its own cluster, merging them iteratively.
• Visualization Tool:
  • Dendrograms to show structure and relationships.

DP Means Clustering

• Begins with one cluster, adding points iteratively.
• New cluster formed if point doesn't fit an existing cluster well.

Sequence Clustering

• Clusters sequences rather than single data points.
• Dynamic Time Warping:
  • Accounts for sequences of different lengths.
  • Matches events while preserving order.

Latent Class Analysis (LCA)

• Not strictly clustering, but closely related.
• Type of structural equation modeling.
• Features:
  • Finds data points that are statistically independent when controlling for the group.
  • Allows statistical analysis of latent classes.
• Drawbacks:
  • Requires substantial data.
  • Slower process.
  • Finds fewer latent classes than traditional clustering.

Conclusion

• Choice of clustering method depends on data characteristics and desired pattern recognition.
• Each method has its strengths and suitable use cases.

Next Steps

• Future lectures to provide examples of clustering in EDM research.

---

Big D Education with Ryan Baker.