CS224 Advanced Algorithms Key Takeaways

Sep 19, 2024

CS224 Advanced Algorithms Lecture Notes

Course Information

  • Instructor: Jelani Nelson
  • TF: Jeffrey
  • Contact: cs224-f14-staff@cs.harvard.edu
  • Course Website: Google CS224 or Jelani Nelson
  • Mailing List: Register on the course website

Logistics

  • Grading Components:
    • Scribing: 10%
    • Problem Sets (P-sets): 60%
    • Final Project: 30%
  • Scribing: Students take turns. Notes are written in LaTeX, using a provided template.
  • P-sets: Must be submitted in LaTeX by email, with page limits.
  • Final Project: Proposal due October 30th, project due last day of reading period.
  • Grading: Students also take turns being graders.

Course Goals

  • Increase ability to analyze and create algorithms.
  • Learn different techniques for analyzing algorithms not covered in CS124.
  • Focus on theoretical models and efficiency measures.

Key Concepts Discussed

Sorting and Algorithms

  • Sorting Complexity: It's known that you cannot sort n numbers faster than n log n using comparison-based sorting.
  • WordRAM Model: Assumes integers fit in a word, W, allowing operations beyond comparisons (e.g., bitwise operations).

Van Emde Boas Trees

  • A data structure for dynamic predecessor queries.
  • Structure:
    • Recursive structure with sub-clusters.
    • Stores minimum and maximum.
  • Operations:
    • Predecessor Query: Uses divide & conquer; time complexity is O(log log U)
    • Insertion: Uses swaps and recursive insertion; time complexity is O(log log U)
  • Space Complexity: Originally U space, can be reduced to linear space with hashing.

Fusion Trees

  • A more advanced data structure for static predecessor queries.
  • Efficiency: Logarithmic time based on word size.
  • Uses linear space.

Advanced Sorting

  • Sorting can be improved in the WordRAM model:
    • Deterministic: O(n log log n)
    • Randomized: O(n sqrt(log log n))

Data Structure Models Discussed

  • Static vs Dynamic: Static doesn't change items, dynamic allows insertions and deletions.
  • Predecessor Problem: Finding the largest value less than a given number.
  • Binary Search Trees: Used for dynamic cases with log n time complexity.

Hashing and Dictionaries

  • Discussed briefly in context of reducing space.
  • Dictionary problem: storing key-value pairs.

Additional Concepts

  • Bitwise Operations: Exploited in algorithms for efficiency beyond comparisons.
  • Indirection Technique: Used to optimize space and time complexity in data structures.

Questions and Clarifications

  • Discussed potential optimizations and concerns about time/space efficiency.
  • Addressed how to implement certain operations using bit manipulation and trees.

Conclusion

  • The lecture concluded with outlining the constraints and potential improvements of the discussed data structures, as well as preliminary discussion on upcoming topics.