PageRank Algorithm in Big Data Analytics

Introduction

• PageRank: A system to determine the importance of a web page, helping to evaluate the quality of a website.
• Origin: Developed by Lawrence Page and Sergey Brin.

PageRank Concept

• Vote System: Pages on the web "vote" for others by linking to them, indicating the importance of a page.
• Link Analysis: The number of links to a page contributes to its value, both inbound (back-links) and outbound.

Functionality

• Random Surfer Model: Assumes a user clicks links randomly without regard to content. PageRank is calculated using:
  • Equation: PR(A) = (1 - D) + D(PR(T1)/C(T1) + ... + PR(Tn)/C(Tn))
  • Components:
    • PR(A): PageRank of page A.
    • PR(Ti): PageRank of pages linking to A.
    • C(Ti): Number of outbound links on page Ti.
    • D: Damping factor, typically set at 0.85.

PageRank Calculation

• Iterative Process: PageRank calculation is iterative, considering:
  • Inbound Links: Increase page rank.
  • Outbound Links: Lack can lead to dangling links.
  • Dangling Links: Pages without outbound links do not contribute to PageRank.

Example Calculation

• Initial Setup:
  • Default initial PageRank for all pages is 1.
  • Damping factor is usually 0.85 unless otherwise specified.
• Iteration Process:
  • Use inbound links and outbound links to calculate iterative PageRanks.
  • Do not round off values during calculations.

Advanced Example

• Matrix Representation: PageRank can also be calculated using matrices for larger web structures.
  • Matrix Elements: Represent the probability of moving from one page to another.
  • Damping Factor in Matrix: Typically does not affect the immediate matrix calculation but influences the overall factor adjustments.

Complex Page Structures

• Handling Dangling Links:
  • Impacts on web page ranking if not managed correctly.
  • PageRank calculations consider both inbound and outbound dynamics.

Conclusion

• Importance of PageRank: Helps determine the relevance and quality of web pages.
• Iterative Nature: Continued iterations until stable PageRank values are achieved.
• Practical Application: Useful for identifying valuable web pages and optimizing web structures.

Thank you for attending the lecture on PageRank Algorithm!