Lecture Notes: Payment Processing System for an E-commerce Website

Introduction

• Topic: Designing a payment processing system for an e-commerce site.
• Reference: Alex Xu book for design inspirations
• Motivations: Common misconception to just use Stripe; there are complexities to consider.

Problem Requirements

• Account for all payments and profits, ordered by time.
• Maintain a perfect record of all orders and payments.
• Prevent duplicate orders (ensure idempotency).
• Payment service details are not covered (e.g., using Stripe for transactions).

Design Considerations

• Latency: Not a primary concern; optimally we should not lose any data.
• Approval Delays: Some payments may take hours or days to get approved.
• Source of Truth Table: A single ledger or payments table can derive all necessary data.

Achieving Strong Consistency

• Synchronous Replication: Ensuring data write on a leader is acknowledged by a follower before acknowledging the client.
• Issues: Lack of fault tolerance; if either node goes down, writes cannot happen.
• Distributed Consensus Algorithms (e.g., Raft): Address the fault tolerance issue with leader and follower nodes ensuring data consistency via quorum (majority agreement).
• Drawbacks: Slower writes due to message overhead and network delays.

Idempotent Payments

• Importance: Prevent duplicate submissions on multiple clicks.
• Item Potency Key: Used for identification of transactions
  • Generation Options: Pre-materialize keys or generate using hash functions.
  • Database Handling: Make checks in the database for key conflicts.

Improving Table Efficiency

• Partitioning: Separate payment tables to increase throughput.
  • Rights are independent; no need for cross-partition commits.
  • Distribute load using hash ranges.

Polling Strategy

• Webhook from Stripe: Listen for payment confirmations from Stripe.
• Failure Scenarios: Server crashes before sending to Stripe or during webhook response.
• Solution: Poll Stripe periodically for pending payments and update the status.
• Pending Payments Cache: Use memory to store pending payments avoiding database lock contentions.

Derived Data and Database Design

• Change Data Capture (CDC): Capture changes from the main payments table.
• Data Streams: Use systems like Kafka for processing changes.
• Read-Optimized Databases: Store derived data efficiently (buyer orders and seller revenues).

Complete Workflow and System Architecture

• Client Interaction
  • Request idempotency key
  • Submit payment using the key
  • Receive Stripe webhook for payment status update
• System Components
  • Payment server: Communicates with Stripe, manages state transitions.
  • Payments Database: Stores item potency keys and payment records with strong consistency.
  • Derived Data Systems: Use CDC to keep user orders and seller revenues updated.
  • Pending Payments Cache: Manages pending payments, reduces read locks.

Final Words

• Encouragement to provide detailed functional requirements for future system designs.

Note: Diagrams and visual aids were referenced but not textually detailed.