🛰️

Open Telemetry for Full Stack Observability

May 23, 2025

Lecture on Open Telemetry

Course Introduction

  • Focus: Achieving full stack observability using Open Telemetry.
  • Purpose: Diagnose performance issues, improve code quality.

Understanding Open Telemetry

  • Open: Refers to open source.
  • Telemetry: Collection of remote measurements and data.
  • Origin: Greek "tele" (remote) and "metron" (measure).
  • Objective: Measure app performance remotely.

Components of Telemetry

  1. Data Generation and Transmission
    • How to generate and transmit data.
  2. Data Analysis
    • How to analyze the collected data.

Standardization with Open Telemetry

  • Challenge: Lack of standardized method to describe system operations.
  • Solution: Open Telemetry standardizes data description across languages and systems.
  • Outcome: Easier transition between analysis tools without affecting historic data.

Open Source Community

  • Transparent governance via GitHub.
  • Encouragement to participate in community meetings.

Course Outline

  1. Microservices
  2. Observability
  3. MELT
  4. History of Open Telemetry
  5. Project Setup
  6. Tracing and Distributed Tracing
  7. Context and Propagation
  8. Metrics
  9. Distributed Projects
  10. Course Conclusion

Microservices and Observability

  • Monoliths: Single complex code base; problematic as apps grow.
  • Microservices: Smaller, independent modules; easier maintenance.
  • Observability: Understanding system's internal state via outputs.

Key Data Types for Observability

  1. Metrics: Regular measurements, e.g., error rate.
  2. Events: Discrete actions, e.g., purchase moment.
  3. Logs: Detailed event context.
  4. Traces: Follow request lifecycle, highlight inefficiencies.

Challenges and Solutions

  • Problem: Manual instrumentation is laborious.
  • Solution: Open source projects simplify tracing.
    • Open Tracing (2016)
    • Open Census (2018)
    • Merge into Open Telemetry (2019)

Distributed Tracing

  • Definition: Debugging tool for microservice architectures.
  • Mechanism: Uses context and propagation to correlate events.

Implementation Steps

  1. Initialization: Set up tracing exporter (e.g., Zipkin).
  2. Start Application: Setting up using Node.js.

Metrics Collection

  • Objective: Use Prometheus for metrics.
  • Process: Install and configure Prometheus, initialize metrics library.

Issues Detected with Open Telemetry

  • Backend, Frontend, Infrastructure issues.

Practical Project Implementation

  • Structure: Dashboard and Movie services.
  • Goal: Trace service interactions and measure performance.

Using Open Telemetry Collector

  • Purpose: Forward data to observability tools (e.g., New Relic).

Conclusion

  • Accomplishments: Implemented tracing and metrics.
  • Next Steps: Explore infrastructure and digital experience monitoring.
  • Further Resources: New Relic link provided for continued exploration.

Full transcript