Creating a Microservice Application with Kafka.js

Overview

• Objective: Create a microservice application in NestJS using Kafka.js as the transporter.
• Components:
  • API Gateway
  • Billing Service
  • Auth Service

Setting Up Kafka

• Tools Required:
  • Kafka
  • Calf Drop (Web GUI for Kafka)
• Setup Steps:
  1. Run Kafka on your machine.
  2. Use Docker Compose to run Kafka and Calf Drop simultaneously.
  3. Clone the Calf Drop repository: git clone <repository-url> cd calf-drop cd docker-compose docker-compose up
  4. Access Kafka Web GUI at localhost:9000 to view topics and messages.

Creating NestJS Applications

• Open Terminals: Three terminals for three separate NestJS applications.
• Generate Projects:
  • API Gateway: nest new api-gateway
  • Billing Service: nest new billing
  • Auth Service: nest new auth

Install Required Packages

• For each service, install the necessary packages: yarn add @nestjs/microservices kafka.js

Configuring API Gateway

• Create Post Route for creating orders.
• Request DTO:
  • Define CreateOrderRequest class with userId and price properties.
• Emit Event:
  • Emit an event from API Gateway to Billing Service after order creation.

App Module Setup

• Import ClientsModule from @nestjs/microservices.
• Register Billing Service with Kafka transport: ClientsModule.register([ { name: 'BILLING_SERVICE', transport: Transport.KAFKA, options: { client: { clientId: 'billing', brokers: ['localhost:9092'], }, consumer: { groupId: 'billing-consumer', }, }, }, ]);

Billing Service Configuration

• Set up Kafka consumer to listen for order created events.
• Main.ts: const app = await NestFactory.createMicroservice(AppModule, { transport: Transport.KAFKA, options: { client: { brokers: ['localhost:9092'], }, consumer: { groupId: 'billing-consumer', }, }, });
• Log incoming order created events in the controller.

Handle Order Created Logic

• Implement function to handle events and interact with Auth Service to retrieve user info.
• Log billing operation details.

Auth Service Configuration

• Similar setup to Billing Service with a different group ID.
• Message Pattern: Listen for get user requests from the Billing Service.
• Implement user retrieval logic from a simple in-memory array.

Testing the Setup

• Use Postman to send POST requests to the API Gateway:
  • Example body: { "userId": "12345", "price": 100 }
• Confirm that the Billing Service logs the billing operation correctly.

Conclusion

• Successfully created a microservices architecture using Kafka.js and NestJS.
• Covered event and message patterns for communication between services.
• Encouraged viewers to ask questions and interact.