Lecture on Queue Processor in Pega

Introduction

• The Job Scheduler runs defined tasks at set intervals as a background process.
• Challenges arise when processes need to be executed for different purposes at varying intervals.

Business Requirement Example

• Claims Application: Email notifications based on customer type:
  • Silver: 8 hours post-submission
  • Gold: 6 hours post-submission
  • Diamond: 4 hours post-submission
• Queue Processor is recommended over a Job Scheduler for such specific, non-fixed scheduling needs.

Kafka Overview

• Kafka: A distributed streaming platform for real-time data feeds.
• Developed by LinkedIn, donated to Apache Software Foundation.
• Functions similarly to a message queue or enterprise messaging system.
  • Key components: Topic, Message, Publisher/Producer, Subscriber/Consumer.

Queue Processor in Pega

• Replaces the standard agent in Pega v8.
• An internal background process that runs business logic based on set intervals.
• Mapped to a topic (queue) to process messages.
• Works on a publish-subscribe model with real-time data flow.
• Processes messages across up to 20 partitions.

Configuring a Queue Processor

Rule and Privileges

• Instances of class RULE-ASYNC-QUEUEPROCESSOR.
• Requires pzQueueProcessorAdministrator privilege to manage.
• Accessible under SysAdmin category in Records explorer.

Key Configuration Areas

1. Node Configuration
   • Tag queue processor to a node type.
   • Can only run on one type of node.
2. Business Logic
   • Use activity rules for processing messages.
3. Execution Mode
   • Immediate or Delayed processing.
   • Define number of threads per node and max attempts for processing.

Types of Queue Processors

• Standard Queue Processor
  • For simple queue management, reuses pzStandardProcessor.
  • No delayed processing, max 3 attempts.
• Dedicated Queue Processor
  • For complex processes needing customization.
  • Supports both immediate and delayed execution.

Pega and Kafka Integration

• Apache Kafka is included with Pega v8.
• Topics are created for each queue processor rule.
• Uses Kafka's 20 partitions to ensure efficient processing.

Invoking a Queue Processor

• Use Queue-For-Processing method in an activity rule or Run in Background smart shape in a flow rule.
• Options for locking, snapshots, and alternate access groups available.

Execution and Debugging

• Messages published to Kafka topic are processed by real-time data flow.
• Queue processors can be traced through Admin Studio.
• Errors logged in System-Message-QueueProcessor-BrokenItem.

Transition from Standard Agents

• Standard agents can still function but new processes should use queue processors.
• Transition involves creating queue processor rules for existing agents and updating code to publish messages to topics.

Key Takeaways

• Queue processors require ASYNCPROCESSOR or System Runtime Context access.
• Must have at least one stream node for processing.
• Ensure 0% data loss through Kafka.

This lecture provides foundational knowledge on configuring and using Queue Processors in Pega, their integration with Kafka, and how they enhance processing efficiency.