Asynchronous Ticket Generation Engine

01.Problem Statement

When organizers create events with thousands of tickets, generating those records synchronously in the main API flow caused connection timeouts and degraded performance. Massively generating 10,000+ unique ticket records on-the-fly would block the event manager service, leading to a poor experience for major event launches.

02.Architecture Overview

Leveraged Kafka to decouple ticket definition from physical record generation. An 'event.created' message triggers a consumer group that processes generation in controlled batches. This architecture allows the event creation API to return instantly while the heavy database write operations are handled by a dedicated background worker pool.

     [ Event Manager ] -- (Produce) --> [ Kafka: generate-tickets ]
                                              |
      +---------------------------------------+
      |                   |                   |
      v                   v                   v
[ Ticket Worker 1 ] [ Ticket Worker 2 ] [ Ticket Worker N ]
      |                   |                   |
      +---------+---------+---------+---------+
                |                   |
        [ MongoDB Session ]   [ Transaction Log ]
    

03.Database Design

Used an index-heavy MongoDB collection for tickets, optimized for range queries. Mapped ticketDefinitionId to individual ticket records for fast lookups during the high-load ticket purchase flow.

04.Key Decisions & Tradeoffs

05.Scaling Considerations

Horizontal scaling of consumer groups allowed us to handle multiple large-scale event launches simultaneously. The worker pool is dynamically adjusted based on the consumer lag metric from the Kafka partition.

06.Failure Scenarios & Mitigation

• Broker Failure: Kafka's log-append model ensures messages are persisted and retried until the generation is acknowledged as successful.
• DB Write Failure: Batch inserts are wrapped in transactions; if the DB fails midway, the entire batch is rolled back and retried by the consumer.
• Network Jitter: The heartbeat() mechanism ensures that transient network spikes between the worker and Kafka don't cause the worker to be evicted from the consumer group.

07.Engineering Challenges

• Kafka Rebalance issues: Long-running generation loops were triggering Kafka rebalances before completion. Solved by integrating a heartbeat mechanism inside the processing loop to keep the consumer active.
• Ensuring no over-generation: Multiple consumers picking up the same message due to broker failure could lead to duplicate ticket definitions. Implemented a robust pre-generation check to calculate the exact remaining capacity of the current ticket set.

08.Implementation Subsystem

async (payload: Payload, heartbeat: () => Promise<void>) => {
  const session = await mongoose.startSession();
  try {
    session.startTransaction();
    
    // 1. Check for existing generation to ensure idempotency
    const currentCount = await ticketModel.countDocuments({ definitionId });
    const remaining = payload.totalCapacity - currentCount;
    
    // 2. Perform batched generation to optimize DB writes
    for (let i = 0; i < payload.batchSize; i += BATCH_LIMIT) {
      const tickets = Array.from({ length: BATCH_LIMIT }, () => ({
        definition: definitionId,
        event: eventId
      }));
      
      await ticketModel.insertMany(tickets, { session });
      
      // 3. Inform Kafka we are still processing to avoid rebalance
      await heartbeat();
    }
    
    await session.commitTransaction();
  } catch (err) {
    await session.abortTransaction();
    throw err; // Trigger Kafka retry
  } finally {
    session.endSession();
  }
}