ADR-005: Message Queue

Status

Accepted

Context

The platform needs asynchronous, event-driven communication between services. The primary driver is the notifications service: the match-gateway publishes domain events (goal scored, lineup announced, card given) and the notifications service consumes them to dispatch push notifications.

Requirements:

• Decoupled routing: Publishers have no knowledge of consumers. New consumers can subscribe to existing events without changing the publisher.
• Durable delivery: Messages must survive broker restarts and consumer downtime.
• Self-hostable: Must run in Docker with minimal operational overhead.
• TypeScript support: Mature client library with good DX.
• Multiple consumers per event: A single event (e.g. GoalScored) may need to be processed by multiple independent services simultaneously.
• Swappable broker: The broker implementation should be replaceable without changing application code.

Options Considered

Kafka

Good at high-throughput event streaming with log retention and replay. However:

• Much higher operational complexity (ZooKeeper/KRaft, partition management, consumer group coordination)
• Replay and event sourcing are not required; once a notification is dispatched, the original event is not re-read
• Overkill for the current message volumes (hundreds of events per match, not thousands per second)

RabbitMQ

Mature message broker with native support for topic exchanges, wildcard routing, and dead letter queues. However:

• Requires explicit topology management (exchanges, queues, bindings, DLQs) in application code
• Adding a new consumer requires binding a new queue; operational overhead that grows with service count
• Application code is coupled to the AMQP protocol; swapping brokers requires rewriting consumer and publisher code
• Dead letter queues and retry logic must be implemented manually per queue

Dapr Pub/Sub

Dapr provides a pub/sub building block that abstracts over multiple brokers behind a uniform HTTP/gRPC API.

• Publisher calls POST /v1.0/publish/{pubsubname}/{topic}; no broker-specific code
• Consumer registers subscriptions via GET /dapr/subscribe and handles events via POST routes; no broker-specific code
• The underlying broker (Redis Streams, RabbitMQ, Kafka, Azure Service Bus, etc.) is swapped by changing a single component YAML file, with zero application code changes
• Dapr handles retries, at-least-once delivery, and dead lettering at the sidecar level
• Runs as a lightweight sidecar container per service

Decision

We adopt Dapr pub/sub with Redis Streams as the message broker for inter-service event-driven communication.

Rationale

Broker abstraction: Application code publishes and subscribes via the Dapr API. The broker is a configuration concern, not an application concern. Switching from Redis Streams to RabbitMQ or Azure Service Bus in production requires changing one YAML file, not rewriting publishers and consumers.

Decoupled routing via topics: Publishers publish to a topic with no knowledge of consumers. Each CloudEvent type string is used directly as the topic name (e.g. com.feyenoord.match.goal-scored.v1). Consumers subscribe to exactly the topics they care about. Adding a new consumer requires no changes to the publisher.

Topic-per-event-type: Using the CloudEvent type string as the topic name gives each event version its own independently subscribable topic. This makes versioning clean: v1 and v2 of the same event are separate topics. Consumers migrate at their own pace.

Programmatic subscriptions: Consumers declare their subscriptions in application code via GET /dapr/subscribe. Each domain module owns its subscription declarations and handler routes. The composition root collects them without knowing the details. This keeps the subscription list decoupled from the infrastructure.

Redis Streams for local development: Redis is already in the stack for caching. Using Redis Streams as the pub/sub backend avoids adding a separate broker container for local development. The pubsub Dapr component is defined in infra/dapr/resources/redis-pubsub.yaml and can be replaced with a production-grade broker by swapping this file.

Topology

Publisher (match-gateway)
  └── Dapr sidecar (match-gateway-dapr)
        └── Redis Streams
              ├── topic: com.feyenoord.match.goal-scored.v1
              │     └── Dapr sidecar (notifications-dapr)
              │           └── POST /events/match/goal-scored → notifications
              ├── topic: com.feyenoord.match.lineup-announced.v1
              │     └── ...
              └── topic: com.feyenoord.match.card-given.v1
                    └── ...

Publishers write to a topic. Any number of services can subscribe to the same topic independently. Each service's Dapr sidecar handles delivery, retries, and acknowledgement.

Dapr Component

The pub/sub component is defined in infra/dapr/resources/redis-pubsub.yaml:

apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: pubsub
spec:
  type: pubsub.redis
  version: v1
  metadata:
    - name: redisHost
      value: "redis:6379"

Swapping to RabbitMQ in production means replacing pubsub.redis with pubsub.rabbitmq and updating the connection metadata. Application code is unchanged.

Retry and Error Handling

Dapr handles at-least-once delivery at the sidecar level. Consumers signal the outcome by returning a status in the HTTP response body.

Retry policy and dead lettering are configured in the Dapr component or subscription manifest, not in application code.

Consequences

Positive

• Publishers are fully decoupled from consumers; adding new consumers requires zero changes to publishers
• The broker is swappable by changing a single YAML file; no application code changes required
• Retry and delivery semantics are handled by Dapr, not by application code
• Redis Streams reuses an existing infrastructure dependency for local development
• Topic-per-event-type gives each event version an independently subscribable, independently versioned channel

Negative

• Each service requires a Dapr sidecar container, increasing container count in local development
• Dapr sidecar adds a network hop between the app and the broker (HTTP vs direct TCP)
• No event replay; once a message is acked it is gone. If replay becomes necessary, the broker can be swapped to Kafka via the component YAML

Risks

• Schema coupling across consumers: if multiple services consume the same event type, schema changes require coordination. Mitigated by the versioning strategy documented in Event Contracts.
• Redis durability: Redis Streams data is lost if Redis is cleared. For local development this is acceptable. In production, possibly use a durable broker (RabbitMQ, Azure Service Bus) by swapping the Dapr component, or use a seperate Redis instance for messaging and caching.

Related

• ADR-001: Microservices Architecture
• Event Contracts