Hey folks!

In the previous post, I introduced the overview of My Broker B3. Today we'll talk about the foundation that supports the entire ecosystem: the infrastructure.

For a microservices system of this complexity, manually configuring each database, message broker and monitoring tool would be unmanageable. The solution was to use Docker Compose to create a reproducible local environment that faithfully replicates the needs of a distributed system.

🗺️ The Infrastructure Map

The docker-compose.yml orchestrates 12 containers organized in 5 layers:

┌─────────────────────────────────────────────┐
│           RELATIONAL LAYER (SQL)            │
│  identity-db  wallet-db  order-db  asset-db │
│     MySQL        MySQL    MySQL     MySQL   │
│                  b3-core-db (PostgreSQL)    │
├─────────────────────────────────────────────┤
│              NOSQL LAYER                    │
│           broker-mongodb (Mongo 6.0)        │
├─────────────────────────────────────────────┤
│              CACHE LAYER                    │
│    broker-asset-cache    b3-market-cache    │
│         Redis Alpine          Redis Alpine  │
├─────────────────────────────────────────────┤
│             MESSAGING LAYER                 │
│    kafka (KRaft)        rabbitmq (AMQP)     │
├─────────────────────────────────────────────┤
│           OBSERVABILITY LAYER               │
│        prometheus            grafana        │
└─────────────────────────────────────────────┘

🏗️ Design Decision: Domain Isolation

The most important infrastructure decision was data isolation by domain. Instead of a monolithic database, each microservice has its own instance:

# Each service has its own database — failures are isolated
broker-identity-db:
  image: mysql:8.0
  ports:
    - '3306:3306'  # identity

broker-wallet-db:
  image: mysql:8.0
  ports:
    - '3307:3306'  # wallet

broker-order-db:
  image: mysql:8.0
  ports:
    - '3308:3306'  # orders

broker-asset-db:
  image: mysql:8.0
  ports:
    - '3309:3306'  # assets

b3-core-db:
  image: postgres:15
  ports:
    - '5432:5432'  # B3 core

Why does this matter? If the orders database has a problem, the identity service keeps running. Failures stay isolated in their domain.

📦 Cache: Isolated Instances by Context

Two Redis instances with distinct purposes:

broker-asset-cache:
  image: redis:7.0-alpine
  ports:
    - '${REDIS_BROKER_ASSET_PORT}:6379'  # prices for the broker

b3-market-cache:
  image: redis:7.0-alpine
  ports:
    - '${REDIS_B3_MARKET_PORT}:6379'    # prices for the matching engine

The b3-market-sync-api writes to b3-market-cache and the b3-matching-engine-api reads from it to decide whether to execute or reject an order. The separation ensures that broker-side problems don't affect the B3 side and vice versa.

📡 Messaging: Two Brokers, Two Purposes

Apache Kafka — Internal Event Bus

Configured in modern KRaft mode (no Zookeeper), lighter and more stable for local development:

kafka:
  image: apache/kafka:3.7.0
  environment:
    KAFKA_PROCESS_ROLES: broker,controller
    KAFKA_LISTENERS: EXTERNAL://0.0.0.0:9092,INTERNAL://0.0.0.0:29092,CONTROLLER://0.0.0.0:9093
    KAFKA_ADVERTISED_LISTENERS: EXTERNAL://localhost:9092,INTERNAL://finance-kafka:29092
    KAFKA_CLUSTER_ID: MkU3OEVBNTcwNTJENDM2Qk

Used for the ecosystem's internal topics:

• assets-market-data-v1 — asset quotes
• order-events-v1 — order lifecycle events

RabbitMQ — Broker ↔ B3 Integration

rabbitmq:
  image: rabbitmq:3-management
  ports:
    - '5672:5672'    # AMQP
    - '15672:15672'  # Management UI

Used exclusively for communication between the broker and the B3 simulator:

• mq-broker-to-b3 — orders sent to the matching engine
• mq-b3-to-broker — execution feedback

Why two messaging systems? Kafka is ideal for event streams with replay and multiple consumers. RabbitMQ is ideal for work queues with point-to-point delivery guarantees. Each where it makes the most sense.

📊 Observability from Day One

prometheus:
  image: prom/prometheus:latest
  volumes:
    - ./config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml
  ports:
    - '9090:9090'

grafana:
  image: grafana/grafana:latest
  ports:
    - '3000:3000'

All Spring Boot microservices export metrics via /actuator/health and /actuator/prometheus. Prometheus collects, Grafana visualizes. From the very first service that starts, we already have observability.

🔐 Security: No Hardcoded Secrets

All sensitive configuration lives in a .env file (ignored by Git):

# docker-compose.yml — references only
broker-identity-db:
  environment:
    MYSQL_DATABASE: ${BROKER_IDENTITY_DB_NAME}
    MYSQL_USER: ${BROKER_DB_USER}
    MYSQL_PASSWORD: ${BROKER_DB_PASS}
    MYSQL_ROOT_PASSWORD: ${BROKER_DB_ROOT_PASS}

A .env.example is committed as a template, but actual values never enter the repository.

🚀 How to Run

cp .env.example .env
# fill in the variables in .env

docker-compose up -d

A single command brings up all 12 containers. The development environment is identical for anyone who clones the repository.

What's Next?

With the infrastructure running, the next post shows the first microservice: broker-market-data-api — a Python service that fetches real quotes from Brapi, persists them in MongoDB and publishes to Kafka.

🔎 About the Series

⬅️ Previous Post: Building a Microservices Ecosystem

➡️ Next Post: Market Data Integrator: Consuming Real-Time Data with Python, MongoDB and Kafka

📘 Series Index: Series Roadmap

Links:

• LinkedIn
• GitHub