When Kafka Becomes Your Database

Event Sourcing is not a storage optimization.
It is a truth model.

If you adopt it casually, it will break your system.
If you adopt it deliberately, it will change how you think about data forever.

1️⃣ The Fundamental Shift (This Is the Whole Game)

Traditional CRUD Model

“Store the current state.”

Order {
  status = SHIPPED
  total = 100
}

Event Sourcing Model

“Store everything that happened.”

OrderCreated
ItemAdded
PaymentCaptured
OrderShipped

State is derived. Events are the source of truth.

2️⃣ Why Kafka Fits Event Sourcing Naturally

Kafka provides:

Append-only log
Ordered per partition
Durable history
Replayability

That maps perfectly to event sourcing requirements.

Kafka doesn’t store rows.
It stores facts.

3️⃣ Core Principle (Non-Negotiable)

Events are immutable.

You do NOT:

Update events
Delete events
Fix events

You only:

Append new events

If this scares you — good. It should.

4️⃣ The Event Store (Kafka as DB)

Topic as Table

orders-events

Partitioning Rule

partition key = orderId

This guarantees:

Per-order ordering
Parallelism across orders

5️⃣ Commands vs Events (Critical Distinction)

Command

“Please do this.”

CreateOrder { orderId: 42 }

Event

“This happened.”

OrderCreated { orderId: 42 }

Commands can fail.
Events never lie.

6️⃣ Building State by Replaying Events

Example: Order Aggregate

function replay(events) {
  let state = { status: "NEW", items: [] };

  for (const e of events) {
    switch (e.type) {
      case "OrderCreated":
        state.id = e.orderId;
        break;
      case "ItemAdded":
        state.items.push(e.item);
        break;
      case "OrderShipped":
        state.status = "SHIPPED";
        break;
    }
  }
  return state;
}

The database is now a function of history.

7️⃣ Why This Is Powerful (Correctness Wins)

You gain:

Full audit log
Perfect traceability
Temporal queries (“what did we know then?”)
Replay after bugs
Deterministic recovery

Debugging becomes re-running history, not guessing.

8️⃣ Event Sourcing vs Traditional DB (Truth Table)

Dimension	CRUD DB	Event Sourcing
History	❌ Lost	✅ Preserved
Audit	Hard	Free
Replay	❌	✅
Fix bugs	Manual	Reprocess
Simplicity	✅	❌

9️⃣ The First Big Trap — Read Performance

Replaying 10,000 events on every read is not acceptable.

❌ Naive

Read → replay all events

✅ Solution — Projections (Read Models)

1️⃣0️⃣ Projections (Materialized Views)

Projections turn events into queryable state.

events → projection → read API

Example Projection

on("OrderCreated", e => db.insert({ id: e.orderId }));
on("ItemAdded", e => db.addItem(e.orderId, e.item));
on("OrderShipped", e => db.updateStatus(e.orderId));

Kafka is the write database.
Projections are read databases.

1️⃣1️⃣ Eventual Consistency (You Must Accept This)

Writes:

Immediate

Reads:

Eventually consistent

If you need:

Strong read-after-write
Immediate consistency

Event sourcing may be a bad fit.

1️⃣2️⃣ Schema Evolution (The Silent Killer)

Events live forever.

You cannot:

Change event meaning
Rename fields casually

Rule

Events are public contracts.

1️⃣3️⃣ Versioned Events (Required)

OrderCreatedV1 { orderId }
OrderCreatedV2 { orderId, currency }

Consumers must handle all historical versions.

1️⃣4️⃣ The Second Big Trap — Exactly-Once Illusion

Kafka gives:

At-least-once delivery

Your projection must be:

Idempotent
Offset-aware

if (alreadyApplied(eventId)) return;
apply(event);

You cannot delete events.

Common Solutions:

Encrypt PII and delete keys
Redaction events
Data minimization

Event sourcing + GDPR requires design-time planning.

1️⃣6️⃣ When Kafka Becomes Your Database (Danger Zone)

You are now responsible for:

Backups
Retention policies
Compaction strategy
Disaster recovery
Cross-region replication

Kafka is a database, whether you admit it or not.

1️⃣7️⃣ When Event Sourcing Is the Right Choice

Use it when:

Auditability is critical
Business logic is complex
Reprocessing is valuable
History matters
You can afford complexity

Examples:

Payments
Financial ledgers
Order lifecycles
Trading systems

1️⃣8️⃣ When Event Sourcing Is a Bad Idea

Avoid it when:

CRUD is simple
Strong consistency required
Team is small
Schema changes frequent
Operational maturity low

🎯 Mental Model to Remember

Traditional DBs store answers.
Event sourcing stores evidence.

📘 Event Sourcing: When Kafka Becomes Your Database