Own Every Layer

No Kafka in V1: the complexity we refused to buy

Belief 05 · 8 min read ·

We run a modular monolith. Our async work goes through a Postgres table, polled with row-level locking, rather than a message broker. There is no service mesh, no Kafka, and no distributed tracing across services that do not exist.

This is not an argument that microservices are wrong. It is an argument that most complexity in modern software is not solving a problem the company has — it is solving a problem the company has read about.

What a broker would have cost us

A message broker is not one decision. It is a cluster to operate, a delivery semantics question to answer, a schema-evolution policy, a dead-letter strategy, a replay story, a monitoring surface, and a new class of incident where the system is up but eleven minutes behind. Every one of those is tractable. None of them is free, and at our scale none of them buys anything a Postgres table does not.

The queue we have is a table with a status column and locking that lets multiple workers claim rows without stepping on each other. It is transactional with the rest of our data, which removes an entire category of bug — the one where a database write commits and the corresponding message does not.

The part that makes this safe

The reason this is not technical debt is that the boundaries are drawn now, even though the split is deferred. The modules have real seams. The queue sits behind the same kind of owned interface as every other provider category, so swapping Postgres for a broker later is a configuration change and an adapter, not a rewrite of the calling code.

Deferred complexity and absent structure look identical from outside and behave completely differently under growth. The difference is entirely in whether the boundaries exist before you need them.

What would change our minds

Specific, measured conditions — not a headcount and not a funding round:

  • Queue throughput where Postgres polling contention is measurably the bottleneck, after the obvious indexing and batching work is done.
  • A genuine need to replay a stream, or for several independent consumers of the same event with different retention needs.
  • Deployment coupling that is demonstrably slowing releases — teams blocked on each other, not merely sharing a repository.

Until one of those is true and measured, adding a broker would be paying a permanent operational tax to look like a larger company.

Simple until proven otherwise

The word doing the work in that sentence is proven. It is not a commitment to stay small or a rejection of distributed systems. It is a requirement that the evidence arrives before the complexity does.

Own every layer.