الدرس 2 من 2
Read Models, Staleness, and the Cost of Level 4
Once the read store is updated asynchronously, you have accepted eventual consistency. That is a product decision before it is a technical one, and it needs designing rather than discovering.
Level 4 — a separate store updated asynchronously — is where CQRS stops being a code organisation choice and becomes an architectural one in the Chapter 1 sense. It buys independent scaling of reads and the freedom to use a store suited to the query shape. It costs a property users rely on without ever being told about it: that what they just did is what they now see.
Level 4, with the failure points named
Every arrow after the write transaction is a place where the read model can fall behind or diverge.
A command handler writes to the write store and enqueues an event in the same transaction through an outbox. A projector consumes events and updates a read store. Queries read only from the read store. Three failure points are marked: the projector falling behind, the projector failing on a poison event, and the read model diverging after a bug.
Write side
Command handler
invariants enforced
Write store + outbox
one transaction
Propagation
Projector
can lag, can fail, can skip
Read side
Read store
eventually consistent
Queries
never touch the write store
Synchronous projection versus asynchronous projection
Updated in the write transaction
The read model is written as part of the same commit. Levels 3 and some level 4 designs.
- No staleness: a read after a write always sees it
- No projector to operate, monitor or replay
- Failures are ordinary transaction failures
- Every write pays for the projection
- A projection bug can fail a business write
- Read and write stores must be transactionally reachable — in practice, the same database
Choose when: The read model lives in the same database and the write volume can absorb the extra work. This is the right default.
Updated asynchronously from events
A projector consumes events after commit and updates a separate store, possibly of a different kind.
- Reads scale independently of writes
- A store can be chosen for the query shape — search index, cache, analytical store
- A projection failure does not fail the business operation
- Eventual consistency, with all the product consequences
- A projector to monitor, with lag alerts and replay tooling
- Rebuild procedures needed for when the projection is wrong
Choose when: Read load genuinely exceeds what the write store can serve, or the query shape needs a store the write side cannot provide.
The operational work level 4 requires
- 1
Make projections idempotent
Delivery is at-least-once in every realistic setup, so applying the same event twice must be harmless. Carry a version or a sequence number and ignore anything already applied.
- 2
Monitor lag as a first-class metric
Projection lag in seconds, alerted against a threshold you have agreed with the product owner. Without it, the read model can be an hour behind and nobody knows until a customer notices.
- 3
Build the rebuild path before you need it
Every derived store eventually diverges — a bug, a skipped event, a bad deployment. A tested procedure that regenerates the read model from source is the only real answer, and building it during an incident is not the moment.
- 4
Handle poison events explicitly
One event the projector cannot process must not stop every other event forever. Route it aside, alert, and continue, accepting that one record is stale rather than all of them.
- 5
Decide staleness per screen and put it in the interface
Where the user could see stale data, tell them: a timestamp, a refresh control, or an explicit "updating" state. Silent staleness is what turns a design decision into a defect report.
In practice
Level 4 justified, and the cost paid honestly
A job board served two very different workloads from one PostgreSQL instance: applicant tracking, with modest write volume and strict correctness, and job search, with faceted full-text queries running at 200 times the write rate.
Constraints
- Search traffic 200x the write traffic and growing
- Faceted search across skills, location and salary bands
- Employers must see a job they just posted in their own dashboard immediately
- Candidates seeing a new posting a minute late is acceptable
Decision
Keep PostgreSQL as the write store with the full domain model. Project job postings asynchronously into a search index that serves candidate search only. The employer dashboard continues to read from PostgreSQL.
Why
The read and write workloads had genuinely different shapes and volumes, and no amount of indexing makes a relational store good at faceted search at that ratio. Crucially, the staleness question was answered per audience: employers need read-your-own-writes and were kept on the write store, while candidates do not and were moved to the index.
What it cost
They now operate a search cluster, a projector, lag monitoring and a rebuild job — roughly a quarter of one engineer's ongoing time. Twice in the first year the index diverged and had to be rebuilt, taking about 40 minutes each time during which search results were incomplete. The team documented the trade as "we accepted a rare, visible, recoverable degradation of candidate search in exchange for search latency falling from 3 seconds to 90 milliseconds", which is the form a trade-off record should take.
A proposal to "move reads to a separate store"
As a developer
Evaluates the technology: which store fits the query shape, how the projector is written, how events are delivered. All necessary questions.
As an architect
Starts with a different one: which screens may show stale data, and who decides that? It is a product question, and answering it usually reshapes the proposal — often into "these three screens move, the rest stay", which is much cheaper and far less risky than moving everything. Then asks who operates the projector at three in the morning and what the rebuild procedure is, because those costs are permanent while the migration is temporary.
Your read model has been serving incorrect totals for two days because a projector bug dropped one event type. What does this incident tell you about the design, beyond the bug itself?RevealHide
Three things, and none of them is "someone wrote a bug". First, nothing detected the divergence: a derived store needs a consistency check — a periodic comparison of aggregate counts or checksums against the source — because a wrong read model is silent by construction. Second, two days suggests there was no tested rebuild path, or the team was reluctant to use it, and both are fixable before the next incident. Third, and most important, the write store still holds the truth, which is the property that makes recovery possible at all; if the projection had been the only copy, the data would simply be wrong. That is the strongest argument for keeping a canonical write model whatever else you build on top of it.
Key takeaways
- Level 4 introduces eventual consistency, which is a product decision to be made per screen before adoption.
- Read-your-own-writes is an unstated expectation; resolve it by returning the value, routing the read, or waiting.
- Synchronous projection in the same transaction is the right default; asynchronous is for genuinely different read scale or store shape.
- Asynchronous projections require idempotency, lag monitoring, poison-event handling and a tested rebuild path.
- Keep a canonical write store: a divergent projection is recoverable only because the truth lives somewhere else.