Mobile architecture
Eventual consistency in mobile apps, per field
"The app is eventually consistent" describes no app. Each field gets its own choice: strong, provisional, eventual or human. Then come the parts users notice: what the screen shows while data converges, how an optimistic change rolls back, and which edit wins. Last write wins, hybrid logical clocks and CRDTs, with what each costs, for iOS and Android.
15 min read
The bug report says the app is broken. A user renamed a trip on the phone, opened the tablet, and saw the old name for a few seconds. Nothing is broken. That is eventual consistency doing what it promises. The defect is that nobody decided what the tablet should show during those seconds, or whether this field could afford them.
I have run 500+ technical interviews from the hiring side over 15 years in mobile, and the same gap shows up in design reviews: "the app is eventually consistent" is offered as a property of the whole app. Eventual consistency is not an app setting. It is a choice per field, with a window the user can see and a rule for which edit wins.
Two pages on this site sit next to this one, and each keeps its own job. For the interview round, with cursors, tombstones and resync, see the offline sync system design answer. For picking or building the sync layer, see how to choose a sync solution. This page is the decision underneath both: which consistency each field gets, what the screen shows while data converges, and what each conflict policy costs. The model comes from my book Controlled Change.
- The choice: strong, provisional, eventual or human, per field.
- The screen: freshness, pending overlays and a rollback that cannot lie.
- The policy: last write wins, merges and CRDTs, what each costs, and a test that proves convergence.
Eventual consistency is a promise with a window
Eventual consistency says one thing: if writes stop, every copy reaches the same value. It says nothing about how long that takes, which value the copies agree on, or what the user sees in the meantime. Those three are product decisions, and a mobile app makes them more often than a backend does, because every device keeps its own copy for hours or days.
Controlled Change defines consistency in its glossary in a way that fits mobile better than a database textbook: "Mobile systems commonly combine authoritative server state, local provisional intent, eventual convergence, and explicit conflict resolution." The word that matters is combine. One app holds all four at once, on different fields.
The book's decision canvas asks the question per decision: strong, eventual, provisional, or human reconciliation. In terms of what the user experiences:
| Choice | What the user experiences | Use it for |
|---|---|---|
| Strong | Waits for the server; nothing shows as done until it is | Charging a card, granting access, anything you cannot take back |
| Provisional | Sees the request at once, marked as not yet confirmed; the server may say no | Bookings, orders, joining something with a limit |
| Eventual | Sees the change at once as done; other devices catch up | Names, notes, ticks, preferences |
| Human reconciliation | Sees both versions and picks one | Rare, high-value disagreements a machine should not settle |
The window has two budgets, and the book's offline chapter names both: staleness, how old data may be for display and for action, and uncertainty, how long an outcome may stay unknown. Write a number next to each field. A tablet showing a trip name five seconds late is fine. A tablet showing a revoked permission five minutes late may not be.
Choose consistency per field, not per screen
One screen usually mixes all four choices. A trip screen shows a title the user owns, a shared packing list, a price the server owns and a booking only the server can confirm. One consistency for the whole screen either makes the title wait for the server or lets the booking pretend.
This is my starting table for a typical app. It is a default to argue with, not a rule:
| Field | Owner | Write | Conflict policy | While it converges, the user sees |
|---|---|---|---|---|
| Display name, bio | The user | Local, queued | Last write wins per field | The new value at once |
| Note body | The user | Local, queued | Last write wins per note, or a text merge if people co-edit | The new text, marked saved on this device until confirmed |
| Checklist item done | The user, or shared | Desired state: done is true | Last write wins per item | The tick at once |
| Tags on a shared item | Shared | Add and remove operations | Set union; a concurrent add beats a remove | The tag at once; a removed tag can return if someone added it meanwhile |
| Order of a shared list | Shared | Move operations | A sequence merge, server order, or no offline reorder | Depends on the choice; often reorder is online only |
| Price, availability | The server | None | Revalidate before acting | The last known value with its age |
| Booking, order | The server | Command with a version and an idempotency key | Server decides; a stale version is refused | Request saved, never confirmed, until a receipt arrives |
| Permissions | The server | None | Server wins | Revoked content disappears; no grace period in the UI |
Two habits in that table do most of the work. Write desired state, not changes: the book's example is that setting a quantity to 3 replays safely where incrementing it does not. And merge per field, not per record: with the record as the unit, a name edited on one device and a bio edited on another become a conflict, and one edit is lost for no reason.
The table also shows what to keep out of offline. The book's own scoping list requires a connection to price or authorize payment, and delays collaborative reordering when its merge rules would confuse people. Leaving a field online only is a legitimate answer. In the book's words: "A narrow offline contract is often more valuable than a universal queue hidden behind optimistic UI."
What the user sees while data converges
Stale data is not the problem. Stale data presented as current is. Controlled Change makes freshness part of the type: a value carries when it was fetched, how long it stays valid, its source and its sync status, so the screen can tell the truth about it. Its example sets the per-field rule: "Displaying an old title may be harmless. Confirming an old price is not."
- Freshness where it changes a decision. An age label on a price or a seat count; nothing on a trip title. Revalidate before the action, not on every render.
- Pending where the user acted. A small marker on the row the user changed, not a global spinner. The book's state names are worth copying: saved on this device, waiting to send, sending, confirmation delayed, completed, needs attention.
- Absence as a state. A cache miss and an item the server says is gone are different states. A deleted item must leave the other device's screen, not linger because a stale row survived.
- Caught up, not connected. The book's chapter on real-time data separates the two: connected is a transport condition, caught up through a cursor is a data condition. A green dot that means connected tells the user nothing about whether the list is current.
One guarantee should never break: read your own writes. The device that made a change shows it at once and never flickers back to the old value while it syncs. The next section is how to keep that promise without lying when the server says no.
Optimistic UI and its rollback
Optimistic UI is a claim that the change will succeed. It is the right claim for eventual fields, where the user owns the value and the server rarely refuses. For provisional fields it must read as a request. For strong fields it should not be made at all. The book lists reporting optimistic UI as authoritative success among the common failures of state management.
The rollback is where most implementations break. The usual version writes the new value into the local row, then tries to restore the old one when the request fails. By then a pull may have overwritten the row, a second edit may have stacked on it, and the old value may be gone. Controlled Change describes what its fictional reference system, Atlas, does instead: render the confirmed local copy plus an overlay of pending intent, each overlay linked to the operation that created it. The book's verdict on the usual version: "This is safer than mutating the same row and later trying to infer which fields came from the server."
struct Note: Sendable {
let id: String
let title: String
let serverVersion: Int
}
enum PendingState: Sendable {
case waitingToSend, sending, confirmationDelayed
case rejected(reason: String)
}
struct PendingEdit: Sendable {
let operationID: String
let noteID: String
let title: String
let state: PendingState
}
enum RowStatus: Equatable, Sendable {
case confirmed
case pending
case needsAttention(reason: String, yourTitle: String)
}
struct NoteRow: Equatable, Sendable {
let id: String
let title: String
let status: RowStatus
}
// Confirmed rows are never mutated by the UI. Rolling back means dropping an overlay.
func compose(confirmed: [Note], pending: [PendingEdit]) -> [NoteRow] {
let latest = Dictionary(pending.map { ($0.noteID, $0) }, uniquingKeysWith: { _, newer in newer })
return confirmed.map { note in
guard let edit = latest[note.id] else {
return NoteRow(id: note.id, title: note.title, status: .confirmed)
}
switch edit.state {
case .waitingToSend, .sending, .confirmationDelayed:
return NoteRow(id: note.id, title: edit.title, status: .pending)
case .rejected(let reason):
// Show the server's value, and keep what the user typed for repair.
return NoteRow(id: note.id, title: note.title,
status: .needsAttention(reason: reason, yourTitle: edit.title))
}
}
}- Rollback is dropping an overlay. The confirmed row was never touched, so there is nothing to restore and nothing to get wrong.
- A rejection keeps the user's work. Show the server's value and keep what the user typed next to it, with the reason. The book's two options are to remove the overlay or offer repair; for anything the user wrote, offer repair.
- A timeout is not a rejection. The book is direct about it: "Unknown is essential, because a timeout does not mean failure." Keep the overlay as confirmation delayed and find out what happened, by looking the operation up or by an idempotent replay, before you roll anything back.
One more detail prevents the flicker users report as a bug: remove the overlay in the same local transaction that writes the confirmed value. Remove it first and the screen shows the old value for a moment. The book's test list states the rule as pending overlays disappearing only after terminal resolution. The same structure works on Android: a Flow of confirmed rows from the database combined with a Flow of pending operations.
Last write wins: per field, and ordered by something you trust
Last write wins has a bad reputation it only half deserves. It converges, it is cheap, and for a value one person owns it matches what users expect: the latest edit stands. The trouble sits in two details, the unit and the meaning of last.
The unit. Per record, it loses work for no reason, as above. Per field, independent edits both survive. The cost is a stamp per field instead of per record.
The meaning of last. Three common answers behave differently after time offline:
| Ordered by | What last means | Where it breaks |
|---|---|---|
| Device wall clock | The device that claims the latest time | Any clock set wrong: a phone a few minutes fast beats every correction made after it |
| Server order | The write the server accepted last | An edit made offline in the morning and synced at night beats an edit made at noon on another device |
| Hybrid logical clock | Close to when the edit was made, and never before an edit the device had already seen | A clock far ahead pushes every stamp ahead with it |
Server order is the right default when every write passes through your server and offline windows are short. Pair it with the version the edit started from, and a stale edit becomes a conflict to resolve instead of a silent overwrite. The book's line on that check: "A version mismatch is information, not transport noise."
A hybrid logical clock, from Kulkarni and colleagues in 2014, is for the case where edits must merge in the order they were made, including on the device. Each stamp is a wall time, a counter and a device ID. A device never issues a stamp lower than one it has seen, so an edit made after reading another always orders after it:
data class Hlc(val wallMillis: Long, val counter: Int, val nodeId: String) : Comparable<Hlc> {
override fun compareTo(other: Hlc): Int =
compareValuesBy(this, other, Hlc::wallMillis, Hlc::counter, Hlc::nodeId)
}
// One clock per device. Call it from one thread or actor.
class HlcClock(private val nodeId: String, private val now: () -> Long) {
private var last = Hlc(0, 0, nodeId)
// A local edit: stamp it.
fun tick(): Hlc {
val physical = now()
last = if (physical > last.wallMillis) Hlc(physical, 0, nodeId)
else last.copy(counter = last.counter + 1)
return last
}
// A remote stamp arrives: move past it, so the next local edit orders after it.
fun receive(remote: Hlc): Hlc {
val physical = now()
val wall = maxOf(physical, last.wallMillis, remote.wallMillis)
val counter = when {
wall == last.wallMillis && wall == remote.wallMillis -> maxOf(last.counter, remote.counter) + 1
wall == last.wallMillis -> last.counter + 1
wall == remote.wallMillis -> remote.counter + 1
else -> 0
}
last = Hlc(wall, counter, nodeId)
return last
}
}With stamps in place, per-field last write wins is a few lines, and the device ID breaks the last ties so every replica picks the same value:
data class Stamped<T>(val value: T, val stamp: Hlc)
fun <T> lww(a: Stamped<T>, b: Stamped<T>): Stamped<T> = if (a.stamp >= b.stamp) a else b
// Per field, not per record: an edit to the name never erases an edit to the bio.
data class Profile(val displayName: Stamped<String>, val bio: Stamped<String>)
fun mergeProfiles(a: Profile, b: Profile) =
Profile(lww(a.displayName, b.displayName), lww(a.bio, b.bio))Two things the code does not fix. A hybrid logical clock still trusts clocks to be roughly right: a device whose clock is a day ahead stamps a day ahead, and every device that receives its edit follows. Have the server compare incoming stamps with its own clock and reject or flag the ones too far ahead. And last write wins, ordered any way, still picks one value. When both values matter, you need a different policy, not a better clock.
Merges and CRDTs: where they fit, what they cost
When both concurrent edits must survive, you need a merge. Many merges need no special data type: tags merge by union, ticks merge by last write per item, an append-only history merges by putting entries in order. The book's policy table puts set union with observed remove on tags and membership-like collections, and reserves domain-specific transformation for reordering and collaborative text, the two places merges get hard.
A CRDT, a conflict-free replicated data type, is a structure whose merge converges whatever the delivery order, with no coordinator. That is a real property, and for the right data it is the best tool there is: two people typing into the same paragraph, a shared list both reorder offline, a counter several devices increment. The book's glossary adds the caveat that decides most cases: "It does not decide whether its merge semantics are correct for the product domain."
The cost list is what vendor pages leave out. Price it before you choose:
- Metadata. Text and list CRDTs give every element an identity and keep deleted elements as tombstones, so a document weighs more than its visible content.
- Cleanup needs agreement. A tombstone can go only when every replica has seen the delete. On mobile, an install nobody has opened for months is still a replica.
- Merge code ships in the binary. The merge runs on every device, so changing its rules means old app versions keep merging the old way. Controlled Change's chapter on schema evolution and version skew is the bill: queued work and its rules outlive the release that wrote them.
- Limits need coordination. A merge cannot enforce at most eight travelers or a balance that never goes below zero. Two devices can each add a traveler within the limit, and the merged result exceeds it. Those fields stay with the server.
- The server still authorizes. A merged operation is still a write, and the server checks that this user may make it.
- Debugging needs history. "Why does it say this?" is answered by the operation history, which you then keep, compact and protect like user data.
My rule of thumb: per-field last write wins for values one person owns, set union for tags and memberships, the server for anything with a limit, money or access, and a CRDT only for collaborative text and shared ordered lists where losing either edit is the bug. If conflicts on a field are rare, a choice for the user may be cheaper than any merge. The book's own example is a trip title, where "Keep mine" and "Use updated" may be enough.
Test convergence, not the happy path
Eventual consistency fails in delivery orders nobody tries by hand. The book's advice for conflict tests is the most useful line on the subject: "Model two or more replicas, permute delivery, and assert the chosen convergence rule." For small cases, try every order outright:
fun <T> permutations(items: List<T>): List<List<T>> =
if (items.size <= 1) listOf(items)
else items.indices.flatMap { i ->
permutations(items.take(i) + items.drop(i + 1)).map { listOf(items[i]) + it }
}
// Every delivery order, plus one duplicate, must reach the same state.
fun assertConverges(start: Profile, edits: List<Profile>) {
val deliveries = edits + edits.first()
val outcomes = permutations(deliveries).map { order -> order.fold(start, ::mergeProfiles) }.toSet()
check(outcomes.size == 1) { "replicas diverged: $outcomes" }
}Every order grows fast, so keep this to a handful of edits, and let a property-based testing library generate random orders for larger cases and shrink a failure to its smallest form. The harness catches a merge that depends on arrival order, the bug that looks fine in every demo.
Then test what the user sees, because convergence can be correct while the screen is wrong. From the book's list of invariant tests, the ones that map to this page:
- Replaying one operation produces one effect.
- Two independent edits merge without dropping either.
- Coupled fields conflict instead of merging into nonsense.
- A delete does not come back after a stale update.
- Pending overlays leave only after a final outcome.
- Server and client converge when pull pages arrive out of order.
For failures around the network and the process, such as a kill right after the local commit or a response lost after the server commits, use the failure list in how to choose a sync solution.
Measure the window in production
A consistency choice is a hypothesis until production data backs it. Four numbers cover most of it, all without logging content:
- Convergence time: from the server accepting a write to a second device showing it, at p50 and p99.
- Age of the oldest pending operation, and how long outcomes stay unknown.
- Conflicts per thousand edits, by field.
- Rollbacks shown to users, by reason.
Compare each with the budgets you wrote next to each field. A field whose conflict rate keeps climbing is telling you its owner or its unit is wrong. Before a cleverer merge, the book suggests smaller aggregates, operations that commute, reservation windows or a single writer.
A per-field consistency sheet
Before you build, write one row per field. It fits in a design doc or an ADR:
- The field, and who owns its truth.
- Its consistency choice: strong, provisional, eventual or human.
- Its staleness budget for display and for action, and its uncertainty budget.
- The write: desired state, an operation, or a command with a version.
- The conflict policy, and what orders last if it is last write wins.
- What the screen shows while pending, after a rejection and after a timeout.
- The test that proves convergence and the metric that watches it.
If a row has no answer on line five, the field is using last write wins by device clock, whether anyone chose it or not. For the interview version of this ground, see the worked offline sync answer; for who owns state inside an iOS screen, SwiftUI architecture is state ownership.
Questions engineers ask about eventual consistency on mobile
How do you handle eventual consistency in a mobile app?
Decide per field, not per app. For each field, name who owns the truth and pick one of four choices: strong (the user waits for the server), provisional (the app shows the request as pending and the server may refuse it), eventual (accepted locally, converges on its own) or human reconciliation. Then design what the screen shows while data converges, how an optimistic change rolls back, and a test that delivers the same edits in every order.
CRDT vs last write wins: which should a mobile app use?
Per-field last write wins is the default for values one person owns, where losing one of two concurrent edits is acceptable. Order it by the server or by a hybrid logical clock, never by the raw device clock. Use a CRDT where people edit the same text or ordered list concurrently and both edits must survive. CRDTs cost metadata, storage and merge code that ships in every app version, and they cannot enforce a limit such as a seat count on their own.
How do you roll back an optimistic update?
Do not write the optimistic value into the confirmed row. Keep it as a pending overlay linked to its operation. Rolling back is then dropping the overlay, and a rejection can show the server's value next to what the user typed. A timeout is not a rejection: keep the overlay as confirmation delayed until you know the outcome.
Do I need a hybrid logical clock in a mobile app?
Only when edits must merge in the order they were made and the server's arrival order would let an old offline edit win. If every write passes through your server and offline windows are short, server order is simpler. A hybrid logical clock still trusts clocks to be roughly right, so have the server reject or flag stamps far ahead of its own time.
