Understanding Eventually Consistency

This is from my series of System design for Developers and based on the book I am reading "Database Internals".

Before we jump into understanding "eventually consisteny". We need to have a clear understanding of 2 questions:

1 . What is consistency?

2 . Why do we need it?

For the above questions, let's try to understand by a simple daily life analogy - Coffee shop.

Your Coffee Shop Becomes a Chain

Imagine you own a local coffee shop that rewards customer through an app. Every time someone buys a coffee, they earn credits. This means 1 shop, 1 app, 1 database - everything is perfect.

and then your coffee becomes so popular that you expand into a nationwide chain. Now your system needs to scale.

Suddenly, you’re dealing with:

• Multiple locations

• More users logging in, especially during peak hours (mornings & weekends)

This means we have to scale our App to manage the lodas. It means - More databases, load balancers, and monitoring tools and we ended up with the distributed system.

Now here comes the problem.

A customer buys a coffee in Location A, but doesn’t immediately see their reward points reflected in the app. Is this a bug? Why? is it happening?

Because in a distributed system, writes (like earning credits) may take time to propagate across replicas.

This delay is known as the consistency delay window — the time between the data being written and it becoming visible across all nodes.

So, this is not a bug but a trade-offs (we will see this ahead).

What is Consistency?

In databases, consistency means showing the most recent and correct data to the reader. When consistency is guaranteed, users always get the latest view of the data.

Why Do We Need It?

From the example you must have understood that

To avoid:

1 . Stale reads (seeing outdated data)

2 . Confusing user experiences (e.g., missing points or duplicate messages)

3 . Conflicting writes

Consistency ensures that your app behaves as expected, especially when it spans multiple systems.

Now that we understand what consistency is and why it matters, let’s explore eventual consistency.

Distributed databases often follow one of the following consistency models:

1 . Strong Consistency – Always shows the latest data.

2 . Eventual Consistency – Guarantees that, eventually, all replicas will converge to the latest data.

3 . Loose (or Weak) Consistency – Does not guarantee a consistent view at all times.

Every model comes with its pros and cons. Eg: Strong consistency will cost performance. So, as an architect one needs to evaluate trade-offs and decide accordingly which consistency model to have.

This blog will focus on "eventually" consistency.

What is Eventual Consistency?

In systems with eventual consistency, the data might be out of sync temporarily (remember delay/inconsistency window??), but it will converge over time.

Examples: Banking apps (a pending transaction might not show instantly), Social media likes/comments, Messaging apps (seen/read receipts can lag), Shopping carts in e-commerce platforms

in our coffee app: A user earns credits at Store A. The app may not reflect it instantly across all devices or stores but eventually, once all replicas sync up, the correct balance is shown.

This model works well for high-traffic systems where availability and scalability are more important than instant consistency.

How Do Systems Handle It?

Is eventual consistency a problem? Not necessarily. Like any design decision, “it depends.” As I mentioned earlier, every consistency model has trade-offs. One need to evalaute the trade-offs accordingly.

For eventually Consistency, goal should be - Minimize inconsistency windows.

But how can we handle such "inconsitency"?

1 . Ensure "read your own write" – where the user sees their most recent action. Eg: When the user bought a coffee, in the app we can show coffee is purchase and points are credit (which will eventually in future will synced)

2 . Repair data automatically through background sync (active/passive repair): This is a way to repair the replicas - Active repair : running job at scheduled time. If there are any mistamtches in replicas then sync them. Passive repair: App notices missing points at a store and silently fixes it

3 . Allow tunable consistency – some databases let developers choose the level of consistency per operation (like Cassandra, DynamoDB).

Summary

When we scale databases across nodes and regions, maintaining consistency becomes harder. Eventually consistent systems trade off immediacy for scalability and availability.

As a developer, it’s crucial to:

• Understand the trade-offs

• Know your system’s consistency model

• Design features with the delay window in mind

• Use monitoring and repair mechanisms wisely

In short, not all apps need strong consistency—but all developers need to understand consistency.