Building block
Database replication
Copies data from the primary database to one or more replicas. It scales reads and enables fast recovery after a failure, but introduces replication lag.
Primary and replicas
In the most common setup all writes go to the primary, which ships changes to replicas. PostgreSQL does this with streaming replication over WAL, MySQL with binlog replication. Replicas serve reads only.
If reads outnumber writes 10 to 1 (course catalog, lesson pages), two replicas take most of the load off the primary. Reports and analytics should also run on a separate replica so heavy queries do not slow down production.
Synchronous or asynchronous
Asynchronous replication: the primary acknowledges a write immediately and the replica catches up later. Fast, but if the primary dies, the last milliseconds or seconds of writes can be lost.
Synchronous: the primary waits for at least one replica to confirm (synchronous_commit and synchronous_standby_names in PostgreSQL). Nothing is lost, but every write is slower by a round trip to the replica, so the synchronous standby is kept in the same region.
Replication lag and failover
Lag is usually milliseconds, but under heavy writes or a long query on the replica it grows to seconds. The classic bug: a user changes their name, the page rereads the profile from a replica and shows the old one. The fix is read-your-writes: read from the primary for a few seconds after the user's own write.
Failover: when the primary dies, one replica is promoted to primary and the app switches to it. Patroni does this for PostgreSQL, orchestrator for MySQL, and managed databases (RDS, Cloud SQL) do it automatically in 1 or 2 minutes.
Pros
- Read scaling without changing the data model.
- Fast recovery when the primary fails.
- A separate replica for reports and backups does not disturb production.
Cons
- Writes do not scale: they all go to one primary.
- Replication lag gives stale reads.
- Asynchronous replication can lose the last writes on failover.
- Failover has to be set up and tested regularly.
When to choose it
- Reads far outnumber writes.
- You need high availability for the database.
- Heavy reports interfere with the main workload.
- As the first step in scaling a database, long before sharding.
Where it is used
- GitHub: MySQL high availability with orchestrator, Consul and GLB (2018 write-up).
Tools
- PostgreSQLA powerful open source relational database with extensions, JSONB, full-text search and strong SQL support.
- MySQLOne of the most widespread relational databases, the foundation of many web projects and the LAMP stack.
- MariaDBAn open fork of MySQL by its original founders, protocol compatible with extra engines and features.
- MongoDBThe most popular document NoSQL database, storing data as JSON-like documents (BSON).
- AWSThe largest cloud platform, with hundreds of services from virtual machines and storage to databases, queues and ML.
Related topics
Sources
Logos belong to their owners and are used for identification only. Icons: Simple Icons (CC0), Devicon (MIT).