Skip to main contentSkip to navigation
ThisIsHowItWorks.in

Complex systems, clearly explained.

An independent visual publication explaining the invisible protocols, networks, infrastructure, and mechanisms that run our world.

Explainers

  • How UPI Works
  • Offline UPI Mechanisms
  • All Explainers (Archive)
  • Topics & Roadmap
  • Curated Library

Publication

  • About Publication
  • Editorial Principles
  • Theme Index
  • Technical Lexicon
  • Open Questions
  • Changelog
  • RSS / Atom Feed

Legal & Contact

  • Privacy Policy
  • Terms of Use
  • Editorial & Legal Notice
  • Contact Us

Connect

  • Instagram
  • Discord Community
© 2026 ThisIsHowItWorks.in. All rights reserved.
Durable technical understanding built from first principles.
ThisIsHowItWorks.in
ExploreTopicsPathsAbout
  1. Home
  2. /Topics
  3. /Computing & Digital Architecture
  4. /Computing & Digital Architecture
  5. /How Relational Databases Guarantee ACID Transactions Without Corrupting Data
Computing & Digital Systems · computing/ Explainer

How Relational Databases Guarantee ACID Transactions Without Corrupting Data

From Write-Ahead Logging and page buffer pools to Strict Two-Phase Locking, MVCC snapshot isolation, and ARIES crash recovery

25 min read·Updated for clarity
The Short AnswerFirst-Principles Core

“How do relational database engines guarantee that multi-step transactions execute completely or leave zero trace, while hundreds of concurrent clients read and write the same tables and server power can vanish at any microsecond?”

A relational database processes tens of thousands of concurrent financial transactions every second without losing a cent or corrupting records, even if server power is severed mid-write. This data reliability is achieved through ACID semantics, Write-Ahead Logging under a steal/no-force buffer pool policy, Strict Two-Phase Locking and Multi-Version Concurrency Control (MVCC), and the landmark ARIES crash recovery algorithm.

What This Explainer Answers
  • What do Atomicity, Consistency, Isolation, and Durability (ACID) strictly guarantee at the physical storage level?
  • Why would writing modified database pages directly to disk on every commit destroy storage throughput?
  • What are the Steal/No-Steal and Force/No-Force buffer management policies, and why do modern engines choose Steal and No-Force?
  • What is Write-Ahead Logging (WAL), and how do Log Sequence Numbers (LSNs) tie in-memory dirty pages to on-disk log records?
  • How does the landmark ARIES algorithm use Analysis, Redo, and Undo passes to recover from catastrophic hardware crashes?
  • Why do Compensation Log Records (CLRs) prevent infinite crash loops if the system crashes repeatedly during recovery?
  • How does Strict Two-Phase Locking (SS2PL) prevent dirty reads, non-repeatable reads, and cascading transaction rollbacks?
  • How does Multi-Version Concurrency Control (MVCC) allow readers to read data without locking out concurrent writers?
  • How do distributed databases coordinate transactions across separate servers using Two-Phase Commit (2PC)?
Recommended Background
Relationship: Requires understanding of

To build a complete first-principles mental model of this mechanism, we recommend starting with these foundational explainers:

Requires understanding of
How Computer Memory (RAM) Actually Works
Understanding How Computer Memory (RAM) Actually Works is required before reading How Relational Databases Guarantee ACID Transactions Without Corrupting Data
Requires understanding of
How an Operating System Actually Runs Software
Understanding How an Operating System Actually Runs Software is required before reading How Relational Databases Guarantee ACID Transactions Without Corrupting Data
Requires understanding of
How Solid-State Drives (SSDs) and Flash Memory Store Data Without Power
Understanding How Solid-State Drives (SSDs) and Flash Memory Store Data Without Power is required before reading How Relational Databases Guarantee ACID Transactions Without Corrupting Data
In this Explainer10 Sections
  1. 01The ACID Guarantees: Defining Data Integrity
  2. 02The Physical Storage Reality: RAM vs. Disk and the Buffer Pool
  3. 03The Write-Ahead Logging (WAL) Protocol
  4. 04The ARIES Crash Recovery Algorithm: Rising from the Ashes
  5. 05Concurrency Control: The Race Against Anarchy
  6. 06How Isolation Is Enforced: Strict 2PL vs. MVCC
  7. 07The Layered Storage Architecture of a Modern Database
  8. 08Distributed Transactions: Two-Phase Commit (2PC)
  9. 09Summary: The Engineering Foundations of Trust
  10. 10Related Knowledge & Prerequisites

How Relational Databases Guarantee ACID Transactions Without Corrupting Data

Consider a routine banking operation: transferring $100 from Alice's account to Bob's account. In SQL, this appears deceptively simple—two consecutive updates executed within a single transaction block:

BEGIN TRANSACTION;
  UPDATE accounts SET balance = balance - 100 WHERE id = 'Alice';
  UPDATE accounts SET balance = balance + 100 WHERE id = 'Bob';
COMMIT;

Now consider what can go wrong in the physical world while these two statements execute:

  1. The server's power cord is ripped from the wall between the first update and the second update.
  2. The operating system suffers a kernel panic while the disk drive is writing Bob's new balance.
  3. Ten other concurrent clients attempt to read Alice's balance or withdraw money from Bob's account at the exact same millisecond.
  4. The server runs out of disk space, or a foreign key constraint triggers a runtime validation failure.

In a naïve system, Alice would lose $100 while Bob never receives it—creating or destroying money out of thin air. Or a concurrent transaction might read Alice's debited balance before Bob is credited, reporting an inconsistent total sum.

Yet commercial relational database management systems—such as PostgreSQL, MySQL InnoDB, Oracle, and Microsoft SQL Server—process millions of financial transactions every day without losing a single cent or corrupting a single table page.

They achieve this through the four guarantees of ACID: Atomicity, Consistency, Isolation, and Durability.

To understand how software achieves ironclad mathematical reliability atop fallible silicon, volatile RAM, and mechanical or flash storage, we must explore the database storage engine from first principles: from the buffer pool trade-offs that govern disk access, to the write-ahead log that records every keystroke of history, the mathematical proofs of concurrency control, and the landmark ARIES recovery algorithm that reconstructs truth from ashes after a sudden crash.


1. The ACID Guarantees: Defining Data Integrity

The term ACID was popularized by computer scientist Jim Gray in the late 1970s and formalized by Andreas Reuter in 1983. Each letter represents a mathematically distinct guarantee:

                                 THE ACID CONTRACT
                                 
      A - Atomicity           C - Consistency         I - Isolation           D - Durability
   -------------------     ---------------------   -------------------     --------------------
    All-or-Nothing; no      Preserves declared      Concurrent queries      Committed changes
    partial execution       invariants, types,      execute as if run       survive power loss,
    survives failure.       and constraints.        in pure isolation.      crashes, and reboots.

Atomicity: The All-or-Nothing Invariant

A transaction is treated as an indivisible unit of work. Either every single operation within the transaction successfully completes and takes effect, or the entire transaction is rolled back with zero trace left in the database. There is no intermediate state where Alice's account is debited but Bob's account is not credited.

Consistency: Valid State Transitions

A database starts in a valid state that satisfies all defined schema constraints: primary keys, unique indices, foreign key references, and table check constraints. A transaction can only commit if it transitions the database from one valid state to another valid state. If any constraint is violated (e.g., an account balance drops below zero when a CHECK (balance >= 0) constraint exists), the transaction is aborted and rolled back.

Isolation: Concurrent Execution Without Interference

In production, hundreds or thousands of clients execute transactions simultaneously against the same tables. Isolation guarantees that the concurrent execution of transactions produces the exact same system state as if those transactions had been executed sequentially, one after another (Serializability). Intermediate, uncommitted changes made by transaction T1 are invisible to transaction T2.

Durability: Survivability Across Catastrophe

Once the database engine returns a success status for a COMMIT statement to the application client, the changes made by that transaction are guaranteed to be permanently recorded. Even if the building loses power one microsecond later, or the operating system crashes, the data will remain intact and uncorrupted when the system reboots.


2. The Physical Storage Reality: RAM vs. Disk and the Buffer Pool

To understand why ACID is challenging, we must confront the physical hierarchy of computer memory, as examined in How Computer Memory (RAM) Works and How Solid-State Drives and Flash Memory Work:

                        THE LATENCY AND VOLATILITY GAP
                        
   Memory Layer        Typical Latency         Bandwidth            Volatility
   ---------------------------------------------------------------------------------
   CPU L1/L2 Cache     0.5 - 4 ns             > 1,000 GB/s          Volatile (Lost on power cut)
   DRAM (RAM)          50 - 100 ns            ~ 50 - 100 GB/s       Volatile (Lost on power cut)
   NVMe SSD (Flash)    10 - 50 µs             ~ 3 - 7 GB/s          Non-Volatile (Permanent)
   Mechanical HDD      5 - 15 ms (5,000 µs)   ~ 150 - 250 MB/s      Non-Volatile (Permanent)

Notice the critical trade-off:

  • RAM is volatile: It loses all contents instantly when electrical power is severed. But it is fast (nanosecond latency).
  • Disk/SSD is non-volatile: It retains bits indefinitely without power. But it is orders of magnitude slower (microseconds to milliseconds) and organizes data in discrete chunks called pages (typically 4 KB, 8 KB, or 16 KB).

If a database wrote every updated table row directly to disk before acknowledging each query, storage throughput would collapse. If a bank processes 10,000 transactions per second, performing 20,000 random disk page writes per second would saturate the storage bus and stall the CPU.

The Page Buffer Pool

To maximize performance, database storage engines maintain a large region of RAM called the Buffer Pool (or Buffer Cache). The buffer pool caches disk pages in memory:

                         THE DATABASE BUFFER POOL ARCHITECTURE
                         
           [ Application Transactions / SQL Engine ]
                            |
                            | Reads and updates rows in memory
                            v
   +--------------------------------------------------------+
   |             BUFFER POOL (RAM - Fast, Volatile)         |
   |                                                        |
   |   [ Page 1 (Clean) ]      [ Page 2 (DIRTY: Alice) ]    |
   |   [ Page 3 (Clean) ]      [ Page 4 (DIRTY: Bob)   ]    |
   +--------------------------------------------------------+
                            |
                            | Asynchronous background flush (Dirty pages)
                            v
   +--------------------------------------------------------+
   |          PERSISTENT DISK STORAGE (Non-Volatile)        |
   |                                                        |
   |   [ Table Data Files ]          [ WAL Log Files ]      |
   +--------------------------------------------------------+

When a query modifies Alice's balance, the database does not write to disk. Instead:

  1. It locates Page 2 in the buffer pool (or reads it from disk if not cached).
  2. It modifies the row directly inside Page 2 in RAM.
  3. Page 2 is now marked as Dirty (its in-memory content differs from its on-disk content).

Eventually, a background thread (the page writer or checkpoint daemon) flushes dirty pages back to disk. But this raises two fundamental questions about buffer management policies:

Steal vs. No-Steal Policy

  • STEAL: Can the buffer manager evict a dirty page to disk to free up RAM for other queries before the transaction that modified that page has committed?
    • If YES (Steal): The buffer pool can handle transactions larger than physical RAM. But if the transaction later aborts or the server crashes, uncommitted data sits on the permanent disk, violating Atomicity!
    • If NO (No-Steal): Dirty pages must stay pinned in RAM until commit. This prevents uncommitted data from touching disk, but severely limits transaction size to available RAM.

Force vs. No-Force Policy

  • FORCE: Must the buffer manager force every dirty page modified by a transaction to be flushed to disk before the COMMIT call returns success?
    • If YES (Force): Durability is trivial. When COMMIT returns, data is guaranteed to be on disk. But every commit requires multiple random I/O writes, destroying performance.
    • If NO (No-Force): COMMIT returns immediately while modified pages remain dirty in RAM. Performance is phenomenal, but if power cuts out one millisecond later, committed data in RAM is wiped out, violating Durability!
Buffer Pool Management Policies and Durability Trade-offs

No-Steal / Force

Poor RAM scalability; every commit stalls on synchronous random disk I/O; trivial crash recovery; toy educational engines only.

No-Steal / No-Force

Poor RAM scalability for large transactions; fast commits without data page writes; moderate recovery redoing committed changes; rarely used.

Steal / Force

High RAM scalability; poor commit throughput stalling on disk writes; moderate recovery undoing dirty pages; obsolete legacy architecture.

Steal / No-Force

Maximum RAM scalability supporting arbitrarily large transactions; blazing commit throughput with async disk flushing; complex recovery requiring WAL and ARIES; universal industry standard across PostgreSQL, MySQL InnoDB, Oracle, and SQL Server.

Comparison table contrasting Steal vs No-Steal and Force vs No-Force buffer policies

Every major production relational database uses STEAL (to allow arbitrarily large transactions) and NO-FORCE (to avoid stalling commits on random disk I/O).

How can a database possibly guarantee Atomicity (no uncommitted data survives) and Durability (no committed data is lost) while running under a Steal / No-Force policy?

The answer is the Write-Ahead Log.


3. The Write-Ahead Logging (WAL) Protocol

The Write-Ahead Logging (WAL) protocol—often referred to in journaling file systems as the redo/undo log—solves the Steal/No-Force dilemma by introducing a secondary, append-only file on disk.

Instead of writing whole 8 KB random data pages to disk during a transaction, the database writes a compact, sequential stream of log records describing the change:

               ANATOMY OF A WRITE-AHEAD LOG (WAL) RECORD
               
   [ LSN: 1042 ]  [ PrevLSN: 1038 ]  [ TxID: T1 ]  [ Type: UPDATE ]
   [ PageID: Page 2 ]  [ Offset: Row 4 ]
   [ Redo Payload: New balance = $900 ]  <--- Needed if power cuts out (Durability)
   [ Undo Payload: Old balance = $1000 ] <--- Needed if transaction aborts (Atomicity)

Each log record contains:

  • LSN (Log Sequence Number): A monotonically increasing 64-bit integer giving the exact byte offset of the record in the log file.
  • PrevLSN: The LSN of the previous log record written by this same transaction (forming a reverse linked list of actions for that transaction).
  • TxID: The transaction identifier.
  • Redo Information: The data necessary to re-apply the change if committed data in volatile RAM was lost before reaching disk.
  • Undo Information: The data necessary to reverse the change if uncommitted data was written to disk by a Steal policy.

The Fundamental WAL Invariants

The Write-Ahead Logging protocol enforces two inviolable rules:

  1. The Undo Rule (Before Stealing): Before any dirty data page in RAM can be written to disk, all log records updating that page (up to and including the page's current state) must already be written to non-volatile log storage.
  2. The Redo Rule (Before Committing): Before a transaction is acknowledged as committed to the client, all log records generated by that transaction—concluding with a COMMIT record—must be flushed to non-volatile disk storage.
                           THE WAL TIMING INVARIANT
                           
            1. Transaction modifies Page in RAM
                     |
                     v
            2. WAL Record written to WAL Buffer in RAM
                     |
                     v  (MUST FLUSH LOG FIRST!)
            3. WAL Buffer flushed to Disk (Sequential I/O: fsync)
                     |
                     v  (ONLY NOW IS STEAL / COMMIT PERMITTED!)
            4. Dirty Data Page flushed to Data Files on Disk (Random I/O)

Why is this protocol so fast? Because appending to a log file is pure sequential I/O! Writing sequential log records to an NVMe drive or hard disk takes microseconds and achieves gigabytes-per-second throughput. The database does not wait for slow, random data pages to be written across the disk platter or flash blocks; it flushes a few hundred bytes of sequential log records using an operating system fsync() call and immediately confirms the transaction as committed!

Log Sequence Numbers and PageLSN

How does the storage engine coordinate between data pages and the log?

Every data page on disk and in memory carries a dedicated header field: PageLSN.

Whenever a transaction modifies a page in the buffer pool, it records the LSN of the generated log record into that page's PageLSN field:

   Page 2 Header: [ PageID: 2 | PageLSN: 1042 | FreeSpace: 2048 | Rows... ]

When the buffer manager later decides to flush Page 2 to disk (to free up RAM), it checks:

FlushedLSN ≥ PageLSN

If the highest LSN currently flushed to disk (FlushedLSN) is less than the page's PageLSN (1042), the buffer manager refuses to write the page to disk. It first triggers a synchronous flush of the WAL buffer up to LSN 1042. Only once the log is secure on disk does it permit the dirty data page to be written.

This guarantees that disk storage never contains a dirty change without the corresponding undo record already safely preserved on disk!


4. The ARIES Crash Recovery Algorithm: Rising from the Ashes

What happens when disaster strikes? The power fails, the operating system panics, and the database restarts.

The disk now contains an unpredictable mixture:

  • Some committed transactions had their dirty pages flushed to disk before the crash; others did not.
  • Some uncommitted transactions had their dirty pages stolen and flushed to disk before the crash.

To restore absolute consistency, modern databases implement ARIES (Algorithms for Recovery and Isolation Exploiting Semantics), developed by IBM researcher C. Mohan in 1992.

ARIES operates on a three-phase recovery protocol:

                       THE THREE PASSES OF ARIES RECOVERY
                       
            Crash Point
                |
                v
   Log: ... [Checkpoint] .................................... [CRASH]
                |
                +=== Phase 1: ANALYSIS PASS (Scan Forward) ===>
                |    Reconstructs active transactions & dirty pages at crash
                |
     <----------+=== Phase 2: REDO PASS ("Repeating History") ====>
     (Oldest unwritten Page)
                     Replays all logged changes (committed & uncommitted)
                     
     <================================ Phase 3: UNDO PASS (Scan Backward)
                                       Rolls back all uncommitted "loser" transactions

Phase 1: The Analysis Pass

The database inspects the log starting from the most recent Checkpoint.

  • It scans forward to the end of the log (the moment of the crash).
  • It reconstructs the Transaction Table (TT): identifying all active transactions that were running when the crash occurred (called loser transactions).
  • It reconstructs the Dirty Page Table (DPT): identifying every page that was dirty in memory at the time of the crash, along with the earliest LSN that modified it (RecLSN).

Phase 2: The Redo Pass ("Repeating History")

ARIES finds the smallest RecLSN across all pages in the Dirty Page Table. This represents the oldest modification in the log that might not have made it to disk.

Starting from this point, ARIES scans forward through the log, reapplying every single logged change up to the crash point.

Crucially, ARIES redoes changes for both committed and uncommitted transactions! This is called Repeating History. By reapplying every operation, the database reconstructs the exact physical and memory state that existed at the precise microsecond of the crash.

How does ARIES avoid re-doing changes that were already safely on disk before the crash? It reads each page's header:

  • If PageLSN ≥ LogRecord.LSN, the change is already present on the page. Skip it!
  • If PageLSN < LogRecord.LSN, apply the change and update PageLSN = LogRecord.LSN.

Phase 3: The Undo Pass

At the end of the Redo pass, the database is in the exact state it was in when power failed. But uncommitted loser transactions are still present!

ARIES now scans backward from the crash point, rolling back every modification made by the loser transactions.

  • As it undoes each update, it restores the row's prior value using the log record's Undo payload.
  • It writes a special Compensation Log Record (CLR) to the log for every undone action.

The Magic of Compensation Log Records (CLRs)

What happens if the server crashes again while executing the Undo pass of recovery?

Without CLRs, the system could enter an infinite crash-and-rollback loop, repeatedly trying to undo an action that was already partially undone.

A CLR contains a field called UndoNextLSN, which points directly to the next older log record that needs to be undone for that transaction, skipping the action just reversed. CLRs are never undone. If the system crashes during recovery, the subsequent recovery pass reads the CLRs, instantly skips already-reversed operations, and continues backward without repetition!

The ARIES Three-Pass Crash Recovery Lifecycle
processServer Crash & RebootOperating system reboots; database detects ungraceful shutdown.
processPhase 1: Analysis PassScans forward from latest checkpoint to identify dirty pages and active loser transactions.
decisionRedo Scan PointLocates smallest RecLSN in Dirty Page Table; begins repeating history.
processPhase 2: Redo PassReplays all logged operations (committed and uncommitted) up to crash point; checks PageLSN to prevent redundant writes.
decisionActive Losers Identified?Identifies all transactions that never wrote a COMMIT record before crash.
dataPhase 3: Undo PassScans backward, reversing loser updates and appending Compensation Log Records (CLRs).
processDatabase Open for TrafficCore achieves total consistency; buffer pool opened for new incoming client queries.
Flowchart showing the Analysis, Redo, and Undo passes of the ARIES recovery algorithm

5. Concurrency Control: The Race Against Anarchy

Atomicity and Durability guarantee that individual transactions execute reliably across crashes. But what happens when hundreds of transactions run concurrently on the same tables?

Without concurrency control, simultaneous interleaving of SQL statements produces disastrous concurrency anomalies:

                       CLASSIC CONCURRENCY ANOMALIES
                       
   Anomaly Name          Mechanism                                    Real-World Impact
   ---------------------------------------------------------------------------------------------
   Dirty Read (G1)       T2 reads uncommitted data written by T1;     T2 sees money that never
                         T1 then aborts and rolls back.               existed once T1 aborts.
                         
   Non-Repeatable Read   T1 reads row X; T2 modifies or deletes X     T1 sees row change beneath
   (Fuzzy Read)          and commits; T1 re-reads X and sees new data. its feet within one query.
   
   Phantom Read (A3)     T1 queries a range of rows (e.g. balance>0); Range calculation yields
                         T2 inserts new row matching range and commits; inconsistent counts within
                         T1 re-runs range query and sees new row.     same transaction.
                         
   Lost Update           T1 and T2 read row X simultaneously; both    T2 overwrites T1's write
                         modify X and write it back.                  without knowing T1 updated it.

To standardize isolation behavior, the ANSI SQL-92 standard defined four formal Isolation Levels, ordered from weakest to strongest:

Isolation LevelDirty Reads Allowed?Non-Repeatable Reads Allowed?Phantom Reads Allowed?Performance Overhead
Read UncommittedYesYesYesMinimal
Read CommittedNoYesYesLow (Default in PostgreSQL, Oracle)
Repeatable ReadNoNoYesModerate (Default in MySQL InnoDB)
SerializableNoNoNoHighest (True ACID Isolation)

How do database engines physically enforce these isolation levels? Production systems employ two primary strategies: Locking (2PL) and Versioning (MVCC).


6. How Isolation Is Enforced: Strict 2PL vs. MVCC

Strategy 1: Strict Two-Phase Locking (SS2PL)

The traditional method of concurrency control is Two-Phase Locking (2PL), governed by the database Lock Manager.

The engine distinguishes between two fundamental lock modes:

  • Shared Lock (S): Acquired before reading a row. Multiple transactions can hold shared locks on the same row concurrently (S is compatible with S).
  • Exclusive Lock (X): Acquired before modifying (updating, deleting, inserting) a row. Only one transaction can hold an exclusive lock, and it blocks all other readers and writers (X conflicts with both S and X).
                         LOCK COMPATIBILITY MATRIX
                         
                              Requested Lock Mode
                                Shared (S)    Exclusive (X)
                -------------------------------------------
   Current      Shared (S)       GRANT           BLOCK
   Lock Mode    Exclusive (X)    BLOCK           BLOCK

Under Strict Two-Phase Locking (SS2PL):

  1. Growing Phase: A transaction acquires locks as needed when reading or writing rows.
  2. Shrinking Phase: A transaction never releases any exclusive locks until it completely terminates (COMMIT or ROLLBACK)!
                       STRICT TWO-PHASE LOCKING TIMELINE
                       
   Lock Count
        ^
        |             [ Transaction Executing ]
        |           Acquires S-locks and X-locks
        |                 /--------------------\
        |                /                      \
        |               /                        \  ALL LOCKS HELD UNTIL COMMIT!
        |              /                          \
        0 ------------+----------------------------+-----------------------------> Time
                    Begin                        Commit (All locks released at once)

Holding all exclusive locks until commit guarantees Conflict Serializability and completely eliminates cascading rollbacks.

However, locking introduces two major drawbacks:

  1. Readers block Writers, and Writers block Readers: If a slow analytics query reads a table, no updates can occur on those rows until the report finishes.
  2. Deadlocks: Transaction T1 holds row A and waits for row B; Transaction T2 holds row B and waits for row A. The database Lock Manager must continuously construct Wait-For Graphs and run cycle-detection algorithms (e.g., Tarjan's algorithm) or timeouts, killing one transaction to break the circular dependency.

Strategy 2: Multi-Version Concurrency Control (MVCC)

To solve the contention problem where readers block writers, modern databases (including PostgreSQL, MySQL InnoDB, and Oracle) rely on Multi-Version Concurrency Control (MVCC).

The core philosophy of MVCC is simple:

Readers never block writers, and writers never block readers.

Under MVCC, when an update statement modifies a row, the database does not overwrite the existing row in place. Instead, it creates a new version of the tuple and appends it to the table page!

Each tuple header contains metadata tracking its temporal visibility:

  • xmin (or creation transaction ID): The transaction ID that inserted this version.
  • xmax (or deletion transaction ID): The transaction ID that deleted or replaced this version (initially 0 or NULL).
                      HOW MVCC STORES MULTIPLE TUPLE VERSIONS
                      
   Initial State: Alice has $1,000 (Created by Tx 100)
   +----------------------------------------------------------------------+
   | Tuple Version 1: [ xmin: 100 | xmax: 0   ] Alice -> Balance = $1,000 |
   +----------------------------------------------------------------------+
   
   Transaction 105 Updates Alice to $900:
   +----------------------------------------------------------------------+
   | Tuple Version 1: [ xmin: 100 | xmax: 105 ] Alice -> Balance = $1,000 |  (Old version)
   | Tuple Version 2: [ xmin: 105 | xmax: 0   ] Alice -> Balance = $900   |  (New version)
   +----------------------------------------------------------------------+

Snapshot Isolation

When a reader transaction starts at timestamp Tsnapshot, the database takes an in-memory snapshot of active transaction IDs.

When the reader queries Alice's balance, it evaluates each version against its snapshot:

  • Is Version 2's xmin (105) greater than my start timestamp or currently uncommitted? Ignore Version 2!
  • Is Version 1's xmin committed, and its xmax either uncommitted or committed after my snapshot? Read Version 1!

The reader reads the old $1,000 balance without acquiring any locks, while Transaction 105 is actively writing the new $900 balance!

The Cost of MVCC: Vacuuming and Garbage Collection

Because old tuple versions are left on disk, high-write workloads cause table bloat. Over time, dead versions must be reclaimed.

  • In PostgreSQL, an asynchronous daemon called VACUUM scans table pages, identifies dead tuples whose xmax is older than all currently running transactions, marks their space as reusable, and freezes transaction IDs to prevent wrap-around.
  • In MySQL InnoDB, old versions are stored in separate Undo Logs forming an undo chain, and a purge thread frees them once snapshots no longer require them.

7. The Layered Storage Architecture of a Modern Database

When we combine the buffer manager, WAL logger, lock manager, and execution engine, we arrive at the complete multi-tiered architecture of a modern transactional RDBMS:

Multi-Tiered Relational Database Storage Engine Architecture
SQL Parsing & Query OptimizerValidates SQL grammar, verifies table schema constraints, and generates relational algebra execution plans.
Transaction & Lock Manager (Concurrency)Tracks active transaction IDs, grants shared and exclusive locks (SS2PL), maintains MVCC visibility snapshots, and detects deadlocks.
Page Buffer Pool Manager (RAM Caching)Maintains in-memory page frame hash tables, executes LRU/Clock eviction under a Steal/No-Force policy, and coordinates PageLSN dirty tracking.
Write-Ahead Logging Subsystem (WAL)Collects redo/undo log records into append-only sequential ring buffers, synchronizing to disk via synchronous fsync calls before commits.
Physical Storage & Tablespace LayerOrganizes data pages, B+Tree index structures, and log files across file systems, NVMe storage controllers, and disk partitions.
Layered architectural diagram showing SQL parser, lock manager, buffer pool, WAL logger, and physical disk storage

8. Distributed Transactions: Two-Phase Commit (2PC)

In modern cloud architectures, a single logical transaction often spans multiple distinct physical database nodes (for example, debiting an account on Node A in New York and crediting an account on Node B in London).

How can a system guarantee Atomicity when network cables can be severed between nodes?

Distributed databases implement the Two-Phase Commit (2PC) protocol, coordinated by a designated Transaction Coordinator:

                         THE TWO-PHASE COMMIT (2PC) PROTOCOL
                         
          Coordinator                              Cohort Node A            Cohort Node B
          -----------                              -------------            -------------
               |                                         |                        |
               | ===== 1. PREPARE ("Can you commit?") ==>|                        |
               | ================================================================>|
               |                                         |                        |
               |                                   Flushes WAL              Flushes WAL
               |                                   to disk (VOTE)           to disk (VOTE)
               |                                         |                        |
               | <==== "VOTE_COMMIT" (Yes, ready!) ===== |                        |
               | <================================================ "VOTE_COMMIT" |
               |                                         |                        |
        All Votes Received:                              |                        |
        Coordinator flushes                              |                        |
        GLOBAL_COMMIT to WAL                             |                        |
               |                                         |                        |
               | ===== 2. COMMIT ("Commit now!") =======>|                        |
               | ================================================================>|
               |                                         |                        |
               |                                   Applies commit           Applies commit
               |                                   Releases locks           Releases locks
               |                                         |                        |
               | <==== "ACK" =========================== |                        |
               | <======================================================= "ACK" = |
               v                                         v                        v

Phase 1: The Prepare Phase

  1. The Coordinator sends a PREPARE message across the network to all participant nodes (Cohorts).
  2. Each cohort checks whether it can safely commit its portion of the transaction (evaluates constraints, acquires all necessary locks).
  3. If ready, the cohort writes its modifications and a PREPARED log record to its local WAL on disk, and replies with VOTE_COMMIT.
  4. Once a cohort votes VOTE_COMMIT, it has entered a binding contract: it guarantees it can commit even if power fails!

Phase 2: The Commit Phase

  1. If every cohort votes VOTE_COMMIT, the coordinator writes a GLOBAL_COMMIT record to its own WAL, and broadcasts a COMMIT message to all cohorts.
  2. Each cohort applies the commit locally, releases its locks, and sends an ACK.
  3. If even one cohort votes VOTE_ABORT (or times out), the coordinator writes a GLOBAL_ABORT record and commands all cohorts to execute an ABORT rollback.

The Limitation of 2PC: The Blocking Coordinator

The critical vulnerability of classic Two-Phase Commit is that it is a blocking protocol. If the Coordinator crashes after cohorts have voted VOTE_COMMIT but before broadcasting the decision, the cohorts are trapped in limbo: they cannot abort (the coordinator might have decided to commit), and they cannot commit (the coordinator might have decided to abort). They must hold all row locks indefinitely until the coordinator recovers.

Modern distributed systems like Google Spanner, CockroachDB, and YugabyteDB overcome this by replacing single coordinators with distributed consensus groups (using Paxos or Raft), ensuring that leader nodes can failover instantly without stalling active transactions.


9. Summary: The Engineering Foundations of Trust

The reason you can trust a database with your bank account, medical records, and airline reservations is not that computers never crash. It is that database architects assume crashes are an inevitable physical certainty.

By designing the system from the premise of physical failure, relational database engines achieve ironclad reliability:

  1. Write-Ahead Logging (WAL) turns slow, random page modifications into high-speed sequential log records, ensuring every change is permanently recorded before dirty data pages can be stolen to disk or transactions committed.
  2. The Steal / No-Force Buffer Policy enables engines to cache massive datasets in RAM without requiring synchronous disk page writes on every commit.
  3. The ARIES Algorithm guarantees that no matter when the server loses power, an automated three-pass scan (Analysis, Redo repeating history, and Undo rolling back losers) will reconstruct the exact correct state of truth without human intervention.
  4. Strict Two-Phase Locking and MVCC eliminate data race conditions and concurrency anomalies, allowing thousands of simultaneous clients to query and modify tables without corrupting each other's data.

The database engine is one of the most sophisticated achievements in computer science—an architectural fortress that transforms unreliable, volatile hardware into an unshakeable foundation of data integrity.


Related Knowledge & Prerequisites

To deepen your understanding of how software coordinates with underlying hardware and file systems:

  • How Operating Systems Run Software: Explore virtual memory, page tables, system calls, and how the OS kernel manages physical resources.
  • How Solid-State Drives and Flash Memory Work: Learn how NAND flash memory cells, wear leveling, and flash translation layers (FTL) store non-volatile data.
  • How Microprocessors Execute Pipelined Instructions: Discover how CPU pipeline stages, branch predictors, and out-of-order execution maintain high instruction throughput.
  • How Computer Memory (RAM) Works: Understand the physics of capacitor-based dynamic RAM (DRAM), refresh cycles, and memory controllers.
  • How Compilers Translate Code into Machine Instructions: Discover how abstract syntax trees, semantic analysis, and intermediate representations transform high-level code into executable binaries.
Core Concepts Introduced12 Concepts
ACID Transaction SemanticsDatabase Page Buffer Pool ManagementSteal vs No-Steal & Force vs No-Force PoliciesWrite-Ahead Logging (WAL) ProtocolLog Sequence Numbers (LSN) & PageLSNARIES Crash Recovery (Analysis, Redo, Undo)Compensation Log Records (CLRs)Strict Two-Phase Locking (SS2PL)Lock Manager Wait-For Graphs & Deadlock DetectionMulti-Version Concurrency Control (MVCC)Snapshot Isolation & Vacuum Garbage CollectionTwo-Phase Commit (2PC) Distributed Consensus
Knowledge Graph Connections

Where to Go From Here

Follow downstream mechanisms, contrasting architectures, or applications of these principles.

Answers next question25 min read

How Hardware Security Modules (HSM) Protect Cryptographic Keys

Why can't private encryption keys be safely kept in computer memory, and how do Hardware Security Modules physically self-destruct secrets when attacked?

Explore How Hardware Security Modules (HSM) Protect Cryptographic Keys
Answers next question25 min read

How Quantum Computers Execute Algorithms

How does a quantum computer actually calculate, and why can't a classical supercomputer simply simulate it?

Explore How Quantum Computers Execute Algorithms
Answers next question25 min read

How Virtual Memory and the MMU Isolate Processes and Expand RAM

How does a computer allow multiple hostile programs to run simultaneously in identical memory addresses without overwriting each other or crashing the operating system?

Explore How Virtual Memory and the MMU Isolate Processes and Expand RAM
Builds upon24 min read

How a CPU Executes an Instruction

Deep-dive following foundational explainer How a CPU Executes an Instruction

Explore How a CPU Executes an Instruction
Research Grounding & Primary Sources

Verified Specifications & Architectural References

4 Authoritative References

This explainer is grounded in primary-source engineering specifications, regulatory circulars, and standard documentation.

Primary SourceACM Transactions on Database Systems (C. Mohan, Don Haderle, Bruce Lindsay, Hamid Pirahesh, Peter Schwarz)• 1992

ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging

The seminal foundational paper introducing the ARIES recovery algorithm, physiological logging, repeating history during redo, and compensation log records.

Primary SourceMorgan Kaufmann (Jim Gray & Andreas Reuter)• 1993

Transaction Processing: Concepts and Techniques

The definitive reference textbook on transaction management, buffer pool architectures, serializability theory, two-phase locking, and distributed consensus.

Primary SourceFoundations and Trends in Databases (Joseph M. Hellerstein, Michael Stonebraker, James R. Hamilton)• 2007

Architecture of a Database System

Comprehensive survey of commercial relational database internals, covering process models, storage engines, buffer managers, and concurrency controllers.

Primary SourcePearson (Hector Garcia-Molina, Jeffrey D. Ullman, Jennifer Widom)• 2008

Database Systems: The Complete Book (2nd Edition)

Standard academic treatise on relational algebra, index structures, query execution, transaction processing, and concurrency control.

Previous ExplainerHow Microprocessors Execute Pipelined InstructionsNext Explainer How Virtual Memory and the MMU Isolate Processes and Expand RAM
More from Computing & Digital Architecture•Topic Hub: Computing & Digital Architecture
Ground Truth Engineering Publication