Glossary of Precise Terms

The fraction of time a system is operational and able to serve requests correctly. Expressed as a percentage (99.9%, 99.99%) or in terms of allowed downtime per period.

Atomicity, Consistency, Isolation, Durability — the four properties that database transactions are traditionally said to guarantee. Each letter describes a different guarantee: Atomicity (all-or-nothing), Consistency (valid state transitions only), Isolation (concurrent transactions behave as if sequential), Durability (committed data survives crashes).

A background process that detects and repairs divergence between replicas by comparing data checksums (typically via Merkle trees) and copying missing or differing values. Used in systems like Dynamo/Cassandra to ensure eventual consistency even after network partitions.

A mechanism by which a downstream system signals to an upstream system that it is overloaded, causing the upstream to slow down or stop sending data. The correct response to overload — as opposed to dropping messages silently or crashing.

A failure mode where a node behaves arbitrarily — it may send different values to different peers, lie about its state, or act in ways designed to subvert the protocol. Named after the Byzantine Generals Problem. More severe than crash faults or omission faults.

A theorem (proved by Gilbert and Lynch, 2002) stating that a distributed system cannot simultaneously guarantee all three of: Consistency (linearizability), Availability (every request gets a response), and Partition tolerance (the system continues operating despite network partitions).

A consistency model that preserves causally-related operations in order — if operation A happened before operation B (in the happens-before sense), then every process that sees B must also have seen A. Concurrent operations (with no causal relationship) may be seen in any order.

A design pattern that prevents a service from repeatedly calling a dependency that is failing. Like an electrical circuit breaker, it has three states: Closed (requests flow normally), Open (requests fail immediately without attempting the call), and Half-Open (a probe request is tried to see if the dependency has recovered).

The problem of getting a group of nodes to agree on a single value, even when some nodes may fail. A consensus algorithm must satisfy: Agreement (all non-faulty nodes decide the same value), Validity (the decided value was proposed by some node), and Termination (all non-faulty nodes eventually decide).

An architectural pattern that separates the data model used for writes (commands) from the data model used for reads (queries). Allows each to be independently optimized — the write model for consistency and integrity, the read model for query performance.

The guarantee that once a transaction is committed, it will survive system failures. In practice, this means data is written to non-volatile storage (disk) before the commit is acknowledged to the caller.

The acceptable amount of unreliability remaining in a given period, derived from an SLO. If the SLO is 99.9% availability, the error budget is 0.1% — approximately 43 minutes of downtime per month. When the budget is consumed, reliability work takes priority over feature work.

A consistency model where replicas are not guaranteed to be consistent at any given moment, but will eventually converge to the same value if no new updates are made. The weakest consistency model that still provides a useful guarantee.

An approach to persistence where the system stores a sequence of events (things that happened) rather than the current state. Current state is derived by replaying events. The event log is the source of truth; state is a derived projection.

The process of propagating one write to many destinations. In social systems, a single post might be fanned out to the feeds of all followers. Fan-out amplifies write QPS: one input write can become thousands of output writes.

A partial ordering relationship between events in a distributed system, introduced by Lamport. Event A happens-before event B (written A → B) if: A and B are on the same process and A occurred before B, or A is the sending of a message and B is the receiving of that message, or there exists C such that A → C and C → B (transitivity).

A partition, shard, or key that receives disproportionately more traffic than others, becoming a bottleneck that limits overall system throughput even if other partitions have spare capacity.

A property of an operation where applying it multiple times produces the same result as applying it once. An idempotent operation can be safely retried without risk of unintended side effects.

The degree to which concurrent transactions are hidden from each other. The SQL standard defines four isolation levels: Read Uncommitted, Read Committed, Repeatable Read, and Serializable — from weakest to strongest guarantee.

The strongest single-object consistency model. A system is linearizable if every operation appears to take effect instantaneously at some point between its invocation and completion, and the result is consistent with a single global ordering of all operations.

A storage engine design that accumulates writes in memory (memtable), flushes them to immutable sorted files (SSTables) on disk, and periodically merges these files (compaction). Optimized for high write throughput at the cost of read amplification and periodic compaction overhead.

A session guarantee that ensures if a client has read a value at time T, any subsequent reads will not return a value from before T. Prevents the disorienting experience of "time going backwards" when a client's requests hit different replicas at different replication states.

A TLS configuration where both the client and server present certificates and authenticate each other, as opposed to standard TLS where only the server is authenticated. Used for service-to-service authentication in zero-trust network architectures.

The ability to understand the internal state of a system from its external outputs. A system is highly observable if you can diagnose novel failures — failures you have never seen before — from its telemetry alone, without needing to add new instrumentation first.

An extension of CAP proposed by Daniel Abadi. States that in the case of a network Partition, choose between Availability and Consistency (as in CAP); Else (when no partition), choose between Latency and Consistency. Recognizes that latency, not just partition tolerance, is a fundamental trade-off dimension.

A failure scenario where two or more subsets of nodes in a distributed system cannot communicate with each other, even though nodes within each subset can communicate internally. The system is split into isolated islands.

A structured analysis of an incident conducted after it is resolved, aimed at understanding what happened, why it happened, and what systemic changes will prevent recurrence or reduce impact. In a blameless culture, the focus is on systemic causes rather than individual actions.

In a system with N replicas, a quorum is a subset of replicas that must agree for an operation to be considered complete. For a write quorum W and read quorum R, the system can handle inconsistency-free reads if W + R > N (the write set and read set must overlap).

A data structure or service that holds messages until they are consumed. Used to decouple producers from consumers, absorb traffic spikes, and enable asynchronous processing. Key properties: ordering guarantees (FIFO vs. best-effort), delivery guarantees (at-most-once, at-least-once, exactly-once), and retention policy.

A session guarantee that ensures a client will always see the results of its own previous writes. Prevents the confusing experience where a user submits data and then immediately queries for it, only to find it absent.

The delay between when a write is committed on the primary/leader and when it is applied on a replica. In asynchronous replication, lag can range from milliseconds to minutes. Under load or during network issues, it can grow significantly.

A pattern for managing long-running transactions across multiple services without distributed locking. A saga is a sequence of local transactions where each step publishes an event or message that triggers the next step. If a step fails, compensating transactions undo the previous steps.

The strongest multi-object transaction isolation level. A schedule of concurrent transactions is serializable if the outcome is equivalent to some serial (one-at-a-time) execution of those transactions. Does not require real-time ordering (see: Strict Serializability).

SLI (Service Level Indicator): A specific metric measuring service behavior (e.g., request success rate, p99 latency).
SLO (Service Level Objective): A target for an SLI (e.g., "99.9% of requests must complete in under 200ms").
SLA (Service Level Agreement): A contractual commitment, typically to customers, with financial penalties for violation. An SLA is usually a weaker commitment than internal SLOs — teams set SLOs tighter than SLAs to have a buffer.

A failure scenario where a network partition causes two or more nodes to simultaneously believe they are the primary/leader, leading to independent and potentially conflicting writes on each partition. One of the most dangerous failure modes in leader-based replication systems.

The latency experienced by the slowest requests in a distribution — typically measured at the 99th or 99.9th percentile. p99 latency of 500ms means 1% of requests take 500ms or longer. Tail latency is more important than average latency for user-facing systems because it affects the slowest (often most important) requests.

Work that is manual, repetitive, automatable, tactical, and produces no lasting value beyond getting you to the next iteration. Defined precisely in Google's SRE framework to distinguish it from overhead (inherent management cost) and engineering work (building value).

A distributed commit protocol that coordinates atomic commitment across multiple nodes. Phase 1 (Prepare): the coordinator asks all participants if they can commit. Phase 2 (Commit/Abort): if all say yes, the coordinator sends commit; if any say no, it sends abort. Guarantees atomicity but blocks if the coordinator fails mid-protocol.

A mechanism for tracking causality in distributed systems. Each node maintains a vector of counters, one per node. On each event, the local counter increments. On message send, the current vector is included. On message receive, the receiver takes the element-wise maximum of its vector and the received vector, then increments its own counter.

The ratio of data written to storage vs. data logically written by the application. An LSM tree with compaction may write each byte of user data 10–30× due to repeated merging and rewriting of SSTables. High write amplification reduces SSD lifespan and limits write throughput.

A technique for durability where every change is first written to an append-only log on disk before being applied to the main data structure. On crash recovery, the log is replayed to reconstruct the most recent state. Used by virtually every durable database (PostgreSQL, MySQL, SQLite).