Glossary

The Raft RPC that carries new log entries from the leader to followers.

A transaction either commits entirely or has no effect at all — there is no partial result.

Scheduling multiple workloads onto the same nodes to fill cluster capacity efficiently, rather than dedicating one node per service.

The fraction of the commitment period a reserved unit must be running to recover its premium over on-demand. ~60% for one-year, ~40–50% for three-year reservations.

A single server in a Kafka cluster that hosts partitions and serves producer and consumer traffic.

The in-memory page cache the database keeps for hot data. The single most important sizing variable — when the working set fits, tail latency stays in microseconds.

A synchronized surge of backend traffic caused by simultaneous cache misses — typically after a cache restart or coordinated expiration. Mitigated by request coalescing, jittered TTLs, and probabilistic early refresh.

Services publish and subscribe to events; each participant knows only its own next step, no central coordinator.

A disaster-recovery posture where the standby region has no live infrastructure; recovery requires provisioning from scratch. Cheapest, slowest to recover.

The latency a serverless platform adds when it spins up a new function instance — typically hundreds of milliseconds, sometimes seconds.

Bertrand Meyer's 1988 principle: a method either changes state or returns a result, never both.

The fraction of total compute spend covered by reservations or spending commitments; one of FinOps's core rate-optimization levers.

The business operation that undoes the effect of an earlier saga step (e.g., refund a charge).

Middleware (e.g., PgBouncer, ProxySQL) that multiplexes many client connections onto a small pool of backend database connections — essential past a few hundred client connections.

A group of nodes agreeing on a single value despite failures and dropped messages.

The gap between the most recently written message and the most recently read message for a given consumer. Lag is paid for in retention budget.

The maximum acceptable consumer lag for the workload — driven by the SLO. A stream-processing job may tolerate seconds; an analytics consumer may tolerate hours. Sets the minimum retention budget.

The orchestrator's management layer that handles scheduling, health checks, and cluster state. On managed Kubernetes this is a fixed monthly fee per cluster.

A reservation that allows the instance family or shape to be swapped within constraints during the term, at the cost of a smaller discount than a standard reservation.

Resource tags applied at creation that let cost-management tools attribute hidden cross-service costs to specific applications, teams, or environments.

Command Query Responsibility Segregation — separating the model that handles writes from the model that handles reads.

Network traffic between availability zones within the same region, billed per gigabyte, often in each direction.

Create / Read / Update / Delete — the basic operations of most simple data-driven applications.

Coined by Dave McCrory (2010): the bigger a dataset, the more expensive it is to move and the harder it is to move away from — explaining why data-movement costs dominate cloud bills.

A queue for messages or operations that cannot be processed and require human intervention.

A read that returns the intermediate state of an in-flight transaction, which may be rolled back.

The fraction of time a system is actively doing work (as opposed to idle).

Data leaving a cloud provider's network for the public internet, billed per gigabyte; the most expensive form of data movement on every major cloud.

Storing state as a log of immutable events instead of a current snapshot; current state is computed from the log.

The read model converges to match the write model over time, never instantly.

A cross-discipline practice (FinOps Foundation, 2019) for shared accountability of cloud costs across engineering, finance, and product teams.

A versioned suite of representative test cases scored against every prompt, model, or retrieval change — the regression suite of an LLM system.

A periodic message from the leader to followers signalling that the leader is still alive.

Metric dimensions with many distinct values (e.g., per-user IDs); priced per unique time series in most observability platforms and a common driver of unexpected observability bills.

The fraction of cache lookups served from cache (hit) vs. forwarded to the backend (miss). Every percentage point of hit rate translates directly to backend load.

A single cache key that absorbs disproportionate traffic, saturating the node that holds it before memory fills. Sharding by key does not help if the hot key is one key.

A partition receiving disproportionately more traffic than its peers, caused by a partition-key choice that concentrates load on one broker.

A control where model output routes to a person before it takes effect — triggered by low confidence, high stakes, or explicit policy.

An operation that has the same effect whether called once or many times.

A value or record that cannot be changed after it is created; updates create new records instead.

Input-Output Operations Per Second — the count of distinct disk operations the storage tier can serve. Cloud volumes have a hard ceiling above which every operation queues.

The transactional property that in-flight transactions don't see each other's intermediate state.

John Little (1961): the average number of items in a stable system equals the arrival rate times the average time each item spends in the system (L = λW). Applied to caches: concurrent backend load = miss rate × miss-handling latency.

A single choke point between every application and every model API — centralizing authentication, rate limits, cost attribution, guardrails, logging, and model routing so the model behind it can change without re-platforming.

The operational discipline for LLM systems: versioned prompts, eval-gated releases, cost and latency telemetry, and drift detection — regression discipline applied to non-deterministic software.

Discarding old entries that have been captured in a snapshot, to keep log size bounded.

A cloud-provider-managed network address translation gateway for outbound traffic from private subnets — AWS NAT Gateway, GCP Cloud NAT, Azure NAT Gateway. Charges hourly + per gigabyte processed.

Adding or removing nodes from a Raft cluster safely, without violating consensus guarantees.

Multi-Version Concurrency Control — instead of overwriting a row, the database writes a new version and marks the old one for cleanup later, letting readers and writers proceed without blocking.

The sustained network throughput a single broker can absorb across producer writes, consumer reads, and inter-broker replication combined. The cluster's slowest broker becomes the bottleneck.

A change that cannot be undone (e.g., sending an email, shipping a package).

A single coordinator service drives the saga, calling each step and walking compensations on failure.

The control system that decides where each containerized workload runs and how it scales — Kubernetes is the dominant example.

A power-law distribution where a small fraction of items (~20%) account for most of the traffic (~80%). The classic access pattern that makes caching worth its cost.

A single ordered, append-only log within a topic — the smallest unit one consumer can read in isolation, and the unit of parallelism.

The field whose hash determines which partition a message lands on; choosing a key with low cardinality or skewed distribution causes hot partitions.

The original distributed consensus algorithm (Lamport, 1989). Correct but notoriously hard to implement — every production system using Paxos has its own subtle variant. Named after a Greek island where Lamport set a fictional parliamentary procedure as a teaching device.

A DR posture sitting between warm and cold: the critical data and minimal services run continuously in the standby region, with the rest provisioned at failover time.

Governance rules expressed as executable checks enforced by the platform, not as documents enforced by meetings.

A private network connection that routes cloud-service traffic through the cloud's internal fabric, bypassing managed NAT — AWS VPC Endpoint, GCP Private Service Connect, Azure Private Link.

A cache strategy that refreshes a value before its TTL with a probability that rises near expiration, smoothing the load curve and eliminating the expiry cliff.

A function that walks every event in order and accumulates a derived state used for reads.

An attack where untrusted content instructs a model to ignore its rules. No perfect defense exists today; layered filtering plus least-privilege design limits the blast radius.

Pre-warmed serverless instances kept ready to serve traffic; billed whether or not they fire. Mitigates cold starts at the cost of giving up the "pay only when running" property.

The minimum number of nodes (usually a majority) that must agree before a decision is committed.

A distributed consensus algorithm (Ongaro & Ousterhout, Stanford, 2014), designed to be more understandable than Paxos. Not an acronym — the name was chosen as a humble visual metaphor (a simple floating craft that survives storms), deliberately contrasting Paxos's Greek-island academic vibe. The popular backronym "Reliable, Replicated, Redundant, And Fault-Tolerant" came after the name was chosen.

Retrieval-augmented generation: grounding a model's answer in documents fetched at query time, so responses reflect your current data instead of the model's training snapshot.

The guarantee that a user reads back the state they just wrote, even if the broader system is eventually consistent.

The number of copies of each partition kept across brokers; production default is three for durability against single-broker failures.

A cache pattern where only one in-flight backend call per missing key is allowed; subsequent requests wait on the same response. Prevents stampedes.

A one- or three-year contractual commitment to specific instance shapes in exchange for a 30–60% discount off on-demand pricing. AWS Reserved Instances, GCP Committed Use Discounts, Azure Reserved VM Instances.

The duration the broker keeps each message before deleting it; configurable per topic as either a time bound or a size bound.

A sequence of local transactions, each paired with a compensating action that semantically reverses it on failure.

A flag on a record indicating it is participating in an in-flight saga, blocking other transactions from reading or modifying it.

A committed checkpoint of derived state used to avoid replaying the full event log on every read.

A commitment to a dollar-per-hour spend across an account, applied automatically as a discount across qualifying compute. AWS Savings Plans is the canonical example.

A partition in which two halves of a cluster both elect leaders and accept divergent writes.

Unused cloud capacity sold at 60–90% off on-demand, with the provider's right to reclaim the instance on short notice (typically two minutes). Suitable only for fault-tolerant or restartable workloads.

The latency at high percentiles (p99, p999) — the rare slow requests that determine user experience at scale, distinct from the median.

Total Cost of Ownership — the fully-loaded annual cost of running a system, including compute, platform fees, operations, monitoring, and failure-mode costs.

A logical period that acts like Raft's clock; each term has at most one leader.

The duration a cached item remains valid before the cache considers it stale. Trades hit rate (longer TTL) against staleness (shorter TTL).

A named stream of messages that producers write to and consumers read from; divided internally into partitions.

A distributed transaction protocol where a coordinator asks all participants to prepare, then commit (or abort) atomically.

Code that translates older versions of stored events into the current schema shape during replay.

The process that reclaims storage from MVCC tombstones or merges write-ahead segments; required maintenance work whose cost rises with write volume.

A database indexed for nearest-neighbor search over embeddings, so semantically similar content can be retrieved even when it shares no keywords with the query.

A disaster-recovery posture where the standby region runs a scaled-down but live copy of production, ready to take over quickly.

The portion of a dataset that is read frequently enough to matter for performance. When it fits in the buffer pool, p99 is microseconds; when it spills to disk, p99 jumps two orders of magnitude.

The ratio of physical disk writes to logical application writes; driven by indexes, write-ahead logs, replication, and storage-engine internals like LSM-tree compaction.

A sequential log of every change written before the change is applied to the main data files, providing durability and crash recovery.