For years, Discord's message store was — by most accounts — one of the largest production Cassandra deployments anywhere. It also became the team's single biggest operational problem. Hot partitions on popular channels caused tail-latency spikes that bled into the user experience: a 100,000-member server posting a meme could blow out p99 read latency for everyone connected. Garbage-collection pauses on the JVM produced second-long stalls that looked, to a user, like the app had frozen. A standing rotation of engineers spent more time tending to the cluster than building features. Anyone who's run Cassandra at scale will recognise the shape of the pain.

The team made the call to move off Cassandra entirely. The replacement was ScyllaDB — a C++ rewrite of the Cassandra query model with a shard-per-core threading architecture, no JVM, and dramatically lower tail latencies under load. But the database swap was the easy part. The hard part was the migration itself: trillions of messages, no acceptable downtime, no acceptable data loss. The pattern Discord built for that migration is now widely copied across the industry. They wrote a custom data-services layer in Rust that fronted both Cassandra and ScyllaDB simultaneously. Every new message dual-wrote to both stores. In parallel, a separate streaming back-fill copied historical data from Cassandra into Scylla, row-by-row, with verification. When the back-fill caught up, the team flipped the read path to Scylla, monitored, and finally cut Cassandra out.

The single most interesting trick in that intermediary, though, has nothing to do with the database swap. Discord engineers call it request coalescing: when many readers ask for the same row at almost the same moment (think a popular channel where everyone is loading the same set of recent messages), the Rust intermediary collapses them into a single underlying database query and fans the result back out to all waiting clients. On hot partitions this dropped read load by an order of magnitude before the database even saw it. Half the win from the migration, by Discord's own framing, came from this single trick at the service layer — independent of whether the storage engine underneath was Cassandra or Scylla. The Rust language choice was deliberate too: predictable latency under load is exactly what a high-fanout proxy needs, and the GC-free runtime mattered.

After cutover, Discord reported a smaller cluster, much lower tail latency on the read path, and a dramatically quieter pager. The Cassandra fleet had reportedly grown to roughly 170–180 nodes; the ScyllaDB replacement does the same work at around half that node count. The data-model itself was redesigned during the migration — channel-id-based partitioning, with the bucket strategy that finally retired some legacy schema quirks. Discord published a detailed blog post in 2023 titled "How Discord Stores Trillions of Messages" walking through the architecture; their earlier post from 2017, "How Discord Stores Billions of Messages", is worth reading first to understand what they were running away from.

Lesson — The database is rarely the bottleneck on its own; the access pattern is. Request coalescing at the service layer pays for itself before you even change databases. If you do have to migrate, do it behind a thin intermediary you control, with dual-writes plus per-row verification. The boring choreography is the migration that actually ships.

When Facebook acquired WhatsApp in 2014 for nineteen billion dollars, the engineering team was reportedly around thirty-five people. The product was carrying several hundred million users and growing at a rate that would have buried most companies. By the time WhatsApp crossed a billion users, the engineering headcount had grown — but only to something like fifty. To put that in perspective: a Series-B SaaS startup with the same headcount and a tiny fraction of the user count would call itself "lean". WhatsApp ran the world's largest messaging network with roughly the same number of engineers as a typical product team at a mid-size enterprise. The ratio is one of the more astonishing data points in modern infrastructure.

The choices that made it possible were defiantly unfashionable. Erlang/OTP for the application layer — a runtime originally designed at Ericsson for telecom switches, with a concurrency model built around millions of cheap processes, a supervision hierarchy, and a "let it crash" philosophy that turned what would have been incidents elsewhere into routine supervisor restarts here. FreeBSD on bare-metal servers, not Linux on the cloud. The network-stack tuning that Rick Reed and team described in his now-famous "That's Billion with a B" talks at the Erlang Factory let a single box handle an order of magnitude more concurrent TCP connections than the same hardware would on a stock kernel. A culture so anti-feature-bloat that team meetings reportedly opened with "what can we delete today?" — Brian Acton's published anti-features post is required reading.

The architecture itself was almost embarrassingly simple. A WhatsApp server ran a heavily-customised XMPP variant (descended from ejabberd, the Erlang XMPP implementation). Each server held an enormous number of persistent TCP connections — the often-cited figure is around two million concurrent connections per box, achieved through careful FreeBSD kernel tuning and Erlang's preemptive scheduler. Routing between users was a function call inside the Erlang VM where both users happened to be on the same server, and a message-passing protocol across machines where they weren't. Persistence used Mnesia (Erlang's distributed in-memory database) for in-flight session state and Postgres for durable records. No microservices. No service mesh. No Kubernetes. The opposite of what the rest of the industry was building during the same years.

The numerical consequence is hard to absorb if you've only worked at companies that scaled the conventional way. At peak the ratio of users per engineer was something on the order of fourteen million to one — a number you can re-derive from the public headcount and user-count figures. After the Facebook acquisition, the engineering team migrated parts of the stack to Linux (Facebook standardises on Linux) over a period of years, but the philosophical core stayed Erlang and the team stayed small. WhatsApp's voice and video calling features, added later, sit on top of the same fundamental architecture — adding capability without adding much headcount.

Lesson — Choose the runtime before you choose the architecture. Erlang's process model, hot-code reload, and supervision trees collapsed what would have been a five-hundred-person ops organisation elsewhere into a small on-call rotation at WhatsApp. Operational simplicity is a competitive advantage — especially when it looks unfashionable from outside.

The received wisdom of the early 2010s was that at "real" scale, you graduated from Postgres or MySQL to a wide-column store — Cassandra, HBase, Riak, DynamoDB. Instagram, at the time the world's most photographically active app, declined to play. Their primary metadata store was sharded PostgreSQL, and over time they moved more workloads onto Postgres rather than away from it. The aesthetic was a quiet but consistent one: prefer the boring database you understand to the exciting one you don't. The Instagram engineering blog of that era reads like a counterculture document compared to most other social-media engineering writing.

The specific tactic that made it scale was described in Mike Krieger's now-famous 2012 post "Sharding & IDs at Instagram". The team needed globally unique 64-bit IDs that were time-sortable (so feeds could be ordered cheaply), encoded the shard the row lived on (so the application could route reads), and were generated without coordination (so ID generation didn't become a bottleneck or a single point of failure). The scheme they landed on packed an epoch-relative timestamp into the high bits, a logical shard ID into the middle bits, and a per-shard auto-increment counter into the low bits — all inside a single 64-bit integer, generated by a PL/pgSQL stored procedure on the database where the new row would live. No central ID server. No coordination overhead. New shards could be added without changing the ID format.

Around this skeleton, a thousand small choices made the architecture work. Thousands of logical shards mapped onto a much smaller number of physical Postgres instances — the indirection meant you could split a physical machine into two by reassigning half of its logical shards to a new box, without touching application code. Cross-shard queries were forbidden as a matter of engineering policy; features that would have required them got redesigned at the product level until they didn't. PgBouncer in front of every Postgres instance kept the connection count manageable — without it, Postgres's per-connection memory cost would have buried the cluster long before the data layer did. The Django routing layer figured out the shard for each query from the ID; the database itself didn't have to coordinate.

The dividend was outsized. Instagram was acquired by Facebook in April 2012 for $1B with around thirteen engineers reportedly on the team and tens of millions of users. The architecture scaled — with extensions and rewrites, of course — into hundreds of millions and then a billion-plus users. Later, after a few painful Cassandra operational incidents in the late 2010s, some workloads explicitly moved back from Cassandra onto Postgres rather than the other direction. The team's running maxim — "is the load too big for Postgres?" is almost never the right question; "is our access pattern too greedy for Postgres?" usually is — has been borrowed by half the infrastructure teams in the Valley.

Lesson — Boring tech is a strategy, not a fallback. The constraint of "no cross-shard queries" reshaped Instagram's product features in ways that would have made the alternative architecture viable too — but starting from a constraint forces the redesigns to happen, instead of letting an exotic database absorb laziness in the access pattern.

Geospatial matching at city scale is easy. Geospatial matching at Uber scale — every ride, every surge zone, every ETA on every driver in every city, recomputed continuously around the clock — is where naive approaches collapse. A lat/long bounding-box query against a B-tree index works for thousands of requests per second. An R-tree helps. Geohash strings (used by Foursquare and others in that era) help further. None of those approaches survive Uber's combination of urban density and global breadth gracefully. So Uber built its own geospatial indexing system — H3, a hexagonal hierarchical index — and then, in 2018, open-sourced it. The project is at github.com/uber/h3 with documentation at h3geo.org.

The reason for hexagons rather than squares is a property that sounds trivial until you've worked with the alternative: every neighbour of a hexagonal cell is exactly the same distance away. With a square grid, the four "side" neighbours are one unit away but the four "diagonal" neighbours are √2 units. That asymmetry quietly poisons "nearest" queries, gradient computations, and any aggregation that involves rings of expanding neighbours. With hexagons, the rings are clean. There are also exactly three types of edges in a hex grid versus two types in a square grid, which sounds esoteric but matters for flow algorithms and contiguity queries.

H3 splits the surface of the earth into 122 base cells — 110 hexagons and 12 pentagons. You can't tile a sphere with regular hexagons alone (this is a topological constraint, not a software limitation); the pentagons are an unavoidable compromise, and Uber's H3 designers deliberately placed them in the middle of oceans where they cause the least trouble. Each base cell can be recursively subdivided into seven children, up to sixteen resolutions deep. At the finest resolution, cells are roughly a square metre in area. Each cell has a 64-bit ID encoding its base cell and the path of subdivisions to reach it — so "what's near this rider" becomes a constant-time set of integer comparisons. K-ring queries (give me all cells within distance K of cell X) are deterministic. Compaction lets you collapse contiguous regions of fine cells into a coarser parent cell efficiently.

Inside Uber, H3 powers dispatch matching, surge pricing zones, supply-demand heatmaps, ETA models, and geofencing. Outside Uber, it's been adopted by DoorDash, Foursquare, Tesla, and a long list of others. Bindings exist for Python, JavaScript, Java, R, Go, C, and more. The H3 indexing model has become something close to a default for geospatial work in modern systems — it shows up in published architectures for ride-share, food delivery, mapping, network planning, and even some HFT routing systems.

Lesson — The right data structure can collapse an entire complexity class. H3 turns "find drivers within radius R of this rider" from a quadratic candidate-pair search into a constant-time set lookup over a fixed number of pre-indexed cells. Don't reinvent geospatial grids — pick one of the published ones.

At any given peak hour in North America, Netflix is responsible for somewhere around a third of all downstream internet traffic — the specific share moves year to year, but the magnitude has been roughly stable since the late 2010s. Sandvine's "Global Internet Phenomena" reports have made this a well-cited stat. Paying a commercial CDN like Akamai or Limelight for that volume would have eaten Netflix's gross margin alive. So Netflix built its own CDN — Open Connect — and then did the genuinely clever thing: rather than charge ISPs for delivering the traffic, they hand the appliances to ISPs for free.

The economics drive the architecture. A pizza-box-sized appliance ("Open Connect Appliance", or OCA) loaded with Netflix's most-watched content sits inside the ISP's own data center, on the ISP's own network. When a customer in that ISP's network streams a show, the bytes never leave the ISP's premises — there's no transit cost to either side. Comcast, AT&T, your local ISP — they all save on transit. Netflix saves on CDN bills. Customer quality-of-experience improves because the content is one hop from the user. The deal aligns every ISP's commercial incentives with Netflix's; the alternative — paying CDNs as a customer, or fighting ISPs over peering — was strategically much worse.

Inside the appliance, the engineering is as interesting as the deal. Open Connect Appliances run a heavily-tuned FreeBSD with kernel-level optimisations for sequential streaming workloads: in-kernel TLS termination (kTLS) so encryption happens without copying bytes back and forth to userspace, zero-copy sendfile from NVMe to NIC, NUMA-aware thread pinning, custom TCP congestion control. The hardware is dense — 100 Gbps NICs, terabytes of NVMe, AMD EPYC or Intel Xeon CPUs depending on the generation. Each box can saturate its 100G link with HTTPS video streams continuously, a number that would be miraculous on a generic Linux box. The Open Connect team's papers and FreeBSD conference talks on the kernel work have become required reading in the network-systems community.

Content is pre-positioned during off-peak hours through what Netflix calls the "fill window" — every night, the popularity-prediction system decides which titles each OCA should hold, and the boxes pull those titles from Netflix's central origins over the public internet during the hours nobody is watching. By the late 2010s, Netflix reported that essentially all of its video bytes were served from Open Connect, with only metadata, recommendations, and tiny payloads passing through the public clouds (AWS). The Open Connect team also runs an academic / research arm that places OCAs at universities and research networks at zero cost, partly as goodwill and partly as a perpetual source of operational telemetry from non-commercial deployments.

Lesson — Sometimes the architecture decision is also the business model. Open Connect isn't just an engineering optimisation; it's a procurement strategy that turns CDN cost into ISP cost-savings, aligning every partner's incentives with Netflix's. The boundary between "infrastructure" and "go-to-market" is more porous than most teams treat it.

Payments are the canonical unforgiving workload. The network blips. The client retries. A request that already succeeded but never returned an acknowledgement now arrives a second time at the server. If you handle this wrong, somebody pays twice for their coffee, twice for their software subscription, twice for the flight they booked at 3am. Stripe's answer to this problem is the single most-copied design pattern in modern API engineering: the Idempotency-Key header. Almost every payment API and many non-payment APIs now use the same pattern; Brandur Leach's 2017 blog post on the topic at brandur.org is required reading for anyone designing APIs that handle money.

The contract is simple to state. Every mutating API call (charge, refund, transfer, payout) accepts a client-generated UUID in the Idempotency-Key header. The server stores the response for that key when it first computes it. If the same key arrives again — because the client retried after a timeout, because a load balancer replayed the request, because the user double-clicked the button — Stripe replays the stored response instead of executing the operation a second time. The customer's card is charged exactly once, regardless of how many times the request makes it across the network.

The implementation behind that one-line contract is anything but simple. Concurrent requests bearing the same key need to be serialised — only one wins the right to actually execute; the others wait or replay. Partial failures must be recoverable — if the operation crashes between the "charge the card" step and the "record the response" step, the second call has to be able to pick up where the first one left off, not start over. The pattern Stripe uses is sometimes called recovery points: the operation is broken into atomic stages, each stage's outcome is recorded in a durable journal table, and a re-execution skips already-completed stages. Idempotency keys have a TTL — typically twenty-four hours — after which the key is forgotten and a fresh request with the same key will execute fresh. The state machine uses Postgres-level locking (SELECT FOR UPDATE) to serialise concurrent attempts cleanly.

Stripe today reportedly processes hundreds of billions of API requests per year. The double-charge rate is, by all available indications, vanishingly small. The contract has been generalised: Stripe applies idempotency to non-payment endpoints too, because the cost of "handle retries safely" is one schema column and the upside is "your client SDK can retry aggressively without you panicking". Every modern payment API — Square, Adyen, Razorpay, the bank APIs that bypass card networks entirely — copies the pattern, often with the same header name and the same TTL window. The pattern has even leaked into REST APIs that have nothing to do with money: GitHub's REST API supports idempotency keys on certain mutations now, and so do most newer cloud-provider APIs.

Lesson — At scale, the network is your worst enemy. Idempotency keys aren't an optional polish — they're the contract that lets you sleep when the on-call pager flips. Build the recovery-point pattern into any API where a duplicate execution would cause real harm; you'll get this back tenfold the first time a network event saves you from a refund queue.

Slack's defining product promise is real-time. Every member of every channel of every workspace expects the new-message ping in milliseconds. At Slack's scale that means millions of persistent WebSocket connections, all of which need to know about every event that's relevant to them. The naive solution — let the monolith hold every connection and broadcast every event to everyone — does not work past the first few thousand workspaces, and even at that scale the GC pressure on Slack's PHP/Hack backend was reportedly punishing.

Slack's answer is a service called Flannel, sometimes described in their engineering blog posts as a "WebSocket cache" or "application-level edge cache". Flannel is a regional, WebSocket-aware proxy that sits between clients and the core application backend. Each client connects to a Flannel server, not directly to the monolith. Each Flannel server holds a local cache of the metadata it needs to route events: which channels each connected user is in, which DMs are active, which workspace they belong to, who their teammates are. When the core service emits a workspace event — for example, "user X posted message Y in channel Z" — Flannel decides, in memory, which of its connected clients care about that event and pushes only to those connections.

The architectural payoff is dramatic. The core monolith no longer has to maintain millions of WebSocket connections; it talks to a much smaller number of Flannel servers, each of which fans out to its connected clients in parallel. Flannel itself scales horizontally — add more boxes, route by workspace ID, done. Cache invalidation — channel memberships change, users get added to DMs, workspaces add new members — is handled by an event stream from the core to Flannel, with a small but important amount of cleverness around making sure the cache is consistent enough that you don't push a message to someone who got removed from the channel a millisecond ago. Slack's engineering blog detailed this around 2017 in a post worth tracking down.

Presence — the "who's online" sidebar — is handled as a separate, lighter-weight problem on top of the same architecture, because presence has different consistency requirements than message delivery (stale presence is OK; stale message delivery is not). The pattern Slack pioneered with Flannel has since become more or less standard for chat-style products at scale; the architecture diagrams of competitors like Microsoft Teams' chat layer, Mattermost in cluster mode, and many in-house chat products at large companies look structurally similar.

Lesson — Real-time is a tax on every backend. Push connection state out to an edge layer that's designed for it, and your core services can pretend the world is request-response. The cost is a new tier of infrastructure to operate; the benefit is that the rest of your stack stays sane.

Around 2011-2012, Pinterest was in real scaling trouble. The growth curve was vertical and the data-layer choices in front of the team were all painful. They tried the era's fashionable answers in order — MongoDB, Cassandra, MySQL Cluster, HBase. All four failed for Pinterest's specific workload, for different reasons: instability and operational complexity in MongoDB's then-young auto-sharding, the inherent cluster-administration cost of Cassandra at the time, rough edges in MySQL Cluster, and the operational pain of running HBase on top of HDFS for a team that hadn't already invested deeply in Hadoop. After a series of painful weekends in late 2011 / early 2012, Marty Weiner and the data team made what looked at the time like a deliberately conservative call: shard MySQL by hand.

The architecture they landed on — described in Pinterest Engineering's famous "Sharding Pinterest: How we scaled our MySQL fleet" Medium post in 2015 — splits data across 8192 logical "virtual shards" that map onto a much smaller number of physical MySQL instances (initially dozens; later many more). Every primary entity (user, pin, board) has a shard ID computed deterministically from its primary key — the high-order bits of a snowflake-style ID encode the shard, much like the Instagram pattern. Routing happens in the application: when you load Pinner #42, the app reads the shard bits, looks up which physical MySQL holds that shard right now, and routes the query there. Cross-shard queries are simply forbidden. Product features that would have required them get redesigned at the product level until they don't.

The cleverness is in what happens when you need more capacity. Because each physical MySQL instance is hosting many logical shards, you can split a physical instance into two without changing application code — copy a subset of the virtual shards to a new instance, update the shard-to-host mapping in ZooKeeper (or whatever coordinator the team is on at the moment), point reads to the new place. The application keeps thinking in terms of "virtual shard 4732"; the ops team handles "which physical box does virtual shard 4732 live on this week". This was the move that let Pinterest scale through a decade-plus without rewriting the data layer. New shard splits became routine maintenance, not architectural events.

The dividend is mostly invisible — and that's the point. When something goes wrong with a Pinterest database in production, an on-call engineer can SSH in, run SHOW PROCESSLIST, look at the slow log, look at the InnoDB buffer pool, and fix it the same way they would have in 2008. There is no exotic operational mode to learn. There are no proprietary admin tools without documentation. The database is MySQL, and MySQL behaves like MySQL. The architecture has survived a decade-plus of growth in approximately the same shape — features and indexes have evolved, but the sharding model is the same one Marty Weiner described in 2015.

Lesson — The database you understand at three in the morning is more valuable than the database that is theoretically faster. Hand-rolled sharding is unglamorous, but it scales without surprises and your on-call rotation will love you.

For most of its early history, Dropbox stored user files on Amazon S3. By the time the company crossed half a billion users, the AWS bill had become Dropbox's largest single infrastructure expense — and S3's per-byte pricing wasn't budging. The infrastructure team made the call that most cloud-native companies say they'd never make: build their own multi-exabyte object store and migrate everything off S3. They called it Magic Pocket. The reported cost savings in the first year after the cutover were over seventy-five million dollars. The migration itself stands as one of the largest data movements in the history of the internet.

The choreography of the migration is worth understanding even if you'll never do one. For each file, Dropbox's automation would atomically copy the file from S3 to Magic Pocket, verify the copy against a content hash, atomically flip the pointer in the metadata layer to point at Magic Pocket, and only then delete the S3 copy. The verification step caught real corruptions and was non-negotiable. The whole thing ran in parallel across hundreds of workers, with backpressure on the metadata service to keep it from being overwhelmed by atomic-flip writes. Roughly 90% of user-uploaded content moved from S3 to Dropbox-owned data centers over about two and a half years — with no user-visible disruption, and reportedly zero file loss across the migration.

The architecture inside Magic Pocket is purpose-built around the access pattern of file-storage workloads. Storage runs on custom-designed racks the team called "Diskotech" — extremely dense (over a petabyte per rack at the time of the original deployment), optimised for the cold-write/warm-read access pattern that user files actually have. Instead of three-way replication (which is roughly what S3's default-equivalent does internally), Magic Pocket uses erasure coding — specifically Locally Repairable Codes (LRC), which trade some durability overhead for cheaper repair when a single disk dies. The metadata layer is a Go service running on Postgres for durability with in-memory caching for hot lookups. Most of the system is written in Go, with the lowest-level disk-handling layers in Rust.

The team has published several papers and conference talks on the design — the LRC strategy, the migration choreography, the operational learnings from the first year of production. Magic Pocket has since absorbed not just Dropbox's primary storage but also third-party object/block storage products that Dropbox now sells to other companies (the "Storage" product line). The infrastructure that began as a cost-cutting exercise turned into a new product line — a result worth pausing on for any team weighing a similar build-vs-buy decision.

Lesson — At hyperscale, the cloud bill becomes the architecture brief. The "build versus buy" question flips once you're S3's biggest single customer. The bigger lesson is that even cloud-exit migrations can be incremental — Dropbox didn't do a flag-day cutover; they ran both stacks in parallel for years and migrated file-by-file with verification.

The classic Twitter interview question writes itself. When @KatyPerry — or whichever celebrity is currently sitting at the top of the follower-count rankings — tweets and has on the order of 100 million followers, do you push that tweet into 100 million precomputed timelines (fanout-on-write — cheap reads, catastrophic writes), or do you compute every follower's timeline lazily when they open the app (fanout-on-read — cheap writes, catastrophic reads)? The pure-push approach falls over at the long tail of high-follower accounts; the pure-pull approach falls over for everyone because computing a 200-tweet timeline from scratch on every app-open is expensive even for normal users. Twitter's actual answer, refined over more than a decade by engineers like Raffi Krikorian and his successors, is neither. It's hybrid.

For most users, Twitter precomputes timelines on write. When you tweet, a fanout service ("Earlybird" and its descendants — the names have evolved) pushes your tweet ID into a Redis-backed timeline list for every one of your followers. Each list is bounded — Twitter caches roughly the last several hundred tweets per user, beyond which the cache falls back to a slower data source. When you open the app and pull your timeline, it's just a Redis list-read against your own pre-built list; the median timeline-load latency is dominated by the network round-trip from your phone to Twitter's data center, not by computation. This is genuinely fast — sub-100ms p99 for the timeline-read alone.

The exception is high-follower accounts. Pushing a tweet into 100 million precomputed lists in milliseconds is not free; it would dominate the write fleet's CPU budget and create hot-spot fanouts on the Redis cluster. So tweets from accounts above a follower threshold are not fanned out at write time. Instead, when you open your timeline, the timeline service merges your pre-computed list (covering tweets from your normal-follower friends) with a freshly-pulled set of "tweets from high-follower accounts you follow" pulled from a separate index. From your perspective the merge happens server-side and the response shape is identical. The cost is some additional read latency for the small slice of users who follow many high-follower accounts.

The exact follower threshold for the high-follower carve-out has moved over the years and is, by all accounts, dynamically tuned based on current cluster capacity. The separate "Earlybird" search index is its own pipeline entirely — a sharded Lucene-derived service that powers tweet search and was historically the world's largest production Lucene deployment. Real-time recommendations, the "For You" timeline, ad insertion, and various ranking models sit on top of the same fanout substrate, treating the timeline as their input rather than rebuilding it from scratch.

Lesson — There's no "right" answer to the read-versus-write tradeoff; there's only the right cut for your data distribution. Twitter's follower-count distribution is power-law, so its timeline architecture is hybrid. If your distribution is uniform, your architecture probably won't need to be.

The serverless industry's first answer to "run user code on demand" was the container. AWS Lambda spins up a container per request (with warm-pool reuse), suffers a 100–1000ms cold start the first time, and tears the container down a few minutes after the last request. Google Cloud Functions and Azure Functions work essentially the same way. At Cloudflare's scale — over 300 cities, hundreds of millions of HTTP requests per second across the network — that model would be unaffordable. The math doesn't work: there isn't enough RAM to keep enough containers warm in enough cities to make first-request latency acceptable to a globally-distributed userbase, especially when many customer Workers may be invoked only once per day.

Cloudflare Workers uses a fundamentally different unit of isolation: the V8 isolate. This is the same primitive that lets Chrome run multiple browser tabs safely in a single process — separate JS heaps, separate execution contexts, but no process boundary between them. Cloudflare runs that primitive on the server side. Each Worker — your code — runs inside its own V8 isolate. The isolates share a single host process per machine, which means there's no per-container memory tax, no per-container Linux process overhead, no namespace setup. Spinning up a new isolate to handle a request is on the order of single-digit milliseconds. Tearing it down is similarly cheap. Kenton Varda's blog post "Cloud computing without containers" (2018) is the canonical writeup of the design.

The tradeoff is real but specific. A V8 isolate can run JavaScript and WebAssembly; it cannot run arbitrary Linux binaries. The API surface is curated — no filesystem, no raw outbound sockets to anywhere on the internet, no fork. Instead Workers gets a strictly-defined set of capabilities: fetch (with allowlist), the Cache API, Durable Objects (for stateful coordination), R2 (for storage), KV (for key-value), Service Bindings (for service-to-service calls within the same account). For Cloudflare's target customer — a web developer writing latency-sensitive logic that augments their CDN — those constraints are fine. For a data-engineering ETL pipeline or anything that needs persistent disk or arbitrary network egress, they're not. The architecture is opinionated and aware of it.

The economics flow directly from the primitive. Cloudflare can offer 100,000 free Worker invocations per day on the free plan and a generous paid tier because the marginal cost of an additional invocation is genuinely small. A single physical edge server hosts tens of thousands of distinct customer Workers concurrently. The closed-down API surface that drives some developers crazy is exactly what makes the unit economics work — without the per-container overhead, isolation can be enforced cheaply by V8 itself rather than by the kernel. Subsequent cold-start work has driven first-request latency down further, with techniques like pre-warming common dependencies and "no cold starts" for actively-used Workers.

Lesson — The unit of isolation determines your unit economics. If the per-unit cost of your platform is too high, the answer is often not to optimise within the existing primitive but to switch primitives entirely. Containers are great for some workloads; isolates are great for others; the choice is upstream of architecture.

YouTube launched in 2005 on MySQL. By the time the load got serious — millions of videos uploaded, hundreds of millions of views per day, then billions — the team faced the standard fork in the road that every successful MySQL deployment eventually faces: rewrite for a NoSQL or distributed database, or build a layer that makes MySQL look infinite. They chose the second path. The result became Vitess. It was originally an internal Google project, was open-sourced in 2012, joined the CNCF (Cloud Native Computing Foundation) in 2018, and graduated as a top-level CNCF project in November 2019. Today Vitess runs primary databases at GitHub, Slack, Square, Shopify, JD.com, HubSpot, and many more.

Vitess is structurally a proxy in front of a fleet of MySQL shards. The application thinks it's talking to a single MySQL database; the proxy — VTGate — parses each query, decides which shards it touches, dispatches to the appropriate underlying MySQL instances, and reassembles the result before returning it to the client. A per-shard agent called VTTablet sits next to each MySQL process and handles things like connection pooling (essential — MySQL connections are expensive, and Vitess multiplexes thousands of client connections onto a much smaller pool of actual MySQL connections), query rewriting, and online schema-change orchestration. Topology metadata (which shard lives where, which is primary versus replica) lives in etcd or ZooKeeper depending on the deployment.

The killer feature is online resharding. Splitting a sharded database in production has historically been one of the most painful operations in infrastructure — Vitess automates it. It uses MySQL's own binary log to stream changes from the old shards to new ones while traffic continues normally, verifies row-by-row equivalence between old and new, and atomically switches the routing at the end. A reshard that would have been a multi-week, three-engineer project becomes a button-click that drains in hours. Online schema changes work similarly — the schema migration is applied to a shadow table, populated by binlog tailing, and swapped in atomically when caught up. The same machinery powers VStream, Vitess's change-data-capture system, which downstream consumers can subscribe to.

The thesis behind Vitess is worth absorbing carefully: MySQL didn't fail to scale. What didn't scale was a human operator's ability to think about hundreds of MySQL shards manually. Vitess solves that and only that problem. GitHub's migration to Vitess for its primary MySQL fleet in the 2019-2022 timeframe is, by their own published writeups, one of the highest-confidence infrastructure changes the company has ever made — Vitess took something operationally fragile (manually sharded MySQL with a custom routing layer) and made it routine. PlanetScale, the company started by some of the original Vitess engineers, now offers Vitess as a managed cloud product, which has further accelerated adoption.

Lesson — A thin abstraction over the boring database often beats replacing it. The lesson generalises: whenever you're tempted to swap your storage engine, ask whether a proxy that makes the operational surface of the existing engine bearable would solve the same problem at a fraction of the risk.

Shopify hosts well over a million merchant stores. On Black Friday Cyber Monday — Shopify's Olympics, the four days a year when the entire architecture is on its toes — traffic to the busiest 0.1% of stores can hit somewhere on the order of 10,000x their normal volume. If a single one of those stores blows up its database or melts a cache, the blast radius could in principle take down every other store running on the same shared infrastructure. The architectural answer Shopify chose is cellular — they call it Pods, and the design has been refined over more than a decade.

A Pod is a self-contained slice of Shopify infrastructure: a MySQL primary plus replicas, a Redis cluster, a job queue (Sidekiq, traditionally), a fleet of application servers, all sized for a defined number of merchant stores. Every store on Shopify is assigned to exactly one Pod. A request from a store routes to that store's Pod and stays there. If a Pod's database falls over, only the stores in that Pod are affected — the other thousand-plus Pods keep humming. Blast radius is bounded by design. A failing Pod can be drained, repaired, or even rebuilt from scratch without touching the rest of the platform.

The operational engineering on top of the architecture is where the real cleverness lives. Shopify has a "shop mover" — an automated system that migrates a store from one Pod to another, online, with no merchant-visible downtime. Ahead of BFCM, the capacity team identifies which stores are likely to spike (based on historical pattern, recent traffic, and marketing schedules they get from merchants who opt in), and pre-emptively moves them onto dedicated or lightly-loaded Pods. During the event itself, if a Pod gets hotter than expected, individual heavy stores can be lifted off mid-event in minutes rather than hours. The team also runs "war games" before BFCM — chaos-engineering exercises that simulate Pod failures under realistic load — to make sure the runbooks actually work.

The numbers Shopify publishes during BFCM are worth absorbing as a sanity check for what cellular architecture buys you. In recent years they have reported peak request rates of around 80 million requests per minute and total merchant sales above $11 billion over the four-day window. The architecture is the reason a single noisy merchant has never (publicly) caused a global Shopify outage. The Pods pattern has since been articulated more broadly as "cell-based architecture" by AWS for its own internal services — the AWS Builders' Library has a public document on it — and has been copied by other large multi-tenant SaaS platforms (Slack's enterprise-grid architecture and Stripe's internal cells are conceptually similar).

Lesson — Multi-tenancy plus blast-radius management is a first-class design problem, not a Day-2 afterthought. Cellular architectures cost extra at small scale (more replicas, more operational pieces to track) and pay off catastrophically at large scale (a bad actor never takes down everyone). If you're a SaaS that hopes to grow to a million tenants, design for it now — retrofitting cells onto a shared-everything platform is much harder than starting cellular.

Figma's defining feature isn't the design tool itself — competitors had vector design tools first, and arguably some of them were technically more capable. It's the multiplayer experience. Hundreds of designers can be in the same file at once, cursors visible in real time, every move propagating with the responsiveness of Google Docs. No manual save. No merge conflicts. No "Sue is editing this file, please wait". For plain text, real-time multiplayer is a more or less solved problem — Operational Transform or CRDTs, well-published. For design files, with nested components, auto-layout, vector paths, prototype connections, fills that reference styles — it's much harder.

The Figma architecture is, in one sentence, deliberate centralisation. Every Figma file is served by a single multiplayer server process. That server is the authoritative source of operation order. Clients send "delta" operations — "move node 42 to (x,y)", "set fill of node 87 to colour C", "add child node N to parent P at index I" — to the server. The server assigns each delta a sequence number, persists it, and broadcasts to all currently connected clients in that order. Clients apply incoming deltas in sequence; their local state stays consistent with the server's state by construction. There's no merge function on the client. There's no concurrent-edit conflict to resolve at the document level.

This is, importantly, the opposite of classic CRDT-style architectures, where every client is "equal" and conflicts are resolved by some commutative merge function. Figma's argument — articulated in a famous engineering blog post by founder/CTO Evan Wallace titled "How Figma's multiplayer technology works" — is that the merge semantics for design-file operations are weird (what does it mean to merge two concurrent "move node X" operations? do you average the positions? pick the later one? both feel wrong) and that pinning one server per file gives you cleaner semantics with simpler client code at the cost of slightly worse fault tolerance. The cost is acceptable because no single design file is hot enough to need horizontal scaling within itself; the fleet scales by sharding files across servers.

The multiplayer server itself was originally written in TypeScript on Node.js and was famously rewritten in Rust for predictable latency under load — Figma's "Rust in production at Figma" engineering post is worth reading. The Postgres backing store gets periodic state snapshots; the operation log between snapshots is what new joining clients replay. The "one server per file" pinning is handled by a higher-level routing layer — your client opens the file, gets routed to the box currently hosting that file's multiplayer process, and stays there until the editing session ends. If the server crashes mid-session, the file's last snapshot plus its operation log is enough to reconstruct state on a new server box; clients reconnect transparently.

Lesson — Not every system needs to scale horizontally. If your unit of work has bounded contention — one document, one chat room, one game session, one design file — pinning that unit to one machine and walking away is often the right architecture. The trick is recognising when your problem has that shape and resisting the cargo-cult pull toward "scale out everything".

By 2017 Airbnb's monolithic Ruby on Rails codebase — known internally as "the monorail", half affectionately and half in despair — had grown to roughly 500,000 lines of code, was deployed by 200+ engineers per day, and was increasingly impossible to test or scale. Every regression had a wide blast radius. Every deploy was a moment of breath-held suspense. The CI pipeline took hours. The leadership team made the call that eventually gets made at every Rails monolith that grows big enough: move to service-oriented architecture. What's interesting about Airbnb specifically is that they did it slowly and deliberately, over several years, in a way most companies attempting the same migration fail to.

The migration was guided by three principles, articulated by Jessica Tai and other Airbnb engineers in subsequent talks and blog posts. First: every new feature ships as a service, not as monorail Ruby code — a hard wall against the existing codebase getting any bigger. Second: existing critical paths (booking, search, payments, listings) get rewritten one at a time, with extensive A/B testing of the new implementation against the old before final cutover — shadow traffic to the new service, compare responses, iterate until they match. Third: the boundaries between services are asymmetrically protected — services own their own data, expose Thrift APIs (later gRPC), and never share a database. Cross-service interactions go through published APIs or async event streams (Kafka).

The booking flow extraction took roughly two years. The search rewrite took longer. Each cutover had a documented rollback plan; some new services hit production and were rolled back within hours when shadow metrics diverged from the legacy code's behaviour in ways the team couldn't explain. The infrastructure team built internal tooling — service discovery (SmartStack with Synapse/Nerve in the early years, later replaced by Kubernetes-native equivalents), config management, observability platforms, traffic mirroring — partly to enable the migration and partly because the existing monolithic tooling didn't support service-oriented operation.

The harder part of the migration was cultural, not technical. The shift from "daily Rails deploys, blame the deployer if it broke" to "services own their own on-call, you own your service forever" required a different set of habits. Some teams adapted faster than others. Some services that got extracted are, by the team's own retrospectives, more painful to operate today than the monolith chunk they replaced — the per-service on-call cost wasn't fully appreciated in advance. The lesson the team distilled afterwards has become widely-cited industry wisdom: monolith-to-services is mostly an organisational problem, not a technical one. The technical pieces are well-understood; getting humans to align on ownership, on-call, and inter-team contracts is the hard part.

Lesson — The companies that move from monolith to services successfully do it in fifty small migrations, not one big rewrite. The ones that try to do it as a Big Rewrite usually stall and end up with a worse monolith plus a half-built microservice ecosystem nobody trusts. Slow, deliberate, with rollback plans at every step — boring as it sounds.