Unplugging Cassandra nodes to see what still answers
Three nodes, three replication settings, every combination of failures. A table of exactly when "consistent" stops being "available."
- Built with
- Apache Cassandra, Docker
The question
Everyone says “replicate your data and you’ll survive failures.” How many failures, exactly, and what do you give up to survive them?
What I tried
I ran a three-node Cassandra cluster in Docker, set the replication factor to 1, 2 and 3, and paused nodes one and two at a time. For every combination I tried both reading and writing, and when an operation failed I checked which consistency level would rescue it.
What I found
- One copy is available exactly as long as that one node is alive. No surprises, no mercy.
- Two copies plus quorum turned out to be the worst of both worlds. Quorum needs both copies, so losing either node fails the request. It was less available than one copy in some cases.
- Three copies plus quorum survived any single failure, but not two.
- Writes have an escape hatch reads don’t. With
ANY, a write can be parked on whichever node is alive and delivered later, so writes survived failures that reads could not.
RF=2 with QUORUM is a trap. wrote that one down.
What I kept
“Replicated” and “available” aren’t the same word. Before choosing a replication factor, I now write down how many failures we need to survive and which operation has to keep working, then do the quorum math.