Disk Spilling in a Vectorized Execution Engine

4 years ago dzone.com

Summary: This is a summary of an article originally published by the source. Read the full original article here →

by v1 of the engine didn't support disk spilling, which meant it couldn't execute certain memory-intensive queries if there was not enough memory available.

This memory, also called "work" memory, is limited to 64MB in CockroachDB, by default.

In this case, spilling to disk is very simple, as the only thing that is needed is an append-only log that will be replayed multiple times.

Queries that could previously use an unbounded amount of memory now use up to a constant amount of work memory and spill to disk if this amount is not enough.

DevOps Articles