Source: thenewstack.io
Data Streaming, for When Micro-batching Just Isn’t Fast Enough
Micro-batching is as close to real-time as batch can get, allowing latencies in mere minutes in some cases while running on the same infrastructure as longer latency batch systems. Back in October, I had the pleasure of attending the debate “If Streaming is the Answer, Why Are We Still Doing Batch?”
Micro-batching isn’t cheating no matter what anyone says, but it does have severe limitations and it isn’t a real-time streaming system. Micro-batching is a low-latency batch system that can send batch data in mere minutes in some cases.
Akidau often meets clients who want real-time but end up realizing they don’t need real-time when he explains the costs and complexity of real-time systems versus batch systems.
Micro-batching isn’t cheating no matter what anyone says, but it does have severe limitations and it isn’t a real-time streaming system. Micro-batching is a low-latency batch system that can send batch data in mere minutes in some cases.
Akidau often meets clients who want real-time but end up realizing they don’t need real-time when he explains the costs and complexity of real-time systems versus batch systems.
Related Articles
Community Partners
DevOps Careers