Unless you’re a serious data developer, chances are you haven’t heard of https://flink.apache.org/. But I can guarantee that you’ve interacted with it.
Unbounded data streams, though, is a horse of an entirely different color, and increasingly it’s how companies want their data ingested.
There’s no waiting for all the data to arrive because the data stream never stops coming, and events in the data stream can arrive out of order.
At this scale, it’s not easy to keep up unless you have robust technology to handle streaming data.