Source: thenewstack.io
Processing Large Data Sets in Fine-Grained Parallel Streams with SQL
SQL is broadly used as a data access language, and Trino provides a powerful engine for SQL access to multiple data sources. However, as more and more real-time data becomes available, developers will need to process large data sets at an unlimited scale with predictable performance. The Aerospike Trino Connector leverages specific Aerospike mechanisms for accessing large data sets over parallel streams.
To process large data sets, a common scheme is to split the data into partitions and assign a worker task to process each partition.
Aerospike allows for data splits larger than 4,096 by allowing a partition to be divided into subpartitions efficiently.
To process large data sets, a common scheme is to split the data into partitions and assign a worker task to process each partition.
Aerospike allows for data splits larger than 4,096 by allowing a partition to be divided into subpartitions efficiently.
Related Articles
Community Partners
DevOps Careers