Source: medium.com
Optimising AWS RedShiftCategory: Database, Data
We had big complicated queries that used multiple joins on multiple tables. We had huge data in our tables and the query rate remains around 20–50 queries per second.
To begin, We started by analysing all the queries that were running, especially the ones that were taking a lot of time.
After further digging, we also found that most of our queries used recent transactions only(not more than 6 months data) but we had all the data in our tables ranging for 3–4 years which we didn’t want to remove as they were required too.
Also, these temp tables were sort of read-only due to which we were able to leverage max utilization AWS concurrent scaling on the fly for our read queries.
To begin, We started by analysing all the queries that were running, especially the ones that were taking a lot of time.
After further digging, we also found that most of our queries used recent transactions only(not more than 6 months data) but we had all the data in our tables ranging for 3–4 years which we didn’t want to remove as they were required too.
Also, these temp tables were sort of read-only due to which we were able to leverage max utilization AWS concurrent scaling on the fly for our read queries.
Related Articles
Community Partners
DevOps Careers