Skip to content
Free since 2015. Built for the engineers who keep things running.

cat /articles/how-cybercrx-cut-ml-processing-time-from-8-days-to-56-minutes-with-aws-step.md

Cloud

How CyberCRX cut ML processing time from 8 days to 56 minutes with AWS Step

Last December, Sébastien Stormacq wrote about the availability of a distributed map state for AWS Step Functions, a new feature that allows you to orchestrate large-scale parallel workloads in the cloud.

AWS DevOps Blog27 Apr 2023 · summary

article --summary

The summary

https://aws.amazon.com/polly/ Last December, https://aws.amazon.com/blogs/aws/step-functions-distributed-map-a-serverless-solution-for-large-scale-parallel-data-processing/ https://aws.amazon.com/step-functions, a new feature that allows you to orchestrate large-scale parallel workloads in the cloud. In addition, the new implementation with AWS Step Functions Distributed Map costs less than what it did originally. Whenever there’s a new company or the data changes for a company, they regenerate their predictive model by processing their entire dataset. Over time, CyberGRX data scientists improve the model or add new features to it, which also requires the model to be regenerated. And to make sure that the steps were run in parallel, he used Step Functions and the https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-map-state.html.

Editorial note: This is a summary of an article first published by AWS DevOps Blog. The full piece is on their site.

articles --latest

Latest articles

New from the blogs we follow and the community.

All articles

newsletter --weekly

DevOps Weekly, minus the noise.

The best reads, new jobs and upcoming events, in one email a week.