4 years ago

Category: Data, machine-learning

A how-to guide on quantifying concept drift for Machine Learning models Machine Learning models are increasingly at the core of products or product features. [...] While this will allow you to compute accuracy metrics, it comes at the cost of degraded user experience or lost revenue and in some cases is not possible (when detecting credit card fraud for example). [...] By defining key slices and tracking metrics for these, you can detect issues that affect only a subset of customers and that would have be lost in the noise of the globals metrics you are tracking. [...] The end goal of concept drift metrics is to detect changes in multivariate distributions and quantify how different today’s inputs are compared to yesterday’s for example.