3 months ago

Category: Kubernetes, Docker

I recently deployed a 500MB Pytorch model. It was surprisingly hard! In this post, I document the pitfalls and tradeoffs I made. Running on CPU was nearly as fast as GPU for non-batch processing, so I recommend starting with that if you can.