benchmark Archives - UbiOps - AI model serving, orchestration & training

How to benchmark and optimize LLM inference performance (for data scientists)

May 3, 2024 / May 3, 2024 by UbiOps

Introduction Optimizing inference is a machine learning (ML) engineer’s task. In a lot of cases, though, it tends to fall into the hands of data scientists. Whether you’re a data scientist deploying models as a hobby or whether you work in a team that lacks engineers, at some point you will probably have to start […]

Tagged