Top 10 Articles To Learn About MLOps

Ten articles UbiOps recommends for you to read to get started with machine learning operations

Machine learning is a hot topic: the release of ChatGPT fired the imagination of many. How can artificial intelligence (AI) help us make progress in our organisations? Your customers can benefit from improved products and services powered by AI solutions. However, incorporating artificial intelligence in your operations is easier said than done. Common challenges include:

• • Complexity of machine learning models: complex models can be challenging to maintain
  • Data dependencies: we all know “garbage in, garbage out”
  • Lack of standardisation: because of its novelty, there is currently no universal method for the deployment of machine learning models
  • Security: machine learning models often deal with sensitive data
  • Scalability: dealing with an increasing number of AI models and data sources makes everything more difficult

Machine learning operations (MLOps) is a set of practices that aims to streamline the development, deployment, and maintenance of machine learning models in a production environment.

At UbiOps, we specialise in helping you run your AI models in the cloud, instantly, easily and at scale. MLOps is our bread and butter and we are continuously looking for interesting new articles in the space. Below, we present ten articles from experts in the field that we feel you should read if you want to familiarise yourself with MLOps.

Why read these (MLOps) articles?

If you are interested in incorporating AI in your organisation, or you’re just curious to learn more about what others are up to, these ten papers will give you a great foundation for your MLOps journey. After reading them, you will have a good holistic understanding of MLOps, be familiar with common pitfalls, and have ideas of where to start if you want MLOps to be a part of your operations. Here’s the list:

————————————–

1. 1. Machine Learning Operations (MLOps): Overview, Definition, and Architecture

Dominik Kreuzberger et al.

MLOps is an umbrella term. It encompasses many different fields of expertise that all require different technical components and responsibilities within your organisation. In this paper, Dominik Kreuzberger et al. introduce the nine principles of MLOps. They also explain multiple technical components, responsibilities, workflows, and how those all interact. Understanding their MLOps flowchart will definitely give you a great idea of how to organise your organisation to work with AI. It can be a bit overwhelming, though, as they really try to cover all the bases. But don’t worry about that, we promise it will make sense when you dive deeper. 

You can find the full paper here.

————————————–

2. 2. Challenges of deploying machine learning: a survey of case studies

Andrei Paleyes, Raoul-Gabriel Urma, Neil D. Lawrence

MLOps has many moving parts, and each of those parts comes with its own set of challenges and considerations. Consequently, familiarising yourself with these challenges will help you understand what to look out for when developing your MLOps pipelines. This article aims to discuss considerations, issues and concerns for all machine learning deployment stages on the basis of case studies done by real companies. Among many other things, they discuss data management, computational cost, and security. No need to reinvent the wheel if you can learn from those that came before you, right?

You can find the full paper here.

————————————–

3. 3. Ease.ML: A Lifecycle Management System for Machine Learning

Leonel Aguilar et al.

As I have noted at the first article in this list, getting started with machine learning applications can be a daunting task. The Ease.ML process is a system for non-experts to do just that! It is a holistic approach to developing ML apps. With step-by-step guidance Ease.ML gets you from day zero to a deployed model with certain quality guarantees. If you know how to handle your dataset, can write simple python scripts and understand basic concepts of ML you are good to go. Your first projects will not be complex, but they will be a good foundation for expansion.

You can find the full paper here.

————————————–

4. 4. Building Continuous Integration Services for Machine Learning

Bojan Karlaš et al.

Continuous integration (CI) has been a staple feature of DevOps for developing industrial-strength software for a while. It helps ensure that changes made by multiple developers are integrated and tested regularly. Ideally, MLOps reaches the same maturity as DevOps in the near future. Unfortunately, the DevOps framework can not be copy-pasted to machine learning solutions. Continuous Integration requires, among other things, continuous work on the datasets. But, research has shown that the fidelity of a test dataset decreases when it is being accessed over and over. For MLOps, it is especially important to prevent this from happening, because a bad test set leads to badly fitted machine learning models. Therefore, this paper discusses two changes in the DevOps framework to make it work for ML: probabilistic software testing instead of deterministic, and dataset refreshing to prevent weakened data influencing the model. Additionally, this paper discusses how to implement a CI system that includes these features.

You can find the full paper here.

————————————–

5. 5. Asset Management in Machine Learning: A Survey

Samuel Idowu, Daniel Strüber, Thorsten Berger

The development of machine learning applications is an iterative process. You will have many different models, with different architectures and hyperparameters, while continuously using different dataset and generating various statistics. These are called ML assets, and it’s a lot to keep track of! What types of ML assets do you need to manage? What tools exist to assist you in this? How can you integrate these tools into your ML development systems? Read this survey to find out!

You can find the full paper here.

————————————–

6. 6. Building a Reproducible Machine Learning Pipeline

Peter Sugimura, Florian Hartl

Reproducibility is an important quality of any iterative system. For that reason, it is important to be able to quantify changes and improvements. In machine learning, this means that a model needs to be able to give the same output every time, given the same input. In this paper, several causes of reproducibility problems are discussed, and possible solutions are presented. They discuss their modelling pipeline and its different layers that are specifically designed to yield a robust machine learning model.

You can find the full paper here.

————————————–

7. 7. How to avoid machine learning pitfalls: a guide for academic researchers

Michael A. Lones

Newcomers in any field tend to make the same types of mistakes. This guide is targeted at students and other researchers who are new to machine learning. It categorises mistakes into different parts of development: data analysis, building a machine learning model, evaluation and comparing different models. Then, for each category, they discuss some important do’s and don’ts that teach you how to avoid making mistakes! Because of its target audience, this article is easy to read if you have some pre-existing ML knowledge.

You can find the full paper here.

————————————–

8. 8. Data Engineering for Data Analytics: A Classification of the Issues, and Case Studies

Alfredo Nazabal et al.

A machine learning model’s output is only ever as good as its input data. Thus, an important step is to prepare data for the model training process. Not understanding the data, not cleaning it properly, or acquiring it from the wrong places will end your ML project quickly, without any presentable results. Such preparation is called data engineering, and this paper specifically discusses how to perform this task for creating high-quality machine learning input data. 

You can find the full paper here.

————————————–

9. 9. A Data Quality-Driven View of MLOps

Cedric Renggli et al.

How does bad data propagate through an ML system? How do you assess the probability of creating a successful machine learning application? By means of four examples, you read how to clean data, perform a feasibility study by estimating the Bayes error rate for your ML problem, and how to organise an efficient system for quality testing. These processes affect both the pre-training and post-training stages of MLOps. Therefore, understanding these will prevent you from doing a lot of fruitless effort.

You can find the full paper here.

————————————–

10. 10. Towards Automating the AI Operations Lifecycle

Matthew Arnold et al.

Getting started with your ML application is a lot of manual work. You need a lot of knowledge and human interaction with your systems to prepare data, design a machine learning model, and maintain it. For that reason, ideally, a lot of the maintenance necessary after deployment is done automatically. This paper presents some technologies that help you work towards just that. Automation can be challenging to set up, but when it works it will save you a lot of time, effort and money.

You can find the full paper here.