Buy or Build for MLOps

Where to get started?

Let’s start with admitting that there is no 100% waterproof pros and cons list of buying vs building an ML Ops stack that works for everyone. It always depends on your specific situation. In this article, I try to highlight some of the most important considerations to make and explain the pros and cons for the majority of companies that face this challenge, including ourselves 4 years ago.  Having said that, the buy versus build decision is an important one, even in the early stages of your AI journey. It has a major impact on how you can leverage AI and the bottom line ROI you can achieve.

1. Long-term versus short-term

Think long-term. AI is something that is here to stay, and if it’s not yet on your agenda, it probably is on that of your competitor. Building an AI stack yourself is a lot of work, ranging from at least a year to multiple years and costs. Maintaining it, updating it and improving it can again take up much of your team’s time. But if AI enables you to do what you do, but better, faster or more efficiently, perhaps buying a platform (or a combination of) is better for you than building it. It allows you to quickly deliver value with AI and take the first mover advantage compared to your competitors. Is it worth it to move your innovative projects further down the road, lowering your competitive advantage? It’s therefore crucial to think about this, even if you’re about to conduct a first proof of concept.

2. DIY or a great SLA?

Do the costs of maintenance outweigh the licensing costs? What’s often underestimated is the time and costs involved with updating your system. I once talked to a data team lead and he said ‘we spend 40% of our budget on learning google cloud, but after half a year everyone on our team knows how to use it. And then we just all spend 20% on learning new things’. 20%, if he even manages to do that, that’s still a lot of time that can perhaps better be spent on delivering value. 20% of the costs of 10 full-time employees one can purchase a pretty big license and a top-notch SLA. Still, he decided to build it himself, taking us to the next point…

3. The ‘build it yourself’ mentality

Developers want to build. Especially with hot technologies, like ML. At UbiOps, our team consists mostly of developers, and for sure I know they like to build, and damn they can be critical when I drop ideas for everything and nothing. Your team may want to build everything themselves too, and they’ll tell you that this is the best solution. Think of 15 years ago, when every website and application was handmade. Think now, how we have WordPress where you can use pre-made templates and Mendix or Out Systems that offer low-code solutions to develop applications easily. Developers were fiercely against that. And some still may be. But the bottom line is what counts: how do we create the most value for our company or client, as fastest and efficiently as possible? Figure 1: A PHP king in 2021?

4. Opportunity costs versus licensing costs

Do the costs of licensing/purchasing a solution offset the costs of building time, and the opportunity costs of delaying any new AI projects? Every month spent on discussing, experimenting and building cannot be spent on running your models in practice and delivering value. And then we don’t yet account for the costs of any unexpected problems that occur. If you have any experience with software development, you probably know that never everything goes as planned.

5. Your solution versus the existing solution

How certain are you that your solution is better than what’s already out there? How well do you know what’s out there and what’s the certainty that what you will build is superior to other solutions available on the market? Solutions that are probably developed over a longer period of time and with more resources. Of course, one must make other considerations as well, such as the SLA, but still. And remember that as humans we are sensitive to the sunk cost fallacy. If we already invested in something, we prefer to continue investing in it simply because ‘we already invested in it’. We all know that should not be the reason, and shouldn’t let ourselves be tricked by that emotion. Build it if you are certain and convinced that what you need is not out there yet.

6. Ensure you have right knowledge

Do you have the right technical resources to successfully complete the build project? 4 years back we developed models for our customers. The reason why we had difficulty deploying our models for and at our customers, was mainly due to the scarcity of technical knowledge to do so. As a result, we pivoted and built UbiOps to solve that. In 2021, the right technical resources are even more scarce and most importantly, more expensive. Besides technical knowledge, ensure there is sufficient cultural support and understanding. Devoting resources (budget, time) to a project only one part of the company understands, or worse, only the data science team understands, creates friction with other stakeholders e.g. the IT team. Therefore you should make sure that others understand why resources are spent on it and why it is important to do so.  For example, include a project manager and an executive sponsor to create an environment that facilitates the project, rather than criticises it

7. The ROI on your team’s time

Consider what is the optimal use of your team’s or company’s budget and time. Is it spending it on developing an ML system from square one? Or is it developing models and using existing systems to create turnover and value? Which of the two has the highest (chance of a high) ROI is something you should consider before making the buy/build decision.

8. Scoping the project properly

Determining the scope sounds easy. But do you really know which functionality, which security feature and which CI/CD configuration you will need? For many organisations this is unclear. Existing solutions have a variety of functionalities and are usually eager to think along with you about which functionality best suits your project. Moreover, vendor solutions continue being developed while you’re delivering value with your AI projects and doing what you do best. Perhaps after some time you need to change direction, and what your initial requirements of a solution were, do not suffice anymore. In that case, see the following point. 

9. Prevent technical debt

If you go for the ‘buy’ decision, please don’t dig yourself in. Staying away from technical debt is easier said than done. When building something yourself creating technical debt is more likely to occur, but also when licensing existing solutions it could happen. Ask yourself: how easy is it to get my models running in another tool? The vendor (like us) of course is benefitted by continued usage of their solution. But we believe that rather than providing you with the entire stack we should offer a piece of a puzzle and be the best at that piece.

Our view on things

We want to allow you to connect different pieces together, and build the entire ML stack according to your needs, whereby you can switch out pieces if your needs change. In the end, no one benefits from unhappy customers. For this reason, we actively participate in the AI Infrastructure Alliance, which aims to create a canonical stack for machine learning using a Darwinistic approach. UbiOps is a serving and hosting tool made with the data scientist in mind. We experienced firsthand the difficulties of MLOps and built UbiOps to solve a piece of the puzzle. Are you unsure if our solution is something that helps you to get a higher ROI on your AI projects? Feel free to reach out to us and we’re happy to think along. Wouter Hollander Connect with me via LinkedIn!