New UbiOps features January 2024 

On the 25th of January 2024 we have released new functionality and made improvements to our UbiOps SaaS product. An overview of the changes is given below.

Python client library version for this release: 4.3.0
CLI version for this release: 3.3.0

 

✔️ Custom metrics

After many requests, we have added functionality for tracking your own custom metrics in UbiOps! You can now create and visualize custom metrics for training runs, deployments and pipelines. This way you can keep track of model specific metrics like accuracy, loss and precision.

 

training metrics

 

To start tracking a new custom metric, it first needs to be defined in UbiOps. You can do so by navigating to Monitoring > Metrics > Create custom metric in the WebApp. After you’ve defined it, you can log metrics to it with the Python Client Library.

 

In the case of a training run you can adjust your training run as follows:

				
					from ubiops.utils.metrics import MetricClient

    def train(training_data, parameters, context):

        training_run_id = context["id"]
        project_name = context["project"]
        metric_client = MetricClient(project_name=project_name)
        metric_client.start()

        # <YOUR TRAINING CODE>

        example_value = 0
        metric_client.log_metric(
            metric_name = "custom.example", # Make sure you created a metric with this name beforehand
            labels = {"request_id": training_run_id},
            value = example_value
        )
				
			

For a deployment you can do the following:

				
					from ubiops.utils.metrics import MetricClient

    class Deployment:
        def __init__(self, base_directory, context):
            self.metric_client = MetricClient(project_name=context["project"])
            self.metric_client.start()
            self.context = context

        def request(self, data, context):
          example_value = 0
          metric_client.log_metric(
              metric_name = "custom.example", # Make sure you created a metric with this name beforehand
              labels = {"deployment_version_id": self.context["version_id"]},
              value = example_value
          )
				
			

 Check out the documentation for all the details and example code snippets.

 

✔️ A customizable monitoring dashboard

We also made the monitoring dashboard in the WebApp (under Monitoring > General) customizable so you can decide what you want to see. You can combine both custom and default metrics in this view.

monitoring page

 

✔️ A dark theme for the WebApp

Do you prefer working in a dark theme? It is now possible to adjust the them of the WebApp. In the page header you will find a small sun/moon icon with which you can toggle between the themes.

dark mode

 

✔️ Updated widgets for the request interface

We have added an image drawer and an image preview widget for file type input/output to the request interface. You can configure this on the Use deployment tab of your deployment.

 

✔️ Improved usage charts

We changed our organization and project usage charts to stacked bar charts so you can better see where your usage is coming from. On your subscription page you can see the organization usage chart, where the usage is split between the different projects. On the project settings page you can find a project usage chart where you can see which deployments are contributing to your credit usage.

 

✔️ Expanded the info in the context parameter for training runs

The training run context parameter now also contains the key project which provides the name of the project the code is currently being run in. This way the project parameter doesn’t need to be hardcoded when using the UbiOps client library from within the code. We also added the key experiment which contains the experiment name.

Do you need more information? Head over to our documentation, or schedule a call with us. 

Latest news

Turn your AI & ML models into powerful services with UbiOps