Tracking pipeline metrics on UbiOps¶

When you have created a production pipeline with UbiOps, you might want to track some custom metrics over time for a performance indication. This how to guides you through the process of creating a new custom metric and logging data to it from your pipeline objects. We cover the following steps

1. Define a new metric in UbiOps
2. Adjust your deployment code to log datapoints on pipeline level if it's called from a pipeline.
3. Tracking your metric in the WebApp

Defining a new metric in UbiOps¶

Before you can track any custom metric in UbiOps, the metric needs to be defined. You can either do this in the WebApp by navigating to Monitoring > Metrics > Create new metric, or via code. Let's walk through both.

When you click Create new custom metric in the WebApp, you will be redirected to a form where you need to provide the following information:

• Name: the name of your custom metric. It always has to begin with custom.. This is the name that will be used for the titles of associated graphs later. In our example we can call our metric custom.example
• Description (optional): the description of your custom metric.
• Unit (optional): the unit of measurement for your custom metric In our case we can use count.
• Metric type: how this metric should be processed. We support Gauge and Delta metrics (see metrics for more details). For our example we can use Gauge.
• Metric level (referred to as labels in the API): on what level do you plan to store the metric? In our case we need the metric on pipeline level.

When you click Create the metric will be created and you can start logging data to it. Alternatively, you can also do the same with the Python client library:

import ubiops

configuration = ubiops.Configuration()
# Configure API token authorization
configuration.api_key['Authorization'] = "Token <YOUR_API_TOKEN>"
api_client = ubiops.ApiClient(configuration)
core_api = ubiops.CoreApi(api_client)

project_name = 'project_name_example' # str
data = ubiops.MetricCreate(
    name = 'custom.example',
    description = 'An example metric',
    metric_type = 'gauge',
    unit = 'count',
    labels = ['pipeline_version_id']
)

# Create the metric
api_response = core_api.metrics_create(project_name, data)
print(api_response)

Logging datapoints from your deployment code¶

Now that the metric has been created in UbiOps, you can start logging data to it from your pipeline. When you want to log a metric on pipeline level, you need to adjust the deployment code of your pipeline object. In your deployment code it's possible to check whether the deployment was called as part of a pipeline or as a standalone deployment. We can use this information to only log our metric data when it's actually called as part of a pipeline. The basic code for logging a custom metric on pipeline level is as follows:

from ubiops.utils.metrics import MetricClient

metric_client = MetricClient(project_name = "your_project_name")
metric_client.start()
metric_client.log_metric(
  metric_name = "custom.example",
  labels = {"pipeline_version_id": "your_deployment_version_id"},
  value = your_metric_value
)

Let's have a look at how this would work with in your deployment code.

from ubiops.utils.metrics import MetricClient

class Deployment:

    def __init__(self, base_directory, context):
        print("Initialising deployment")
        self.metric_client = MetricClient(project_name=context["project"])
        self.metric_client.start()
        self.context = context

    def request(self, data, context):
        print(f"Processing request {context['id']}")

        example_value = 1

        # log the metrics to UbiOps only if this deployment is called as part of a pipeline
        if "pipeline_version_id" in context.keys():
            print("Logging metrics...")
            self.metric_client.log_metric(metric_name="custom.example", labels={"pipeline_version_id": context["pipeline_version_id"]}, value=example_value)        

        return {}

The project name can be retrieved from the context parameter passed in the __init__ function, and the pipeline version id from the request function. These values can be used to log the metric value correctly to your pipeline.

Viewing your data¶

When you define your metric and adjust your code to track it as defined in the previous steps, you will be able to see your custom metric data in the UbiOps WebApp. Simply navigate to your pipeline version details page and click on the Metrics tab. The graph in the WebApp will have a resolution of maximum one value per minute.