Skip to content

Blob handling

A blob is any type of file that can be used as input to deployments and pipelines, or produced as output. Blobs can also be used as intermediate and temporary storage of data files within a single deployment or to access and share data within the same project.

Working with blobs

Blobs are files that are uploaded to UbiOps and that can be used in requests to deployments and pipelines. A deployment or pipeline output file is also stored as a blob, which can be downloaded via the WebApp or API.

Within an organization, blobs are defined at project level. Each blob is uniquely identified in a project using its id , a unique identifier string in UUID format.

Storing and retrieving blobs

The blobs stored for each project can be retrieved using the UbiOps API endpoint /projects/{project_name}/blobs. To upload a new blob to your project you can use the /projects/{project_name}/blobs API endpoint. This will return the blob id that can be used as deployment or pipeline input.

To use blobs in a deployment, define the deployment with type structured and create one or more input fields with data type blob.

When using the WebApp, blobs can be uploaded using the Upload file button when making a request. Additionally, you can use a blob file previously uploaded or available in your project by selecting it using the Choose file button. This button will list all the blobs available within your project.

Blob time-to-live (TTL)

Blobs are stored by default for 72 hours and can be used, within the same project, for different deployment and pipeline requests. You can specify a longer or shorter time-to-live (ttl) for the blob by defining the ttl parameter when uploading the blob via the UbiOps API. The minimum value allowed for the blob ttl is 900 seconds, whereas a maximum value of 72 hours can be specified.

Sending blobs as input data

Blobs can be passed as input to your requests in a JSON format by specifying the blob id as value field in the request. UbiOps will automatically retrieve the files based on the blob id For example, if your deployment expects two input blob files, i.e., file_1 and file_2, from the WebApp select the JSON tab when creating the request and specify

{
  "file_1": "67ee50f2-e8e2-46f6-b9c5-998880b4b159",
  "file_2": "3f170234-ded9-4930-b876-ca80b9bc3ba7",
  "other_not_blob_input": "my-other-input"

}
in which 97aa50f2-e8e2-46f6-b9c5-998880b4b759 and 4g771243-ded9-4970-b876-ca80b9bc3ba7 are the blob identifiers generated when uploading or creating file_1 and file_2.

Here an example on how to use blobs passed as inputs to your deployment. The example also shows how to use blobs as output.

class Deployment:

    def __init__(self, base_directory, context):
        """
        Initialization method for the deployment
        """

        # Initialization steps for the deployments
        ...

    @staticmethod
    def process_files(data):
        """
        Example method to process files
        """

        with open(data['file_1']) as f:
            # Do something

        with open(data['file_2']) as f:
            # Do something

        # Process the files and create output_file
        ...
        return output_file

    def request(self, data):
        """
        Example method for deployment requests
        """

        processed_file = self.process_files(data)

        return {
            "output_file": processed_file
        }

When using the API, a blob is first uploaded using the blob management endpoints. A request can then be made by passing the unique identifier of the blob as the value for the field, in the same way as shown for the WebApp JSON request.

More information about using blobs via the API can be found on our Swagger API explorer.

See platform limits for limitations on the usage of blobs.

Managing access to blobs

Users and service accounts can be given permissions to blobs on a project level. The following default roles are available:

  • blob-viewer - can list and download blobs.
  • blob-editor - can list, download, and upload blobs.
  • blob-admin - can list, download, upload, and delete blobs.

These permissions can only be granted to a user for the entire project, not per individual blob.

Blob access privileges

Because blob permissions can only be granted at project level, allowed users can list all blob files available within a project. These can include blob outputs from deployments or pipelines to which the user does not have direct permission.

Learn more about permissions in UbiOps on Permissions and roles.