Skip to content

Howto propagate blobs

How to propagate and use blobs

In UbiOps the storage of deployments is not persistent. Deployments are stateless to ensure efficient scaling on our platform. Locally stored files, or blobs, are wiped from the deployment's memory, after a request has finished. In this how-to, we show how we can send blobs to our storages in such a way, that the blob can be used by another deployment, for example during a following request, or by the next deployment in your pipeline.

There is multiple ways to implement this. We will first show how to create a blob, and pass the unique identifier of the created blob to our output.

Creating blobs from a local file

Assume that we have created a test file during a request, and that we have stored it locally inside inside our instance. During the request, we upload the blob via:

api_response = api.blobs_create(project_name, "10mb_testfile.txt", blob_ttl=blob_ttl)

where api is an api connection to our api (that we have e.g. set-up in the __init__ of the deployment) with sufficient authorization to write blobs to our project, called project_name and 10mb_testfile.txt is the file that we have created locally. Finally, blob_ttl is the time-to-live of our blob, in seconds. The unique identifier is stored as a variable to the api response object: api_response.id.

Using the blob elsewhere

There are multiple to pass this blob to another deployment, and the exact way depends on your situation and your needs.

For example, we can pass this id in the output of our request, and connect this output to the input of a next deployment in a pipeline.

return {
    "testfile" : api_response.id
}

If the deployments are not connected in a pipeline, still we want to introduce statefulnes, we can start by listing blobs that are available inside the project:

api_response.api.blobs_list(project_name, range = -5)
In this example, setting range = -5, causes the api response to provide the metadata of the five blobs that were added to the project most recently. The object api_response has the original filename attached as a variable. We can then search for our blob, by searching for the original filename:

for blob in api_response:
    if blob.file_name == "10mb_testfile.txt":
        blob_id = blob.id

Finally, we can use this blob_id to import the content of our original file, using the blobs_get method

with api.blobs_get(project_name, blob_id) as response:
               text_file_content = response.read()

How you can read the content of a blob, depends on its filetype.