Upgrade to the new file system¶

In release 2.21.0 the new file system was introduced which allows for better and more functionalities than the legacy blob system. This page will walk you through upgrading old deployments and pipelines that use the legacy blob system.

1. Updating the input/output configuration¶

Navigate to your deployment or pipeline that you want to upgrade and go to "edit". In the input and output sections, simply change the datatype of your Blob fields to use File instead.

If your deployment is used in a pipeline, make sure you detach any connections in the pipeline to that object before updating it, and attach it again afterwards.

2. Updating the way you make requests¶

If you are sending automated requests to the deployment or pipeline that you upgraded in step 1, you will need to adjust the code that you use for sending the request. Previously, requests with blob input required a blob id, whereas requests with file input require a file URI. UbiOps file URI's are formatted as follows: ubiops-file://{bucket-name}/{path-to-file-in-bucket}. This file URI is returned whenever you upload a new file with the client library. Below you can see a working example with the Python client library. Make sure you update your client library version to the latest version before updating your scripts.

# Specify a bucket to upload a file to, and the path to your file
bucket_name = 'bucket_name_example'
path_to_file = 'example.csv'

# Upload a file
file_uri = ubiops.utils.upload_file(api_client, project_name, path_to_file, bucket_name)

# Now we can make the request:
data = {'csv': file_uri}
response = api.deployment_requests_create(
    project_name=project_name, deployment_name=deployment_name, data=data
)

When upgrading your deployments and pipelines to the new file system, make sure you check all your automated scripts that send requests to them and update the data formatting accordingly.

3. Updating the way you handle a request response with files¶

If you have automated scripts that do something with the request output of your upgraded deployment or pipeline, make sure you adjust them accordingly. Previously, a deployment would return a blob id if it had a blob in its output. With the new system the file URI is returned instead, which is formatted as follows: ubiops-file://{bucket-name}/{path-to-file-in-bucket}. If you need to download the returned file you can do that as follows:

# We make a request:
response = api.deployment_requests_create(
    project_name=project_name, deployment_name=deployment_name, data={}
)

# Now we get the file URI
file_uri = response.results["csv"]

# And here we download the csv file
api.utils.download_file(
    api_client,
    project_name,
    file_uri=file_uri,
    output_path='.',
    stream=True,
    chunk_size=8192
)

4. Double checking permissions¶

By default, every deployment and pipeline has access to the default bucket which every project has. In case you want your deployment to read from or write to a separate bucket, you need to grant permission to that deployment. You can do so by navigating to your deployment and clicking "edit". In the "Bucket permissions" section you can grant your deployment access to other buckets.

If you are working with service users (API tokens) to work with files, you also need to make sure that these actually have the right permissions. See managing bucket permissions for more info.

With these four steps you can upgrade your deployments and pipelines to work with the new file system!