Working with files¶
Files are uploaded, listed, downloaded or deleted in UbiOps using an API request, either directly or using the WebApp, CLI or Python client library. In the WebApp you can browse through existing files, upload new files, or organize them in folders. When using the WebApp, the uploading of the file and passing the URI in the deployment request is handled automatically in the background.
See platform limits for limitations on the usage of files.
Uploading & downloading files from UbiOps¶
To support the process of uploading and downloading files, we have added relevant helper functions to our Python client in a module
named ubiops.utils. The module contains the methods ubiops.utils.download_file and ubiops.utils.upload file. These methods can be used to download or upload a file in one step. The following example uses the API Client api_client to upload/download a file file to the location files/file in bucket my-bucket in project my-project. When using the WebApp, the uploading of the file and passing the URI in the deployment request is handled automatically in the background.
from ubiops import utils
#Upload a file
file_uri = ubiops.utils.upload_file(
client = api_client, #a UbiOps API client
project_name = "my-project",
file_path = "file",
bucket_name = "your-bucket",
file_name = "files/file"
)
from ubiops import utils
#Download a file
file_uri = ubiops.utils.download_file(
client = api_client, #a UbiOps API client,
project_name= "my-project",
output_path = "file"
bucket_name= "your-bucket",
file_name= "files/file"
)
In the WebApp a download button is available after completing a request or by navigating to a file in your bucket.
Alternatively, you can use the API directly to download or upload files. When using the API directly to download or upload files, please note that it is a two-step process that uses "signed URLs".
For downloading:
- Request a signed URL using the "Download a file" API endpoint.
- Download the file using a standard HTTP GET request.
For uploading:
- Request a signed URL using the "Upload a file" API endpoint.
- Use the signed URL to upload the file using a HTTP PUT upload request. An example using cURL is:
curl -X PUT <signed URL> --upload-file file.ext. - Azure buckets require two additional request headers to be sent with the upload, see the "Upload a file" endpoint on our Swagger documentation for more information.
A signed URL is valid for 5 minutes. Note that anyone in possession of this URL can upload or download the file during the 5-minute window.
Managing files in your bucket¶
You can manage files in your bucket via the API or with the Python client library. Below is an example to delete files that are stored in your bucket. Authentication to the API can be done via environment variables or with authorization parameters
import ubiops
core_api = ubiops.CoreApi()
project_name = "my-project"
bucket_name = "your-bucket"
file = 'file_example'
# Delete a file
core_api.files_delete(project_name, bucket_name, file)
# Close the connection
core_api.api_client.close()
import ubiops
configuration = ubiops.Configuration()
# Configure API token authorization
configuration.api_key['Authorization'] = "Token <YOUR_API_TOKEN>"
# Defining host is optional and default to "https://api.ubiops.com/v2.1"
configuration.host = "https://api.ubiops.com/v2.1"
api_client = ubiops.ApiClient(configuration)
core_api = ubiops.CoreApi(api_client)
project_name = 'project_name_example' # str
bucket_name = 'bucket_name_example' # str
file = 'file_example' # str
# Delete a file
core_api.files_delete(project_name, bucket_name, file)
# Close the connection
api_client.close()
File operations¶
End-to-end examples of more common file operations are provided in the how-to work with buckets & files in UbiOps How-to. We will show one example here: It is possible to list all the files that are currently in a specific directory of a bucket. This can be useful when you want different deployments to interact with different directories in the same bucket. Authentication to the API can be done via environment variables or with authorization parameters
import ubiops
core_api = ubiops.CoreApi()
project_name = 'project_name_example' # str
bucket_name = 'bucket_name_example' # str
prefix = 'prefix_example' # str (optional, prefix to filter files)
delimiter = 'delimiter_example' # str (optional, delimiter used with prefix to emulate hierarchy to filter files)
continuation_token = 'continuation_token_example' # str (optional, a token that indicates the start point of the returned the files)
limit = 56 # int (optional, the maximum number of files returned, default is 100)
# List files
api_response = core_api.files_list(project_name,
bucket_name,
prefix=prefix,
delimiter=delimiter,
continuation_token=continuation_token,
limit=limit
)
print(api_response)
# Close the connection
core_api.api_client.close()
import ubiops
configuration = ubiops.Configuration()
# Configure API token authorization
configuration.api_key['Authorization'] = "Token <YOUR_API_TOKEN>"
# Defining host is optional and default to "https://api.ubiops.com/v2.1"
configuration.host = "https://api.ubiops.com/v2.1"
api_client = ubiops.ApiClient(configuration)
core_api = ubiops.CoreApi(api_client)
project_name = 'project_name_example' # str
bucket_name = 'bucket_name_example' # str
prefix = 'prefix_example' # str (optional, prefix to filter files)
delimiter = 'delimiter_example' # str (optional, delimiter used with prefix to emulate hierarchy to filter files)
continuation_token = 'continuation_token_example' # str (optional, a token that indicates the start point of the returned the files)
limit = 56 # int (optional, the maximum number of files returned, default is 100)
# List files
api_response = core_api.files_list(project_name, bucket_name, prefix=prefix, delimiter=delimiter, continuation_token=continuation_token, limit=limit)
print(api_response)
# Close the connection
api_client.close()
We provide other how-to's on