There are various settings related to requests that can be changed in the advanced settings section of your deployment version. They fall into two categories: request storage and request queue size limits.
UbiOps can store the data of requests made to deployments and pipelines, both for direct requests and for batch requests.
The data that can be stored for a request consists of two elements:
- Request metadata: the metadata is all information related to a request, except the data sent as input or produced as output. Specifically, metadata includes the creation and completion times of a request, its status, whether it was successful and who created it.
- Request input and output: the data that is sent as input or produced as output.
What data is stored and for how long can be configured for each deployment or pipeline version separately. The following settings are available:
- The request retention mode. The retention mode determines what data is stored for each request. There are three possible values:
- Full: everything is stored, both metadata and input and output.
- Metadata: only the metadata is stored.
- None: nothing is stored.
- The request retention time. The time the request is stored, counting from the start of the request. It is automatically deleted within an hour after this time expires and the request is completed. Currently, the maximum retention time is 2419200 seconds (28 days).
These settings can be configured in the WebApp or via the API when creating or editing a deployment or pipeline version. Changing the retention time or mode does not influence the retention of already stored requests.
Enabling request storage does not influence the response times of your deployment or pipeline, as the data is stored asynchronously. This does mean that it can take a couple of seconds for a request to become visible in the WebApp or API after it completes.
UbiOps does not retain any data of requests that have retention mode none or that are deleted, except for possible logs and metrics generated by the request, and deployment requests that are part of a pipeline request for the time the pipeline request is still in progress.
Retention mode of deployments in pipelines
The retention mode configured for deployments applies to the deployment requests that are made as part of a pipeline request as well.
The retention settings apply to both direct and batch requests. Therefore, if you want to collect the result of batch requests when they are completed, you have to configure the retention mode to full with a retention time longer than the request duration.
Request queue size limits¶
UbiOps buffers requests between request creation and processing. This greatly improves the reliability and performance of the UbiOps API. It is possible to control how many requests can be in the queue before the queue is full and new requests are blocked using the parameters of a deployment version
maximum_queue_size_batch. If the queue is full when creating a request then the UbiOps API will return an HTTP 503 response code and the request will not be created.
These settings can be configured in the WebApp or via the API when creating or editing a deployment version. Changing the maximum express or batch queue size does not influence the already created requests.
Using the queue size limits it is possible to limit the maximum number of queued requests and have a more predictable upper bound for the time it takes to process a request already before creating the request. The HTTP 503 response code can be used to know if the deployment version is 'busy' and notify an end-user directly without having to wait for a timeout. It is also possible to control the maximum number of pending batch requests to avoid having a huge backlog.
Queue size limits for pipelines
It is not possible do directly control the queue size for pipeline requests. However, if the queue of a deployment versions in a pipeline is full the pipeline request will not continue and stop processing.