Skip to content

Introduction to Services

Services let you expose your deployments through custom HTTP endpoints. Unlike the standard deployment endpoints that follow the UbiOps API structure, services enable you to send direct HTTP requests to your deployments.

This is particularly useful when combined with our bring your own Docker feature, as it allows you to deploy any server-based application — such as VLLM, Ollama, or custom servers built with Flask, FastAPI, and others — and expose it directly via a service.

Services integrate with other UbiOps features, such as request logging, the permission system and monitoring. It provides you the flexibility of running custom APIs and applications, while maintaining control with observability and access management.

You can create and manage services using the UbiOps WebApp, Python Client Library, and Command Line Interface (CLI).

How Services Work

A service establishes an HTTP connection to your deployment on a configured port. Your deployment code can run any webserver or API that provides content on this port.

The service is exposed via a dedicated subdomain. This URL provides direct access to your application without going through UbiOps' standard request/response API structure. Externally the service is always reachable on HTTPS port 443, independent of what port the deployment uses internally. A TLS certificate is automatically provisioned.

The exact URL endpoint will be displayed in the WebApp and can be retrieved via the API or CLI by querying the service details. It will be in the following format, in this example with the URL of UbiOps Cloud:

https://[service-id].services.ubiops.com

Requests are automatically load balanced between replicas of your deployment. You are able to configure health checks that UbiOps can use to determine whether replicas are ready and should be included in the load balancing.

Creating a Service

A service connects to a specific deployment version and upon creation, requires at least the following parameters:

  • Name: A descriptive name for your service.
  • Deployment: The deployment to connect to.
  • Port: The port number (1025-65535) your application listens on.

Note that the port must match the port your application is configured to use within the deployment.

Optionally, you can also specify the specific deployment version to use:

  • Version: The specific deployment version to use, if not specified, the default version of the deployment will be used.

Other optional configurations are discussed on separate pages:

Best Practices

  1. Port Management: Ensure your application code uses the same port configured in the service, to avoid connectivity issues
  2. Health Endpoints: Implement health check endpoints in your application for monitoring service status
  3. Security: Choose appropriate authentication methods for your use case and secure your applications accordingly
  4. Rate Limiting: Implement rate limiting to protect against abuse and ensure fair usage