Skip to content

Set up CI/CD from Git

On this page we will show you how to push changes from Git to UbiOps automatically. More specifically, we will show you how to set up a GitLab CI/CD, a GitHub Actions or an Azure pipelines workflow that pushes your code to a new deployment version of your UbiOps deployment every time you push a commit to the main branch of your repository.

If you want to route new requests from the main deployment endpoint URL to your new version, do ensure to update the default version of your deployment to this new deployment version.

This guide will cover three steps:

  1. Create an API Token with the permissions to update a deployment.
  2. Set up a CI workflow in either GitLab, GitHub or Azure DevOps
  3. General tips and tricks that can help you create a proper workflow from development to production

Requirements

  • A UbiOps account (you can create a free trial account here using your email address or Google or Microsoft account)

  • A UbiOps deployment to which you want to push your code

  • A GitLab, GitHub or Azure DevOps repository where your code is located

Creating the API token

Start by going to Permissions in the sidebar, then navigate to the API tokens tab. Here you can click the [+]Add button to create a new API token. Give the new token a name, for instance "Git-token".

User & permissions

Now you will be prompted to add roles to the token. Click on the [+]Assign roles to user button and select deployment-editor from the dropdown menu. You can set the role level on Deployment and select the specific deployment you want to control with Git CI in the Applied to field. Lastly, copy the API token and save it in a secure spot. You will not be able to retrieve this token again.

Setting up the CI workflow

Now that we have our token, we can set up the Git CI. We will first cover GitLab, if you are interested in GitHub CI or Azure DevOps, scroll down to the next section.

GitLab

Go to the GitLab repository that contains the code you want to push to UbiOps. To make sure GitLab has the right permissions to push to UbiOps, we need to add the API token we created in the previous step as an environment variable to GitLab. You can do so by navigating to Settings, then clicking on the tab CI/CD and then going to variables. Name the variable UBIOPS_TOKEN and paste the saved token as value in the format "Token abcd123". For more information on GitLab variables see here.

Now that the token is added as a variable to your repository, we can make a gitlab-ci.yml that pushes the code from our GitLab repository to UbiOps. Below is the gitlab-ci.yml we need.

stages:
  - deploy

deploy:
  stage: deploy
  image: python:3.11
  variables:
    # Below are the parameter settings of your UbiOps deployment
    DEPLOYMENT_NAME: <your-deployment-name>
    # Version specific parameters
    ENVIRONMENT: python3-12
    # The path to your deployment package in your repository
    DIR_PATH: ./path/to/deployment-package
  before_script:
    - pip install ubiops-cli
    # Authorize the CLI using an API token 
    - ubiops signin --token -p "${UBIOPS_TOKEN}"
  script:
    # Construct the version name from the Commit SHA
    - VERSION_NAME="v-${CI_COMMIT_SHA:0:7}"
    # Deploy your new deployment package to UbiOps using the CLI
    - ubiops dpl deploy ${DEPLOYMENT_NAME} -v ${VERSION_NAME} -dir ${DIR_PATH} 
      --environment ${ENVIRONMENT}
      --labels git-user:${GITLAB_USER_LOGIN},git-commit:${CI_COMMIT_SHA}
  only:
    - main

The script deploys the deployment package in your repository as a new version of the given deployment every time new commits are pushed to the main branch. The version name is constructed automatically from the first seven characters of the Commit-SHA. We create a new deployment version using the UbiOps CLI. Inside the script part of this .yaml file you can call any function of the CLI, as long as your used token permits it.

To make this script work, you first need to configure some variables. At minimum, you need to provide your deployment name, environment and path to the deployment package in your repository. Use path ./ if your deployment file (the deployment.py) is in the root of your repository. Configure your parameters in the following lines of the script:

variables:
    # Below are the parameter settings of your UbiOps deployment
    DEPLOYMENT_NAME: <your-deployment-name>
    ENVIRONMENT: python3-12
    # The path to your deployment package in your repository
    DIR_PATH: ./path/to/deployment-package

If you want, you can also specify additional parameters, such as the instance type group and the minimum and maximum number of instances that you want to have running. For more information on that, please see scaling and resource allocation.

Once you have added the .gitlab_ci.yml, your CI workflow is all set up!

GitHub

Go to the GitHub repository that contains the code you want to push to UbiOps. To make sure GitHub can communicate with UbiOps we need to add the API token we created in step one as a repository secret. You can do so by navigating to Settings, then to Secrets and lastly click the button New repository secret. Name the variable UBIOPS_TOKEN and paste the saved token as value in the format "Token abcd123" and click Add secret. For more information on repository secrets in GitHub see here.

Now that the token is added as a secret to your repository, we can make a new GitHub action to push code from our GitHub repository to UbiOps. Go to Actions and click create a new workflow. You will be redirected to .github/workflows/name-of-your-workflow.yml. Below is the .yml we need.

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      # Below are the parameter settings of your UbiOps deployment
      DEPLOYMENT_NAME: <your-deployment-name>
      UBIOPS_TOKEN: ${{ secrets.UBIOPS_TOKEN }}
      # Version specific parameters
      ENVIRONMENT: python3-12
      # The path to your deployment package in your repository
      DIR_PATH: ./path/to/deployment-package

    steps:
      - uses: actions/checkout@v2

      - name: Set up Python 3.11
        uses: actions/setup-python@v2
        with:
          python-version: 3.11

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install ubiops-cli

      # Authorize the CLI using an API token
      - name: Sign in
        run: ubiops signin --token -p "$UBIOPS_TOKEN"

      # Deploy your new deployment package to UbiOps using the CLI
      - name: Deploy
        run: |
          VERSION_NAME="v-${GITHUB_SHA:0:7}"
          ubiops dpl deploy "$DEPLOYMENT_NAME" -v "$VERSION_NAME" -dir "${DIR_PATH}" \
          --environment "$ENVIRONMENT"  \
          --labels git-user:"$GITHUB_ACTOR",git-commit:"$GITHUB_SHA"

The script deploys the deployment package in your repository as a new version of the given deployment every time new commits are pushed to the main branch. The version name is constructed automatically from the first seven characters of the Commit-SHA. We create a new deployment version using the UbiOps CLI. Inside the script part of this .yaml file you can call any function of the CLI, as long as your used token permits it.

To make this script work, you first need to configure some variables. At minimum, you need to provide your deployment name, environment and path to the deployment package in your repository. Use path ./ if your deployment file (the deployment.py) is in the root of your repository. Configure your parameters in the following lines of the script:

 env:
      # Below are the parameter settings of your UbiOps deployment
      DEPLOYMENT_NAME: <your-deployment-name>
      UBIOPS_TOKEN: ${{ secrets.UBIOPS_TOKEN }}
      # Version specific parameters
      ENVIRONMENT: python3-12
      # The path to your deployment package in your repository
      DIR_PATH: ./path/to/deployment-package

If you want, you can also specify additional parameters, such as the instance type group and the minimum and maximum number of instances that you want to have running. For more information on that, please see scaling and resource allocation.

When you've added the .yml file and configured your parameters you can click Start commit to add the new action yaml to your repository. Your GitHub CI workflow is now all set up!

Azure DevOps

For Azure DevOps, the process to configure CI/CD can be done through creating a azure-pipelines.yml file in your repository and creating a pipeline in Azure DevOps, which links to the azure-pipelines.yml file.

The process of setting up CI/CD on Azure DevOps involves two key steps:

  1. Configure the azure-pipelines.yml file
  2. Create a pipeline in Azure DevOps.

Configuring the Azure DevOps Pipeline

Create an azure-pipelines.yml at the root of your repository. A template azure-pipelines.yml file is provided below.

trigger:
- main

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.11'
  displayName: 'Set up Python 3.11'

- script: |
    python -m pip install --upgrade pip
    pip install ubiops-cli
  displayName: 'Install dependencies'

- script: |
    ubiops signin --token -p "$(UBIOPS_TOKEN)"
  displayName: 'Authorize UbiOps CLI'
  env:
    UBIOPS_TOKEN: $(UBIOPS_TOKEN)

- script: |
    VERSION_NAME="v-${BUILD_SOURCEVERSION:0:7}"
    ubiops dpl deploy "$(DEPLOYMENT_NAME)" -v "$(VERSION_NAME)" -dir "$(DIR_PATH)" --environment "$(ENVIRONMENT)"  --labels git-user:$(Build.Repository.Name),git-commit:$(BUILD_SOURCEVERSION)
  displayName: 'Deploy to UbiOps'
  env:
    DEPLOYMENT_NAME: <your-deployment-name>
    UBIOPS_TOKEN: $(UBIOPS_TOKEN)
    ENVIRONMENT: python3-12
    DIR_PATH: ./path/to/deployment-package
    BUILD_SOURCEVERSION: $(Build.SourceVersion)

The script deploys the deployment package in your repository as a new version of the given deployment every time new commits are pushed to the main branch. The version name is constructed automatically from the first seven characters of the Commit-SHA. We create a new deployment version using the UbiOps CLI. Inside the script part of this .yaml file you can call any function of the CLI, as long as your used token permits it.

To make this script work, you first need to configure some variables. At minimum, you need to provide your deployment name, environment and path to the deployment package in your repository. Use path ./ if your deployment file (the deployment.py) is in the root of your repository.

If you want, you can also specify additional parameters, such as the instance type group and the minimum and maximum number of instances that you want to have running. For more information on that, please see scaling and resource allocation.

Create a pipeline in Azure DevOps

To create a pipeline in Azure DevOps, do the following steps:

  1. Go to your Azure DevOps project and click on Pipelines in the left menu
  2. Click on New pipeline
  3. Select Use the classic editor and configure your repository
  4. Select YAML and click on Apply
  5. Set the name, agent pool and specify the path to your azure-pipelines.yml file
  6. Go to the variables tab and add a variable UBIOPS_TOKEN with your UbiOps API token (as a secret variable)
  7. Click on Save&queue and then on Save&run to save and run your pipeline

Your Azure DevOps CI workflow is now all set up! You can now push new commits to your repository and see the pipeline running. If you want, you can also specify additional parameters, like memory allocation and maximum number of instances. For more information on that, please see scaling and resource allocation.

Tips and Tricks

We hope you have been able to deploy your first deployment version via GIT. The provided scripts and workflow serves as a baseline example. You can tailor it to your requirements.

  1. A development/production workflow can be set-up by working on different branches and using conditional statements and different API Tokens to deploy your new versions to different projects or deployments.
  2. In the provided scripts, we only made use of one ubiops dpl deploy step. You can always include more steps, to include unit tests inside or to add environment variables to your deployment in the same process. Check out our CLI documentation for different options.
  3. In the provided scripts, we opted to set deployment-specific variables as environment variables within the CI scripts. However, using a dedicated configuration.yaml file can help structure your repository better by storing all deployment specifications into a single file, leaving only CI-related steps within the CI scripts. You can then for example declare an environment variable DEPLOYMENT CONFIG that points to this file, and update your ubiops dpl deploy command to ubiops dpl deploy -f ${DEPLOYMENT_CONFIG} -dir ${CPU_DIR_PATH} --labels git-user:${GITLAB_USER_LOGIN},git-commit:${CI_COMMIT_SHA}. See the relevant documentation of the ubiops dpl deploy command for more information on how to set that up. See an example configuration.yaml and .gitlab-ci.yml below.
  4. You may want to add a .ubiops-ignore file in the deployment package directory to exclude files from pushing to UbiOps. It works similar as the well-known .gitignore file.
  5. Always feel free to reach out to UbiOps Support to discuss and optimize your deployment process with our Support team!
Click here for an example `configuration.yaml` 
deployment_name: <your-deployment-name>
version_name: <your-deployment-version-name>
version_description: This deployment version was deployed via a CI script!
environment: python3-12
minimum_instances: 0 
maximum_instances: 1
maximum_idle_time: 300
request_retention_mode: full
request_retention_time: 31536000
maximum_queue_size_express: 100
maximum_queue_size_batch: 100000
static_ip: false
Click here for an example `.gitlab-ci.yml` that makes use of a `configuration.yaml` 
stages:
  - deploy

deploy:
  stage: deploy
  image: python:3.11
  variables:
    # Below are the parameter settings of your UbiOps deployment
    CONFIG_FILE_PATH: ./path/to/config/file.yaml
    # The path to your deployment package in your repository
    DIR_PATH: ./path/to/deployment-package
  before_script:
    - pip install ubiops-cli
    # Authorize the CLI using an API token 
    - ubiops signin --token -p "${UBIOPS_TOKEN}"
  script:
    # Deploy your new deployment package to UbiOps using the CLI
    - ubiops dpl deploy -f ${CONFIG_FILE_PATH} -dir ${DIR_PATH} \
      --labels git-user:${GITLAB_USER_LOGIN},git-commit:${CI_COMMIT_SHA}
  only:
    - main