Skip to content

Set up CI/CD from Git

On this page we will show you how to push changes from Git to UbiOps automatically. More specifically, we will show you how to set up a GitLab CI/CD or a GitHub Actions workflow that pushes your code to your UbiOps deployment every time you push a commit to the main branch of your repository.

This guide will cover two steps:

  1. Create an API Token with the permissions to update a deployment.
  2. Set up CI workflow in either GitLab, GitHub or Azure DevOps

Requirements

  • A web browser

  • A UbiOps account (you can create a free trial account here using your email address or Google or Microsoft account)

  • A UbiOps deployment to which you want to push your code

  • A GitLab, GitHub or AzureDevOps repository where your code is located

  • 5 minutes of your time

Creating the API token

Start by going to Permissions in the sidebar, then navigate to the API tokens tab. Here you can click the [+]Add button to create a new API token. Give the new token a name, for instance "Git-token".

User & permissions

Now you will be prompted to add roles to the token. Click on the [+]Assign roles to user button and select deployment-editor from the drop down menu. You can set the role level on Deployment and select the specific deployment you want to control with Git CI in the Applied to field. Lastly, copy the API token and save it in a secure spot. You will not be able to retrieve this token again.

Setting up CI workflow

Now that we have our token, we can set up the Git CI. We will first cover GitLab, if you are interested in GitHub CI scroll down to the next section.

GitLab

Go to the GitLab repository that contains the code you want to push to UbiOps. To make sure GitLab has the right permissions to push to UbiOps, we need to add the API token we created in the previous step as an environment variable to GitLab. You can do so by navigating to Settings, then clicking on the tab CI/CD and then going to variables. Name the variable UBIOPS_TOKEN and paste the saved token as value in the format "Token abcd123". For more information on GitLab variables see here.

Now that the token is added as a variable to your repository, we can make a gitlab-ci.yml that pushes the code from our GitLab repository to UbiOps. Below is the gitlab-ci.yml we need.

stages:
  - deploy

deploy:
  stage: deploy
  image: python:3.8
  variables:
    # Below are the parameter settings of your UbiOps deployment
    DEPLOYMENT_NAME: <your-deployment-name>
    VERSION_NAME: <your-version>
    # Version specific parameters
    ENVIRONMENT: python3-8
    # The path to your deployment package in your repository
    DIR_PATH: ./path/to/deployment-package
  before_script:
    - pip install ubiops-cli
    # Authorize the CLI using an API token 
    - ubiops signin --token -p "${UBIOPS_TOKEN}"
  script:
    # Deploy your new deployment package to UbiOps using the CLI
    - ubiops dpl deploy ${DEPLOYMENT_NAME} -v ${VERSION_NAME} -dir ${DIR_PATH} 
      --environment ${ENVIRONMENT} -y --overwrite 
      --labels git-user:${GITLAB_USER_LOGIN},git-commit:${CI_COMMIT_SHA}
  only:
    - main

This script deploys the deployment package in your repository as a new revision of the given deployment version every time new commits are pushed to the main branch. If there is no version yet, a new version with VERSION_NAME will be created. We do so using the UbiOps CLI. Inside the script part of this yaml file you can call any function of the CLI, as long as your used token permits it.

To make this script work, you first need to configure some variables. At minimum you need to provide your deployment name, version, environment and path to the deployment package in your repository. Use path ./ if your deployment file (deployment.py for Python) is in the root of your repository. You may want to add a .ubiops-ignore file in the deployment package directory to exclude files from pushing to UbiOps. It works similar as the well-known .gitignore file. Configure your parameters in the following lines of the script:

variables:
    # Below are the parameter settings of your UbiOps deployment
    DEPLOYMENT_NAME: <your-deployment-name>
    VERSION_NAME: <your-version>
    # Version specific parameters
    ENVIRONMENT: python3-8
    # The path to your deployment package in your repository
    DIR_PATH: ./path/to/deployment-package

If you want, you can also specify additional parameters, like memory allocation and maximum number of instances. For more information on that, please see scaling and resource allocation.

Once you have added the gitlab_ci.yml your CI workflow is all set up!

GitHub

Go to the GitHub repository that contains the code you want to push to UbiOps. To make sure GitHub can communicate with UbiOps we need to add the API token we created in step one as a repository secret. You can do so by navigating to Settings, then to Secrets and lastly click the button New repository secret. Name the variable UBIOPS_TOKEN and paste the saved token as value in the format "Token abcd123" and click Add secret. For more information on repository secrets in GitHub see here.

Now that the token is added as a secret to your repository, we can make a new GitHub action to push code from our GitHub repository to UbiOps. Go to Actions and click create a new workflow. You will be redirected to .github/workflows/name-of-your-workflow.yml. Below is the .yml we need.

name: CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    env:
      # Below are the parameter settings of your UbiOps deployment
      DEPLOYMENT_NAME: <your-deployment-name>
      VERSION_NAME: <your-version>
      UBIOPS_TOKEN: ${{ secrets.UBIOPS_TOKEN }}
      # Version specific parameters
      ENVIRONMENT: python3-8
      # The path to your deployment package in your repository
      DIR_PATH: ./path/to/deployment-package

    steps:
      - uses: actions/checkout@v2

      - name: Set up Python 3.8
        uses: actions/setup-python@v2
        with:
          python-version: 3.8

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install ubiops-cli

      # Authorize the CLI using an API token
      - name: Sign in
        run: ubiops signin --token -p "$UBIOPS_TOKEN"

      # Deploy your new deployment package to UbiOps using the CLI
      - name: Deploy
        run: |
          ubiops dpl deploy "$DEPLOYMENT_NAME" -v "$VERSION_NAME" -dir "${DIR_PATH}" \
          --environment "$ENVIRONMENT" -y --overwrite \
          --labels git-user:"$GITHUB_ACTOR",git-commit:"$GITHUB_SHA"

This script deploys the deployment package in your repository as a new revision of the given deployment version every time new commits are pushed to the main branch. If there is no version yet, a new version with VERSION_NAME will be created. We do so using the UbiOps CLI. Inside the script part of this yaml file you can call any function of the CLI, as long as your used token permits it.

To make this script work, you first need to configure some variables. At minimum you need to provide your deployment name, version, environment and path to the deployment package in your repository. Use path ./ if your deployment file (deployment.py for Python) is in the root of your repository. You may want to add a .ubiops-ignore file in the deployment package directory to exclude files from pushing to UbiOps. It works similar as the well-known .gitignore file. Configure your parameters in the following lines of the script:

 env:
      # Below are the parameter settings of your UbiOps deployment
      DEPLOYMENT_NAME: <your-deployment-name>
      VERSION_NAME: <your-version>
      UBIOPS_TOKEN: ${{ secrets.UBIOPS_TOKEN }}
      # Version specific parameters
      ENVIRONMENT: python3-8
      # The path to your deployment package in your repository
      DIR_PATH: ./path/to/deployment-package

If you want, you can also specify additional parameters, like memory allocation and maximum number of instances. For more information on that, please see scaling and resource allocation.

When you've added the .yml file and configured your parameters you can click Start commit to add the new action yaml to your repository. Your GitHub CI workflow is now all set up!

Azure DevOps

For Azure DevOps, the process to configure CI/CD can be done through creating a azure-pipelines.yml file in your repository and creating a pipeline in Azure DevOps, which links to the azure-pipelines.yml file.

The process of setting up CI/CD on Azure DevOps involves two key steps:

  1. Configure the azure-pipelines.yml file
  2. Create a pipeline in Azure DevOps.

Configuring the Azure DevOps Pipeline

Create an azure-pipelines.yml at the root of your repository. A template azure-pipelines.yml file is provided below.

trigger:
- main

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: UsePythonVersion@0
  inputs:
    versionSpec: '3.8'

- script: |
    python -m pip install --upgrade pip
    pip install ubiops-cli
  displayName: 'Install dependencies'

- script: |
    ubiops signin --token -p "$(UBIOPS_TOKEN)"
  displayName: 'Authorize UbiOps CLI'
  env:
    UBIOPS_TOKEN: $(UBIOPS_TOKEN)

- script: |
    ubiops dpl deploy "$(DEPLOYMENT_NAME)" -v "$(VERSION_NAME)" -dir "$(DIR_PATH)" --environment "$(ENVIRONMENT)" -y --overwrite --labels git-user:$(Build.Repository.Name),git-commit:$(Build.SourceVersion)
  displayName: 'Deploy to UbiOps'
  env:
    DEPLOYMENT_NAME: <your-deployment-name>
    VERSION_NAME: <your-version>
    UBIOPS_TOKEN: $(UBIOPS_TOKEN)
    ENVIRONMENT: python3-10
    DIR_PATH: ./path/to/deployment-package

The script deploys the deployment package in your repository as a new revision of the given deployment version every time new commits are pushed to the main branch. If there is no version yet, a new version with VERSION_NAME will be created. We do so using the UbiOps CLI. Inside the script part of this yaml file you can call any function of the CLI, as long as your used token permits it.
Replace <your-deployment-name>, <your-version>, and ./path/to/deployment-package with the appropriate values for your deployment. If your deployment file (deployment.py for Python) is in the root of your repository, use path ./.

Create a pipeline in Azure DevOps

To create a pipeline in Azure DevOps, do the following steps:

  1. Go to your Azure DevOps project and click on Pipelines in the left menu
  2. Click on New pipeline
  3. Select Use the classic editor and configure your repository
  4. Select YAML and click on Apply
  5. Set the name, agent pool and specify the path to your azure-pipelines.yml file
  6. Go to the variables tab and add a variable UBIOPS_TOKEN with your UbiOps API token (as a secret variable)
  7. Click on Save&queue and then on Save&run to save and run your pipeline

Your Azure DevOps CI workflow is now all set up! You can now push new commits to your repository and see the pipeline running. If you want, you can also specify additional parameters, like memory allocation and maximum number of instances. For more information on that, please see scaling and resource allocation.