Set up CI/CD from Git¶
On this page we will show you how to push changes from Git to UbiOps automatically. More specifically, we will show you how to set up a GitLab CI/CD or a GitHub Actions workflow that pushes your code to your UbiOps deployment every time you push a commit to the main branch of your repository.
This guide will cover two steps:
- Create an API Token with the permissions to update a deployment.
- Set up CI workflow in either GitLab, GitHub or Azure DevOps
Requirements¶
-
A web browser
-
A UbiOps account (you can create a free trial account here using your email address or Google or Microsoft account)
-
A UbiOps deployment to which you want to push your code
-
A GitLab, GitHub or AzureDevOps repository where your code is located
-
5 minutes of your time
Creating the API token¶
Start by going to Permissions in the sidebar, then navigate to the API tokens tab. Here you can click the [+]Add button to create a new API token. Give the new token a name, for instance "Git-token".
Now you will be prompted to add roles to the token. Click on the [+]Assign roles to user button and select deployment-editor from the drop down menu. You can set the role level on Deployment and select the specific deployment you want to control with Git CI in the Applied to field. Lastly, copy the API token and save it in a secure spot. You will not be able to retrieve this token again.
Setting up CI workflow¶
Now that we have our token, we can set up the Git CI. We will first cover GitLab, if you are interested in GitHub CI scroll down to the next section.
GitLab¶
Go to the GitLab repository that contains the code you want to push to UbiOps. To make sure GitLab has the right permissions to push to UbiOps, we need to add the API token we created in the previous step as an environment variable to GitLab. You can do so by navigating to Settings, then clicking on the tab CI/CD and then going to variables. Name the variable UBIOPS_TOKEN and paste the saved token as value in the format "Token abcd123". For more information on GitLab variables see here.
Now that the token is added as a variable to your repository, we can make a gitlab-ci.yml that pushes the code from our GitLab repository to UbiOps. Below is the gitlab-ci.yml we need.
stages:
- deploy
deploy:
stage: deploy
image: python:3.8
variables:
# Below are the parameter settings of your UbiOps deployment
DEPLOYMENT_NAME: <your-deployment-name>
VERSION_NAME: <your-version>
# Version specific parameters
ENVIRONMENT: python3-8
# The path to your deployment package in your repository
DIR_PATH: ./path/to/deployment-package
before_script:
- pip install ubiops-cli
# Authorize the CLI using an API token
- ubiops signin --token -p "${UBIOPS_TOKEN}"
script:
# Deploy your new deployment package to UbiOps using the CLI
- ubiops dpl deploy ${DEPLOYMENT_NAME} -v ${VERSION_NAME} -dir ${DIR_PATH}
--environment ${ENVIRONMENT} -y --overwrite
--labels git-user:${GITLAB_USER_LOGIN},git-commit:${CI_COMMIT_SHA}
only:
- main
This script deploys the deployment package in your repository as a new revision of the given deployment version every time new commits are pushed to the main branch. If there is no version yet, a new version with VERSION_NAME
will be created. We do so using the UbiOps CLI. Inside the script
part of this yaml file you can call any function of the CLI, as long as your used token permits it.
To make this script work, you first need to configure some variables. At minimum you need to provide your deployment name, version, environment and path to the deployment package in your repository. Use path ./
if your deployment file (deployment.py for Python) is in the root of your repository. You may want to add a .ubiops-ignore
file in the deployment package directory to exclude files from pushing to UbiOps. It works similar as the well-known .gitignore
file. Configure your parameters in the following lines of the script:
variables:
# Below are the parameter settings of your UbiOps deployment
DEPLOYMENT_NAME: <your-deployment-name>
VERSION_NAME: <your-version>
# Version specific parameters
ENVIRONMENT: python3-8
# The path to your deployment package in your repository
DIR_PATH: ./path/to/deployment-package
If you want, you can also specify additional parameters, like memory allocation and maximum number of instances. For more information on that, please see scaling and resource allocation.
Once you have added the gitlab_ci.yml your CI workflow is all set up!
GitHub¶
Go to the GitHub repository that contains the code you want to push to UbiOps. To make sure GitHub can communicate with UbiOps we need to add the API token we created in step one as a repository secret. You can do so by navigating to Settings, then to Secrets and lastly click the button New repository secret. Name the variable UBIOPS_TOKEN and paste the saved token as value in the format "Token abcd123" and click Add secret. For more information on repository secrets in GitHub see here.
Now that the token is added as a secret to your repository, we can make a new GitHub action to push code from our GitHub repository to UbiOps. Go to Actions and click create a new workflow. You will be redirected to .github/workflows/name-of-your-workflow.yml
. Below is the .yml we need.
name: CI
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
deploy:
runs-on: ubuntu-latest
env:
# Below are the parameter settings of your UbiOps deployment
DEPLOYMENT_NAME: <your-deployment-name>
VERSION_NAME: <your-version>
UBIOPS_TOKEN: ${{ secrets.UBIOPS_TOKEN }}
# Version specific parameters
ENVIRONMENT: python3-8
# The path to your deployment package in your repository
DIR_PATH: ./path/to/deployment-package
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
uses: actions/setup-python@v2
with:
python-version: 3.8
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install ubiops-cli
# Authorize the CLI using an API token
- name: Sign in
run: ubiops signin --token -p "$UBIOPS_TOKEN"
# Deploy your new deployment package to UbiOps using the CLI
- name: Deploy
run: |
ubiops dpl deploy "$DEPLOYMENT_NAME" -v "$VERSION_NAME" -dir "${DIR_PATH}" \
--environment "$ENVIRONMENT" -y --overwrite \
--labels git-user:"$GITHUB_ACTOR",git-commit:"$GITHUB_SHA"
This script deploys the deployment package in your repository as a new revision of the given deployment version every time new commits are pushed to the main branch. If there is no version yet, a new version with VERSION_NAME
will be created. We do so using the UbiOps CLI. Inside the script
part of this yaml file you can call any function of the CLI, as long as your used token permits it.
To make this script work, you first need to configure some variables. At minimum you need to provide your deployment name, version, environment and path to the deployment package in your repository. Use path ./
if your deployment file (deployment.py for Python) is in the root of your repository. You may want to add a .ubiops-ignore
file in the deployment package directory to exclude files from pushing to UbiOps. It works similar as the well-known .gitignore
file. Configure your parameters in the following lines of the script:
env:
# Below are the parameter settings of your UbiOps deployment
DEPLOYMENT_NAME: <your-deployment-name>
VERSION_NAME: <your-version>
UBIOPS_TOKEN: ${{ secrets.UBIOPS_TOKEN }}
# Version specific parameters
ENVIRONMENT: python3-8
# The path to your deployment package in your repository
DIR_PATH: ./path/to/deployment-package
If you want, you can also specify additional parameters, like memory allocation and maximum number of instances. For more information on that, please see scaling and resource allocation.
When you've added the .yml
file and configured your parameters you can click Start commit to add the new action yaml to your repository. Your GitHub CI workflow is now all set up!
Azure DevOps¶
For Azure DevOps, the process to configure CI/CD can be done through creating a azure-pipelines.yml
file in your repository and creating a pipeline in Azure DevOps, which links to the azure-pipelines.yml
file.
The process of setting up CI/CD on Azure DevOps involves two key steps:
- Configure the
azure-pipelines.yml
file - Create a pipeline in Azure DevOps.
Configuring the Azure DevOps Pipeline¶
Create an azure-pipelines.yml
at the root of your repository. A template azure-pipelines.yml
file is provided below.
trigger:
- main
pool:
vmImage: 'ubuntu-latest'
steps:
- task: UsePythonVersion@0
inputs:
versionSpec: '3.8'
- script: |
python -m pip install --upgrade pip
pip install ubiops-cli
displayName: 'Install dependencies'
- script: |
ubiops signin --token -p "$(UBIOPS_TOKEN)"
displayName: 'Authorize UbiOps CLI'
env:
UBIOPS_TOKEN: $(UBIOPS_TOKEN)
- script: |
ubiops dpl deploy "$(DEPLOYMENT_NAME)" -v "$(VERSION_NAME)" -dir "$(DIR_PATH)" --environment "$(ENVIRONMENT)" -y --overwrite --labels git-user:$(Build.Repository.Name),git-commit:$(Build.SourceVersion)
displayName: 'Deploy to UbiOps'
env:
DEPLOYMENT_NAME: <your-deployment-name>
VERSION_NAME: <your-version>
UBIOPS_TOKEN: $(UBIOPS_TOKEN)
ENVIRONMENT: python3-10
DIR_PATH: ./path/to/deployment-package
The script deploys the deployment package in your repository as a new revision of the given deployment version every time new commits are pushed to the main branch. If there is no version yet, a new version with VERSION_NAME
will be created. We do so using the UbiOps CLI. Inside the script part of this yaml file you can call any function of the CLI, as long as your used token permits it.
Replace <your-deployment-name>
, <your-version>
, and ./path/to/deployment-package
with the appropriate values for your deployment. If your deployment file (deployment.py
for Python) is in the root of your repository, use path ./
.
Create a pipeline in Azure DevOps¶
To create a pipeline in Azure DevOps, do the following steps:
- Go to your Azure DevOps project and click on Pipelines in the left menu
- Click on New pipeline
- Select
Use the classic editor
and configure your repository - Select
YAML
and click on Apply - Set the name, agent pool and specify the path to your
azure-pipelines.yml
file - Go to the variables tab and add a variable
UBIOPS_TOKEN
with your UbiOps API token (as a secret variable) - Click on Save&queue and then on Save&run to save and run your pipeline
Your Azure DevOps CI workflow is now all set up! You can now push new commits to your repository and see the pipeline running. If you want, you can also specify additional parameters, like memory allocation and maximum number of instances. For more information on that, please see scaling and resource allocation.