Run Jupyter in a UbiOps deployment¶

When you're still experimenting with your code and it's not yet completely production ready, you might want to use Jupyter notebooks to help you iterate more quickly. It can also help you develop your code in the runtime environment of your deployments. By using the port forwarding functionality of deployments it is possible to run Jupyter directly in a deployment instance in UbiOps. This how-to will describe how to do that step by step.

To run Jupyter inside a deployment, you need to perform the following steps:

1. Create a deployment that spawns a Jupyter environment
2. Turn on the deployment
3. Open Jupyter URL in browser

Creating a deployment that spawns a Jupyter environment¶

To create a deployment that spawns a Jupyter environment, we need to put together a deployment.py, our environment specifications and then deploy it to UbiOps with the right configuration.

Creating the deployment.py¶

To spin up a Jupyter environment from Python code we can use:

import subprocess
import uuid

token = str(uuid.uuid4())
proc = subprocess.Popen(['jupyter', 'notebook', '--ip', '0.0.0.0', '--IdentityProvider.token', token], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)

This starts a Jupyter environment in the deployment. We need to obtain the IP address of the deployment to be able to access the Jupyter interface, which can be done in the following way:

# Get the IP address and print to the logs
http_request = urllib.request.urlopen("https://whatismyipv4.ubiops.com")
ip_address = http_request.read().decode("utf8")
http_request.close()

print(f"Notebook URL: http://{ip_address}:8888/tree?token={token}")

Let's put this together in a deployment.py with some error handling. The deployment will print the Jupyter link to the logs, but it will also return it in its request output for easy accessibility.

import subprocess
import urllib.request
import uuid


# This class allows us to have custom error messages when the deployment fails
class UbiOpsError(Exception):
    def __init__(self, error_message):
        super().__init__()
        self.public_error_message = error_message


class Deployment:

    def __init__(self):

        token = str(uuid.uuid4())

        try:
            self.proc = subprocess.Popen(['jupyter', 'notebook', '--ip', '0.0.0.0', '--IdentityProvider.token', token], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
        except FileNotFoundError:
            raise UbiOpsError("Unable to start a Jupyter notebook, are the packages jupyterlab and notebook installed?")

        # Get the IP address and print to the logs
        http_request = urllib.request.urlopen("https://whatismyipv4.ubiops.com")
        ip_address = http_request.read().decode("utf8")
        http_request.close()


        self.notebook_url = f"http://{ip_address}:8888/tree?token={token}"
        print(f"Notebook URL: {self.notebook_url}")

    def request(self, data):
        return {"notebook_url": self.notebook_url}

    def stop(self):
        # Stop the Jupyter environment when the deployment is shutting down
        if self.proc is not None:
            self.proc.kill()

Preparing the environment specifications¶

To be able to run the deployment.py from the previous section, we need Jupyter to be available in the deployment environment. We can ensure that with the following requirements.txt:

jupyterlab==4.0.11
notebook==7.0.7

Creating the deployment in UbiOps¶

Now that we have our files ready, we can create a deployment in UbiOps. You can use the following settings:

Once you click Next: create a version you can fill in the form with the following settings:

Any settings that aren't mentioned in the table can be left however you prefer. We set the minimum number of instances to one to ensure that an instance will start running directly after creation of the deployment version.

Click Create and your deployment will start building!

Accessing the Jupyter Environment¶

To access the environment you can either go to the logs and copy the Jupyter URL in your browser from there, or you can send a request to the deployment and copy the URL fromt he request results. Once you open the URL in your browser, you will have access to all the functionalities of Jupyter, with the compute power of your selected instance type! You can thus use the environment to start developing and iterating over your code.

To shut down the Jupyter deployment, you can set the minimum number of instances back to zero.