Connectors can be used to insert the output data from your Models to your own databases or other storage solutions. To do this, you can use a pipeline in which a model is connected to the connector.
- Connectors for structured data insert each data point as a row in a table or similar structure.
- Connectors of blob data insert each entry as a single object into the storage solution. We define a blob as any binary file.
|Amazon Web Services S3||Blob|
|Azure Blob Storage||Blob|
|Google Cloud Storage||Blob|
|Google Cloud BigQuery||Structured|
Structured/Blob vs. Structured/Plain
In our models we speak of structured and plain models, but for connectors we define structured and blob models. What is the difference exactly?
Essentially a structured model can output multiple fields, of which one (or more) can be of
blob (file) type. Following this, a structured model can be linked to both a structured as blob connector: When attaching the connectors in a pipeline, you are asked to provide a mapping of the fields. This allows you to map the fields of the model to the structured connector. Moreover, the field of type
blob of the structured model can be linked to the field
blob of the blob connector.
Before connecting UbiOps to your storage, make sure that:
- Your storage has the correct structure to store the data (such as a database and table).
- You have credentials and access rights are setup correctly.
- Your firewall is configured to allow access from UbiOps.
IP address of connectors
UbiOps connectors will query through a proxy with the following IPv4 address: 220.127.116.11. Make sure your firewall allows access to your storage from this IP address.
In the left navigation bar, click on Connectors, on the connectors page then click CREATE.
Each connector requires the following fields to be set:
name: name of the connector.
type: type of the data store to connect to.
credentials: access credentials for the data store.
configuration: where in the data store the data should be written, for example, the table name.
fields: how the data store is organized (required only for structured connectors).
See Connector specific instructions for detailed instructions on each type of connector that UbiOps supports.
The connectors authenticate with external systems using Credentials. UbiOps provides the ability to reuse existing credentials when creating new connectors. This way your login details only have to be entered once, and only one set of credentials need to be changed if something about the settings needs to be updated.
You can create these credentials during the creation of a Connector. Alternatively you can create credentials from the credentials overview page. You can go to this page by first clicking on Connectors on the left sidebar menu and then clicking on Credentials. On this page you can view, edit and create credentials. These credentials are matched to connector types.
See Connector specific instructions for detailed instructions on each type of credentials that UbiOps supports.