Create a Connector

Integrating your data pipelines via platform connectors allows you to add data to the platform and take data off the platform for use within your own environments. Connectors form the bridge between your organization’s data systems and the Harbr platform, enabling secure, controlled data movement.

It’s often best to set up your cloud connectors early in your platform onboarding, as this process may require assistance from your organization’s technical teams. Once configured, connectors make it possible for users to create assets and products that rely on live, managed, or imported data.

How Connectors Work

Connectors use secure credentials to reach out to a specified source location, receive data from that source, and make it available on the Harbr data plane for use within the platform.

Depending on the configuration, data may either be:

• On-platform: copied into Harbr-managed storage, or

• At-source: left in its original location and accessed remotely.

Both approaches make data accessible and usable through Harbr; the difference lies in where the data physically resides and how it’s accessed and controlled.

As an early access feature, some platforms may have Databricks' Lakehouse Federation available, whereby users can query data across multiple external sources, like other databases and data warehouses, without needing to move the data into the Databricks lakehouse.

Further information on how different connector types function on the platform is contained below.

Connector Types

While availability varies by environment, the following connector types are currently supported on the Harbr platform.

On-Platform Connectors

Definition:
Data is copied from the source into Harbr-managed storage, allowing full control and visibility from within the platform.

Control:
The Harbr platform manages the catalog and indexing, ensuring consistent access across features like Query, Spaces, and Data Shares.

Security:
Credentials are managed securely by the platform.

Use Case:
Ideal for sharing, collaboration, or analytical scenarios where having a managed copy of data improves performance, accessibility, and governance.

At-Source Connectors

Definition:
Data remains in its original location, with Harbr securely managing the connection and credentials required to read it.

Control:
Operates through isolated infrastructure (such as Trino jobs and clusters) that query the data without copying it.

Security:
Credentials are managed exclusively by Harbr, ensuring they remain secret and inaccessible to users.

Use Case:
Ideal for high-security or compliance-sensitive scenarios where data must remain in place and only be accessed on demand.

Behavioral Notes:
At-source assets differ slightly from on-platform assets in how certain metadata is handled:

• Data Dictionary: Loaded dynamically from the source’s current schema; automatically reflects changes such as new columns.

• Sample Data: Displays the first rows in the source table rather than a random sample.

• Metadata Size: Estimated from the external table rather than scanned in full.

Federated Connectors (Early Access)

In selected environments, Harbr supports Databricks Lakehouse Federation as an early access feature. Federated connectors enable users to query data across multiple external systems - such as other databases or data warehouses, without moving the data into the Databricks lakehouse.

This capability creates a unified view and governance layer across distributed data sources by establishing read-only “foreign catalogs” that mirror external databases, managed via Unity Catalog.

Federated connectors support the creation and consumption of assets that stay at their original source, but with central visibility and access through Harbr.

Supported Connectors

While availability will vary based on your specific environment, the following connectors are currently supported by the Harbr platform: