Working with Assets

Assets on the Harbr platform are units of data that can be managed, distributed and consumed by users.

Users with the correct permissions are able to create assets using pre-configured connectors. These connectors provide an authenticated connection to data where it is stored at source.

Assets are created using connectors that have been configured on a number of supported storage solutions

• AWS S3

• GCP GCS

• Azure Blob

• Snowflake

• BigQuery

There are two main types of asset that users can choose to create

1. On platform

2. At source

On platform asset

The creation of these assets result in data copy from their original source to storage on the platform. This allows for a larger amount of up front processing to be done so that more is understood about the data when it comes to analysis on the platform. Namely collaborative use in spaces.

At source asset

These assets reference data that stays in its source location. This type of asset is useful in scenarios where it is important that data is not moved unnecessarily from its source either at all or until it is required for collaborative analysis on the platform.

Creating an asset on the platform

To create an asset, a user must have asset creation permissions. From the manage menu, users can access the asset management page and start the creation of a new asset. During asset creation, the asset goes through a number of stages before it is ready for consumption

1. In setup - The basic details of the asset are established such as asset name, source connector, data type and whether the asset is to be at source or on platform. Access and format checks are also performed during this stage to ensure the data is accessible and details of its structure are obtained for information and access later on.

2. Draft - Assets stay in a draft state while the final details like metadata, sample data and schema are finalised. Once those processes are complete, asset managers can adjust any final details, share the asset if required and then release it to Live.

3. Live - Once an asset is live, it can be consumed by users on the platform. This means it can be used in a Space or exported. Asset managers can update the asset details at any time and release those to establish a new live version.

Automatic asset updates

On platform assets can be configured to update on a consistent schedule to enable fresh data loads for consumers. This is done by configuring an update schedule that ingests data and either replaces the existing data entirely or appends to it.