Snowflake

Snowflake Connectors are secure gateways through which data can be moved in and out of the Platform. Rather than a storage connector, that connects to a GCP or S3 bucket for example, a Snowflake connector is referred to as a database connector.

By connecting to your Snowflake instances, you may access tables and views within your Snowflake account and to then create them as Data Assets. In doing so, you are defining the location of the data that you reference from the platform.

Harbr supports Snowflake both as source for creating assets (requires only read access) or as a destination for exports (requires write access). However, if a Snowflake reader account is used to setup the connector, it will not be possible to use it for export.

Pre-requisites

To connect to a data source in Snowflake, you need:

1. The ability to manage the Snowflake account containing your source data via the Snowflake management console

2. Account URL.

3. Username.

4. Password.

Create the Connector

1. Click Manage on the Navigation bar.

2. Select Connectors to view the Manage Connectors screen

3. Click the Create Connector button at the top right

4. Enter a Name for your Connector and a Description (optional)

5. Choose Type > Snowflake .

6. Insert your Snowflake details

   • Account URL

   • Username

   • Pasword

7. Add any Integration Metadata needed for programmatic integration.

8. Click Create.

9. Click Close

Note: Use the Snowflake web interface (Snowpipe) or other supported Snowflake ETL software to integrate data using your connector. A direct connection to Snowflake is coming soon.

Snowflake Data Assets

A Data Asset can be added to a Space when you specify what you need, just like a product. Snowflake Data Assets are remote which means that the data within the asset is queried in it’s remote location and is not transferred to the platform.

Once an asset has been added to a space, you can query it. Current naming conventions to access an asset is:

• Asset Name: Mortgage Portfolio

• Asset Source: Snowflake

• Hue Catalog Name : Mortgage Portfolio

• Trino Catalog Name: snwfl_mortgage_portfolio

select * from <trino_catalog_name>.<schema>.<table> limit 100;

where

<trino_catalogue_name> is the technical 'Trino Catalog Name' name shown above

<schema> is the "schema name" from the source system (e.g. Snowflake)

<table_name> is the name of a table in the schema.

Note: 

• Use show catalogs command to get a list of catalogs and their names 

• Do use <catalog name>; show schemas; commands to get a list of the schemas within a catalogQuery performance may be impacted if:

• full volume table results are returned (>10 million records).

• a query joins a Data Asset and data within a product.

• the configuration of the Snowflake warehouse referenced by the Data Asset is sub-optimal.