Automation Best Practices

On the platform you can easily update data your assets when your source data is refreshed through the use of a transfer notification file, or TNF. You can also assign data products or assets to be exported whenever they are updated, on an event (refresh) or schedule basis (for example once a week).

Tasks allow you to turn what once were static assets into repeating engineered assets. This feature allows you to automate their Spaces code and dynamically create engineered data assets automatically. Automated Data Assets can update on a schedule, be triggered by an update of another Data Product/Asset, or be run manually. Using code assets and Tasks saves you countless hours of updating their engineered data assets as well as opening up new opportunities for discovery.

We recommend the following when you set up automation for running tasks:

• Structure your workflow in smaller, manageable Spaces, with a limited number of products and/or assets, instead of one large Space with all the automation inside which is very challenging to maintain. You can create multiple Spaces to suit your needs.

• Separate your automation by tasks that are independent from each other.

• If you are working on many use cases, combining several data sets, create one automation per use case instead of one large automation that tries to address everything at once. This is easier to maintain, to update, and to get support should you need help.

• There are different ways to trigger an automated task on the platform :

  • Update manually;

  • Tie updates to data product or asset updates;

  • Schedule updates by date, time and interval.

• Automating one task can also be used to trigger another. For example, you can trigger a task to update and then use this to trigger the automation of a use case.

• You can always change when an automated asset updates in the Trigger section.