Automation

Using the Automation feature in the main navigation bar, you can transform your existing code into an updating asset that refreshes automatically—either when contributing data products or assets are updated, or based on a defined schedule or trigger.

To do this, select a code asset from a Space and define a task that runs the code on a recurring or event-driven basis. Once the task executes successfully, it can be used as the source for creating an updating asset.

You can also configure the task to automatically export the resulting data to your local environment, enabling operational use of the output.

Note: You must have an Automation Creator or Administrator role to perform these action. The Automation icon on the navigation bar is not visible without one of these roles.

You must have saved and successfully executed a query in a Space.

Create a Code Asset

A Code Asset is code that has been written in a Space and has been transformed into a Code Asset, so it can be used as part of a Task.

The code used in your Code Asset is a copy of the code you saved in your Space that created a table or tables in publish_db. Any changes you make to your saved code within the Space are not reflected in your code asset unless you refresh the Code Asset. For others to be able to edit the code for your Code Asset you they must be added to your Space as a collaborator.

1. Click on the Automation icon on the navigation bar.

2. Click on the Code Assets option.

3. Click Create Code Asset.
   The New Code Asset page appears.

4. Enter a Title.

5. Enter a Description.

6. Select the Code you want to turn into a code asset:

   • Select the relevant Space from the drop down list.

   • Select the code you saved.
     It must have run successfully in the Space to ensure it is available to be used as a Code Asset.

7. Data Products and assets display a list of all products and assets in your Space.

   • If your Code Asset does not reference all products or assets in your Space then consider removing those that are not required to avoid downstream lineage restrictions.

8. Code Asset Sharing:

   • Add collaborators who can edit the Code Asset title and description.

9. Click Save.

Create a Task

A Task is an automated process for executing a code asset against target products and/or assets to create an output that can be used to create a new asset. The process can be run on a scheduled basis or be triggered when one or more of the target products or assets update.

Tasks can take several minutes to complete because behind the scenes a new, temporary compute instance is activated to execute the code in your Code Asset. As a user this is only really noticeable when the Task is run for the first time as all subsequent updates are executed automatically. You can return to check the status of your Task at any time.

To schedule the execution of your Code Asset:

1. Click on the Automation icon on the navigation bar.

2. Click on the Tasks option.

3. Click Create Task.
   The Create Task dialog page appears.

   • In the Details section:

     • (Required) Enter a title for the task.

     • (Optional) Enter a description for the task.

   • In the Code Asset section:

     • Click Select Code Asset. The Code Asset selection page appears.

     • Select the required Code Asset.

     • Click Save. The dialog box closes and you return to the Create Task page.

   • In the Products section: select any additional products, if available.

   • In the Assets section: select any additional Assets, if available.

   • In the Task Sharing section: select any users you wish to add as collaborators on this Task.

   • In the Trigger section: select how the Task will trigger. Available options are:

     • Manual: your Task must be triggered by user input.

     • Product or sset Update: an update to a linked Product or asset will trigger this task. At least one contributing Product or asset must be updating.

     • Schedule: set a schedule to automatically trigger the Task repeatedly.

   • Select if a notification email should be sent for a successful run as well as for failed runs. If a Task fails then check the status of it to find out more information.

   • Click Save or Save and Run.

     • A Task must run successfully for it to be available as a Source to create an automated data product

Now you are ready to create an asset from a Task, as shown here.

Manage a Task

Note: Again, you must have an Automation creator or Administrator role to perform these actions. The Automation icon on the navigation bar is not visible without one of these roles.

Check the Status of a Task

You can return to check on the status of a Task at any time.

1. Click on the Automation icon on the navigation bar.

2. Click on the Tasks option.

3. View Task details or search for a specific Task. Available details are:

   • Name

   • Description

   • Trigger

   • Creator

   • In Use (Yes or No)

   • Last Run

4. Click on a Task.
   The Details tab appears.

   • Click on the three dots at the top right to edit the Name and Description of the Task.

   • Click on the three dots at the top right to delete the Task.

   • The Task Run History tab:

     • Shows the date the task was executed and what triggered it.

     • Click View Run Details to see more information and download it to share with Support if there are any issues.

   • Check the Usage tab information matches your expectations.

Edit or Delete a Task

Certain descriptive attributes of a Task can be edited. A Task can also be deleted which means it no longer executes and the asset data no longer updates.

1. Click on the Automation icon on the navigation bar.

2. Click on the Tasks option.

3. For each Task you can:

   • Click three dots > Delete Task.

     • Your Task is deleted and no longer executes.

   • Click three dots > Edit Task:

     • Edit Title and Description

4. Click Save to save your changes.