Integration Best Practices
The platform provides various integration capabilities so that you can connect your work on the platform with your own environment.
To ensure that this integration remains simple and can grow with your needs, we recommend the following best practices
Separate your inputs from your outputs
Each connector that you define in the platform is connected to one on your cloud storage.
If the connector is used for a recurrent publication, the platform will scan the storage on a regular basis to look for new data revisions requests (TNF) and automatically trigger the import.
By separating your inputs, you can apply the storage policy that works best for your environment and decide if you want to remove the temporary data after import.
If the connector is used for a recurrent export, the platform will write a new dataset after each successful export.
By separating your outputs, you can quickly identify that this data can be re-created by the platform and apply the appropriate storage policy based on your usage of data.
Document your platform integrations
When you create a connector, the platform provides you with instructions to update your cloud data storage security to allow data movement.
Even if you can access all the configuration information in the administration of the platform, we recommend you store the security information in your documentation system and the part of your system that is connected to it.
If you are multi-cloud, prioritize exchange from the native platform storage for high traffic
Our platform is able to connect with the major cloud providers (e.g. AWS, Azure and GCP). As all the cloud providers charge a fee for the outgoing traffic, if you use multi-clouds (exchanging data from several cloud providers) and plan to have high volume exchange with the platform, we recommend using the same storage as our native platform to limit those cloud provider fees and ensuring a better transfer speed. Check with Support to find out which cloud provider the platform is hosted on.
If you are not familiar with cloud provider storage, use our documented articles
We fully understand that the first cloud exchange setup can be intimidating. Our documentation provides an initial guide on how to use each of the cloud providers and links to resources to get more information.
Look at the publishing or export information to understand your cloud traffic
You can view the start/end and size of each data operation performed on the platform on the data product / asset or on the export.
With this information, you can correlate the cloud traffic with your system logs and associate a data size to the time observed to transmit the data.
Do not hesitate to contact Support
Our support team can help you configure your connectors, guide you in your cloud to cloud integration and provide more best practices advice based on your specific cloud deployment.