Connector Limits
This describes the typical file size limits and transfer speeds for bringing data to the platform and exporting it, by data source.
Note that these may be different depending on users internet connection, file type, region, your native cloud provider and other factors and should only be used as a guideline.
The limits when transferring data from or to a desktop are:
Desktop upload = 7GB
Desktop download = 8GB
If your data exceeds these limits then it is recommended you use a cloud endpoint instead.
AWS
If the native cloud provider for the platform is AWS, these are the transfer limits.
Connector | Transfer file size | Transfer speed |
|---|---|---|
S3 | No specific limit | up to 2GB/s |
GCS | No specific limit, recommended size up to 50-100GB due to transfer speed time | up to 6MB/s |
Azure Blob Storage | No specific limit, recommended size up to 50-100GB due to transfer speed time | up to 6MB/s |
Snowflake | No specific limit, tested up to 400GB | - |
BigQuery | No specific limit, tested up to 200GB | - |
GCP
If the native cloud provider for the platform is GCP, these are the transfer limits.
Connector | Transfer file size | Transfer speed |
|---|---|---|
Azure Blob Storage | Up to 7-10 GB for a single file If product consists of multiple files, then limitation applies to each individual file | up to 6MB/s |
GCS | Up to 200GB | up to 400MB/s |
S3 | Up to 7-10 GB for a single file If product consists of multiple files, then limitation applies to each individual file | up to 6MB/s |
Snowflake | No specific limit, tested up to 400GB | - |
BigQuery | No specific limit, tested up to 200GB | - |
If you are unsure whether the Platform you are working on is based on GCP or AWS then please contact Support.