Azure agent private preview

Private preview

Data Productivity Cloud agents hosted in your Azure infrastructure is currently a private preview feature. This means that the feature is approved for evaluation and testing by selected customers, but may be subject to certain limitations. These limitations are described in this topic, which may be updated over the course of the preview as new features are added.

We assume you are already familiar with the operation of Data Productivity Cloud before beginning any evaluation of private preview features.

To create your first Azure agent, read Create an agent in your infrastructure.

Private preview scope

The following features are included in this private preview:

Create an Azure hybrid agent.
Configure Snowflake credentials in an Azure Key Vault and access these credentials to configure your Snowflake project in Data Productivity Cloud and connect to an Azure agent.
Execute the Green Wave demo pipeline.
Load from, load to, and stage to Azure blob locations using the Azure credentials you provide. This includes use of the following components: Azure Blob Storage Load, Azure Blob Storage Unload, Data Transfer, File Iterator, and all query components.
Load from, load to, and stage to AWS S3 buckets using AWS cloud credentials you provide. This includes use of the following components: S3 Load, S3 Unload, Data Transfer, File Iterator, and all connectors.
Load from a public S3 bucket via the S3 Load component.
Use all Data Productivity Cloud components, including Python Pushdown.
Use all of the AWS-specific components (for example, RDS Query, SNS Message, SQS Message, S3 Load, S3 Unload) with AWS cloud credentials.

Known limitations

These are the known limitations in the initial private preview release. These limitiations will be addressed in subsequent releases.

Azure applies throttling per account (100 requests in a 15 minute period), which you may hit if you do repeated validations and/or executions.
There is currently no support for multiple key vaults. You must select a key vault when creating an environment, creating secrets, and creating cloud credentials.
There is currently no support for Azure Queue Storage Message. This will be provided in a new component to be released soon.
There may be problems staging, loading, or unloading data in certain connectors if your Azure account has multiple subscriptions associated with it.
There is no support for loading your own libraries in the Python Script component. Python Pushdown is fully functional and provides external library support as normal.
It is not currently possible to stage data to a public AWS S3 bucket (but you can load from such a bucket).