Databricks azure soft delete

If you intend to set your destination as Databricks and your stage platform as Azure Storage, you must turn off the "Enable soft delete for blobs" setting in your Azure account for your pipeline to run successfully. To do this:

1. In the Azure portal, navigate to your storage account.
2. In the menu, under Data management, click Data protection.
3. Clear the Enable soft delete for blobs checkbox. For more information, read Soft delete for blobs.