Databricks azure soft delete

If you intend to set your destination as Databricks and your stage platform as Azure Storage, you must turn off the "Enable soft delete for blobs" setting in your Azure account for your pipeline to run successfully. To do this:

1. Log in to the Azure portal.
2. In the top-left, click ☰ → Storage Accounts.
3. Select the intended storage account.
4. In the menu, under Data management, click Data protection.
5. Untick Enable soft delete for blobs. For more information, read Soft delete for blobs.