Streaming agent installation using a CloudFormation quick-create link

This article details how to install a Streaming agent on Amazon ECS Fargate using a CloudFormation template quick-create link. The quick-create CloudFormation link will pre-fill values into the template for you.

Alternatively, you can download the CloudFormation template and edit as needed.

---

Prerequisites

To get started, you'll need:

• A Hub account. To register, read Registration.
• Access to AWS, with the permissions to create a stack on a billable account. You may need an administrator from your organization to either give access or perform this process with you.
• Permissions to provision cloud resources in the AWS environment that the source database is running in.
• An IAM role that has suitable permissions for Data Productivity Cloud. If you're using the same IAM role, you might need to add cloudformation.amazonaws.com to the Service section in the Trust Relationship section of your IAM role to allow the role to work with the ECS service. An example trusted entity can be found below. Read Roles and permissions for more information about creating IAM roles.

You'll also need to allow the IP addresses listed in Allowing IP addresses.

Example trusted entity:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudformation.amazonaws.com"
            },
            "Action": "sts:AssumeRole"
        }
    ]
}

To learn more about the trust relationship IAM policies, read trust policies in the AWS documentation.

We recommend that you read the documentation and prerequisites before beginning this process. You may require input from your organization's cloud administrator for access and permissions.

For detailed AWS permission information about deploying an agent in the Data Productivity Cloud, read Agent deployment permissions.

---

Created resources

This template will create the following resources in your AWS account:

• ECS task definition.
• ECS Fargate cluster.
• CloudWatch Log group.
• An entry in AWS Secrets Manager.
• IAM Roles with permissions for the following:
  • Task Roles to your ECS, EC2, and Secrets Manager.
  • Task Execution Roles to pull from ECR Public repositories.

---

Visualization

---

Using the CloudFormation link

1. If you haven't already created the agent in the Data Productivity Cloud, follow the procedure in Create a Streaming agent.
2. On the Agent details page, note the following Credentials: client_id and client secret.
3. Ensure you are logged in to your AWS account before proceeding.
4. Under Install using CloudFormation on the Agent details screen, click Launch. This will launch the CloudFormation stack in the AWS Console with pre-filled values for most parameters.

5. Complete the remaining parameters as follows:

   • Stack name: An arbitrary name given to the created stack. This must be unique.
   • OAuth Client ID: Copy the client_id from the Agent details page.
   • OAuth Client Secret: Copy the client_secret from the Agent details page.
   • VPC Id: Select the ID of an existing VPC. For more information, read VPCs.
   • VPC Subnet Ids: Select at least one VPC Subnet to use.
   • Database VPC Security Groups: Select at least one security group associated with your databases.

6. Ensure that the CPU and RAM allocation is appropriate for the workloads your Streaming pipelines will manage. Read Sizing streaming agents for guidance.

7. If you intend to use MySQL as a streaming data source, you will need to complete additional configuration under the Optional MySQL Support section. Read Configure the streaming agent for MySQL for details.
8. Click Create stack.

Once your stack is ready, this means the ECS service has now successfully deployed with 1 ECS task.