Datadog APM in ECS Fargate

Abstract

---

• Datadog APM is used for Application Performance Monitoring (APM)

ECS Fargate Setup

---

Security: whitelist outbound traffic to Datadog Endpoints

In some deployment environment, by default all outbound traffic is denied. Here is a list of datadog endpoints you can use to whitelist the traffic. So the Datadog Agent Sidecar is able to pipe the application Trace to Datadog.

Prerequisites

• AWS Integration
• Datadog ddtrace

The setup is around Task Definition, we need to have the following 3 components inside the task definition:

• Pipe container log to AWS Firelens

Configuration for ECS Container logging

Add the following block inside the container that we want to pipe the log to Datadog. Update the highlighted parts with your own values

"logConfiguration": {
  "logDriver": "awsfirelens",
  "options": {
    "Host": "http-intake.logs.datadoghq.eu",
    "Name": "datadog",
    "TLS": "on",
    "apikey": "<YOUR_API_KEY>",
    "dd_service": "AEGIS-dev-backend",
    "dd_source": "AEGIS-dev-backend-firelens",
    "provider": "ecs"
  }
}

• AWS Firelens - Log Router Container

Configuration for Log Router Container

AWS Firelens container functions as a Log Router. Update the highlighted parts with your own values, you can refer to Hardware Details for the cpu and memory configuration

{
  "name": "log_router",
  "image": "amazon/aws-for-fluent-bit:stable",
  "cpu": 256,
  "memory": 512,
  "portMappings": [],
  "essential": true,
  "environment": [],
  "mountPoints": [],
  "volumesFrom": [],
  "user": "0",
  "firelensConfiguration": {
    "type": "fluentbit",
    "options": {
      "enable-ecs-log-metadata": "true"
    }
  }
}

• Datadog Agent Sidecar Container

Configuration for Datadog Agent Sidecar Container

This Datadog agent sidecar container is needed to pipe the traces collected by Datadog ddtrace to Datadog. Update the highlighted parts with your own values, here is a list of environment variables you can add to further fine tune the agent

{
  "name": "datadog-agent",
  "image": "public.ecr.aws/datadog/agent:latest",
  "cpu": 256,
  "memory": 512,
  "portMappings": [],
  "essential": true,
  "environment": [
    {
      "name": "ECS_FARGATE",
      "value": "true"
    },
    {
      "name": "DD_API_KEY",
      "value": "<YOUR_API_KEY>"
    },
    {
      "name": "DD_SITE",
      "value": "datadoghq.eu"
    },
    {
      "name": "DD_APM_ENV",
      "value": "aegis-stg"
    },
    {
      "name": "DD_APM_IGNORE_RESOURCES",
      "value": "GET /health"
    }
  ],
  "mountPoints": [],
  "volumesFrom": []
}

• DD_APM_ENV overrides DD_ENV
• We can use DD_APM_IGNORE_RESOURCE to ignore Trace from transmitted to Datadog

This Terraform code template below can help you create a task definition with Datadog APM integration

Terraform Code Template

You can use the following Terraform code template to create the task definition with Datadog APM integrated.

resource "aws_ecs_task_definition" "backend_app" {
 family                   = ""
 requires_compatibilities = ["FARGATE"]
 network_mode             = "awsvpc"
 cpu                      = 2048
 memory                   = 4096
 execution_role_arn       = ""
 task_role_arn            = ""
 
 container_definitions = jsonencode([
   {
     name      = ""
     image     = ""
     cpu       = 1024
     memory    = 2048
     essential = true
 
 
 
     portMappings = []
     secrets      = []
 
     environment = [
       {
         "name" : "DD_SERVICE",
         "value" : ""
       },
       {
         "name" : "DD_ENV",
         "value" : ""
       }
     ]
 
     logConfiguration = {
       logDriver : "awsfirelens",
       options : {
         "Host" : "",
         "Name" : "",
         "TLS" : "on",
         "apikey" : "<YOUR_API_KEY>",
         "dd_service" : "",
         "dd_source" : "",
         "provider" : "ecs"
       }
     }
   },
   {
     name : "",
     image : "amazon/aws-for-fluent-bit:stable",
     cpu : 256,
     memory : 512,
     portMappings : [],
     essential : true,
     environment : [],
     mountPoints : [],
     volumesFrom : [],
     user : "0",
     firelensConfiguration : {
       type : "fluentbit",
       options : {
         "enable-ecs-log-metadata" : "true"
       }
     }
   },
   {
     "name" : "",
     "image" : "public.ecr.aws/datadog/agent:latest",
     "cpu" : 256,
     "memory" : 512,
     "portMappings" : [],
     "essential" : true,
     "environment" : [
       {
         "name" : "ECS_FARGATE",
         "value" : "true"
       },
       {
         "name" : "DD_API_KEY",
         "value" : "<YOUR_API_KEY>"
       },
       {
         "name" : "DD_SITE",
         "value" : ""
       },
       {
         "name" : "DD_APM_ENV",
         "value" : ""
       },
       {
         "name" : "DD_APM_IGNORE_RESOURCES",
         "value" : ""
       }
     ],
     "mountPoints" : [],
     "volumesFrom" : [],
   }
 ])
 runtime_platform {
   operating_system_family = "LINUX"
   cpu_architecture        = "X86_64"
 }
}

DD_APM_IGNORE_RESOURCES takes in a list of resources, but I wasn't able to pass a list object to the key-value pair environment variable. Please let me know if you find a way around it 😃

References

---

• Official ECS Fargate Integration
• Official Datadog Agent Configuration