Feb
07

Simple, Cost Effective ECS Service Communication

posted on 07 February 2020 in programming

3 Docker containers want to communicate with each other. They live on a single host with no scaling, it should be a trivial problem right? If you’re hosting them on AWS ECS you can use ECS Service Discovery, but for this simple scenario it’s costly and overly complex. Here’s a cheaper, simpler way.

Scenario

I have 3 Docker containers, A, B and C. The need to be able to communicate with each other via HTTP, with a circular dependency between 2 of them, e.g. A -> B, A -> C, B <-> C.

Container B runs a 3rd party app that expects a URL to talk to C, therefore it needs to be a standard URL using an A record, not a SRV record.

Small scale, cost-effective service discovery

The solution is to use a host network type. This type of network exposes all ports on the host machine directly. Each service will require a dedicated port number. Assuming this isn’t a constraint for you, the containers can then communicate with each other via a port number on localhost, e.g:

  • A exposes localhost:80
  • B exposes localhost:8080
  • C exposes localhost:80801

These port number configurations can be provided as environment variables.

Alternative approaches

Docker links are the simplest solution, and worked perfectly for me in a local development environment. Unfortunately the AWS ECS implementation only supports one way links, and chokes if you have a circular dependency (e.g. B <-> C).

Note also that links are a legacy feature and are deprecated.

User-defined network

A user-defined bridge network in Docker is also perfect, providing automatic DNS resolution between containers. Unfortunately they are not supported at all on AWS ECS.

ECS Service Discovery

ECS Service Discovery is the preferred approach when working with Amazon Web Services. It uses AWS Cloud Map backed by Route 53 DNS to register service IPs / ports and provide service discovery. A private Hosted Zone on Route 53 will cost about $0.50 per month.

If using a bridge or host network, only SRV records are supported. If you’re working with 3rd party application that expects a standard DNS entry, this won’t be sufficient so you’ll need to use awsvpc networking so you get A DNS records.

awsvpc networking requires a private subnet, which in turn requires a NAT Gateway which will set you back around $32.40 per month (at $0.045 per hour).

Custom ECS optimised AMI

Another idea that I didn’t fully explore was running a custom AMI with a full Docker daemon so I could make use of user-defined networks. I’m not sure how much effort would be involved in getting this to work with ECS, but I suppose you could forgo ECS altogether and just deploy Docker compose on an EC2 instance and manually bootstrap the containers from there.