AWS FARGATE ON EKS

3 min readJun 18, 2020

We were using EKS and thinking of using Fargate to reduce load on our EC2 worker nodes. So we tried Fargate and in this article I will be mentioning our experience.

Pricing

With Fargate, you pay only for the amount of vCPU and memory resources that your pod needs to run. This includes the resources the pod requests in addition to a small amount of memory needed to run Kubernetes components alongside the pod. Pods running on Fargate follow the existing pricing model. vCPU and memory resources are calculated from the time your pod’s container images are pulled until the pod terminates, rounded up to the nearest second. A minimum charge for 1 minute applies. Additionally, you pay the standard cost for each EKS cluster you run, $0.20 per hour.

Advantages

The one thing I really liked about Fargate is we can continue to run our EKS with EC2 worker node and at the same time can also utilize fargate advantages, i.e. the best of both world.

Fargate profile can be created for a specific namespace which means all pods in that namespace will run on fargate, or fargate profile can select specific pods from a namespace using labels.
With Fargate, you pay only for the amount of vCPU and memory resources that your pod needs to run.

Limitations

There are currently a few limitations that you should be aware of:

There is a maximum of 4 vCPU and 30Gb memory per pod.
Currently there is no support for stateful workloads that require persistent volumes or file systems.
You cannot run Daemonsets, Privileged pods, or pods that use HostNetwork or HostPort.
The only load balancer you can use is an Application Load Balancer.

In addition to those above mentioned there is some more which I want to mention here:-

Logging

We use fluentbit for sending our pods logs to AWS Cloudwatch. Now for pods running on Fargate profile it is not possible as fargate do not support Daemonsets and fluentbit runs as a Daemonsets.

One solution is we can run fluentbit as sidecar in every pod which we want to run on Fargate, but this will only increase complexity.

Rediness and Liveness Check

Rediness and Liveness check is another difficulty you can face for pods running on Fargate.

As a workaround you can increase initialDelaySeconds and timeoutSeconds.

Daemonsets pods in pending state

As Fargate do not support daemonsets, so if you are using daemonsets for running pods on EC2 worker node. We will see some daemonset pods in pending state because each fargate pod runs on a different fargate node, so if there is 3 fargate pods running that means 3 additional fargate nodes, which means 3 pods of daemonset in pending state. So solution is use node affinity in daemonsets.

Solution is use nodeAffinity

affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
      - matchExpressions:
        - key: eks.amazonaws.com/compute-type
          operator: NotIn
          values:
          - fargate

For setup of AWS Fargate on EKS you can check official AWS documentation.