Running the ARGLabs apps on EKS instead of ECS: effort, cost, monitoring and verdict

Written by Claude Code on 2026-10-14, from the ARGLabs repositories. Prices are us-east-1 on-demand list prices at 730 hours a month, and the baseline is dev, two regions, the full stack. Estimates, not invoices. The 2024 post ECS or EKS chose ECS from the requirements; this one puts numbers on that choice. Both start from Cloud as an Operating System.

What would change for arglabs-workload, statuspage and dependencytrack if they left ECS Fargate for an EKS cluster, what that cluster’s repository would look like, how much it would cost on EC2, Fargate and Auto Mode, and what happens to the sidecars and to the monitoring arglabs-tfmod-ecs-app delivers today.

1. Executive summary

2. What exists today

Three applications run as ECS Fargate services in the aio account, each in its own repository, attached to the ALBs BigBang created. The ECS cluster is an empty, free namespace, created by arglabs-tfmod-bigbang-ecs-cluster (130 lines of Terraform) with Container Insights enhanced in dev, stg and prd.

What arglabs-tfmod-ecs-app delivers per app and region

3,427 lines of Terraform (2,200 without variables.tf), 133 variables, 798 lines of tests. Everything arrives with the application; nothing is installed in the cluster:

• Service and task definition on Fargate, capacity providers FARGATE and FARGATE_SPOT per service (min_ondemand, spot_weight), a circuit breaker with rollback, the native ROLLING, BLUE_GREEN, CANARY and LINEAR strategies, ECS Exec.
• A target group and two listener rules on the shared ALB (host and /<app>/*), a regional Route53 record and, in the repository, the weighted regionless name with a health check.
• Autoscaling on CPU and memory (target tracking at 70%), emergency step scaling, and request count per target.
• 8 alarms (high CPU, high memory, emergency, and 5 for the ALB, off with threshold 0), notifying the account’s alerts SNS topic.
• A 24-widget dashboard: 15 for the ALB, 2 from AWS/ECS, 3 from ECS/ContainerInsights, 3 Logs Insights, and the Cost Explorer cost widget filtered by app_name and env.
• Sidecars: FireLens (Fluent Bit, its configuration baked into the aio-<app>-fluentbit image, output to stdout plus S3 in the account’s log bucket, partitioned for Athena) and ADOT (OTLP on localhost, X-Ray).
• IAM: the execution role (ECR, logs, Secrets Manager) and the task role (X-Ray, Exec, EMF), plus the repository’s workload role, trusting ecs-tasks.amazonaws.com.

The pipeline has nothing ECS-specific: the deploy is a terraform apply (wait_for_steady_state = false), and the images are arm64, built on the self-hosted linux.arm64 runner and pushed only to the dev account’s ECR. That is good news for a migration: the build, lotr and the scripts stay the same.

The network is dual-stack, and the apps subnets have no NAT (outbound only over IPv6, through the egress-only gateway, plus the S3 endpoint), which is why the tasks run in public with a public IPv4. The account’s deployer has no eks:*, no iam:*OpenIDConnectProvider* and no xray:*, and is denied creating NAT gateways and VPC endpoints.

3. What it would look like on EKS, keeping the ARGLabs model

The ARGLabs rule is that BigBang delivers the platform and the team brings everything that matters along with the app. On EKS that means one cluster per region and environment, created by BigBang, and an app module that creates the Kubernetes objects through Terraform’s kubernetes provider, with no Helm by hand and no kubectl apply in the pipeline.

The decisions that keep the model

• The shared ALB stays. The target group and the listener rules remain in the app’s Terraform (code reused from ecs-app), and a TargetGroupBinding of the AWS Load Balancer Controller registers the pods in that TG, in ip mode. The controller neither creates nor touches the ALB. The 15 widgets and the 5 alarms of the ALB stay identical.
• One identity per app. The repository’s workload role gains the pods.eks.amazonaws.com trust and an aws_eks_pod_identity_association with the app’s ServiceAccount. No IRSA, and no OIDC provider per cluster.
• Secrets. The SECRET_* stay in aio/<app>/<env>/app; an ExternalSecret (External Secrets Operator) materializes them as a Kubernetes Secret, and the Deployment injects them as variables, keeping APP_SECRET_KEYS. The reader of Secrets Manager becomes the operator’s identity, not the execution role.
• Logs. Fluent Bit becomes an account DaemonSet (part of the amazon-cloudwatch-observability add-on, or the aws-for-fluent-bit chart), with one configuration per cluster doing what s3-output.conf does today: JSON to CloudWatch, and gzip to logs/<app>/YYYY/MM/DD/ in the account’s bucket, with the namespace as the app’s name.
• Traces. ADOT through the operator’s sidecar injection (an annotation on the pod), or one collector per node. The app only swaps AwsEcsResourceDetector for AwsEksResourceDetector in tracing.py.
• Ephemeral environments (dev-*) become namespaces in the dev cluster instead of whole stacks: cheaper and faster than today. It is the only point where EKS simplifies something ECS does with effort.

4. The repositories: cluster, BigBang and app module

arglabs-tfmod-bigbang-eks-cluster (new, used only by BigBang)

One cluster per call, in one region, named <scope>-eks-<region short>-<env> (aio-eks-use1-dev), following the pattern of arglabs-tfmod-bigbang-ecs-cluster, but with one file per resource group:

terraform/
  terraform.tf              aws >= 6.x, kubernetes, helm, tls
  variables.tf              name, region, kubernetes_version, compute_mode
                            (mng | fargate | auto), node_*, fargate_*,
                            subnet_ids, enable_* per add-on, log retention
  eks-cluster.tf            aws_eks_cluster: authentication_mode API,
                            public endpoint with an allowlist (or private),
                            ip_family ipv4, control plane logs, upgrade_policy
                            STANDARD (avoids extended support by mistake)
  eks-access.tf             access entries: deployer (cluster admin),
                            AWSReservedSSO_AdministratorAccess_* (admin),
                            PowerUserAccess (view), the pipeline runner
  eks-addons.tf             vpc-cni, coredns, kube-proxy, eks-pod-identity-agent,
                            metrics-server, amazon-cloudwatch-observability
  eks-nodegroup.tf          compute_mode = mng: Graviton managed node group,
                            launch template, Spot or on-demand, 1 to N nodes per AZ
  eks-fargate-profile.tf    compute_mode = fargate: one profile per namespace,
                            built-in logging (ConfigMap aws-logging)
  eks-automode.tf           compute_mode = auto: compute_config, a custom
                            arm64 and Spot NodePool
  helm-lb-controller.tf     AWS Load Balancer Controller (Pod Identity),
                            not needed with Auto Mode
  helm-external-secrets.tf  External Secrets Operator + ClusterSecretStore
  helm-adot.tf              ADOT operator and collector (X-Ray)
  iam.tf                    cluster role, node role, add-on roles
                            (Pod Identity associations)
  log_group.tf              /aws/eks/<name>/cluster, retention per env
  cloudwatch_alarms.tf      nodes NotReady, pods Pending, API server 5xx
  cloudwatch_dashboard.tf   the cluster's Container Insights + cost widget
                            (arglabs-tfmod-cost-widget, service Amazon EKS)
  outputs.tf                cluster_name, endpoint, certificate_authority,
                            oidc_issuer, version, node_security_group_id

Estimate: 1,800 to 2,500 lines of Terraform, plus tests. A helm_release and the kubernetes provider in the same root that creates the cluster is the known weak point of this kind of module (the provider needs the endpoint before it exists). BigBang already splits account creation from configuration, and the same cut solves it here: the cluster in one apply, the Helm add-ons in the next, or everything as aws_eks_addon wherever a managed add-on exists.

Changes to arglabs-tfmod-bigbang and its neighbors

arglabs-tfmod-eks-app: the sibling of ecs-app

In the app repository, tfmod-eks-app.tf would be one more parkable file, with the same invariant as the others: it references no other tf*.tf, it reads the cluster from platform, and iam-role-workload.tf gains the second service principal. The SECRET_*, the ENV_* and the image contract do not change.

5. EC2 (managed node groups), Fargate or Auto Mode

The point that rules Fargate out is not only the price. Today the whole platform is arm64, from the linux.arm64 runner to cpu_architecture = "ARM64", and CLAUDE.md already records the missing x86 build as a gap. EKS Fargate would force x86 images (QEMU emulation on the ARM runner, or a second runner), only to pay more and lose Spot.

6. Costs

Comparing totals does not help: Aurora, Valkey, SQS, SNS, the buckets, Lambda, CloudFront, the ALBs, CloudTrail and DNS cost the same under both models. The tables below isolate the part that depends on the orchestrator: the control plane, the apps’ compute, the system pods and the cluster’s observability. The rest of the full stack, by the estimate in Arglabs, by Claude, is between USD 180 and 220 a month in dev with everything on, and goes away with lotr_destroy_all.

Assumptions and prices (us-east-1, 730 hours a month)

Left out of the comparison because they are the same in both: X-Ray (USD 5 per million traces), alarms (0.10), dashboards (3.00 beyond the three free ones), the cost widget (0.01 per Cost Explorer call), the ALBs, DNS. Network data and transfer were not modeled.

What each app costs a month (USD)

In the scenarios with nodes, us-east-1 takes a t4g.large and a t4g.medium, in two AZs, and us-east-2 a t4g.medium (only the workload, like today’s single task). Dependency-Track asks for 2 vCPU and 4 GB, which no t4g.medium holds (4 GiB in total, about 3.4 allocatable). On Kubernetes it runs with requests of 1 vCPU and 3.5 GB and limits of 2 vCPU and 4 GB on the t4g.large, and the two small apps share the nodes with about 1.5 GB of agents and system pods. That gap between requests and limits is a real Kubernetes advantage, because on Fargate you pay for the whole task. Even with it, EKS loses.

Dev, 2 regions, full stack: the part that depends on the orchestrator (USD a month)

Three environments, and the edge cases

The order of magnitude does not change with the assumptions. Even with Container Insights at zero and Spot at 70%, the cheapest EKS stays above USD 180 a month in dev with two regions, because USD 146 of it are the two control planes. In one region, the difference is a fixed USD 73 plus the smallest node, against a USD 2.52 Spot task.

7. Monitoring, item by item

What ecs-app delivers today, what would do the same on EKS, the effort, and whether the result is better, the same or worse. “Worse” here means more pieces to maintain for the same result, not that Kubernetes monitors badly.

Where EKS monitors better

• Prometheus and Grafana are the ecosystem’s standard. Amazon Managed Service for Prometheus with the managed collector, and Amazon Managed Grafana, give metrics per pod, per namespace and per app, with ready-made dashboards (kube-prometheus-stack). The cost: AMP per ingested sample and AMG per active user, plus one more stack to keep versioned.
• EKS Container Insights enhanced gives a view of pods, nodes and namespaces in the console, with more detail than the ECS one.
• kubectl gives a live view (describe, logs, top, events) that on ECS takes the console or several CLI calls.

None of this is free or automatic: every piece is an add-on or a chart with its own version. ecs-app reaches today’s result with nothing but AWS resources in the app’s Terraform.

8. The sidecars in both options

FireLens today

• An essential fluentbit container in every task, 50 MB reserved, its own image with s3-output.conf baked in (aws_fluent_bit_init_file_2), the LOG_S3_* variables.
• It does two things: it prints every JSON record to awslogs (what the ECS console reads), and it sends gzip to logs/<app>/YYYY/MM/DD/ in the account’s bucket.
• A failure of the S3 output takes the task down (essential).
• The cost: memory in every task, and one more image to build and scan (SBOM).

Fluent Bit on EKS with nodes

• One DaemonSet per cluster (the amazon-cloudwatch-observability add-on, or the aws-for-fluent-bit chart), one configuration for every app, read from a ConfigMap: the aio-<app>-fluentbit image and the build_sidecars step go away.
• Memory per node (about 100 to 150 MB) instead of per task; the app keeps its 50 MB.
• The kubernetes filter adds the namespace, the pod and the container; s3_key_format uses the namespace as <app> and keeps the Athena layout. The tag kube.<ns>.<pod>.<container> replaces FireLens’s $TAG.
• A sick Fluent Bit affects the logs of every app on the node, not one task.
• The stdout output stops making sense: kubectl logs reads the container’s stdout directly.

The built-in log router of EKS Fargate

• Managed by AWS, with no pod to pay for, configured by an aws-logging ConfigMap of at most 5,300 characters, with Filter, Output and Parser sections only.
• Allowed outputs: CloudWatch, Firehose, Kinesis and OpenSearch. No direct S3: the road to Athena becomes Firehose with dynamic partitioning (a cost per GB, and prefixes that differ from today’s layout).
• No stdout, no extra plugins.

ADOT today

• An opentelemetry container in every task, 128 MB reserved, ecs-xray.yaml, OTLP on localhost 4317 and 4318; the app gets the OTEL_* variables from the module.
• It cannot pull from public.ecr.aws in the apps subnets, which have no NAT: that is why enable_otel_sidecar is off with app_network = apps.

ADOT on EKS

• With nodes, two modes. The operator with sidecar injection (the annotation sidecar.opentelemetry.io/inject): the same design as today, 128 MB per pod, localhost. Or one collector per node (a DaemonSet), with the app pointing at the node’s IP (status.hostIP): memory per node, one more network hop, simpler to operate.
• On Fargate, sidecar only (no DaemonSets), adding to the pod’s size and to the rounding.
• The identity for X-Ray comes from the ServiceAccount’s Pod Identity, not from a task role.
• The image still comes from public.ecr.aws, so the egress problem of the apps subnets is the same, solved by nodes in a public subnet or by mirroring the image to ECR.

The Dependency-Track case

• wait-for-db becomes a literal initContainer; dependsOn HEALTHY becomes the apiserver’s readiness and startup probes, and the frontend simply waits for the Service.
• The project publishes an official Helm chart, which on EKS would try to pull the app out of the tfmod-eks-app pattern. That decision has to be made before starting.

In short: with nodes, the log and trace sidecars can become cluster agents and give 178 MB back to every workload pod, at the cost of a shared point of failure and more pieces to upgrade. On Fargate, the design stays as it is today, more expensive and with fewer outputs.

9. Migration effort

Working days of focused work (a person plus Claude) to reach what exists today. ecs-app took months of iteration in the pipeline; the numbers below assume its code and tests are reused wherever they fit.

Order and risks

1. The cluster module in dev, one region, compute_mode = mng on Spot. Validate the network egress of the chosen subnets (ECR, STS, CloudWatch, X-Ray) before any app.
2. BigBang with enable_eks_cluster in dev only; measure the apply and destroy times in lotr.
3. tfmod-eks-app with the workload; repeat the ECSAPP001 and SECRETS001 tests on the EKS version.
4. statuspage and dependencytrack last, the second one with the Helm chart versus module decision.

• Network: nodes in a public subnet with a public IPv4 repeat today’s posture; IPv6-only apps subnets require validating the dual-stack endpoints. Fargate does not have the first option.
• The runner and the EKS API: the runner is self-hosted and internal; the public endpoint with an allowlist is the simple path, while the private one requires the runner in the VPC.
• Names: the 64-character limit of role names still applies; aio-workload-eks-cost-widget-lambda-role-use1-dev has 49.
• Calendar: at least one Kubernetes version upgrade a year, with the add-ons aligned, or USD 365 more per cluster a month.
• Two vocabularies in the same root: AWS resources and Kubernetes objects in the same apply works, but an unavailable cluster makes the whole plan fail.

10. Pros and cons

ECS, as it is

• A free cluster, with no version and no add-ons: BigBang delivers an empty namespace and everything arrives with the app.
• Spot per service (min_ondemand, spot_weight), arm64 end to end.
• Monitoring built from AWS resources in the app’s Terraform: 24 widgets, alarms, a per-app cost widget, a secrets dashboard.
• Native deploy strategies with rollback.
• A clear cost boundary: a tag on the task definition, and Cost Explorer answers.
• Cons: no Helm catalog, no operators, no namespaces for ephemeral environments (every branch is a stack), a smaller tool ecosystem, and less learning value for anyone who needs Kubernetes on the job market.

EKS

• The market’s standard API, portability, Helm and operators (Dependency-Track itself has a chart), Argo CD as a managed capability (from USD 22 a month).
• Namespaces as ephemeral environments: faster and cheaper than today.
• Per-pod observability, and the Prometheus and Grafana ecosystem.
• HTTP probes solve the images without a shell.
• Requests smaller than limits pack pods into fewer nodes; on Fargate you pay for the whole task.
• Cons: USD 73 per cluster a month before any pod (146 per environment in two regions); a version calendar; dozens of versioned pieces (CNI, coredns, kube-proxy, LB controller, metrics-server, ESO, ADOT, Fluent Bit, KEDA, Argo); the per-app cost widget stops working without Split Cost Allocation; Spot belongs to the node, not to the service; Fargate is x86 only, with no Spot; the deployer needs new permissions; two infrastructure vocabularies.

When EKS would be worth it

• If most of what runs came from Helm charts or needed operators (databases in the cluster, queues, CRDs).
• If there were dozens of services per account and environment: the control plane gets diluted, and ephemeral environments as namespaces pay for it.
• If the team already spoke Kubernetes, and the platform were used by teams that expect kubectl.
• If the goal were a lab and content: an EKS track in ARGLabs makes a good post, and its cost in dev with one region is close to USD 113 a month while it is on.

None of the four describes ARGLabs today: three small services, one person, daily use in dev only, and the project’s public thesis is precisely that ECS in the team’s own account makes the shared cluster unnecessary.

11. Verdict and recommendation

The incremental path, if it goes ahead

1. arglabs-tfmod-bigbang-eks-cluster with compute_mode = mng and the minimal add-ons (vpc-cni, coredns, kube-proxy, pod identity, metrics-server, CloudWatch observability). No LB Controller yet.
2. BigBang dev with the cluster in us-east-1; measure the apply, the destroy and the real cost for 30 days on the cost dashboard.
3. The LB Controller and tfmod-eks-app with the workload, a TargetGroupBinding on the ext ALB; repeat the tests of TESTS.md on the EKS track.
4. Only then decide about statuspage and dependencytrack, and write the post with measured numbers in place of the estimates here.

12. Sources and assumptions

• Amazon EKS pricing: USD 0.10 per cluster-hour (standard), 0.60 (extended), the examples of the Auto Mode fee (12% of on-demand), capabilities (Argo CD at USD 0.03 an hour).
• AWS Fargate pricing: x86 at 0.000011244 per vCPU-second and 0.000001235 per GB-second; arm64 at 0.0000089944 and 0.0000009889; Spot up to 70% off; a 1-minute minimum.
• EKS on AWS Fargate, considerations: no DaemonSets, no Spot, private subnets only, IP targets only, and the table row “Can run workloads that require Arm processors: No”.
• Fargate logging: the aws-logging ConfigMap, its sections and its allowed destinations.
• EKS Auto Mode and its managed instances: the supported families (t4g included), “more than 1 CPU, instance size is not nano, micro or small”, a 21-day maximum lifetime, no SSH or SSM.
• IPv6 on EKS clusters: the family is chosen at creation, IPv6-only pods with NAT on the node for IPv4, the LB Controller in IP mode.
• TargetGroupBinding: registering pods in a target group created outside Kubernetes.
• Kubernetes version lifecycle on EKS: 14 months of standard support; 1.34 until 2026-12-02, 1.35 until 2027-03-27, 1.36 until 2027-08-02 (the 1.35 and 1.36 dates came from secondary sources).
• CloudWatch pricing: Container Insights enhanced for EKS at 0.21 per million observations; ECS enhanced at 0.07 per metric; custom metrics 0.30; alarms 0.10; dashboards 3.00; X-Ray 5.00 per million traces.
• EC2 t4g.medium at USD 0.0336 an hour (us-east-1, on-demand, from price aggregators checked on 2026-10-14; check the AWS calculator before budgeting).
• ARGLabs facts: arglabs-workload/CLAUDE.md and docs/operations.md, arglabs-tfmod-ecs-app, arglabs-tfmod-bigbang, arglabs-tfmod-bigbang-iam, arglabs-account-sharednetwork, arglabs-bigbang-aio, and the post Arglabs, by Claude, section “What it costs”.

Estimates, not prices: Container Insights per cluster, the control plane logs, Spot at 65% (Fargate) and 60% (EC2), the number of system pods on Fargate, the lines of Terraform of the new module and the days of effort. Data transfer, Route53 and the costs that are the same in both models were left out. Prices were read on the official pages on 2026-10-14; the model is a short Python script that recomputes the tables and the chart when an assumption changes.