Almost every AWS bill that has doubled did so without a single dramatic mistake. No runaway Lambda loop, nobody spun up a GPU cluster by accident. Six or seven small things each grew 15% a quarter for two years, and none of them had an owner.
So you are not hunting one culprit. You are running a diagnostic pass over known failure modes, each individually boring and collectively expensive. They are the same in almost every account, and most are confirmable in ten minutes. If your bill jumped *this month*, skip to the triage at the end.
First: find out where the money actually is
Step 1. Cost Explorer, daily granularity, grouped by service. Billing and Cost Management → Cost Explorer. Last 60 days, Daily, group by Service. Look for which services dominate and for step changes on a specific date. A step means someone did something. A slope means something is growing on its own.
Step 2. Regroup by usage type. This is the step people skip. AWS lumps unrelated spend into a bucket called "EC2 - Other": EBS volumes, snapshots, NAT gateway data processing, elastic IPs, and most of your data transfer. In many accounts it is the second-largest line on the bill and by itself tells you nothing.
Filter to it and regroup by Usage Type. Now you get real names: USE1-NatGateway-Bytes, USE1-EBS:VolumeUsage.gp2, USE1-DataTransfer-Regional-Bytes, USE1-EBS:SnapshotUsage. That is a diagnosis. "EC2 - Other" is not.
01aws ce get-cost-and-usage \02 --time-period Start=2026-06-01,End=2026-07-26 \03 --granularity MONTHLY --metrics UnblendedCost UsageQuantity \04 --filter '{"Dimensions":{"Key":"SERVICE","Values":["EC2 - Other"]}}' \05 --group-by Type=DIMENSION,Key=USAGE_TYPEStep 3. When Cost Explorer is too coarse, use the Cost and Usage Report. It will not tell you *which* volume, bucket or NAT gateway. For that you need the CUR, under Billing → Data Exports. Enable resource IDs when you create it, or it cannot attribute anything to a specific object. Wire it to Athena with the CloudFormation template AWS ships and ask real questions:
01SELECT line_item_product_code, line_item_usage_type,02 line_item_resource_id, SUM(line_item_unblended_cost) AS cost03FROM cur.cur_table04WHERE year = '2026' AND month = '7'05 AND line_item_line_item_type IN ('Usage','DiscountedUsage','SavingsPlanCoveredUsage')06GROUP BY 1,2,3 ORDER BY cost DESC LIMIT 50;Note that unblended cost is what hit the invoice, so an all-upfront Savings Plan shows as one enormous spike in the month you bought it, while amortized spreads it. Mixing the two invents problems that do not exist.
Step 4. Tagging. Untagged resources bleed because nobody can be asked about them. Activate cost allocation tags in Billing → Cost allocation tags before expecting to see them in Cost Explorer; activation applies going forward, though AWS backfills twelve months on request. Enforce with Organizations tag policies and an AWS Config required-tags rule. If you enforce one tag, make it owner.
The seven usual suspects
1. Idle and oversized EC2 and RDS instances
Something was sized for a launch that never came, or for a load test, and nobody revisited it. Check CloudWatch CPU and network over 30 days, not 7, then check AWS Compute Optimizer.
Compute Optimizer under-reports, and you need to know why. It defaults to a 14-day lookback and reads CloudWatch metrics only. Memory is not a default EC2 metric, so unless the CloudWatch agent publishes it, Compute Optimizer judges your memory-bound JVM entirely on CPU: it will either recommend a downsize that OOMs you, or mark an idle box "Optimized". It also never recommends *deletion*, only resizing, so an instance at 2% CPU forever gets "consider a smaller size" instead of the correct answer, which is terminate it. Treat it as a candidate list, and enable enhanced infrastructure metrics for a 93-day lookback. Two structural moves beat any amount of resizing: migrating to Graviton, and moving gp2 volumes to gp3, which is cheaper per GB and decouples IOPS from size.
2. NAT Gateway data processing
Nobody looks at this, and in a private-subnet architecture it is regularly a top-five line item.
A NAT gateway has two charges. An hourly charge, small and predictable, on the order of thirty-something dollars a month per gateway. And a per-GB data processing charge on every byte through it, in both directions, at roughly the rate of an hour of gateway uptime. At real traffic volumes the per-GB charge dwarfs the hourly one, and it is additive with normal data transfer rather than replacing it. The pattern: workloads sit in private subnets, so every S3 read, DynamoDB call and ECR image pull routes through the NAT. You pay per gigabyte to talk to AWS services in the same region.
01aws cloudwatch get-metric-statistics --namespace AWS/NATGateway \02 --metric-name BytesOutToDestination \03 --dimensions Name=NatGatewayId,Value=nat-0abc123def456 \04 --start-time 2026-06-26T00:00:00Z --end-time 2026-07-26T00:00:00Z \05 --period 86400 --statistics SumThen enable VPC Flow Logs on the NAT's ENI and aggregate by destination to find the top talkers.
The fix is VPC endpoints. Gateway endpoints for S3 and DynamoDB are free: no hourly charge, no per-GB charge. You should never pay NAT processing for S3 traffic. Interface endpoints (PrivateLink) cover ECR, Secrets Manager, SSM, CloudWatch Logs and STS, and are not free: hourly per endpoint per availability zone, plus a per-GB charge roughly a quarter of NAT's. High-volume services pay back immediately, something you call twice a day does not, so do the arithmetic before deploying twenty across three AZs.
Two details catch people. ECR pulls fetch layers from S3, so you need ecr.api and ecr.dkr interface endpoints and the S3 gateway endpoint, or you still pay NAT for the bulk of the transfer. And a NAT gateway in another AZ adds cross-AZ transfer on top of NAT processing. Registry storage growth compounds this from the other side, covered in reducing ECR costs.
3. Cross-AZ and inter-region data transfer
Traffic between availability zones inside a region is charged in both directions, roughly a cent per gigabyte each way. Nobody notices at 100 GB a month; at 50 TB a month it is worth a full-time engineer. Look for DataTransfer-Regional-Bytes. The usual sources are replicated stateful systems (Kafka, Cassandra, OpenSearch, anything with a quorum) and microservices scheduled across AZs at random.
- In Kubernetes, enable Topology Aware Routing (
service.kubernetes.io/topology-mode: Auto, orspec.trafficDistributionon recent clusters) so callers prefer same-zone endpoints. - In Kafka, enable rack awareness and follower fetching so consumers read a same-AZ replica.
- Co-locate the tight loops. A worker pool talking to one shard need not span three AZs.
RDS Multi-AZ *replication* traffic is free; your application crossing an AZ boundary to reach that database is not. Inter-region costs several times more, so audit cross-region replication, DynamoDB global tables and peering for anything on by default rather than by decision.
4. Unattached EBS volumes, orphaned snapshots, old AMIs
An EBS volume bills whether or not it is attached. Terminating an instance with DeleteOnTermination set to false leaves the volume behind, billing forever, attached to nothing.
01aws ec2 describe-volumes --filters Name=status,Values=available \02 --query 'Volumes[].{ID:VolumeId,GiB:Size,Type:VolumeType,Created:CreateTime}' --output tableSnapshots are subtler. They are incremental, so deleting one does not necessarily free its apparent size when later snapshots reference the same blocks. And deregistering an AMI does not delete the snapshots behind it: teams bake images nightly for three years, clean up the AMIs, and keep paying for a thousand orphaned snapshots.
01comm -23 \02 <(aws ec2 describe-snapshots --owner-ids self --query 'Snapshots[].SnapshotId' \03 --output text | tr '\t' '\n' | sort) \04 <(aws ec2 describe-images --owners self --query 'Images[].BlockDeviceMappings[].Ebs.SnapshotId' \05 --output text | tr '\t' '\n' | sort)Review before deleting; some are legitimately standalone backups. Move long-retention compliance copies to EBS Snapshot Archive (90-day minimum, restores up to 72 hours), and check Recycle Bin rules, which quietly hold "deleted" snapshots for months.
5. Untagged, forgotten dev and staging environments
Non-production runs 168 hours a week to serve maybe 50 hours of use. Scheduling it off outside working hours removes roughly 70% of that compute cost and is the least risky change on this list. Use EventBridge Scheduler with a small Lambda, or the AWS Instance Scheduler solution. Two caveats: a stopped EC2 instance stops compute charges but EBS keeps billing, and a stopped RDS instance restarts automatically after seven days with storage and backups billing throughout. For variable dev database load, Aurora Serverless v2 scales to zero ACUs.
Then group Cost Explorer by tag and read the untagged bucket. That is where the 2024 proof-of-concept lives.
6. S3 storage classes and incomplete multipart uploads
Start with the invisible one. Incomplete multipart uploads are stored and billed but do not appear in object listings, in the console or via aws s3 ls. A CI pipeline uploading large artifacts that fails 2% of the time accumulates orphaned parts indefinitely, with no signal in the normal UI. Check per bucket, or read incomplete multipart upload bytes estate-wide in S3 console → Storage Lens.
01aws s3api list-multipart-uploads --bucket my-bucket # region-scoped; use --region per bucket0203cat > lifecycle.json <<'EOF'04{"Rules":[{"ID":"abort-incomplete-mpu","Status":"Enabled","Filter":{"Prefix":""},05 "AbortIncompleteMultipartUpload":{"DaysAfterInitiation":7}}]}06EOF07aws s3api put-bucket-lifecycle-configuration \08 --bucket my-bucket --lifecycle-configuration file://lifecycle.jsonThat rule takes five minutes and belongs on every bucket you own. Same for versioning: with versioning on and no NoncurrentVersionExpiration rule, every overwrite since the day you enabled it is still billing. Add ExpiredObjectDeleteMarker cleanup too.
On storage classes, resist the urge to lifecycle everything into Infrequent Access. IA classes have a 128 KB minimum billable object size, a 30-day minimum duration (90 for Glacier Flexible, 180 for Deep Archive), and a per-object charge per transition, so moving ten million 20 KB objects to Standard-IA will *increase* your bill. Intelligent-Tiering is the safer default for unpredictable access. Read request costs separately too: high-frequency LIST and small-object GET storms have their own usage types and can exceed the storage charge.
7. No Savings Plan coverage on steady-state usage, and the over-commitment trap
If a meaningful share of your compute has run continuously for a year at on-demand rates, you are paying a premium for flexibility you are not using. Cost Explorer → Savings Plans → Coverage shows how much.
- Compute Savings Plans apply across EC2, Fargate and Lambda, across instance families, regions, operating systems and tenancy. They survive a container migration or a move to Graviton.
- EC2 Instance Savings Plans discount more deeply but lock you to an instance family in a region. Migrate off it and the commitment strands.
- Savings Plans do not cover RDS, ElastiCache, OpenSearch or Redshift. Those need Reserved Instances, and it is the most commonly missed coverage gap in a mature account.
The trap is over-committing. Commit to your trough, not your average. Take 90 days of hourly usage, find the floor, commit to a fraction of it, then layer more as the floor rises. A Savings Plan cannot be cancelled, sold or resized, unlike Convertible RIs (exchangeable) and Standard EC2 RIs (sellable on the Marketplace). Start with 1-year no-upfront on the confident baseline.
If you would rather have this run end to end, that is what our AWS cost optimization audit covers.
Where the money usually hides
| Line item | Why it grows | How to check | Effort |
|---|---|---|---|
| Oversized EC2 / RDS | Sized for load that never arrived | Compute Optimizer + 30-day CloudWatch | Med |
| NAT gateway processing | Per-GB on S3/ECR/DynamoDB traffic from private subnets | NatGateway-Bytes usage type; Flow Logs | Low |
| Cross-AZ transfer | Replication and topology-blind scheduling | DataTransfer-Regional-Bytes usage type | Med |
| Unattached EBS, orphaned snapshots | Volumes survive termination; deregistering an AMI keeps its snapshots | describe-volumes with status=available | Low |
| Always-on non-production | Nobody turns off what they do not pay for | Cost Explorer grouped by environment tag | Low |
| Incomplete multipart uploads | Failed uploads bill but are invisible in listings | Storage Lens; list-multipart-uploads | Low |
| S3 versioning, wrong storage class | No lifecycle rules; blanket IA transitions on tiny objects | Storage Lens; bucket lifecycle config | Low |
| Missing Savings Plan coverage | Steady-state workloads paying on-demand | Cost Explorer → Savings Plans → Coverage | Med |
| Multi-account sprawl | Infrastructure duplicated per account, no owner | CUR grouped by linked account | High |
| Kubernetes, no attribution | Cluster is one line item; no team sees its share | OpenCost / Kubecost per namespace | High |
The one-hour triage: your bill jumped this month
Run these in order. Most sudden jumps resolve in the first four steps.
- Cost Explorer, daily granularity, last 60 days, grouped by service. Find the exact date the line moved. A step change on one day narrows the search enormously.
- Filter to that service, regroup by usage type. Now you know what kind of charge grew, not just which service.
- Regroup by region and by linked account. New spend in a region you do not operate in is not a cost problem, it is a security incident: rotate credentials and open a support case immediately. Compromised access keys used for crypto mining cause most shocking overnight jumps.
- Check whether a commitment expired. This raises the bill with zero change in usage, and almost nobody checks it first. Check expiring Reserved Instances the same way, and credits under Billing → Credits.
01 aws savingsplans describe-savings-plans \02 --query 'savingsPlans[].{Id:savingsPlanId,Type:savingsPlanType,End:end,Commit:commitment}' \03 --output table- Compare unblended against amortized, and check the account age. An all-upfront commitment shows its whole payment in one unblended month, and around the one-year mark introductory free-tier allowances and startup credits simply lapse. Neither is a usage increase.
- Correlate with CloudTrail around the inflection date, then repeat for
CreateDBInstance,CreateClusterandCreateNatGateway:
01 aws cloudtrail lookup-events \02 --lookup-attributes AttributeKey=EventName,AttributeValue=RunInstances \03 --start-time 2026-07-01T00:00:00Z --end-time 2026-07-05T00:00:00Z \04 --query 'Events[].{Time:EventTime,User:Username,Region:AwsRegion}' --output table- If usage quantity rose with no infrastructure change, you have an application regression: a retry loop, a cache that stopped caching, a cron overlapping itself, a log level left at DEBUG writing to CloudWatch Logs. Always read usage quantity alongside cost.
- Set an AWS Budget with anomaly detection so the next one reaches you in days rather than at invoice time.
When this stops being a DIY problem
Most teams recover a meaningful share of their bill using nothing but the steps above, and you should do that first. Nobody needs to be paid to tell you to add an S3 gateway endpoint. Where it stops being tractable in-house:
- Architectural waste rather than configuration waste. The cost follows from how the system is shaped: synchronous services that should be event-driven, an RDS instance sized for a query pattern one index would fix, a pipeline reprocessing full history nightly.
- Multi-account sprawl. Twenty accounts under Organizations, no consistent tagging, NAT gateways and load balancers duplicated in each, no owner for shared infrastructure. That is organizational work as much as technical.
- Kubernetes cost with no per-team attribution. EKS is one large number with no visibility into which namespace is responsible, so no team has an incentive to change. Getting to per-namespace cost is where the Kubernetes consulting work usually starts.
- Commitment strategy across a changing workload. Layering Savings Plans mid-migration to Graviton or containers is hard, and getting it wrong locks in the mistake for years.
FAQ
How much can I realistically save?
Typical engagements land in the 20–40% range for an account never systematically reviewed. Heavy NAT traffic, no commitment coverage or lots of always-on non-production run higher; well-tended accounts run lower. Anyone quoting a percentage before seeing your Cost and Usage Report is guessing.
Will right-sizing cause downtime?
It can if you do it carelessly. Stateless workloads behind a load balancer resize with zero downtime by draining old instances. Instance family changes need a stop and start, so those are maintenance-window work, and RDS resizing means a failover: seconds on Multi-AZ, minutes on Single-AZ. The real risk is sizing on incomplete data, so install the CloudWatch agent before touching anything memory-bound, use 30 days of metrics, and change one dimension at a time.
Should I buy a FinOps tool?
Not first. A tool shows the same data your CUR already contains, priced as a percentage of spend, and it will not implement anything. Do the diagnostic pass, fix what it surfaces, then see what remains. Tools earn their cost in two cases: Kubernetes, where per-namespace attribution is hard to build yourself, and multi-account estates where showback has to be automated.
Do Savings Plans lock me in?
Yes, more than people expect. A Savings Plan cannot be cancelled, resized or sold; you pay the committed hourly amount for the full term whether you use it or not. Convertible RIs can be exchanged and Standard EC2 RIs sold on the Marketplace, so both offer more of an exit. Mitigate by committing to your usage floor, preferring Compute Savings Plans while your architecture is moving, and starting at one year no-upfront.
Why doesn't Cost Explorer show me the real cause?
Because its default grouping is by service, and AWS's taxonomy does not match how you think about your architecture. "EC2 - Other" bundles EBS, snapshots, NAT processing and data transfer into one meaningless line, and resource IDs are hidden unless you enable that granularity, so it tells you EBS costs a lot but not which volumes. It is a triage tool, and a good one. Root cause lives in the CUR, queried through Athena.
How often should we review this?
Set up guardrails once, then review quarterly. The guardrails are AWS Budgets with anomaly detection, a Config rule enforcing your tagging standard, and lifecycle policies on every S3 bucket and ECR repository. The quarterly review covers what automation cannot catch: commitment coverage as your baseline shifts, instance sizing as workloads change, and whether last quarter's architecture decisions added transfer paths.
If you would rather not run this yourself, we do it as a fixed-scope two-week engagement: we analyse your Cost and Usage Report and hand back a prioritised findings list with the dollar impact and implementation risk of each item, plus the remediation plan. No percentage-of-savings pricing, no tool to buy. Details on the AWS cost optimization page, or send your last three months of Cost Explorer exports and we will tell you if it is worth your time.

