Every quarter, two flavors of the same question land in our inbox. From CFOs: “Our colo bill is up 22% — should we move to the cloud?” From engineers who recently saw their AWS invoice: “We should probably pull this back on-prem, right?” Both are reasonable instincts. Both are usually wrong, because neither side of the comparison is being calculated honestly.
Here is the breakdown we use with clients — the cost lines most spreadsheets forget, on both sides, and the conditions under which each model actually wins.
The wrong question, in two flavors
The most common framing — “is the cloud cheaper than on-prem?” — has no defensible answer because the inputs on both sides are wildly underspecified. A 5-server rack with a senior sysadmin on call can be cheaper than the cloud, or twice as expensive, depending on a dozen variables no one wrote down. The useful question is narrower:
For this specific workload, with this load profile, this regulatory posture, this growth curve, and this in-house operational maturity — what is the total five-year cost of each option, including the costs my CFO doesn’t see?
The on-prem TCO most spreadsheets forget
A typical “on-prem is cheap” calculation goes: hardware + licenses + power. Sometimes someone remembers the colo bill. That’s usually 40% of the actual cost. The line items that usually get omitted:
- Hardware refresh on a 3–5 year cadence. A $40k server is a $40k server every four years, plus the migration weekend, plus the “turns out the new RAID controller has a different firmware path” week. Average this over the asset lifecycle; do not amortize it as “we already paid that.”
- Power and cooling at real utility rates. A mid-range 2U dual-socket server pulls 400–600W under typical enterprise load. That’s roughly $700–$1,200 per server per year in power alone in NYC commercial rates. Add cooling overhead (PUE 1.5+) and you’re at ~$1,500 per server-year before colo space.
- Operations headcount, allocated honestly. The sysadmin who babysits the cluster is not free. If 40% of an engineer’s time goes to on-prem upkeep — patching, firmware, hardware tickets, backup verification, restart rituals — that’s ~$60k of fully-loaded cost annually that belongs in the comparison.
- Downtime risk, priced. The hours you spend recovering from a controller failure or a botched patch are revenue you didn’t make, calls you didn’t take, and deals that quietly slid sideways. Put a dollar value on the hour. Multiply by your honest annual downtime, not your aspirational SLA.
- Software licenses on an aging base. VMware, Oracle, Microsoft per-core licensing, backup-product per-socket fees. Many of these scale with CPU counts that modern servers have made unfavorable. The renewal hits harder every cycle.
- The disaster-recovery site that doesn’t exist.A real DR posture for on-prem means a second site with synchronized data and tested failover. Many on-prem environments do not have this, and the unspoken plan is “recover from backup over the weekend.” That is a DR plan only in the sense that hope is a plan.
- Real estate and security. If you own a server room rather than rent colo, the space, the access control, the fire suppression, and the insurance are all costs. If you rent colo, the cross-connects, remote-hands hours, and after-hours dispatch fees add up.
The cloud TCO that surprises first-timers
The opposite mistake — “cloud is cheap because we only pay for what we use” — survives until the first end-of-month bill. The line items that surprise first-time migrators:
- Egress. Moving data into the cloud is free; moving it out is not. Backup replication to a non-cloud target, big data exports to a BI tool, cross-region replication for DR — these line items can quietly become 15–25% of the bill if architecture isn’t intentional.
- Observability. CloudWatch, Azure Monitor, Datadog, New Relic — pick one, but understand that the observability tier alone often costs 10–20% of the underlying compute. The free tier is for development; production dashboards bill on volume.
- Right-sizing failure. The default behavior of most teams during a lift-and-shift is to provision cloud instances at parity with on-prem hardware specs, which were themselves over-provisioned. You can pay double for two years before someone reviews the instance fleet.
- Managed-service premium. Managed RDS is cheaper than running your own Postgres on EC2 when you account for the ops time you don’t spend, but it is more expensive than the raw infrastructure. Same for managed Kubernetes, managed Elasticsearch, managed Redis. The premium is the point; price it knowingly.
- The bill nobody owns. Three engineering teams each spinning up their own logging stack, their own dev environments, and their own staging databases will produce a monthly bill with no single throat to choke. FinOps governance isn’t optional; it’s the only way the cost story stays defensible.
- Network egress between zones and regions.Multi-AZ is cheap. Multi-region with active-active replication is not. If your DR design has the data crossing regions continuously, that’s a line item the on-prem comparison probably didn’t have.
When cloud genuinely wins
Decisively, on TCO alone, when:
- The workload is bursty. Marketing analytics that runs once a day, a Black Friday traffic curve, a quarterly close that needs 10x normal compute for two days. Cloud’s utility model is built for this; on-prem requires sizing for peak and paying for valley.
- You need geographic distribution. Latency- sensitive products serving multiple regions, compliance regimes requiring data residency in specific jurisdictions, M&A consolidating offices across continents. Cloud regions exist; building 14 colos doesn’t.
- Your in-house operations team is small or junior.Managed services (RDS, ECS/EKS, managed Kafka, managed search) let a small team punch above their weight. The premium you pay is the salary you didn’t.
- You’re consolidating vendors and licenses.A move to M365 + Azure + Defender often replaces 8 separate line items. The unit cost of the platform is offset by the procurement and integration cost saved.
- The application is greenfield. Architecting for cloud from scratch — managed services, infrastructure as code, autoscaling, ephemeral environments — captures essentially all of the cloud’s economics. Lift-and-shift captures the worst of both worlds.
When on-prem still wins
Equally honestly: the cloud is not always cheaper. Cases where we’ve recommended clients stay on-prem (or move workloads back):
- Predictable, steady-state, high-utilization workloads.A database serving a steady 8,000 transactions per second, 24/7, for the next four years. You will not save money by renting it.
- Egress-heavy workloads. Video encoding pipelines, media archives streamed to external CDNs, scientific datasets delivered to partners. The egress bill quietly inverts the TCO comparison.
- Data-residency regimes with brittle audit trails.When the regulator wants to walk in and physically see the disk the data lived on (this still happens in finance and some healthcare contexts), cloud abstraction makes the conversation harder, not easier.
- Latency-sensitive industrial systems. Factory control, real-time analytics on the plant floor, OT-adjacent workloads where a 12 ms cloud round-trip is a deal-breaker.
- Sunk capital with significant useful life. A two-year-old VxRail cluster paid for in cash is not free, but it is closer to free than the migration cost to move workloads off it before depreciation runs out.
A six-line decision framework
Before you spend a quarter modeling spreadsheets, run the workload through these six questions. If the answers cluster, the choice usually makes itself.
- How variable is the load? Highly variable (10×+ peak vs valley) leans cloud. Steady-state at 60%+ utilization leans on-prem.
- How geographically distributed are the users?More than one continent, or strict residency requirements → cloud. Single region, mostly LAN-local → on-prem holds up.
- How thin is the ops bench? A two-person ops team supporting a hundred services should not be running their own Kafka cluster. Managed services exist for a reason.
- What’s the data gravity? If the workload produces or consumes terabytes that have to leave the platform, run an egress calculation before anything else.
- What’s the audit posture? Some regulators are friendlier to documented cloud controls than others. Talk to your auditor before the architect, not after.
- What stage is the application in? Mature and stable → either model works; lean on operational fit. Active development with frequent deploys → cloud, almost always.
Why the answer is usually hybrid
The honest answer for most mid-market firms isn’t cloud or on-prem; it’s a deliberate split. Run the steady-state production database on the cluster you already own; run the bursty analytics, the dev environments, and the customer-facing web tier in the cloud. Use the cloud for backups even if the primary stays on-prem (immutable, off-account, off-tenant — the ransomware playbook makes this almost mandatory in 2026).
The risk with hybrid is the same as the reason for it: complexity. Two operational models, two security postures, two cost models, two sets of tooling. Done thoughtfully, it captures the best of both. Done by accident — which is how most hybrid happens — it captures the worst of both.
We run a honest five-year TCO model for clients in a single 90-minute working session. The deliverable is a one-page spreadsheet you keep, with all the line items above, sized to your actual environment. Want one before you renew your colo or your cloud commitment?