"It does not fit. Not tight — short."

The €100 Lakehouse — 2/12

The plan said co-host Superset on the Airflow box. The measurement said 1,045 MB available vs the 1.2–1.5 GB Superset needs. The ADR verdict: "It does not fit. Not 'tight' — short."

The box was 4 GiB on purpose — Airflow's measured peak was 1.87 GiB, so I bought 4 instead of the planned 8 and deferred Superset until it had a number of its own.

Three options, each priced:

→ A, +€0: drop Celery, free ~700 MB. Rejected — ~300 MB of slack left. "It fits and has no room to be wrong in, which is the same as not fitting."
→ B, +€23: resize to 8 GiB. Rejected — pays double to keep the coupling.
→ C, +€12: a dedicated t4g.small. Accepted.

Because failure isolation IS the product requirement: an Airflow OOM at 03:00 must not take down the dashboard the studio owner is looking at.

When on-demand came up, the honest saving was €10.7/mo, not €12 — AWS bills the public IPv4 (€4.9/mo) even while the instance is stopped.

And the box can stop itself but cannot start itself: the IAM grant is ec2:StopInstances only, tag-scoped to that one instance. An instance that could start itself would eventually find a reason to.

Lesson: decide with the number you can measure tomorrow for free, not the estimate you have today.

What's the smallest amount of money you've ever spent to buy failure isolation — and was it worth it?

Databricks · dbt · Airflow · Terraform · AWS

P.S. New tech post every Wednesday.

#100EuroLakehouse #AWS #FinOps