What should a deployment health check actually test?
By the Keel team · Last checked · 4 min read
Short answer
A deployment health check should prove that a new instance of your app can serve real requests: the process is running, it has loaded its configuration, and it can reach the dependencies it cannot work without, usually the database. It should be fast and should not fail because an optional third-party service is slow. After the deploy, a short smoke test checks the user-facing paths.
Why do health checks matter for ECS deployments?
Health checks are how AWS decides whether a new release is working.
On ECS behind an Application Load Balancer, the load balancer only sends traffic to tasks that pass their health check, and the ECS deployment circuit breaker uses failed health checks to decide when to roll back. A health check that always returns 200 means broken releases go live.
What should a deployment health check endpoint check?
A health check endpoint should confirm that the process responds, its configuration loaded and the database is reachable, and should not depend on optional third-party APIs or slow queries.
| Check | Include? | Why |
|---|---|---|
| The process is up and responding | Yes | The basic signal |
| Configuration and secrets loaded | Yes | Catches missing environment variables |
| Database reachable with a cheap query | Yes | Most apps cannot work without it |
| Migrations at the expected version | Often | Catches code deployed ahead of its schema |
| Cache reachable | Only if required | Some apps run fine without it |
| Third-party APIs such as payments or email | No | Their outage would take your whole service down |
| Slow or heavy queries | No | Health checks run often and must be fast |
The aim is a check that fails when this instance cannot do its job, and passes when it can, even if the outside world is having a bad day.
What is the difference between liveness and readiness checks?
A liveness check answers whether the process should be restarted.
Use a liveness check for
- Deciding whether a container should be restarted.
- Keeping it simple, so a slow database does not cause restart loops.
- The container health check in the ECS task definition.
Use a readiness check for
- Deciding whether a task should receive traffic.
- Checking the dependencies the app cannot work without.
- The load balancer target group health check.
A readiness check answers whether it should receive traffic. On ECS, the container health check in the task definition behaves like liveness, and the load balancer target group health check behaves like readiness. Keep the container check simple, so a slow database does not cause endless restarts, and put the dependency checks in the load balancer check.
How should you configure health checks on ECS?
On ECS, point the target group health check at a readiness endpoint, tune the interval and thresholds, set a health check grace period and turn on the circuit breaker.
- Set the target group health check path to your readiness endpoint, such as
/health. - Choose an interval and unhealthy threshold that detect failure within a minute or two.
- Set the ECS service health check grace period long enough for your app to start, so slow starts are not mistaken for failures.
- Turn on the deployment circuit breaker with rollback, so repeated failures roll back automatically.
- Return a plain status code and a short body. Do not leak versions or internal details publicly.
The rollback side is covered in automatic rollback on ECS with GitHub Actions.
What should a smoke test check after a deploy?
A smoke test runs once after each deploy and checks the paths customers use most.
It complements health checks, which only prove each instance is alive.
- The home page and login page return successfully.
- A test user can log in.
- One key read path works, such as loading a dashboard.
- One key write path works in a safe test account, if you can do it without side effects.
- The API returns the expected version.
Keep smoke tests short, a minute or two, and fail the pipeline if they fail.
How do you roll back an ECS deploy after database migrations ran?
If migrations have already run, rolling back the code is only safe if the old code works with the new schema.
That is why migrations should be additive in the release that introduces them: add new columns and tables, keep the old ones, and remove them in a later release. If a migration was destructive, rolling back the code will not help, and you may need a point-in-time restore to a copy and a careful data fix.
This is the main reason to rehearse restores. The RDS restore guide shows how.
Are your deployment health checks catching broken releases?
Tick each item that is true for your main service.
- The health endpoint fails when the database is unreachable.
- It does not fail when a payment or email provider is slow.
- It responds in well under a second.
- The ECS grace period covers normal start-up time.
- A smoke test runs after every deploy and can fail the pipeline.
Decision point: If the first item is not ticked, a broken release can pass as healthy. Fix that before anything else.
Should the health check endpoint be public?
The load balancer needs to reach it, but it does not need to be useful to anyone else.
Keep the response minimal, a status code and a short word such as ok, and avoid returning version numbers, dependency names or error details publicly. If you want a detailed status page for your team, put it behind authentication on a separate path.
How often should deployment health checks run?
Often enough to detect a failed instance within a minute or two, but not so often that the checks themselves create load.
A common pattern is an interval of tens of seconds with a small unhealthy threshold, so a task is marked unhealthy after a few consecutive failures. Tune the values so normal blips do not cause flapping and real failures are caught quickly.
Sources
- AWS docs, Health checks for Application Load Balancer target groups (checked October 2026)
- AWS docs, How the Amazon ECS deployment circuit breaker detects failures (checked October 2026)
Keep reading
Keel is an AWS release and recovery operations service for SaaS teams of 3 to 15 developers, provided by Bargainn Services Ltd, a UK company registered at Companies House (16497746). Registered office: 3rd Floor, 86-90 Paul Street, London, EC2A 4NE, United Kingdom.