Move health checks into Docker
Replaces our current host-based HTTP healthchecks with Docker healthchecks, and adds a new `healthcheck.cmd` config option that can be used to define a custom health check command. Also removes Traefik's healthchecks, since they are no longer necessary. When deploying a container that has a healthcheck defined, we wait for it to report a healthy status before stopping the old container that it replaces. Containers that don't have a healthcheck defined continue to wait for `MRSK.config.readiness_delay`. There are some pros and cons to using Docker healthchecks rather than checking from the host. The main advantages are: - Supports non-HTTP checks, and app-specific check scripts provided by a container. - When booting a container, allows MRSK to wait for a container to be healthy before shutting down the old container it replaces. This should be safer than relying on a timeout. - Containers with healthchecks won't be active in Traefik until they reach a healthy state, which prevents any traffic from being routed to them before they are ready. The main _disadvantage_ is that containers are now required to provide some way to check their health. Our default check assumes that `curl` is available in the container which, while common, won't always be the case.
This commit is contained in:
@@ -35,6 +35,21 @@ class Mrsk::Configuration::Role
|
||||
argumentize_env_with_secrets env
|
||||
end
|
||||
|
||||
def health_check_args
|
||||
if health_check_cmd.present?
|
||||
optionize({ "health-cmd" => health_check_cmd, "health-interval" => "1s" })
|
||||
else
|
||||
[]
|
||||
end
|
||||
end
|
||||
|
||||
def health_check_cmd
|
||||
options = specializations["healthcheck"] || {}
|
||||
options = config.healthcheck.merge(options) if running_traefik?
|
||||
|
||||
options["cmd"] || http_health_check(port: options["port"], path: options["path"])
|
||||
end
|
||||
|
||||
def cmd
|
||||
specializations["cmd"]
|
||||
end
|
||||
@@ -75,8 +90,6 @@ class Mrsk::Configuration::Role
|
||||
if running_traefik?
|
||||
{
|
||||
"traefik.http.routers.#{traefik_service}.rule" => "PathPrefix(`/`)",
|
||||
"traefik.http.services.#{traefik_service}.loadbalancer.healthcheck.path" => config.healthcheck["path"],
|
||||
"traefik.http.services.#{traefik_service}.loadbalancer.healthcheck.interval" => "1s",
|
||||
"traefik.http.middlewares.#{traefik_service}-retry.retry.attempts" => "5",
|
||||
"traefik.http.middlewares.#{traefik_service}-retry.retry.initialinterval" => "500ms",
|
||||
"traefik.http.routers.#{traefik_service}.middlewares" => "#{traefik_service}-retry@docker"
|
||||
@@ -125,4 +138,8 @@ class Mrsk::Configuration::Role
|
||||
new_env["clear"] = (clear_app_env + clear_role_env).uniq
|
||||
end
|
||||
end
|
||||
|
||||
def http_health_check(port:, path:)
|
||||
"curl -f #{URI.join("http://localhost:#{port}", path)} || exit 1" if path.present? || port.present?
|
||||
end
|
||||
end
|
||||
|
||||
Reference in New Issue
Block a user