[kamal-proxy](https://github.com/basecamp/kamal-proxy) is a custom
minimal proxy designed specifically for Kamal.
It has some advantages over Traefik:
1. Imperative deployments - we tell it to switch from container A to
container B, and it waits for container B to start then switches. No
need to poll for health checks ourselves or mess around with forcing
health checks to fail.
2. Support for multiple apps - as much as possible, configuration is
supplied at runtime by the deploy command, allowing us to have
multiple apps share a proxy without conflicting config.
3. First class support for Kamal operations - rather than trying to
work out how to make Traefik do what we want, we can build features
directly into the proxy, making configuration simpler and avoiding
obscure errors
To speed up deployments, we'll remove the healthcheck step.
This adds some risk to deployments for non-web roles - if they don't
have a Docker healthcheck configured then the only check we do is if
the container is running.
If there is a bad image we might see the container running before it
exits and deploy it. Previously the healthcheck step would have avoided
this by ensuring a web container could boot and serve traffic first.
To mitigate this, we'll add a deployment barrier. Until one of the
primary role containers passes its healthcheck, we'll keep the barrier
up and avoid stopping the containers on the non-primary roles.
It the primary role container fails its healthcheck, we'll close the
barrier and shut down the new containers on the waiting roles.
We also have a new integration test to check we correctly handle a
a broken image. This highlighted that SSHKit's default runner will
stop at the first error it encounters. We'll now have a custom runner
that waits for all threads to finish allowing them to clean up.
Docker does not respect the .dockerignore file when building from a tar.
Instead by default we'll make a local clone into a tmp directory and
build from there. Subsequent builds will reset the clone to match the
checkout.
Compared to building directly in the repo, we'll have reproducible
builds.
Compared to using a git archive:
1. .dockerignore is respected
2. We'll have faster builds - docker can be smarter about caching the
build context on subsequent builds from a directory
To build from the repo directly, set the build context to "." in the
config.
If there are uncommitted changes, we'll warn about them either being
included or ignored depending on whether we build from the clone.
Allow hosts to be tagged so we can have host specific env variables.
We might want host specific env variables for things like datacenter
specific tags or testing GC settings on a specific host.
Right now you either need to set up a separate role, or have the app
be host aware.
Now you can define tag env variables and assign those to hosts.
For example:
```
servers:
- 1.1.1.1
- 1.1.1.2: tag1
- 1.1.1.2: tag2
- 1.1.1.3: [ tag1, tag2 ]
env_tags:
tag1:
ENV1: value1
tag2:
ENV2: value2
```
The tag env supports the full env format, allowing you to set secret and
clear values.
Currently the latest container is the one that was created last. But if
we have had a failed deployment that left two containers running that
would not be the one we want. The second container could be in a
restart loop for example.
Instead we want the container that is running the image tagged as
latest. As we now tag as latest after a successful deployment we can
trust that that is a healthy container.
In the case that there is no container running the latest image tag,
we'll fall back to the latest container.
This could happen if the deploy was halted in between the old container
being stopped and the image being tagged as latest.
If you are deploying more than one destination to a host, the latest
tags will conflict, so we'll append the destination to the tag.
The latest tag is used when booting the app or exec-ing a new container.
If a deploy doesn't complete on a host for all roles then we should
probably not be using it, so move the tagging to the end of the boot
process.
This will allow us to filter for containers that have no destination in
cases where we deploy an empty + a non empty destination to the same
host.
To note:
```
\# Containers with a destination label
$ docker ps --filter label=destination
\# Containers with an empty destination label
$ docker ps --filter label=destination=
```
If no context is specified and we are in a git repo, then we'll build
from a git archive by default. This means we don't need a separate
setting and gives us a safer default build.
Building directly from a checkout will pull in uncommitted files to or
more sneakily files that are git ignored, but not docker ignored.
To avoid this, we'll add an option to build from a git archive of HEAD
instead. Docker doesn't provide a way to build directly from a git
repo, so instead we create a tarball of the current HEAD with git
archive and pipe it into the build command.
When building from a git archive, we'll still display the warning about
uncommitted changes, but we won't add the `_uncommitted_...` suffix to
the container name as they won't be included in the build.
Perhaps this should be the default, but we'll leave that decision for
now.
Secret and clear env variables have different lifecycles. The clear ones
are part of the repo, so it makes sense to always deploy them with the
rest of the repo.
The secret ones are external so we can't be sure that they are up to
date, therefore they require an explicit push via `envify` or `env push`.
We'll keep the env file, but now it just contains secrets. The clear
values are passed directly to `docker run`.
If curl is not available to download the docker install script, try
with wget instead.
If neither is available or both fail, return a simple failing script
so that we don't carry on regardless.
Fixes: https://github.com/basecamp/kamal/issues/395
By default we keep 5 containers around for rollback. The containers
don't take much space, but the images for them can.
Make the number of containers to retain configurable, either in the
config with the `retain_containers` setting on the command line
with the `--retain` option.
If the app container is down or not responding then traefik will return
a 404 response code. This is not ideal as it suggests a client rather
than a server problem.
To fix this, we'll define a catch all route that always returns a 502.
This is not ideal as this route would take priority over a shorter route
with priorty 1.
TODO: up the priority of the app route.
Adding -T to the copy command ensures that the files are copied at the
same level into the target directory whether it exists or not.
That allows us to drop the `/*` which was not picking up hidden files.
Fixes: https://github.com/basecamp/kamal/issues/465
