Commit Graph

272 Commits

Author SHA1 Message Date
Donal McBreen
24031fefb0 Remove proxy only if no apps are installed 2024-09-16 16:44:58 +01:00
Donal McBreen
35fe9c154d Move audits back to run dir so they survive kamal remove 2024-09-16 16:44:58 +01:00
Donal McBreen
c21757f747 Move all files on the host under a common directory
This will make running kamal remove simpler, we can just clean up that
directory.
2024-09-16 16:44:58 +01:00
Donal McBreen
cb73c730f9 No need for run_id 2024-09-16 16:44:58 +01:00
Donal McBreen
f4d309c5cc Rip out Traefik 2024-09-16 16:44:55 +01:00
Donal McBreen
5bca8015bc Map kamal proxy config into .kamal/proxy/config
This will allow us to share files with the proxy via the host.
2024-09-16 16:44:41 +01:00
Donal McBreen
27a7b339a6 Drop run_directory configuration option
We need to drop to be fixed so multiple applications put the config in
the same place.
2024-09-16 16:44:41 +01:00
Donal McBreen
dcd4778dd9 Port -> app_port 2024-09-16 16:44:41 +01:00
Donal McBreen
6f2eaed398 Work out the host and port for the container
Avoid docker inspect:
1. Use the container ID as the host
2. Configure the port, default to 3000
2024-09-16 16:44:41 +01:00
Donal McBreen
2fdc59a3aa Fix tests 2024-09-16 16:44:41 +01:00
Donal McBreen
2056351c38 Use kamal network for accessories 2024-09-16 16:44:41 +01:00
Donal McBreen
63ebeda489 Create proxy and app containers in a kamal network 2024-09-16 16:44:41 +01:00
Donal McBreen
13bdf50ceb Fix tests for proxy defaults and required builder arch 2024-09-16 16:44:41 +01:00
Donal McBreen
eab717e0cf Add kamal-proxy in experimental mode
The proxy can be enabled via the config:

```
proxy:
  enabled: true
  hosts:
    - 10.0.0.1
    - 10.0.0.2
```

This will enable the proxy and cause it to be run on the hosts listed
under `hosts`, after running `kamal proxy reboot`.

Enabling the proxy disables `kamal traefik` commands and replaces them
with `kamal proxy` ones. However only the marked hosts will run the
kamal-proxy container, the rest will run Traefik as before.
2024-09-16 16:44:19 +01:00
Donal McBreen
dc1bbac3c8 Override the entrypoint when extracting assets
When overriding the command, docker will still run the entrypoint. We
want to avoid that here - we just want to get the assets out as quickly
as possible. Otherwise maybe something important is going on when we
stop the container.
2024-09-12 19:31:18 +01:00
Donal McBreen
aed2ef99d0 Use env files for secrets
Add env files back in for secrets - hides them from process lists and
allows you to pick up the latest env file when running
`kamal app exec` without reusing.
2024-09-09 14:43:12 +01:00
Donal McBreen
1522d94ac9 Pass secrets to pre/post deploy hooks 2024-09-04 16:24:10 +01:00
Donal McBreen
56754fe40c Lazily load secrets whenever needed 2024-09-04 09:32:45 +01:00
Donal McBreen
a2549b1f60 Allow disabling of local builds
To disable local builds set:
```
builder:
   local: false
   remote: ssh://docker@docker-builder
```
2024-09-03 14:33:25 +01:00
Donal McBreen
e557eea79c Build and clean remote builders correctly
Check that the builder and context match what we expect, and if not
remove and re-create them.
2024-09-02 15:12:19 +01:00
Donal McBreen
5f2384f123 Use docker info to get arch 2024-08-29 08:46:18 +01:00
Donal McBreen
eab7d3adc5 Keep buildx build, in case of old docker versions which don't default to buildkit 2024-08-29 08:45:51 +01:00
Donal McBreen
d2d0223c37 Require an arch to be set, and default to amd64 in the template 2024-08-29 08:45:51 +01:00
Donal McBreen
56268d724d Simplify the builders configuration
1. Add driver as an option, defaulting to `docker-container`. For a
   "native" build you can set it to `docker`
2. Set arch as a array of architectures to build for, defaulting to
   `[ "amd64", "arm64" ]` unless you are using the docker driver in
   which case we default to not setting a platform
3. Remote is now just a connection string for the remote builder
4. If remote is set, we only use it for non-local arches, if we are
   only building for the local arch, we'll ignore it.

Examples:

On arm64, build for arm64 locally, amd64 remotely or
On amd64, build for amd64 locally, arm64 remotely:

```yaml
builder:
  remote: ssh://docker@docker-builder
```

On arm64, build amd64 on remote,
On amd64 build locally:

```yaml
builder:
  arch:
    - amd64
  remote:
    host: ssh://docker@docker-builder
```

Build amd64 on local:

```yaml
builder:
  arch:
    - amd64
```

Use docker driver, building for local arch:

```yaml
builder:
  driver: docker
```
2024-08-29 08:45:48 +01:00
Donal McBreen
bd1726f305 docker buildx build -> docker build 2024-08-29 08:44:11 +01:00
Donal McBreen
7ddb122a22 Get tests passing 2024-08-29 08:44:11 +01:00
Donal McBreen
d6a5cf3c78 Rip out context_hosts checks
The remote host is now encoded in the builder name so we don't need
to check it. We'll just do an inspect to confirm the builder exists.
2024-08-29 08:44:11 +01:00
Donal McBreen
d48080c772 Dump native builder
We already ensure that buildx is installed, so let's always use it.
2024-08-29 08:44:11 +01:00
Donal McBreen
3f64338929 Move native remote to just remote
It's just a remote builder, that will build whichever platform is asked
for, so let's remove the "native" part.

We'll also remove the service name from the builder name, so multiple
services can share the same builder.
2024-08-29 08:44:11 +01:00
Donal McBreen
0ab838bc25 Combine multiarch and native/cache builders
Combine the two builders, as they are almost identical. The only
difference was whether the platforms were set.

The native cached builder wasn't using the context it created, so now
we do.

We'll set the driver to `docker-container` - it seems to be the default
but the Docker docs claim it is `docker`.
2024-08-29 08:44:11 +01:00
Donal McBreen
0ea07b1760 Merge pull request #878 from pagbrl/main
feat: Use git email as performer when available
2024-07-15 13:41:17 +01:00
Donal McBreen
1369c46a83 Seed docker mirrors by pulling once per mirror first
Find the first registry mirror on each host. If we find any, pull the
images on one host per mirror, then do the remainder concurrently.

The initial pulls will seed the mirrors ensuring that we pull the image
from Docker Hub once each.

This works best if there is only one mirror on each host.
2024-07-11 16:20:37 +01:00
Paul Gabriel
deccf1cfaf feat: Use git email as performer when available 2024-07-11 11:19:44 +02:00
Nick Hammond
58a51b079e Add KAMAL_SERVICE to custom hooks and exclude from auditor 2024-06-27 10:52:55 -06:00
Donal McBreen
b63982c3a7 Allow arrays in args
Just check that args is a Hash without checking the value types.

Fixes: https://github.com/basecamp/kamal/issues/863
2024-06-25 14:18:23 +01:00
Donal McBreen
da599d90c1 Merge pull request #828 from basecamp/configuration-validation
Configuration validation
2024-06-18 08:31:47 +01:00
Donal McBreen
10e3229d7c Merge pull request #817 from nickhammond/grep-context
Add grep options to log commands
2024-06-13 14:38:54 +01:00
Nick Hammond
c7bd377fa5 Swap grep context with grep options 2024-06-06 09:26:12 -07:00
Donal McBreen
b52e66814a Check that we have valid contexts before building
Load the hosts from the contexts before trying to build.

If there is no context, we'll create one. If there is one but the hosts
don't match we'll re-create.

Where we just have a local context, there won't be any hosts but we
still inspect the builder to check that it exists.
2024-06-05 11:52:45 +01:00
Donal McBreen
4f317b8499 Configuration validation
Validate the Kamal configuration giving useful warning on errors.
Each section of the configuration has its own config class and a YAML
file containing documented example configuration.

You can run `kamal docs` to see the example configuration, and
`kamal docs <section>` to see the example configuration for a specific
section.

The validation matches the configuration to the example configuration
checking that there are no unknown keys and that the values are of
matching types.

Where there is more complex validation - e.g for envs and servers, we
have custom validators that implement those rules.

Additonally the configuration examples are used to generate the
configuration documentation in the kamal-site repo.

You generate them by running:

```
bundle exec bin/docs <kamal-site-checkout>
```
2024-06-04 14:19:29 +01:00
Nick Hammond
89994c8b20 Add grep's context option to show lines before and after a match 2024-05-24 08:59:33 -07:00
Donal McBreen
0efb5ccfff Remove the healthcheck step
To speed up deployments, we'll remove the healthcheck step.

This adds some risk to deployments for non-web roles - if they don't
have a Docker healthcheck configured then the only check we do is if
the container is running.

If there is a bad image we might see the container running before it
exits and deploy it. Previously the healthcheck step would have avoided
this by ensuring a web container could boot and serve traffic first.

To mitigate this, we'll add a deployment barrier. Until one of the
primary role containers passes its healthcheck, we'll keep the barrier
up and avoid stopping the containers on the non-primary roles.

It the primary role container fails its healthcheck, we'll close the
barrier and shut down the new containers on the waiting roles.

We also have a new integration test to check we correctly handle a
a broken image. This highlighted that SSHKit's default runner will
stop at the first error it encounters. We'll now have a custom runner
that waits for all threads to finish allowing them to clean up.
2024-05-20 12:18:30 +01:00
Donal McBreen
307750ff70 Build from within a git clone by default
Docker does not respect the .dockerignore file when building from a tar.

Instead by default we'll make a local clone into a tmp directory and
build from there. Subsequent builds will reset the clone to match the
checkout.

Compared to building directly in the repo, we'll have reproducible
builds.

Compared to using a git archive:
1. .dockerignore is respected
2. We'll have faster builds - docker can be smarter about caching the
build context on subsequent builds from a directory

To build from the repo directly, set the build context to "." in the
config.

If there are uncommitted changes, we'll warn about them either being
included or ignored depending on whether we build from the clone.
2024-05-20 09:30:56 +01:00
Donal McBreen
f48c227768 Move env_tags under env key
Instead of:

```
env:
  CLEAR_TAG: untagged
env_tags:
  tag1:
    CLEAR_TAG: tagged
```

We'll have:

```
env:
  clear:
    CLEAR_TAG: untagged
  tags:
    tag1:
      CLEAR_TAG: tagged
```
2024-05-15 10:19:22 +01:00
Donal McBreen
f8b7f74543 Merge pull request #786 from hundredwatt/add-target-option-to-builder
Add --target option to Builder to support multi-stage Docker builds
2024-05-10 15:15:31 +01:00
Donal McBreen
6d062ce271 Host specific env with tags
Allow hosts to be tagged so we can have host specific env variables.

We might want host specific env variables for things like datacenter
specific tags or testing GC settings on a specific host.

Right now you either need to set up a separate role, or have the app
be host aware.

Now you can define tag env variables and assign those to hosts.

For example:
```
servers:
  - 1.1.1.1
  - 1.1.1.2: tag1
  - 1.1.1.2: tag2
  - 1.1.1.3: [ tag1, tag2 ]
env_tags:
  tag1:
    ENV1: value1
  tag2:
    ENV2: value2
```

The tag env supports the full env format, allowing you to set secret and
clear values.
2024-05-09 16:02:45 +01:00
Jason Nochlin
947be0877f add --target option for builder configuration 2024-04-27 10:24:47 -06:00
Donal McBreen
1f5b936fa2 Escape single quotes to fix log following
Fixes: https://github.com/basecamp/kamal/issues/777
2024-04-26 14:16:19 +01:00
Donal McBreen
5e492ecc4d Merge pull request #748 from basecamp/latest-by-tag
Latest by tag
2024-04-03 09:11:03 +01:00
Donal McBreen
8a6a51977f Set env variables when running kamal app exec
Allow additional env variable to be set when running `kamal app exec`.
Works for both new and existing containers.
2024-04-01 15:01:32 +01:00