Commit Graph

614 Commits

Author SHA1 Message Date
Donal McBreen
1d0e81b00a Eager load only CLI for faster commands 2024-09-04 09:32:45 +01:00
Donal McBreen
56754fe40c Lazily load secrets whenever needed 2024-09-04 09:32:45 +01:00
Donal McBreen
6a06efc9d9 Strip out env loading, envify, env push 2024-09-04 09:32:45 +01:00
Donal McBreen
5c4c33e0a8 Replace .env* with .kamal/env*
By default look for the env file in .kamal/env to avoid clashes with
other tools using .env.

For now we'll still load .env and issue a deprecation warning, but in
future we'll stop reading those.
2024-09-04 09:32:45 +01:00
Donal McBreen
a2549b1f60 Allow disabling of local builds
To disable local builds set:
```
builder:
   local: false
   remote: ssh://docker@docker-builder
```
2024-09-03 14:33:25 +01:00
Donal McBreen
e557eea79c Build and clean remote builders correctly
Check that the builder and context match what we expect, and if not
remove and re-create them.
2024-09-02 15:12:19 +01:00
Donal McBreen
9f0b10425c Fix aliases tests 2024-08-29 09:16:07 +01:00
Donal McBreen
5f2384f123 Use docker info to get arch 2024-08-29 08:46:18 +01:00
Donal McBreen
eab7d3adc5 Keep buildx build, in case of old docker versions which don't default to buildkit 2024-08-29 08:45:51 +01:00
Donal McBreen
d2d0223c37 Require an arch to be set, and default to amd64 in the template 2024-08-29 08:45:51 +01:00
Donal McBreen
56268d724d Simplify the builders configuration
1. Add driver as an option, defaulting to `docker-container`. For a
   "native" build you can set it to `docker`
2. Set arch as a array of architectures to build for, defaulting to
   `[ "amd64", "arm64" ]` unless you are using the docker driver in
   which case we default to not setting a platform
3. Remote is now just a connection string for the remote builder
4. If remote is set, we only use it for non-local arches, if we are
   only building for the local arch, we'll ignore it.

Examples:

On arm64, build for arm64 locally, amd64 remotely or
On amd64, build for amd64 locally, arm64 remotely:

```yaml
builder:
  remote: ssh://docker@docker-builder
```

On arm64, build amd64 on remote,
On amd64 build locally:

```yaml
builder:
  arch:
    - amd64
  remote:
    host: ssh://docker@docker-builder
```

Build amd64 on local:

```yaml
builder:
  arch:
    - amd64
```

Use docker driver, building for local arch:

```yaml
builder:
  driver: docker
```
2024-08-29 08:45:48 +01:00
Donal McBreen
cffb6c3d7e Allow the driver to be set 2024-08-29 08:44:11 +01:00
Donal McBreen
bd1726f305 docker buildx build -> docker build 2024-08-29 08:44:11 +01:00
Donal McBreen
7ddb122a22 Get tests passing 2024-08-29 08:44:11 +01:00
Donal McBreen
d6a5cf3c78 Rip out context_hosts checks
The remote host is now encoded in the builder name so we don't need
to check it. We'll just do an inspect to confirm the builder exists.
2024-08-29 08:44:11 +01:00
Donal McBreen
d48080c772 Dump native builder
We already ensure that buildx is installed, so let's always use it.
2024-08-29 08:44:11 +01:00
Donal McBreen
3f64338929 Move native remote to just remote
It's just a remote builder, that will build whichever platform is asked
for, so let's remove the "native" part.

We'll also remove the service name from the builder name, so multiple
services can share the same builder.
2024-08-29 08:44:11 +01:00
Donal McBreen
0ab838bc25 Combine multiarch and native/cache builders
Combine the two builders, as they are almost identical. The only
difference was whether the platforms were set.

The native cached builder wasn't using the context it created, so now
we do.

We'll set the driver to `docker-container` - it seems to be the default
but the Docker docs claim it is `docker`.
2024-08-29 08:44:11 +01:00
Donal McBreen
b7382ceeaf Merge pull request #912 from basecamp/alias
Add aliases to Kamal
2024-08-29 08:43:35 +01:00
Donal McBreen
579e169be2 Allow multiple arguments for exec commands
If you can have an alias like:

```
aliases:
  rails: app exec -p rails
```

Then `kamal rails db:migrate:status` will execute
`kamal app exec -p rails db:migrate:status`.

So this works, we'll allow multiple arguments `app exec` and
`server exec` to accept multiple arguments.

The arguments are combined by simply joining them with a space. This
means that these are equivalent:

```
kamal app exec -p rails db:migrate:status
kamal app exec -p "rails db:migrate:status"
```

If you want to pass an argument with spaces, you'll need to quote it:

```
kamal app exec -p "git commit -am \"My comment\""
kamal app exec -p git commit -am "\"My comment\""
```
2024-08-28 10:58:25 +01:00
Donal McBreen
9ccfe20b10 Fix up tests 2024-08-26 11:20:26 +01:00
Donal McBreen
b8af719bb7 Add aliases to Kamal
Aliases are defined in the configuration file under the `aliases` key.

The configuration is a map of alias name to command. When we run the
command the we just do a literal replacement of the alias with the
string.

So if we have:

```yaml
aliases:
  console: app exec -r console -i --reuse "rails console"
```

Then running `kamal console -r workers` will run the command

```sh
$ kamal app exec -r console -i --reuse "rails console" -r workers
```

Because of the order Thor parses the arguments, this allows us to
override the role from the alias command.

There might be cases where we need to munge the command a bit more but
that would involve getting into Thor command parsing internals,
which are complicated and possibly subject to change.

There's a chance that your aliases could conflict with future built-in
commands, but there's not likely to be many of those and if it happens
you'll get a validation error when you upgrade.

Thanks to @dhnaranjo for the idea!
2024-08-26 10:47:43 +01:00
Donal McBreen
4e811372f8 Integration test insecure registry
The integrations tests use their own registry so avoid hitting docker
hub rate limits.

This was using a self signed certificate but instead use
`--insecure-registry` to let the docker daemon use HTTP.
2024-07-31 16:54:00 +01:00
Jeremy Daer
57d9ce177a Pull once from hosts that warm registry mirrors 2024-07-18 09:14:22 -07:00
Donal McBreen
9db6fc0704 Fix the tests when you have a git config email set
The ran ok on CI where we fall back to `whoami`, but failed locally
where there was a git email set.
2024-07-16 12:09:05 +01:00
Donal McBreen
29c723f7ec Add support for configuration extensions
Allow blocks prefixed with `x-` in the configuration as a place to
declare reusable blocks with YAML anchors and aliases.

Borrowed from the Docker Compose configuration file format -
https://github.com/compose-spec/compose-spec/blob/main/spec.md#extension

Thanks to @ruyrocha for the suggestion.
2024-07-15 20:47:55 +01:00
Donal McBreen
0ea07b1760 Merge pull request #878 from pagbrl/main
feat: Use git email as performer when available
2024-07-15 13:41:17 +01:00
Donal McBreen
03b531f179 Merge pull request #865 from basecamp/clean-envify-env
Ensure envify templates aren't polluted by existing env
2024-07-15 13:41:03 +01:00
Donal McBreen
1369c46a83 Seed docker mirrors by pulling once per mirror first
Find the first registry mirror on each host. If we find any, pull the
images on one host per mirror, then do the remainder concurrently.

The initial pulls will seed the mirrors ensuring that we pull the image
from Docker Hub once each.

This works best if there is only one mirror on each host.
2024-07-11 16:20:37 +01:00
Paul Gabriel
deccf1cfaf feat: Use git email as performer when available 2024-07-11 11:19:44 +02:00
Donal McBreen
1573cebadf Merge pull request #868 from nickhammond/env/service
Add ENV['KAMAL_SERVICE'] to hooks
2024-07-10 10:26:59 +01:00
Nick Hammond
85a2926cde Remove the deprecated docker compose version (#869) 2024-06-28 15:00:23 -07:00
Nick Hammond
58a51b079e Add KAMAL_SERVICE to custom hooks and exclude from auditor 2024-06-27 10:52:55 -06:00
Nick Hammond
f1f3fc566f Add ENV['SERVICE'] to hooks 2024-06-27 10:26:11 -06:00
Jeremy Daer
13409ada5a Ensure envify templates aren't polluted by existing env
Setting `GITHUB_TOKEN` as in the docs results in reusing the existing
`GITHUB_TOKEN` since `gh` returns that env var if it's set:
```bash
GITHUB_TOKEN=junk gh config get -h github.com oauth_token
junk
```

Using the original env ensures that the templates will be evaluated the
same way regardless of whether envify had been previously invoked.
2024-06-25 11:14:34 -07:00
Donal McBreen
b63982c3a7 Allow arrays in args
Just check that args is a Hash without checking the value types.

Fixes: https://github.com/basecamp/kamal/issues/863
2024-06-25 14:18:23 +01:00
Donal McBreen
dde637ffff Merge pull request #846 from basecamp/always-log-boot-errors
Log on boot errors with one role
2024-06-20 08:50:03 +01:00
Donal McBreen
f8f88af534 Log on boot errors with one role
We didn't log boot errors if there was one role because there was no
barrier and the logging is done by the first host to close the barrier.

Let's always create the barrier to fix this.
2024-06-20 08:28:37 +01:00
Donal McBreen
3da7fad9ee Revert "Envify already env pushes" 2024-06-20 08:11:18 +01:00
Donal McBreen
da599d90c1 Merge pull request #828 from basecamp/configuration-validation
Configuration validation
2024-06-18 08:31:47 +01:00
Donal McBreen
10e3229d7c Merge pull request #817 from nickhammond/grep-context
Add grep options to log commands
2024-06-13 14:38:54 +01:00
Nick Hammond
c7bd377fa5 Swap grep context with grep options 2024-06-06 09:26:12 -07:00
Donal McBreen
bdd951b756 Merge pull request #832 from basecamp/registry-skips
Allow registry commands to skip local and remote
2024-06-06 08:12:15 +01:00
Donal McBreen
00e0e5073e Allow registry commands to skip local and remote
- Add local logout to `kamal registry logout`
- Add `skip_local` and `skip_remote` options to `kamal registry` commands
- Skip local login in `kamal deploy` when `--skip-push` is used
2024-06-05 12:10:36 +01:00
Donal McBreen
b52e66814a Check that we have valid contexts before building
Load the hosts from the contexts before trying to build.

If there is no context, we'll create one. If there is one but the hosts
don't match we'll re-create.

Where we just have a local context, there won't be any hosts but we
still inspect the builder to check that it exists.
2024-06-05 11:52:45 +01:00
Donal McBreen
4f317b8499 Configuration validation
Validate the Kamal configuration giving useful warning on errors.
Each section of the configuration has its own config class and a YAML
file containing documented example configuration.

You can run `kamal docs` to see the example configuration, and
`kamal docs <section>` to see the example configuration for a specific
section.

The validation matches the configuration to the example configuration
checking that there are no unknown keys and that the values are of
matching types.

Where there is more complex validation - e.g for envs and servers, we
have custom validators that implement those rules.

Additonally the configuration examples are used to generate the
configuration documentation in the kamal-site repo.

You generate them by running:

```
bundle exec bin/docs <kamal-site-checkout>
```
2024-06-04 14:19:29 +01:00
Donal McBreen
2c2053558a Handle corrupt git clones
When cloning the git repo:
1. Try to clone
2. If there's already a build directory reset it
3. Check the clone is valid

If anything goes wrong during that process:
1. Delete the clone directory
2. Clone it again
3. Check the clone is valid

Raise any errors after that
2024-05-27 11:17:34 +01:00
Nick Hammond
eb79d93139 Run RC 2024-05-24 09:16:14 -07:00
Nick Hammond
89994c8b20 Add grep's context option to show lines before and after a match 2024-05-24 08:59:33 -07:00
Donal McBreen
0e73f02743 Split lock and connection setup
Allow run the pre-connect hook before the first SSH command is executed,
but only run the locking in `with_lock` blocks.
2024-05-21 12:02:16 +01:00