Commit Graph
100 Commits
Author SHA1 Message Date
Donal McBreen 85f1e14b97 Merge pull request #953 from basecamp/inherit-env-for-secrets
Avoid setting env via SSHKit
2024-09-19 15:18:31 +01:00
Donal McBreen 45a58f7e15 Merge pull request #952 from basecamp/app-exec-in-kamal-network
Run app exec in the kamal network
2024-09-19 14:46:58 +01:00
Donal McBreen 9fe1821cae Merge pull request #951 from basecamp/proxy-config-ownership
Fix /home/kamal-proxy/.config/kamal-proxy ownership
2024-09-19 12:57:10 +01:00
Donal McBreen 3ec4ad2ea5 Merge pull request #949 from basecamp/role-proxy-config
Allow role specific proxy config
2024-09-18 18:27:36 +01:00
Donal McBreen 684f7ac148 Merge pull request #948 from basecamp/deploy-drain-timeouts
Simplified deploy/drain timeouts
2024-09-18 15:18:00 +01:00
Donal McBreen 600bbd77ef Merge pull request #947 from basecamp/doc-updates
Update proxy and docs for Kamal 2.0/kamal-proxy 0.3.0
2024-09-18 14:45:32 +01:00
Donal McBreen e07ac070aa Merge pull request #945 from basecamp/kamal-2-upgrade
Upgrade on accessory hosts only with correct messages
2024-09-18 10:27:28 +01:00
Donal McBreen bd54c74682 Merge pull request #944 from basecamp/single-secrets-instance
Ensure we don't load the secrets more than once
2024-09-17 14:33:38 +01:00
Donal McBreen 190dbd1ea3 Merge pull request #943 from basecamp/response-timeout-string
Need a duration string for the response timeout
2024-09-17 14:14:49 +01:00
Donal McBreen d6eda3d741 Merge pull request #942 from basecamp/kamal-2-upgrade
Upgrade commands for Kamal 1.x -> 2.0
2024-09-17 13:34:00 +01:00
Donal McBreen 434490bd0c Merge pull request #940 from basecamp/proxy
Replace Traefik with kamal-proxy
2024-09-17 09:47:16 +01:00
Donal McBreen 53903ddcd2 Read buffer not buffering 2024-09-16 16:44:21 +01:00
Donal McBreen 55756fa6f3 Set request and response headers 2024-09-16 16:44:21 +01:00
Donal McBreen fe0c656de5 Split buffer requests/responses 2024-09-16 16:44:21 +01:00
Donal McBreen 418d8045d8 Add forward headers support 2024-09-16 16:44:21 +01:00
Donal McBreen d63ff8f251 Set extra fields 2024-09-16 16:44:21 +01:00
Donal McBreen eab717e0cf Add kamal-proxy in experimental mode
The proxy can be enabled via the config:

```
proxy:
  enabled: true
  hosts:
    - 10.0.0.1
    - 10.0.0.2
```

This will enable the proxy and cause it to be run on the hosts listed
under `hosts`, after running `kamal proxy reboot`.

Enabling the proxy disables `kamal traefik` commands and replaces them
with `kamal proxy` ones. However only the marked hosts will run the
kamal-proxy container, the rest will run Traefik as before.
2024-09-16 16:44:19 +01:00
Donal McBreen 66d5e25834 Merge pull request #939 from basecamp/secrets-mutex
Add a mutex around loading secrets
2024-09-16 16:26:58 +01:00
Donal McBreen 876eebc7c5 Merge pull request #936 from basecamp/asset-extraction-entrypoint
Override the entrypoint when extracting assets
2024-09-13 17:46:43 +01:00
Donal McBreen 045aa7d167 Merge pull request #934 from basecamp/secrets-dont-exit
Don't exit from failed secrets commands
2024-09-11 16:27:33 +01:00
Donal McBreen debdf00cca Merge pull request #933 from basecamp/common-secrets
Add secrets-common for shared secrets
2024-09-11 14:20:01 +01:00
Donal McBreen c9946808b1 Merge pull request #931 from basecamp/dont-git-ignore-dot-kamal-secrets
Don't git ignore .kamal/secrets
2024-09-11 13:26:07 +01:00
Donal McBreen deb2a6d298 Merge pull request #930 from basecamp/hide-1password-login-error
Hide the 1password login error
2024-09-11 12:18:09 +01:00
Donal McBreen 63d0b5ddfa Merge pull request #928 from basecamp/kamal-secrets-inline-aware
Make the secrets commands inline aware
2024-09-10 11:08:10 +01:00
Donal McBreen 5aa3d1aeb0 Merge pull request #927 from basecamp/revert-903-integration-test-insecure-registry
Revert "Integration test insecure registry"
2024-09-10 10:12:57 +01:00
Donal McBreen 7156c80f34 Merge pull request #924 from basecamp/secrets
Secrets
2024-09-09 15:13:35 +01:00
Donal McBreenandSijawusz Pur Rahnama b99c044327 Update lib/kamal/cli/templates/secrets
Co-authored-by: Sijawusz Pur Rahnama <sija@sija.pl>
2024-09-06 13:25:39 +01:00
Donal McBreen b464c4fd4a Include dotenv upgrade 2024-09-04 09:32:45 +01:00
Donal McBreen 56754fe40c Lazily load secrets whenever needed 2024-09-04 09:32:45 +01:00
Donal McBreen 6a06efc9d9 Strip out env loading, envify, env push 2024-09-04 09:32:45 +01:00
Donal McBreen 5c4c33e0a8 Replace .env* with .kamal/env*
By default look for the env file in .kamal/env to avoid clashes with
other tools using .env.

For now we'll still load .env and issue a deprecation warning, but in
future we'll stop reading those.
2024-09-04 09:32:45 +01:00
Donal McBreen 0b5506f6f2 Merge pull request #923 from basecamp/disable-local-builder
Allow disabling of local builds
2024-09-03 14:53:23 +01:00
Donal McBreen 9b9e60ec7f Merge pull request #921 from basecamp/remote-hybrid-builders-cleanup
Build and clean remote builders correctly
2024-09-02 15:24:28 +01:00
Donal McBreen 6adf3c117f Merge pull request #905 from basecamp/simplify-builders-config
Simplify builders config
2024-08-29 09:28:51 +01:00
Donal McBreen 56268d724d Simplify the builders configuration
1. Add driver as an option, defaulting to `docker-container`. For a
   "native" build you can set it to `docker`
2. Set arch as a array of architectures to build for, defaulting to
   `[ "amd64", "arm64" ]` unless you are using the docker driver in
   which case we default to not setting a platform
3. Remote is now just a connection string for the remote builder
4. If remote is set, we only use it for non-local arches, if we are
   only building for the local arch, we'll ignore it.

Examples:

On arm64, build for arm64 locally, amd64 remotely or
On amd64, build for amd64 locally, arm64 remotely:

```yaml
builder:
  remote: ssh://docker@docker-builder
```

On arm64, build amd64 on remote,
On amd64 build locally:

```yaml
builder:
  arch:
    - amd64
  remote:
    host: ssh://docker@docker-builder
```

Build amd64 on local:

```yaml
builder:
  arch:
    - amd64
```

Use docker driver, building for local arch:

```yaml
builder:
  driver: docker
```
2024-08-29 08:45:48 +01:00
Donal McBreen cffb6c3d7e Allow the driver to be set 2024-08-29 08:44:11 +01:00
Donal McBreen bd1726f305 docker buildx build -> docker build 2024-08-29 08:44:11 +01:00
Donal McBreen 7ddb122a22 Get tests passing 2024-08-29 08:44:11 +01:00
Donal McBreen 98c951bbdb Simplfy choosing a builder 2024-08-29 08:44:11 +01:00
Donal McBreen 374c117b79 Validate multiarch configuration
Remote and local are only allowed when multiarch is enabled.
Remote requires a host and arch, local only requires an arch.
2024-08-29 08:44:11 +01:00
Donal McBreen d6a5cf3c78 Rip out context_hosts checks
The remote host is now encoded in the builder name so we don't need
to check it. We'll just do an inspect to confirm the builder exists.
2024-08-29 08:44:11 +01:00
Donal McBreen 2aeabda455 Move multiarch remote builder to hybrid builder
Include the host name in the builder name, so we can have one builder
per host/arch across all kamal projects.

Inherit from the remote builder. The difference in the hybrid builder
is that we create a local buildx instance and append the remote context
to it.
2024-08-29 08:44:11 +01:00
Donal McBreen c048c097ed Create a context for local builds
This ensures we use the docker-container driver and not whatever the
local default is.
2024-08-29 08:44:11 +01:00
Donal McBreen ed148628fb Local build doesn't need a builder 2024-08-29 08:44:11 +01:00
Donal McBreen d48080c772 Dump native builder
We already ensure that buildx is installed, so let's always use it.
2024-08-29 08:44:11 +01:00
Donal McBreen 3f64338929 Move native remote to just remote
It's just a remote builder, that will build whichever platform is asked
for, so let's remove the "native" part.

We'll also remove the service name from the builder name, so multiple
services can share the same builder.
2024-08-29 08:44:11 +01:00
Donal McBreen 0ab838bc25 Combine multiarch and native/cache builders
Combine the two builders, as they are almost identical. The only
difference was whether the platforms were set.

The native cached builder wasn't using the context it created, so now
we do.

We'll set the driver to `docker-container` - it seems to be the default
but the Docker docs claim it is `docker`.
2024-08-29 08:44:11 +01:00
Donal McBreen b7382ceeaf Merge pull request #912 from basecamp/alias
Add aliases to Kamal
2024-08-29 08:43:35 +01:00
Donal McBreen 69367fbc6b Merge pull request #917 from basecamp/v2.0-alpha
Switch the version on main to 2.0.0.alpha
2024-08-29 08:43:19 +01:00
Donal McBreen e871d347d5 Merge pull request #889 from xiaohui-zhangxh/git-clone-update-submodules
git clone with --recurse-submodules
2024-08-26 11:20:05 +01:00
Donal McBreen b8af719bb7 Add aliases to Kamal
Aliases are defined in the configuration file under the `aliases` key.

The configuration is a map of alias name to command. When we run the
command the we just do a literal replacement of the alias with the
string.

So if we have:

```yaml
aliases:
  console: app exec -r console -i --reuse "rails console"
```

Then running `kamal console -r workers` will run the command

```sh
$ kamal app exec -r console -i --reuse "rails console" -r workers
```

Because of the order Thor parses the arguments, this allows us to
override the role from the alias command.

There might be cases where we need to munge the command a bit more but
that would involve getting into Thor command parsing internals,
which are complicated and possibly subject to change.

There's a chance that your aliases could conflict with future built-in
commands, but there's not likely to be many of those and if it happens
you'll get a validation error when you upgrade.

Thanks to @dhnaranjo for the idea!
2024-08-26 10:47:43 +01:00
Donal McBreen f48987aa03 Merge pull request #903 from basecamp/integration-test-insecure-registry
Integration test insecure registry
2024-08-01 09:57:17 +01:00
Donal McBreen ef051eca1b Merge pull request #904 from galori/main
Fixed typo in `env.yml`: "valies" --> "values"
2024-08-01 09:57:03 +01:00
Donal McBreen 4e811372f8 Integration test insecure registry
The integrations tests use their own registry so avoid hitting docker
hub rate limits.

This was using a self signed certificate but instead use
`--insecure-registry` to let the docker daemon use HTTP.
2024-07-31 16:54:00 +01:00
Donal McBreen ec4aa45852 Bump version for 1.8.1 2024-07-29 09:09:57 +01:00
Donal McBreen 5e11a64181 Merge pull request #891 from basecamp/single-pull
Pull once from hosts that warm registry mirrors
2024-07-22 08:18:48 +01:00
Donal McBreen 8a98949634 Merge pull request #886 from guoard/patch-2
Remove `--update` flag from `apk add` command
2024-07-16 15:46:37 +01:00
Donal McBreen 0eb9f48082 Merge pull request #887 from basecamp/fix-tests-with-git-config
Fix the tests when you have a git config email set
2024-07-16 13:08:18 +01:00
Donal McBreen 9db6fc0704 Fix the tests when you have a git config email set
The ran ok on CI where we fall back to `whoami`, but failed locally
where there was a git email set.
2024-07-16 12:09:05 +01:00
Donal McBreen 27fede3caa Merge pull request #884 from basecamp/x-config
Add support for configuration extensions
2024-07-16 11:38:28 +01:00
Donal McBreen 29c723f7ec Add support for configuration extensions
Allow blocks prefixed with `x-` in the configuration as a place to
declare reusable blocks with YAML anchors and aliases.

Borrowed from the Docker Compose configuration file format -
https://github.com/compose-spec/compose-spec/blob/main/spec.md#extension

Thanks to @ruyrocha for the suggestion.
2024-07-15 20:47:55 +01:00
Donal McBreen fa73d722ea Bump version for 1.8.0 2024-07-15 14:21:23 +01:00
Donal McBreen c535e4e44f Merge pull request #883 from basecamp/revert-840-main
Revert "Add x25519 gem, support Curve25519"
2024-07-15 13:56:49 +01:00
Donal McBreen 0ea07b1760 Merge pull request #878 from pagbrl/main
feat: Use git email as performer when available
2024-07-15 13:41:17 +01:00
Donal McBreen 03b531f179 Merge pull request #865 from basecamp/clean-envify-env
Ensure envify templates aren't polluted by existing env
2024-07-15 13:41:03 +01:00
Donal McBreen d8570d1c2c Merge pull request #847 from basecamp/remove-ruby-2.7-from-ci
Remove Ruby 2.7 from CI
2024-07-15 13:40:37 +01:00
Donal McBreen 3fe70b458d Merge pull request #862 from jeromedalbert/bump-sshkit
Bump sshkit to support unbracketed IPv6 addresses
2024-07-15 13:40:18 +01:00
Donal McBreen ade8b43599 Merge pull request #866 from acidtib/ssh-key-overwrite
Configurable SSH Identity
2024-07-15 13:39:51 +01:00
Donal McBreen 7c244bbb98 Merge pull request #879 from basecamp/seed-mirror
Seed docker mirrors by pulling once per mirror first
2024-07-15 13:30:53 +01:00
Donal McBreen 1369c46a83 Seed docker mirrors by pulling once per mirror first
Find the first registry mirror on each host. If we find any, pull the
images on one host per mirror, then do the remainder concurrently.

The initial pulls will seed the mirrors ensuring that we pull the image
from Docker Hub once each.

This works best if there is only one mirror on each host.
2024-07-11 16:20:37 +01:00
Donal McBreen 1573cebadf Merge pull request #868 from nickhammond/env/service
Add ENV['KAMAL_SERVICE'] to hooks
2024-07-10 10:26:59 +01:00
Donal McBreen 9a1379be6c Bump version for 1.7.3 2024-06-25 15:03:02 +01:00
Donal McBreen 31d6c198da Merge pull request #861 from K4sku/update-docker-setup-sample-hook
Expand on docker-setup.sample hook
2024-06-25 14:44:13 +01:00
Donal McBreen 22afe4de77 Merge pull request #864 from basecamp/allow-arrays-in-args
Allow arrays in args
2024-06-25 14:41:07 +01:00
Donal McBreen b63982c3a7 Allow arrays in args
Just check that args is a Hash without checking the value types.

Fixes: https://github.com/basecamp/kamal/issues/863
2024-06-25 14:18:23 +01:00
Donal McBreen ff03891d47 Bump version for 1.7.2 2024-06-24 10:11:27 +01:00
Donal McBreen f21dc30875 Merge pull request #858 from basecamp/match-does-not-exist
Match a "does not exist" error message
2024-06-24 09:54:25 +01:00
Donal McBreen 69fa7286e2 Match a "does not exist" error message
Only show the warning for building when we are actually going to do that
and match `does not exist` in the error message.

Fixes: https://github.com/basecamp/kamal/issues/851
2024-06-24 08:21:03 +01:00
Donal McBreen e160852e4d Remove Ruby 2.7 from CI
It's EOL since March 2023.
2024-06-20 08:54:55 +01:00
Donal McBreen 4697f89441 Bump version for 1.7.1 2024-06-20 08:50:37 +01:00
Donal McBreen dde637ffff Merge pull request #846 from basecamp/always-log-boot-errors
Log on boot errors with one role
2024-06-20 08:50:03 +01:00
Donal McBreen f8f88af534 Log on boot errors with one role
We didn't log boot errors if there was one role because there was no
barrier and the logging is done by the first host to close the barrier.

Let's always create the barrier to fix this.
2024-06-20 08:28:37 +01:00
Donal McBreen f6a9698f55 Merge pull request #845 from basecamp/revert-815-envify-already-pushes-env
Revert "Envify already env pushes"
2024-06-20 08:22:06 +01:00
Donal McBreen 1109a864d0 Bump version for 1.7.0 2024-06-18 10:33:02 +01:00
Donal McBreen da599d90c1 Merge pull request #828 from basecamp/configuration-validation
Configuration validation
2024-06-18 08:31:47 +01:00
Donal McBreen 6bf3f4888a Allow aliases still 2024-06-18 08:20:27 +01:00
Donal McBreen 0a6b0b7133 Merge pull request #840 from HLFH/main
Add x25519 gem, support Curve25519
2024-06-18 08:17:48 +01:00
Donal McBreen 10e3229d7c Merge pull request #817 from nickhammond/grep-context
Add grep options to log commands
2024-06-13 14:38:54 +01:00
Donal McBreen bdd951b756 Merge pull request #832 from basecamp/registry-skips
Allow registry commands to skip local and remote
2024-06-06 08:12:15 +01:00
Donal McBreen 080897dc4d Merge pull request #831 from basecamp/check-buildx-contexts
Check that we have valid contexts before building
2024-06-06 08:12:05 +01:00
Donal McBreen d652221100 Merge pull request #818 from fabiosammy/fix-header-docker-setup-template
Fix the header template of the docker-setup hook
2024-06-05 12:18:59 +01:00
Donal McBreen 00e0e5073e Allow registry commands to skip local and remote
- Add local logout to `kamal registry logout`
- Add `skip_local` and `skip_remote` options to `kamal registry` commands
- Skip local login in `kamal deploy` when `--skip-push` is used
2024-06-05 12:10:36 +01:00
Donal McBreen b52e66814a Check that we have valid contexts before building
Load the hosts from the contexts before trying to build.

If there is no context, we'll create one. If there is one but the hosts
don't match we'll re-create.

Where we just have a local context, there won't be any hosts but we
still inspect the builder to check that it exists.
2024-06-05 11:52:45 +01:00
Donal McBreen 29fbe7a98f Remove redundant Kamal::Configuration:: 2024-06-04 16:45:39 +01:00
Donal McBreen 4f317b8499 Configuration validation
Validate the Kamal configuration giving useful warning on errors.
Each section of the configuration has its own config class and a YAML
file containing documented example configuration.

You can run `kamal docs` to see the example configuration, and
`kamal docs <section>` to see the example configuration for a specific
section.

The validation matches the configuration to the example configuration
checking that there are no unknown keys and that the values are of
matching types.

Where there is more complex validation - e.g for envs and servers, we
have custom validators that implement those rules.

Additonally the configuration examples are used to generate the
configuration documentation in the kamal-site repo.

You generate them by running:

```
bundle exec bin/docs <kamal-site-checkout>
```
2024-06-04 14:19:29 +01:00
Donal McBreen 6e60ab918a Bump version for 1.6.0 2024-06-03 08:34:12 +01:00
Donal McBreen 90ecb6a12a Merge pull request #821 from basecamp/retry-clone
Handle corrupt git clones
2024-05-28 15:23:36 +01:00
Donal McBreen 2c2053558a Handle corrupt git clones
When cloning the git repo:
1. Try to clone
2. If there's already a build directory reset it
3. Check the clone is valid

If anything goes wrong during that process:
1. Delete the clone directory
2. Clone it again
3. Check the clone is valid

Raise any errors after that
2024-05-27 11:17:34 +01:00
Donal McBreen 10b8c826d8 Merge pull request #755 from basecamp/lock-less
Lock less
2024-05-21 12:33:45 +01:00
Donal McBreen 187861fa60 Space not tab 2024-05-21 12:20:19 +01:00