Compare commits

..

286 Commits

Author SHA1 Message Date
Donal McBreen
3654a7e1be Bump version for 2.2.1 2024-10-09 14:46:44 -04:00
Donal McBreen
6a7783c979 Merge pull request #1086 from basecamp/proxy-1.8.1
Bump proxy to version 0.8.1
2024-10-09 14:46:09 -04:00
Donal McBreen
7dc2609b77 Merge pull request #1082 from graysonchen/patch-1
ERROR (ArgumentError): Unknown boot_config subcommand clear
2024-10-09 14:03:07 -04:00
Donal McBreen
74960499c0 Bump proxy to version 0.8.1
Fixes issue where incorrect status code may be returned when buffering
responses.

https://github.com/basecamp/kamal-proxy/releases/tag/v0.8.1
2024-10-09 14:00:38 -04:00
Grayson Chen
50c96e36c0 typo clear change to reset
kamal proxy boot_config clear
 
 ERROR (ArgumentError): Unknown boot_config subcommand clear
2024-10-09 00:11:54 +08:00
Donal McBreen
7b48648bf2 Bump version for 2.2.0 2024-10-08 08:59:23 -04:00
Donal McBreen
91df935d05 Merge pull request #1076 from basecamp/active-support-require-for-to-sentence
Add Active Support require for to_sentence
2024-10-08 07:46:53 -04:00
Donal McBreen
bbfcbfa94b Merge pull request #1064 from basecamp/kamal-proxy-0.8.0
Update to kamal-proxy 0.8.0
2024-10-08 07:42:48 -04:00
Donal McBreen
440044b900 Merge pull request #1072 from basecamp/proxy-default-10m-logs
Default to keeping 10m of proxy logs
2024-10-08 07:35:38 -04:00
Donal McBreen
06419f8749 Add Active Support require for to_sentence
Fixes: https://github.com/basecamp/kamal/issues/1061
2024-10-08 07:33:03 -04:00
Donal McBreen
67ce1912f7 Default to keeping 10m of proxy logs
Match the defaults for the application containers of 10m of logs.

Allow them to be altered with the proxy boot_config set command.
2024-10-07 16:20:40 -04:00
Donal McBreen
a434b10bfd Update to kamal-proxy 0.8.0
Proxy changes:
- Add option to use custom TLS certificates (#17)
- Don't buffer SSE responses (#36)
- Allow routing to wildcard subdomains (#45)

Custom TLS certificates not supported in Kamal itself yet. Buffering
SSE responses and wildcard subdomains will work without any Kamal
changes.
2024-10-06 13:48:00 -04:00
Donal McBreen
e34031f70c Bump version for 2.1.2 2024-10-06 13:40:53 -04:00
Donal McBreen
23898a5197 Merge pull request #1062 from basecamp/skip-proxy-flag-ssl-false
Skip setting the proxy flag when ssl is false
2024-10-06 18:32:45 +01:00
Donal McBreen
1e9c9e9103 Skip setting the proxy flag when ssl is false
Fixes: https://github.com/basecamp/kamal/issues/1037
2024-10-06 13:22:43 -04:00
David Heinemeier Hansson
4b2c9cdc72 Merge pull request #1026 from ehutzelman/patch-1
Update init description for kamal secrets
2024-10-05 01:54:08 +02:00
David Heinemeier Hansson
80191588c2 Merge pull request #1050 from tiramizoo/template-docker-setup
Update sample template for docker setup hook.
2024-10-05 01:46:09 +02:00
David Heinemeier Hansson
5ca806f4d3 Merge pull request #1054 from tuladhar/cloudflare-ssl
Clarify SSL comment when using Cloudflare
2024-10-04 21:03:23 +02:00
Puru
1d04a6644f Clarify SSL comment when using Cloudflare 2024-10-05 00:45:04 +05:45
Wojciech Wnętrzak
950624d667 Update sample template for docker setup hook.
"kamal" network is already created (in v2.0) so the sample code is
no longer accurate.
2024-10-04 09:27:17 +02:00
David Heinemeier Hansson
81f3508507 Bump version for 2.1.1 2024-10-03 11:39:56 -07:00
David Heinemeier Hansson
9a16873f21 Merge pull request #1035 from basecamp/fix-kamal-setup-on-accessory-hosts
Restore kamal setup to accessory hosts
2024-10-03 19:25:33 +02:00
David Heinemeier Hansson
e5ca53db6e Use new deploy config so as not to update all other tests 2024-10-02 17:34:13 -07:00
David Heinemeier Hansson
82a436fa02 Rubocop 2024-10-02 17:07:51 -07:00
David Heinemeier Hansson
7be2e7e0ba Test accessory_hosts with roles and without filtering 2024-10-02 17:03:30 -07:00
David Heinemeier Hansson
4f7ebd73a3 Specifics#accessory_hosts was being filtered out by role host check 2024-10-02 16:30:32 -07:00
Donal McBreen
279bda2770 Bump version for 2.1.0 2024-10-02 11:35:45 +01:00
Donal McBreen
aa15fa532a Merge pull request #1024 from jeromedalbert/follow-primary-role
Follow logs on primary role by default
2024-10-02 09:29:16 +01:00
Donal McBreen
276b469c2b Merge pull request #1019 from nickhammond/ENV/destination
Set KAMAL_DESTINATION when loading config
2024-10-02 09:28:01 +01:00
Donal McBreen
c10b3fb07a Merge pull request #908 from basecamp/ignorable-ssh-config
SSH: allow setting `config: false` to ignore local user `~/.ssh/config`
2024-10-02 08:28:57 +01:00
Eric Hutzelman
0ff1450a74 Update init description for kamal secrets
No longer uses .env stub, replace with secrets stub in .kamal directory.
2024-10-01 18:49:08 -05:00
Jerome Dalbert
f47fd13e5b Follow logs on primary role by default
Do not default `kamal app logs -f` to use the hardcoded “web” role;
instead use the primary role, which can be different from “web”.
2024-10-01 13:51:09 -07:00
Nick Hammond
1d8c40f5d2 Run RC 2024-10-01 08:20:21 -07:00
Nick Hammond
73c78079bc Set KAMAL_DESTINATION when loading config 2024-10-01 08:16:52 -07:00
Donal McBreen
cd12f95a97 Merge pull request #1018 from basecamp/kamal-proxy-deploy-equals-in-args
Use `=` in kamal-proxy deploy command args
2024-10-01 15:51:43 +01:00
Donal McBreen
641e9056b3 Use = in kamal-proxy deploy command args
`=` is required for boolean values and works for all values.
2024-10-01 15:42:12 +01:00
Donal McBreen
b4bcf35f78 Merge pull request #1000 from kpumuk/hosts
Allow specifying multiple hosts for kamal-proxy via an array
2024-09-30 10:26:15 -04:00
Donal McBreen
7f6095c9eb Merge pull request #1009 from basecamp/secrets-print
Add `kamal secrets print` for secret debugging
2024-09-30 09:55:53 -04:00
Donal McBreen
ef1271df47 Merge pull request #1010 from basecamp/kamal-proxy-0-7-0
Added support for ACME `http-01` challenges
2024-09-30 09:55:34 -04:00
Donal McBreen
df1232d90f Added support for ACME http-01 challenges
Update to kamal-proxy 0.7.0 for ACME `http-01` challenge support.
2024-09-30 14:44:28 +01:00
Dmytro Shteflyuk
e75365c8c6 Simplified deploy options for kamal-proxy as it supports multiple --host arguments 2024-09-30 08:09:05 -04:00
Donal McBreen
e441399255 Add kamal secrets print for secret debugging
Dotenv's variable substitution doesn't work the same way as commands run
in the shell. It needs values to be escaped.

```sh
$ cat /tmp/env
SECRETS=$(cat /tmp/json)
SECRETS2=$(echo $SECRETS | jq)
$ cat /tmp/json
\{\ \"foo\"\ :\ \"bar\" \}
$ SECRETS=$(cat /tmp/json)
$ SECRETS2=$(echo $SECRETS | jq)
jq: parse error: Invalid numeric literal at line 1, column 2
$ ruby -e 'require "dotenv"; puts Dotenv.parse("/tmp/env")["SECRETS2"]'
{
  "foo": "bar"
}
```

Since you then can't use the shell to debug, `kamal secrets print` will
allow you to see what the secrets will be set to.
2024-09-30 12:28:29 +01:00
Donal McBreen
af992ce755 Merge pull request #996 from iximiuz/patch-1
Fix git --add safe.directory command in Dockerfile
2024-09-30 04:36:00 -04:00
Donal McBreen
32caf4b148 Merge pull request #995 from honzasterba/bw_nicer_error_message_on_non_field_fetch
[bitwarden] default fetch raises NoMethodError
2024-09-30 04:35:48 -04:00
Donal McBreen
28a02262df Merge pull request #988 from igor-alexandrov/kamal-proxy-remove-no-target
Fixed kamal-proxy remove command
2024-09-30 04:18:12 -04:00
Donal McBreen
b11fb93a6c Merge pull request #971 from igor-alexandrov/add-app-port-example
Added app_port example to the proxy section
2024-09-30 04:16:34 -04:00
Dmytro Shteflyuk
67ad7662ab Simplified proxy hosts validation and documentation, similar to accessory config 2024-09-29 20:56:23 -04:00
Dmytro Shteflyuk
c63ec39f07 Added a test for colliding hosts passed via hosts array 2024-09-29 20:56:23 -04:00
Dmytro Shteflyuk
8df7d7d92d Do not allow both host and hosts for proxy configuration 2024-09-29 20:43:44 -04:00
Dmytro Shteflyuk
1d48a0fb0a Allow specifying multiple hosts for kamal proxy via an array 2024-09-29 20:43:44 -04:00
David Heinemeier Hansson
f331605efa Merge pull request #1004 from kpumuk/grammar
Backporting changes to the documentation committed directly to kamal-site
2024-09-30 02:22:39 +02:00
Dmytro Shteflyuk
7ea995db91 Added a comment to the front matter of the configuration docs about the generator 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
994a8faf6b Re-applied corrections to configuration YAML files that were merged directly into kamal-site 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
6c75fe40df Removed second newline characters after the section title 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
7567cae964 Added empty lines around YAML code fences 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
ecd842ab9b 'Configuration overview' section was moved to overview.md file 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
91ae1dd7b9 Removed unused variable from bin/doc 2024-09-29 15:33:06 -04:00
Ivan Velichko
0f815e17e4 Relax the safe.directory requirement
Co-authored-by: Jeremy Daer <jeremydaer@gmail.com>
2024-09-28 18:00:10 +02:00
Ivan Velichko
a310aa8fef Fix git --add safe.directory command in Dockerfile
Upgrading kamal from `v1.8.3` to `v1.9.0` broke my [kamal playground](https://labs.iximiuz.com/playgrounds/kamal):

```
laborant@dev-machine:~/svc-a$ kamal setup
  INFO [34d0def6] Running /usr/bin/env mkdir -p .kamal on 172.16.0.3
  INFO [c34cf833] Running /usr/bin/env mkdir -p .kamal on 172.16.0.4
  INFO [34d0def6] Finished in 0.147 seconds with exit status 0 (successful).
  INFO [c34cf833] Finished in 0.204 seconds with exit status 0 (successful).
Acquiring the deploy lock...
Ensure Docker is installed...
  INFO [413ee426] Running docker -v on 172.16.0.4
  INFO [f1acacba] Running docker -v on 172.16.0.3
  INFO [413ee426] Finished in 0.036 seconds with exit status 0 (successful).
  INFO [f1acacba] Finished in 0.076 seconds with exit status 0 (successful).
Log into image registry...
  INFO [94cff492] Running docker login registry.iximiuz.com -u [REDACTED] -p [REDACTED] on localhost
  INFO [94cff492] Finished in 0.077 seconds with exit status 0 (successful).
  INFO [605c535f] Running docker login registry.iximiuz.com -u [REDACTED] -p [REDACTED] on 172.16.0.4
  INFO [6002b598] Running docker login registry.iximiuz.com -u [REDACTED] -p [REDACTED] on 172.16.0.3
  INFO [605c535f] Finished in 0.083 seconds with exit status 0 (successful).
  INFO [6002b598] Finished in 0.083 seconds with exit status 0 (successful).
Build and push app image...
  INFO [9d172b1e] Running docker --version && docker buildx version on localhost
  INFO [9d172b1e] Finished in 0.059 seconds with exit status 0 (successful).
  INFO Cloning repo into build directory `/tmp/kamal-clones/svc-a-2f65914456263/workdir/`...
  INFO [26fb1bd3] Running /usr/bin/env git -C /tmp/kamal-clones/svc-a-2f65914456263 clone /workdir --recurse-submodules on localhost
 ERROR Error preparing clone: Failed to clone repo: git exit status: 32768
git stdout: Nothing written
git stderr: Cloning into 'workdir'...
fatal: detected dubious ownership in repository at '/workdir/.git'
To add an exception for this directory, call:

        git config --global --add safe.directory /workdir/.git
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
, deleting and retrying...
  INFO Cloning repo into build directory `/tmp/kamal-clones/svc-a-2f65914456263/workdir/`...
  INFO [fd4aac0c] Running /usr/bin/env git -C /tmp/kamal-clones/svc-a-2f65914456263 clone /workdir --recurse-submodules on localhost
  Finished all in 0.3 seconds
Releasing the deploy lock...
  Finished all in 0.6 seconds
  ERROR (SSHKit::Command::Failed): git exit status: 32768
git stdout: Nothing written
git stderr: Cloning into 'workdir'...
fatal: detected dubious ownership in repository at '/workdir/.git'
To add an exception for this directory, call:

        git config --global --add safe.directory /workdir/.git
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

laborant@dev-machine:~/svc-a$ kamal version
2.0.0
```

I checked the [v1.8.3...v1.9.0](https://github.com/basecamp/kamal/compare/v1.8.3...v1.9.0) diff, and couldn't find anything even remotely related to the above error.

Then I checked the `git` versions in kamal `v1.8.3` and `v1.9.0` images:

```
docker run -it --rm --entrypoint sh ghcr.io/basecamp/kamal:v1.8.3 
/workdir # git --version
git version 2.38.5
```

vs.

```
docker run -it --rm --entrypoint sh ghcr.io/basecamp/kamal:v2.0.0 
/workdir # git --version
git version 2.39.5
```

Apparently, something changed in between `2.38.5` and `2.39.5` git releases (likely yet another CVE fix), and the `git config --global --add safe.directory /workdir` stopped working.

Here is the mitigation I currently use, but it's a bit awkward to do it:

```
docker build -t ghcr.io/basecamp/kamal:v2.0.0 - <<EOF
FROM ghcr.io/basecamp/kamal:v2.0.0

RUN git config --global --add safe.directory /workdir/.git
EOF
```

Hence, this PR.

To repro, you can start a [kamal playground](https://labs.iximiuz.com/playgrounds/kamal), then `docker pull ghcr.io/basecamp/kamal:v2.0.0` to override my patched image, and `cd svc-a && kamal setup`.
2024-09-28 14:23:30 +02:00
Jan Sterba
29b02f5c30 [bitwarden] default fetch raises NoMethodError
When fetched item is not a login, Bitwarden adapter raises NoMethodError
because the returned JSON does not have the login.password value.

Add a nicer error message for that case.
2024-09-28 13:24:14 +02:00
Igor Alexandrov
6d63c4e9c6 Fixed example with the proxy status after the application has been removed 2024-09-27 21:11:51 +04:00
Igor Alexandrov
472d163cc7 Assert 404 after app is stopped 2024-09-27 19:15:42 +04:00
Igor Alexandrov
dadac999d7 Fixed kamal-proxy remove call 2024-09-27 17:45:35 +04:00
Donal McBreen
5036f8843f Merge pull request #986 from basecamp/rails-8-requires-ruby-3.2
Rails 8 doesn't support Ruby 3.1
2024-09-27 08:23:32 -04:00
Donal McBreen
14d0396581 Rails 8 doesn't support Ruby 3.1
Remove it from the build matrix as its no longer a supported combination.
2024-09-27 08:13:24 -04:00
Donal McBreen
8c32e6af07 Bump version for 2.0.0 2024-09-26 15:34:24 -04:00
Donal McBreen
a765c501a3 Bump version for 2.0.0.rc4 2024-09-26 07:06:51 -04:00
Donal McBreen
ae990efd02 Merge pull request #978 from basecamp/ignore-ssl-false
Handle ssl: false in proxy config
2024-09-26 11:56:00 +01:00
Donal McBreen
b3a6921118 Handle ssl: false in proxy config
Fixes: https://github.com/basecamp/kamal/issues/956
2024-09-26 06:17:45 -04:00
Donal McBreen
325bf9a797 Merge pull request #975 from basecamp/kamal-proxy-0.6.0
Bump to latest version of kamal-proxy
2024-09-25 23:03:25 +01:00
Donal McBreen
7bdf6cd2e8 Bump to latest version of kamal-proxy 2024-09-25 17:54:38 -04:00
Donal McBreen
7633fe0293 Merge pull request #974 from basecamp/proxy-boot-config
Proxy boot config
2024-09-25 20:28:24 +01:00
Donal McBreen
f6851048a6 Proxy boot config
Add commands for managing proxy boot config. Since the proxy can be
shared by multiple applications, the configuration doesn't belong in
`config/deploy.yml`.

Instead you can set the config with:

```
Usage:
  kamal proxy boot_config <set|get|clear>

Options:
      [--publish], [--no-publish], [--skip-publish]   # Publish the proxy ports on the host
                                                      # Default: true
      [--http-port=N]                                 # HTTP port to publish on the host
                                                      # Default: 80
      [--https-port=N]                                # HTTPS port to publish on the host
                                                      # Default: 443
      [--docker-options=option=value option2=value2]  # Docker options to pass to the proxy container
```

By default we boot the proxy with `--publish 80:80 --publish 443:443`.

You can stop it from publishing ports, specify different ports and pass
other docker options.

The config is stored in `.kamal/proxy/options` as arguments to be passed
verbatim to docker run.

Where someone wants to set the options in their application they can do
that by calling `kamal proxy boot_config set` in a pre-deploy hook.

There's an example in the integration tests showing how to use this to
front kamal-proxy with Traefik, using an accessory.
2024-09-25 15:15:26 -04:00
Donal McBreen
f0d7f786fa Traefik should be kamal-proxy in reboot hooks 2024-09-25 14:51:22 -04:00
Donal McBreen
4d8387b1c9 Merge pull request #973 from eroluysal/main
Fix adapter names
2024-09-25 19:48:40 +01:00
eroluysal
0258ac4297 Fix adapter names. 2024-09-25 21:22:59 +03:00
Igor Aleksandrov
2b0810d063 Update lib/kamal/cli/templates/deploy.yml
Co-authored-by: Nick Hammond <nick@nickhammond.com>
2024-09-25 17:19:20 +04:00
Igor Alexandrov
098f1855e2 Added back accidentially removed new line 2024-09-25 12:11:45 +04:00
Igor Alexandrov
88351312bf Added app_port example to the proxy section 2024-09-25 11:25:42 +04:00
David Heinemeier Hansson
4a13803119 Bump version for 2.0.0.rc3 2024-09-23 16:48:07 -07:00
David Heinemeier Hansson
bda252835b Merge pull request #966 from basecamp/cleanup-default-templates
Bring default templates up to par with what Rails generates
2024-09-24 01:46:09 +02:00
David Heinemeier Hansson
0f5dfa204f Rearrange one last time 2024-09-23 16:44:54 -07:00
David Heinemeier Hansson
9dde204480 Rearange 2024-09-23 16:30:16 -07:00
David Heinemeier Hansson
b6cd4f8070 Bring default templates up to par with what Rails generates 2024-09-23 14:41:31 -07:00
David Heinemeier Hansson
e71bfcbadd Bump version for 2.0.0.rc2 2024-09-20 15:41:26 -07:00
David Heinemeier Hansson
567309596a Make the skip of timestamps a boolean 2024-09-20 12:50:46 -07:00
David Heinemeier Hansson
b89ec2bf63 Bump version for 2.0.0.rc1 2024-09-20 11:08:45 -07:00
David Heinemeier Hansson
3172adca30 Merge pull request #958 from basecamp/optional-timestamps
Add option to skip timestamps on logging output
2024-09-20 18:01:14 +02:00
David Heinemeier Hansson
04d21f45bb Fix test 2024-09-20 08:45:40 -07:00
David Heinemeier Hansson
eabd57350c Fix tests 2024-09-20 08:33:14 -07:00
David Heinemeier Hansson
487f6f5f53 Fix excess spacing 2024-09-20 08:31:56 -07:00
David Heinemeier Hansson
d98500982d Update tests 2024-09-20 08:19:38 -07:00
David Heinemeier Hansson
8693e968c1 Timestamps now default on for app logs too 2024-09-20 08:17:19 -07:00
David Heinemeier Hansson
6ab5fc9459 Allow timestamps on/off for app logging too 2024-09-20 08:04:28 -07:00
David Heinemeier Hansson
6fc2915884 Merge branch 'main' into optional-timestamps 2024-09-20 07:58:49 -07:00
David Heinemeier Hansson
afa6898a82 Fix pipe 2024-09-20 07:58:38 -07:00
David Heinemeier Hansson
384b36d158 Add option to skip timestamps on logging output
So it is easier to follow live when you are doing debugging, especially
early days app setup when you are the only user.
2024-09-20 07:42:31 -07:00
Donal McBreen
6df169a4fb Doc updates 2024-09-20 15:27:10 +01:00
Donal McBreen
ab109afc52 Merge pull request #957 from basecamp/numeric-timeouts
Response timeout should be a number
2024-09-20 09:50:05 +01:00
Donal McBreen
a6a48c456c Response timeout should be a number
Kamal will append the `s` for the duration when talking to kamal-proxy
so no need to have it in the config.
2024-09-20 09:26:06 +01:00
David Heinemeier Hansson
a4e5dbe5d4 Bump version for 2.0.0.beta2 2024-09-19 11:37:22 -07:00
Donal McBreen
56e90906b1 Merge pull request #954 from basecamp/two-app-integration-test
Integration test two apps
2024-09-19 16:36:45 +01:00
Donal McBreen
6e65968bdc Integration test two apps
Use localhost for app_with_roles and 127.0.0.1 for app. Confirm we can
deploy both and the respond to requests. Ensure the proxy is removed
once both have been removed.
2024-09-19 16:25:09 +01:00
Donal McBreen
85f1e14b97 Merge pull request #953 from basecamp/inherit-env-for-secrets
Avoid setting env via SSHKit
2024-09-19 15:18:31 +01:00
Donal McBreen
2c829a4824 Avoid setting env via SSHKit
SSHKit puts the env in the command, so leaks them in process listings.
2024-09-19 15:09:17 +01:00
Donal McBreen
45a58f7e15 Merge pull request #952 from basecamp/app-exec-in-kamal-network
Run app exec in the kamal network
2024-09-19 14:46:58 +01:00
Donal McBreen
834b343ded Run app exec in the kamal network
All other containers run in the kamal network, so let's add app exec-ed
containers as well.
2024-09-19 14:29:33 +01:00
Donal McBreen
9fe1821cae Merge pull request #951 from basecamp/proxy-config-ownership
Fix /home/kamal-proxy/.config/kamal-proxy ownership
2024-09-19 12:57:10 +01:00
Donal McBreen
1d7c9fec1d Fix /home/kamal-proxy/.config/kamal-proxy ownership
1. Update to kamal-proxy 0.4.0 which creates and chowns
/home/kamal-proxy/.config/kamal-proxy to kamal-proxy
2. Use a docker volume rather than mapping in a directory, so docker
keeps it owned by the correct user
2024-09-19 12:25:57 +01:00
David Heinemeier Hansson
a6b983de06 Bump version for 2.0.0.beta1 2024-09-18 15:33:21 -07:00
Donal McBreen
3ec4ad2ea5 Merge pull request #949 from basecamp/role-proxy-config
Allow role specific proxy config
2024-09-18 18:27:36 +01:00
Donal McBreen
63f854ea18 Add validations for host/ssl roles
Roles with SSL can only have one server.
Two roles with SSL can't use the same host.
2024-09-18 17:42:45 +01:00
Donal McBreen
fd0cdc1ca1 All role specific proxy configuration
By default only the primary role runs the proxy. To disable the proxy
for that role, you can set `proxy: false` under it.

For other roles they default to not running the proxy, but you can
enable it by setting `proxy: true` for the role, or alternatively
setting a proxy configuration.

The proxy configuration will be merged into the root proxy configuration.
2024-09-18 17:25:35 +01:00
Donal McBreen
d218264b69 Doc output fixes 2024-09-18 15:28:26 +01:00
Donal McBreen
684f7ac148 Merge pull request #948 from basecamp/deploy-drain-timeouts
Simplified deploy/drain timeouts
2024-09-18 15:18:00 +01:00
Donal McBreen
8bcd896242 Simplified deploy/drain timeouts
Remove `stop_wait_time` and `readiness_timeout` from the root config
and remove `deploy_timeout` and `drain_timeout` from the proxy config.

Instead we'll just have `deploy_timeout` and `drain_timeout` in the
root config.

For roles that run the proxy, they are passed to the kamal-proxy deploy
command. Once that returns we can assume the container is ready to
shut down.

For other roles, we'll use the `deploy_timeout` when polling the
container to see if it is ready and the `drain_timeout` when stopping
the container.
2024-09-18 15:08:08 +01:00
Donal McBreen
600bbd77ef Merge pull request #947 from basecamp/doc-updates
Update proxy and docs for Kamal 2.0/kamal-proxy 0.3.0
2024-09-18 14:45:32 +01:00
Donal McBreen
34effef70a Update proxy and docs for Kamal 2.0/kamal-proxy 0.3.0
Update to kamal-proxy 0.3.0 and improve docs making sure they are in
sync with that version.
2024-09-18 14:00:43 +01:00
Donal McBreen
e07ac070aa Merge pull request #945 from basecamp/kamal-2-upgrade
Upgrade on accessory hosts only with correct messages
2024-09-18 10:27:28 +01:00
Donal McBreen
46c0836cd4 Upgrade on accessory hosts only with correct messages 2024-09-18 10:07:07 +01:00
Donal McBreen
bd54c74682 Merge pull request #944 from basecamp/single-secrets-instance
Ensure we don't load the secrets more than once
2024-09-17 14:33:38 +01:00
Donal McBreen
f183419f7a Ensure we don't load the secrets more than once
Secrets were being created twice, which might require logging in twice.
2024-09-17 14:21:43 +01:00
Donal McBreen
190dbd1ea3 Merge pull request #943 from basecamp/response-timeout-string
Need a duration string for the response timeout
2024-09-17 14:14:49 +01:00
Donal McBreen
d6eda3d741 Merge pull request #942 from basecamp/kamal-2-upgrade
Upgrade commands for Kamal 1.x -> 2.0
2024-09-17 13:34:00 +01:00
Donal McBreen
0fe6a17a91 Need a duration string for the response timeout
Add `s` as the timeout is a duration.
2024-09-17 13:23:15 +01:00
Donal McBreen
7f15fd143f Upgrade commands for Kamal 1.x -> 2.0
Adds:
- `kamal upgrade` to upgrade all app hosts and accessory hosts
- `kamal proxy upgrade` to upgrade the proxy on all hosts
- `kamal accessory upgrade [name]` to upgrade accessories on all hosts

Upgrade takes rolling and confirmed options and calls `proxy upgrade`
and `accessory upgrade` in turn.

To just upgrade a single host add -h [host] to the command. But the
upgrade should run on all hosts, not just those running the proxy.

Calling upgrade on a host that has already been upgraded should work ok.

Upgrading hosts causes downtime but you can avoid if you run multiple
hosts by:
1. Implementing the pre-proxy-reboot and post-proxy-reboot hooks to
   remove the host from external load balancers
2. Running the upgrade with the --rolling option

**kamal proxy upgrade**
1. Creates a `kamal` network if required
2. Stops and removes the old proxy (whether Traefik or kamal-proxy)
3. Starts a kamal-proxy container in the `kamal` network
4. Reboots the app containers in the `kamal` network

**kamal accessory upgrade [name]**
1. Creates a `kamal` network if required
2. Reboots the accessory containers in the `kamal` network

A matching `downgrade` command will be added to Kamal 1.9.
2024-09-17 13:02:02 +01:00
Donal McBreen
434490bd0c Merge pull request #940 from basecamp/proxy
Replace Traefik with kamal-proxy
2024-09-17 09:47:16 +01:00
Donal McBreen
267b526438 Switch proxy/hosts to proxy/host
The proxy only supports a single host per app for nowm so make the
config match that.
2024-09-16 20:45:09 +01:00
Donal McBreen
1f721739d6 Use version 0.1.0 of kamal-proxy and add minimum version check 2024-09-16 16:44:58 +01:00
Donal McBreen
6c51e596ae Put locks directories in .kamal so they leave no trace when deleted 2024-09-16 16:44:58 +01:00
Donal McBreen
7f31510aec Set hosts via config rather than options 2024-09-16 16:44:58 +01:00
Donal McBreen
e8ff233e81 Fix default log header tests 2024-09-16 16:44:58 +01:00
Donal McBreen
a316e51eda Add user agent to default headers 2024-09-16 16:44:58 +01:00
Donal McBreen
bf91d6c1ca Fix command description 2024-09-16 16:44:58 +01:00
Donal McBreen
a84ee6315f Rename service -> app directory 2024-09-16 16:44:58 +01:00
Donal McBreen
3c39086613 Not experimental 2024-09-16 16:44:58 +01:00
Donal McBreen
8b965b0a31 Handle polling without the healthcheck config 2024-09-16 16:44:58 +01:00
Donal McBreen
d2672c771e Remove redundant call to env remove 2024-09-16 16:44:58 +01:00
Donal McBreen
24031fefb0 Remove proxy only if no apps are installed 2024-09-16 16:44:58 +01:00
Donal McBreen
35fe9c154d Move audits back to run dir so they survive kamal remove 2024-09-16 16:44:58 +01:00
Donal McBreen
b8972a6833 Remove service directory on kamal remove 2024-09-16 16:44:58 +01:00
Donal McBreen
d7d6fa34b0 Use Volume for kamal proxy config volume 2024-09-16 16:44:58 +01:00
Donal McBreen
c21757f747 Move all files on the host under a common directory
This will make running kamal remove simpler, we can just clean up that
directory.
2024-09-16 16:44:58 +01:00
Donal McBreen
cb73c730f9 No need for run_id 2024-09-16 16:44:58 +01:00
Donal McBreen
109339189a Fix up integration app_test.rb 2024-09-16 16:44:58 +01:00
Donal McBreen
33834a266a Drop sleep after container healthy 2024-09-16 16:44:58 +01:00
Donal McBreen
e1016b2469 No need to wait_for_healthy 2024-09-16 16:44:58 +01:00
Donal McBreen
a40b644145 Check that there's no traefik hooks left behind 2024-09-16 16:44:58 +01:00
Donal McBreen
ccb7424197 Remove stray exit! 2024-09-16 16:44:58 +01:00
Donal McBreen
2125327d54 proxy/host -> proxy/hosts 2024-09-16 16:44:58 +01:00
Donal McBreen
f4d309c5cc Rip out Traefik 2024-09-16 16:44:55 +01:00
Donal McBreen
5bca8015bc Map kamal proxy config into .kamal/proxy/config
This will allow us to share files with the proxy via the host.
2024-09-16 16:44:41 +01:00
Donal McBreen
27a7b339a6 Drop run_directory configuration option
We need to drop to be fixed so multiple applications put the config in
the same place.
2024-09-16 16:44:41 +01:00
Donal McBreen
dcd4778dd9 Port -> app_port 2024-09-16 16:44:41 +01:00
Donal McBreen
6f2eaed398 Work out the host and port for the container
Avoid docker inspect:
1. Use the container ID as the host
2. Configure the port, default to 3000
2024-09-16 16:44:41 +01:00
Donal McBreen
e9d480b514 Add the proxy/ssl config and pass on to kamal-proxy 2024-09-16 16:44:41 +01:00
Donal McBreen
2fdc59a3aa Fix tests 2024-09-16 16:44:41 +01:00
Donal McBreen
b33c999125 Remove envify, make proxy booting work with env files 2024-09-16 16:44:41 +01:00
Donal McBreen
2056351c38 Use kamal network for accessories 2024-09-16 16:44:41 +01:00
Donal McBreen
9c2d5f83f7 Boot latest version when upgrading proxy 2024-09-16 16:44:41 +01:00
Donal McBreen
f347ef7e44 Add proxy upgrade command 2024-09-16 16:44:41 +01:00
Donal McBreen
63ebeda489 Create proxy and app containers in a kamal network 2024-09-16 16:44:41 +01:00
Donal McBreen
13bdf50ceb Fix tests for proxy defaults and required builder arch 2024-09-16 16:44:41 +01:00
Donal McBreen
bd6558630f Fix merge error 2024-09-16 16:44:39 +01:00
Donal McBreen
53903ddcd2 Read buffer not buffering 2024-09-16 16:44:21 +01:00
Donal McBreen
55756fa6f3 Set request and response headers 2024-09-16 16:44:21 +01:00
Donal McBreen
fe0c656de5 Split buffer requests/responses 2024-09-16 16:44:21 +01:00
Donal McBreen
418d8045d8 Add forward headers support 2024-09-16 16:44:21 +01:00
Donal McBreen
d63ff8f251 Set extra fields 2024-09-16 16:44:21 +01:00
Donal McBreen
eab717e0cf Add kamal-proxy in experimental mode
The proxy can be enabled via the config:

```
proxy:
  enabled: true
  hosts:
    - 10.0.0.1
    - 10.0.0.2
```

This will enable the proxy and cause it to be run on the hosts listed
under `hosts`, after running `kamal proxy reboot`.

Enabling the proxy disables `kamal traefik` commands and replaces them
with `kamal proxy` ones. However only the marked hosts will run the
kamal-proxy container, the rest will run Traefik as before.
2024-09-16 16:44:19 +01:00
Donal McBreen
66d5e25834 Merge pull request #939 from basecamp/secrets-mutex
Add a mutex around loading secrets
2024-09-16 16:26:58 +01:00
Donal McBreen
6bbbd81da1 Add a mutex around loading secrets
Loading secrets may ask for use input, so we need to ensure only one
thread does it at a time.
2024-09-16 14:44:39 +01:00
Donal McBreen
876eebc7c5 Merge pull request #936 from basecamp/asset-extraction-entrypoint
Override the entrypoint when extracting assets
2024-09-13 17:46:43 +01:00
Donal McBreen
dc1bbac3c8 Override the entrypoint when extracting assets
When overriding the command, docker will still run the entrypoint. We
want to avoid that here - we just want to get the assets out as quickly
as possible. Otherwise maybe something important is going on when we
stop the container.
2024-09-12 19:31:18 +01:00
Donal McBreen
045aa7d167 Merge pull request #934 from basecamp/secrets-dont-exit
Don't exit from failed secrets commands
2024-09-11 16:27:33 +01:00
Donal McBreen
0660895e75 Don't exit from failed secrets commands
We can let the exception bubble up. We'll still get an error message and
it ensures that any cleanup we need is done (i.e. releasing deploy locks).
2024-09-11 16:10:52 +01:00
Donal McBreen
debdf00cca Merge pull request #933 from basecamp/common-secrets
Add secrets-common for shared secrets
2024-09-11 14:20:01 +01:00
Donal McBreen
9089c41f30 Add secrets-common for shared secrets
Add a shared secrets file used across all destinations. Useful for
things Github tokens or registry passwords.

The secrets are added to a new file called `secrets-common` to highlight
they are shared, and to avoid acciedentally inheriting a secret from the
`secrets` file to `secrets.destination`.
2024-09-11 13:41:36 +01:00
Donal McBreen
c9946808b1 Merge pull request #931 from basecamp/dont-git-ignore-dot-kamal-secrets
Don't git ignore .kamal/secrets
2024-09-11 13:26:07 +01:00
Donal McBreen
deb2a6d298 Merge pull request #930 from basecamp/hide-1password-login-error
Hide the 1password login error
2024-09-11 12:18:09 +01:00
Donal McBreen
0cb69a84f5 Don't git ignore .kamal/secrets
Secrets should be interpolated at runtime so we do want the file in git.

But add a warning at the top to avoid adding secrets or git ignore the
file if you do.

Also provide examples of the three options for interpolating secrets.
2024-09-11 12:16:18 +01:00
Donal McBreen
aa630f156a Hide the 1password login error
Avoid outputting this login error message, it wasn't an error and you
don't need to follow those instructions.

```
[ERROR] 2024/09/11 11:57:08 You are not currently signed in. Please run `op signin --help` for instructions
```
2024-09-11 12:02:53 +01:00
Donal McBreen
63d0b5ddfa Merge pull request #928 from basecamp/kamal-secrets-inline-aware
Make the secrets commands inline aware
2024-09-10 11:08:10 +01:00
Donal McBreen
06f4caa866 Make the secrets commands inline aware
Rather than redirecting the global $stdout, which is not never clever in
a threaded program, we'll make the secrets commands aware they are
being inlined, so they return the value instead of printing it.

Additionally we no longer need to interrupt the parent process on error
as we've inlined the command - exit 1 is enough.
2024-09-10 10:39:44 +01:00
Donal McBreen
5aa3d1aeb0 Merge pull request #927 from basecamp/revert-903-integration-test-insecure-registry
Revert "Integration test insecure registry"
2024-09-10 10:12:57 +01:00
Donal McBreen
a4d668cd39 Revert "Integration test insecure registry" 2024-09-10 10:02:10 +01:00
Donal McBreen
7156c80f34 Merge pull request #924 from basecamp/secrets
Secrets
2024-09-09 15:13:35 +01:00
Donal McBreen
aed2ef99d0 Use env files for secrets
Add env files back in for secrets - hides them from process lists and
allows you to pick up the latest env file when running
`kamal app exec` without reusing.
2024-09-09 14:43:12 +01:00
Donal McBreen
57cbf7cdb5 Inline dotenv kamal secrets calls 2024-09-06 16:56:54 +01:00
Donal McBreen
b99c044327 Update lib/kamal/cli/templates/secrets
Co-authored-by: Sijawusz Pur Rahnama <sija@sija.pl>
2024-09-06 13:25:39 +01:00
Donal McBreen
8ad6a0ed16 Add .kamal/secrets on kamal init 2024-09-06 11:54:12 +01:00
Donal McBreen
8b62e2694a Test non-ascii secret interpolation 2024-09-05 10:01:56 +01:00
Donal McBreen
be1df4356a fetch_from_vault -> fetch_secrets 2024-09-05 09:53:33 +01:00
Donal McBreen
8210e8e768 Drop redundant rescue 2024-09-05 09:53:18 +01:00
Donal McBreen
9b96ef2412 Shellescape command input 2024-09-05 08:37:50 +01:00
Donal McBreen
1522d94ac9 Pass secrets to pre/post deploy hooks 2024-09-04 16:24:10 +01:00
Donal McBreen
a68294c384 Remote test adapter from test_helper.rb 2024-09-04 12:57:25 +01:00
Donal McBreen
31a347c285 Move int parent comment 2024-09-04 12:52:30 +01:00
Donal McBreen
3d502ab12d Add test adapter and interpolate secrets in integration tests 2024-09-04 12:40:27 +01:00
Donal McBreen
5226d52f8a Interrupting parent on error 2024-09-04 12:14:47 +01:00
Donal McBreen
9deb8af4a0 Don't hide command 2024-09-04 09:32:45 +01:00
Donal McBreen
068aaa0bd0 Fix options 2024-09-04 09:32:45 +01:00
Donal McBreen
a726a86a17 Add lastpass, bitwarden adapters 2024-09-04 09:32:45 +01:00
Donal McBreen
b2e1a4d4c1 Secrets test 2024-09-04 09:32:45 +01:00
Donal McBreen
9ade79fc84 OnePassword, LastPass + Bitwarden adapters 2024-09-04 09:32:45 +01:00
Donal McBreen
79731da619 Single fetch command 2024-09-04 09:32:45 +01:00
Donal McBreen
0ae8046905 Add secret tests 2024-09-04 09:32:45 +01:00
Donal McBreen
d5ecca0fd4 Add tests 2024-09-04 09:32:45 +01:00
Donal McBreen
0c6a593554 Remove redundant test 2024-09-04 09:32:45 +01:00
Donal McBreen
3f37fea7c3 Configuration::Secrets -> Secrets 2024-09-04 09:32:45 +01:00
Donal McBreen
7daaabd4d4 One file, no destination env 2024-09-04 09:32:45 +01:00
Donal McBreen
fcdef5fa06 Set KAMAL_DESTINATION for dotenv parsing 2024-09-04 09:32:45 +01:00
Donal McBreen
5480b40ba3 Correct secret files order 2024-09-04 09:32:45 +01:00
Donal McBreen
1d0e81b00a Eager load only CLI for faster commands 2024-09-04 09:32:45 +01:00
Donal McBreen
5910249d02 Add secrets command + 1password integration 2024-09-04 09:32:45 +01:00
Donal McBreen
b464c4fd4a Include dotenv upgrade 2024-09-04 09:32:45 +01:00
Donal McBreen
56754fe40c Lazily load secrets whenever needed 2024-09-04 09:32:45 +01:00
Donal McBreen
6a06efc9d9 Strip out env loading, envify, env push 2024-09-04 09:32:45 +01:00
Donal McBreen
5c4c33e0a8 Replace .env* with .kamal/env*
By default look for the env file in .kamal/env to avoid clashes with
other tools using .env.

For now we'll still load .env and issue a deprecation warning, but in
future we'll stop reading those.
2024-09-04 09:32:45 +01:00
Donal McBreen
0b5506f6f2 Merge pull request #923 from basecamp/disable-local-builder
Allow disabling of local builds
2024-09-03 14:53:23 +01:00
Donal McBreen
a2549b1f60 Allow disabling of local builds
To disable local builds set:
```
builder:
   local: false
   remote: ssh://docker@docker-builder
```
2024-09-03 14:33:25 +01:00
Donal McBreen
9b9e60ec7f Merge pull request #921 from basecamp/remote-hybrid-builders-cleanup
Build and clean remote builders correctly
2024-09-02 15:24:28 +01:00
Donal McBreen
e557eea79c Build and clean remote builders correctly
Check that the builder and context match what we expect, and if not
remove and re-create them.
2024-09-02 15:12:19 +01:00
David Heinemeier Hansson
d7e785cd36 Merge pull request #920 from mblayman/env-docs-typos
Fix typos in "Environment variables" docs.
2024-09-01 14:22:15 -07:00
Matt Layman
5cda3086c4 Found a typo in the healthcheck docs. 2024-08-31 23:38:28 -04:00
Matt Layman
362f5d00f6 Fix typos in "Environment variables" docs. 2024-08-31 23:29:30 -04:00
Donal McBreen
6adf3c117f Merge pull request #905 from basecamp/simplify-builders-config
Simplify builders config
2024-08-29 09:28:51 +01:00
Donal McBreen
9f0b10425c Fix aliases tests 2024-08-29 09:16:07 +01:00
Donal McBreen
5f2384f123 Use docker info to get arch 2024-08-29 08:46:18 +01:00
Donal McBreen
eab7d3adc5 Keep buildx build, in case of old docker versions which don't default to buildkit 2024-08-29 08:45:51 +01:00
Donal McBreen
d2d0223c37 Require an arch to be set, and default to amd64 in the template 2024-08-29 08:45:51 +01:00
Donal McBreen
56268d724d Simplify the builders configuration
1. Add driver as an option, defaulting to `docker-container`. For a
   "native" build you can set it to `docker`
2. Set arch as a array of architectures to build for, defaulting to
   `[ "amd64", "arm64" ]` unless you are using the docker driver in
   which case we default to not setting a platform
3. Remote is now just a connection string for the remote builder
4. If remote is set, we only use it for non-local arches, if we are
   only building for the local arch, we'll ignore it.

Examples:

On arm64, build for arm64 locally, amd64 remotely or
On amd64, build for amd64 locally, arm64 remotely:

```yaml
builder:
  remote: ssh://docker@docker-builder
```

On arm64, build amd64 on remote,
On amd64 build locally:

```yaml
builder:
  arch:
    - amd64
  remote:
    host: ssh://docker@docker-builder
```

Build amd64 on local:

```yaml
builder:
  arch:
    - amd64
```

Use docker driver, building for local arch:

```yaml
builder:
  driver: docker
```
2024-08-29 08:45:48 +01:00
Donal McBreen
cffb6c3d7e Allow the driver to be set 2024-08-29 08:44:11 +01:00
Donal McBreen
bd1726f305 docker buildx build -> docker build 2024-08-29 08:44:11 +01:00
Donal McBreen
7ddb122a22 Get tests passing 2024-08-29 08:44:11 +01:00
Donal McBreen
98c951bbdb Simplfy choosing a builder 2024-08-29 08:44:11 +01:00
Donal McBreen
374c117b79 Validate multiarch configuration
Remote and local are only allowed when multiarch is enabled.
Remote requires a host and arch, local only requires an arch.
2024-08-29 08:44:11 +01:00
Donal McBreen
d6a5cf3c78 Rip out context_hosts checks
The remote host is now encoded in the builder name so we don't need
to check it. We'll just do an inspect to confirm the builder exists.
2024-08-29 08:44:11 +01:00
Donal McBreen
2aeabda455 Move multiarch remote builder to hybrid builder
Include the host name in the builder name, so we can have one builder
per host/arch across all kamal projects.

Inherit from the remote builder. The difference in the hybrid builder
is that we create a local buildx instance and append the remote context
to it.
2024-08-29 08:44:11 +01:00
Donal McBreen
c048c097ed Create a context for local builds
This ensures we use the docker-container driver and not whatever the
local default is.
2024-08-29 08:44:11 +01:00
Donal McBreen
ed148628fb Local build doesn't need a builder 2024-08-29 08:44:11 +01:00
Donal McBreen
d48080c772 Dump native builder
We already ensure that buildx is installed, so let's always use it.
2024-08-29 08:44:11 +01:00
Donal McBreen
3f64338929 Move native remote to just remote
It's just a remote builder, that will build whichever platform is asked
for, so let's remove the "native" part.

We'll also remove the service name from the builder name, so multiple
services can share the same builder.
2024-08-29 08:44:11 +01:00
Donal McBreen
0ab838bc25 Combine multiarch and native/cache builders
Combine the two builders, as they are almost identical. The only
difference was whether the platforms were set.

The native cached builder wasn't using the context it created, so now
we do.

We'll set the driver to `docker-container` - it seems to be the default
but the Docker docs claim it is `docker`.
2024-08-29 08:44:11 +01:00
Donal McBreen
b7382ceeaf Merge pull request #912 from basecamp/alias
Add aliases to Kamal
2024-08-29 08:43:35 +01:00
Donal McBreen
69367fbc6b Merge pull request #917 from basecamp/v2.0-alpha
Switch the version on main to 2.0.0.alpha
2024-08-29 08:43:19 +01:00
Donal McBreen
2515bd705c Switch the version on main to 2.0.0.alpha
All development is now for the 2.0.0 release.
2024-08-29 08:33:21 +01:00
Donal McBreen
579e169be2 Allow multiple arguments for exec commands
If you can have an alias like:

```
aliases:
  rails: app exec -p rails
```

Then `kamal rails db:migrate:status` will execute
`kamal app exec -p rails db:migrate:status`.

So this works, we'll allow multiple arguments `app exec` and
`server exec` to accept multiple arguments.

The arguments are combined by simply joining them with a space. This
means that these are equivalent:

```
kamal app exec -p rails db:migrate:status
kamal app exec -p "rails db:migrate:status"
```

If you want to pass an argument with spaces, you'll need to quote it:

```
kamal app exec -p "git commit -am \"My comment\""
kamal app exec -p git commit -am "\"My comment\""
```
2024-08-28 10:58:25 +01:00
Donal McBreen
d6f5da92be Bump version for 1.8.2 2024-08-28 09:43:06 +01:00
Donal McBreen
9ccfe20b10 Fix up tests 2024-08-26 11:20:26 +01:00
Donal McBreen
e871d347d5 Merge pull request #889 from xiaohui-zhangxh/git-clone-update-submodules
git clone with --recurse-submodules
2024-08-26 11:20:05 +01:00
Donal McBreen
b8af719bb7 Add aliases to Kamal
Aliases are defined in the configuration file under the `aliases` key.

The configuration is a map of alias name to command. When we run the
command the we just do a literal replacement of the alias with the
string.

So if we have:

```yaml
aliases:
  console: app exec -r console -i --reuse "rails console"
```

Then running `kamal console -r workers` will run the command

```sh
$ kamal app exec -r console -i --reuse "rails console" -r workers
```

Because of the order Thor parses the arguments, this allows us to
override the role from the alias command.

There might be cases where we need to munge the command a bit more but
that would involve getting into Thor command parsing internals,
which are complicated and possibly subject to change.

There's a chance that your aliases could conflict with future built-in
commands, but there's not likely to be many of those and if it happens
you'll get a validation error when you upgrade.

Thanks to @dhnaranjo for the idea!
2024-08-26 10:47:43 +01:00
Jeremy Daer
190f4fba28 SSH: allow setting config: false to ignore local user ~/.ssh/config 2024-08-13 12:46:50 -07:00
Donal McBreen
f48987aa03 Merge pull request #903 from basecamp/integration-test-insecure-registry
Integration test insecure registry
2024-08-01 09:57:17 +01:00
Donal McBreen
ef051eca1b Merge pull request #904 from galori/main
Fixed typo in `env.yml`: "valies" --> "values"
2024-08-01 09:57:03 +01:00
Gall Steinitz
173d44ee0a fixed typo in env.yml: valies --> values 2024-07-31 22:12:21 -07:00
Donal McBreen
4e811372f8 Integration test insecure registry
The integrations tests use their own registry so avoid hitting docker
hub rate limits.

This was using a self signed certificate but instead use
`--insecure-registry` to let the docker daemon use HTTP.
2024-07-31 16:54:00 +01:00
Donal McBreen
ec4aa45852 Bump version for 1.8.1 2024-07-29 09:09:57 +01:00
Donal McBreen
5e11a64181 Merge pull request #891 from basecamp/single-pull
Pull once from hosts that warm registry mirrors
2024-07-22 08:18:48 +01:00
Jeremy Daer
57d9ce177a Pull once from hosts that warm registry mirrors 2024-07-18 09:14:22 -07:00
xiaohui
b12de87388 git clone with --recurse-submodules 2024-07-17 10:36:58 +08:00
Donal McBreen
8a98949634 Merge pull request #886 from guoard/patch-2
Remove `--update` flag from `apk add` command
2024-07-16 15:46:37 +01:00
Donal McBreen
0eb9f48082 Merge pull request #887 from basecamp/fix-tests-with-git-config
Fix the tests when you have a git config email set
2024-07-16 13:08:18 +01:00
Donal McBreen
9db6fc0704 Fix the tests when you have a git config email set
The ran ok on CI where we fall back to `whoami`, but failed locally
where there was a git email set.
2024-07-16 12:09:05 +01:00
Donal McBreen
27fede3caa Merge pull request #884 from basecamp/x-config
Add support for configuration extensions
2024-07-16 11:38:28 +01:00
Donal McBreen
29c723f7ec Add support for configuration extensions
Allow blocks prefixed with `x-` in the configuration as a place to
declare reusable blocks with YAML anchors and aliases.

Borrowed from the Docker Compose configuration file format -
https://github.com/compose-spec/compose-spec/blob/main/spec.md#extension

Thanks to @ruyrocha for the suggestion.
2024-07-15 20:47:55 +01:00
Ali Afsharzadeh
2755582c47 Remove --update flag from apk add command 2024-07-15 22:15:25 +03:30
Donal McBreen
fa73d722ea Bump version for 1.8.0 2024-07-15 14:21:23 +01:00
Donal McBreen
c535e4e44f Merge pull request #883 from basecamp/revert-840-main
Revert "Add x25519 gem, support Curve25519"
2024-07-15 13:56:49 +01:00
Donal McBreen
0ea07b1760 Merge pull request #878 from pagbrl/main
feat: Use git email as performer when available
2024-07-15 13:41:17 +01:00
Donal McBreen
03b531f179 Merge pull request #865 from basecamp/clean-envify-env
Ensure envify templates aren't polluted by existing env
2024-07-15 13:41:03 +01:00
Donal McBreen
d8570d1c2c Merge pull request #847 from basecamp/remove-ruby-2.7-from-ci
Remove Ruby 2.7 from CI
2024-07-15 13:40:37 +01:00
Donal McBreen
3fe70b458d Merge pull request #862 from jeromedalbert/bump-sshkit
Bump sshkit to support unbracketed IPv6 addresses
2024-07-15 13:40:18 +01:00
Donal McBreen
ade8b43599 Merge pull request #866 from acidtib/ssh-key-overwrite
Configurable SSH Identity
2024-07-15 13:39:51 +01:00
Donal McBreen
d24fc3ca4e Revert "Add x25519 gem, support Curve25519" 2024-07-15 13:36:50 +01:00
Donal McBreen
7c244bbb98 Merge pull request #879 from basecamp/seed-mirror
Seed docker mirrors by pulling once per mirror first
2024-07-15 13:30:53 +01:00
Donal McBreen
1369c46a83 Seed docker mirrors by pulling once per mirror first
Find the first registry mirror on each host. If we find any, pull the
images on one host per mirror, then do the remainder concurrently.

The initial pulls will seed the mirrors ensuring that we pull the image
from Docker Hub once each.

This works best if there is only one mirror on each host.
2024-07-11 16:20:37 +01:00
Paul Gabriel
deccf1cfaf feat: Use git email as performer when available 2024-07-11 11:19:44 +02:00
Donal McBreen
1573cebadf Merge pull request #868 from nickhammond/env/service
Add ENV['KAMAL_SERVICE'] to hooks
2024-07-10 10:26:59 +01:00
Nick Hammond
85a2926cde Remove the deprecated docker compose version (#869) 2024-06-28 15:00:23 -07:00
Nick Hammond
58a51b079e Add KAMAL_SERVICE to custom hooks and exclude from auditor 2024-06-27 10:52:55 -06:00
Nick Hammond
f1f3fc566f Add ENV['SERVICE'] to hooks 2024-06-27 10:26:11 -06:00
acidtib
44726ff65a overwrite ssh identity 2024-06-26 17:14:13 -06:00
Jerome Dalbert
fd0d4af21f Bump sshkit to support unbracketed IPv6 addresses
Set sshkit minimum version to 1.23.0, which includes an enhancement to
support unbracketed IPv6 addresses.

See https://github.com/capistrano/sshkit/pull/538
2024-06-25 12:17:40 -07:00
Jeremy Daer
13409ada5a Ensure envify templates aren't polluted by existing env
Setting `GITHUB_TOKEN` as in the docs results in reusing the existing
`GITHUB_TOKEN` since `gh` returns that env var if it's set:
```bash
GITHUB_TOKEN=junk gh config get -h github.com oauth_token
junk
```

Using the original env ensures that the templates will be evaluated the
same way regardless of whether envify had been previously invoked.
2024-06-25 11:14:34 -07:00
Donal McBreen
e160852e4d Remove Ruby 2.7 from CI
It's EOL since March 2023.
2024-06-20 08:54:55 +01:00
210 changed files with 4987 additions and 3037 deletions

View File

@@ -24,25 +24,15 @@ jobs:
strategy: strategy:
matrix: matrix:
ruby-version: ruby-version:
- "2.7"
- "3.1" - "3.1"
- "3.2" - "3.2"
- "3.3" - "3.3"
gemfile: gemfile:
- Gemfile - Gemfile
- gemfiles/ruby_2.7.gemfile
- gemfiles/rails_edge.gemfile - gemfiles/rails_edge.gemfile
exclude: exclude:
- ruby-version: "2.7"
gemfile: Gemfile
- ruby-version: "2.7"
gemfile: gemfiles/rails_edge.gemfile
- ruby-version: "3.1" - ruby-version: "3.1"
gemfile: gemfiles/ruby_2.7.gemfile gemfile: gemfiles/rails_edge.gemfile
- ruby-version: "3.2"
gemfile: gemfiles/ruby_2.7.gemfile
- ruby-version: "3.3"
gemfile: gemfiles/ruby_2.7.gemfile
name: ${{ format('Tests (Ruby {0})', matrix.ruby-version) }} name: ${{ format('Tests (Ruby {0})', matrix.ruby-version) }}
runs-on: ubuntu-latest runs-on: ubuntu-latest
continue-on-error: true continue-on-error: true

View File

@@ -14,7 +14,7 @@ COPY Gemfile Gemfile.lock kamal.gemspec ./
COPY lib/kamal/version.rb /kamal/lib/kamal/version.rb COPY lib/kamal/version.rb /kamal/lib/kamal/version.rb
# Install system dependencies # Install system dependencies
RUN apk add --no-cache --update build-base git docker openrc openssh-client-default \ RUN apk add --no-cache build-base git docker openrc openssh-client-default \
&& rc-update add docker boot \ && rc-update add docker boot \
&& gem install bundler --version=2.4.3 \ && gem install bundler --version=2.4.3 \
&& bundle install && bundle install
@@ -33,7 +33,7 @@ WORKDIR /workdir
# Tell git it's safe to access /workdir/.git even if # Tell git it's safe to access /workdir/.git even if
# the directory is owned by a different user # the directory is owned by a different user
RUN git config --global --add safe.directory /workdir RUN git config --global --add safe.directory '*'
# Set the entrypoint to run the installed binary in /workdir # Set the entrypoint to run the installed binary in /workdir
# Example: docker run -it -v "$PWD:/workdir" kamal init # Example: docker run -it -v "$PWD:/workdir" kamal init

View File

@@ -1,25 +1,24 @@
PATH PATH
remote: . remote: .
specs: specs:
kamal (1.7.3) kamal (2.2.1)
activesupport (>= 7.0) activesupport (>= 7.0)
base64 (~> 0.2) base64 (~> 0.2)
bcrypt_pbkdf (~> 1.0) bcrypt_pbkdf (~> 1.0)
concurrent-ruby (~> 1.2) concurrent-ruby (~> 1.2)
dotenv (~> 2.8) dotenv (~> 3.1)
ed25519 (~> 1.2) ed25519 (~> 1.2)
net-ssh (~> 7.0) net-ssh (~> 7.0)
sshkit (>= 1.22.2, < 2.0) sshkit (>= 1.23.0, < 2.0)
thor (~> 1.2) thor (~> 1.3)
x25519 (~> 1.0, >= 1.0.10)
zeitwerk (~> 2.5) zeitwerk (~> 2.5)
GEM GEM
remote: https://rubygems.org/ remote: https://rubygems.org/
specs: specs:
actionpack (7.1.2) actionpack (7.1.3.4)
actionview (= 7.1.2) actionview (= 7.1.3.4)
activesupport (= 7.1.2) activesupport (= 7.1.3.4)
nokogiri (>= 1.8.5) nokogiri (>= 1.8.5)
racc racc
rack (>= 2.2.4) rack (>= 2.2.4)
@@ -27,13 +26,13 @@ GEM
rack-test (>= 0.6.3) rack-test (>= 0.6.3)
rails-dom-testing (~> 2.2) rails-dom-testing (~> 2.2)
rails-html-sanitizer (~> 1.6) rails-html-sanitizer (~> 1.6)
actionview (7.1.2) actionview (7.1.3.4)
activesupport (= 7.1.2) activesupport (= 7.1.3.4)
builder (~> 3.1) builder (~> 3.1)
erubi (~> 1.11) erubi (~> 1.11)
rails-dom-testing (~> 2.2) rails-dom-testing (~> 2.2)
rails-html-sanitizer (~> 1.6) rails-html-sanitizer (~> 1.6)
activesupport (7.1.2) activesupport (7.1.3.4)
base64 base64
bigdecimal bigdecimal
concurrent-ruby (~> 1.0, >= 1.0.2) concurrent-ruby (~> 1.0, >= 1.0.2)
@@ -45,54 +44,55 @@ GEM
tzinfo (~> 2.0) tzinfo (~> 2.0)
ast (2.4.2) ast (2.4.2)
base64 (0.2.0) base64 (0.2.0)
bcrypt_pbkdf (1.1.0) bcrypt_pbkdf (1.1.1)
bigdecimal (3.1.5) bcrypt_pbkdf (1.1.1-arm64-darwin)
builder (3.2.4) bcrypt_pbkdf (1.1.1-x86_64-darwin)
concurrent-ruby (1.2.2) bigdecimal (3.1.8)
builder (3.3.0)
concurrent-ruby (1.3.3)
connection_pool (2.4.1) connection_pool (2.4.1)
crass (1.0.6) crass (1.0.6)
debug (1.9.1) debug (1.9.2)
irb (~> 1.10) irb (~> 1.10)
reline (>= 0.3.8) reline (>= 0.3.8)
dotenv (2.8.1) dotenv (3.1.2)
drb (2.2.0) drb (2.2.1)
ruby2_keywords
ed25519 (1.3.0) ed25519 (1.3.0)
erubi (1.12.0) erubi (1.13.0)
i18n (1.14.1) i18n (1.14.5)
concurrent-ruby (~> 1.0) concurrent-ruby (~> 1.0)
io-console (0.7.1) io-console (0.7.2)
irb (1.11.0) irb (1.14.0)
rdoc rdoc (>= 4.0.0)
reline (>= 0.3.8) reline (>= 0.4.2)
json (2.7.1) json (2.7.2)
language_server-protocol (3.17.0.3) language_server-protocol (3.17.0.3)
loofah (2.22.0) loofah (2.22.0)
crass (~> 1.0.2) crass (~> 1.0.2)
nokogiri (>= 1.12.0) nokogiri (>= 1.12.0)
minitest (5.20.0) minitest (5.24.1)
mocha (2.1.0) mocha (2.4.5)
ruby2_keywords (>= 0.0.5) ruby2_keywords (>= 0.0.5)
mutex_m (0.2.0) mutex_m (0.2.0)
net-scp (4.0.0) net-scp (4.0.0)
net-ssh (>= 2.6.5, < 8.0.0) net-ssh (>= 2.6.5, < 8.0.0)
net-sftp (4.0.0) net-sftp (4.0.0)
net-ssh (>= 5.0.0, < 8.0.0) net-ssh (>= 5.0.0, < 8.0.0)
net-ssh (7.2.1) net-ssh (7.2.3)
nokogiri (1.16.0-arm64-darwin) nokogiri (1.16.7-arm64-darwin)
racc (~> 1.4) racc (~> 1.4)
nokogiri (1.16.0-x86_64-darwin) nokogiri (1.16.7-x86_64-darwin)
racc (~> 1.4) racc (~> 1.4)
nokogiri (1.16.0-x86_64-linux) nokogiri (1.16.7-x86_64-linux)
racc (~> 1.4) racc (~> 1.4)
parallel (1.24.0) parallel (1.25.1)
parser (3.3.0.5) parser (3.3.4.0)
ast (~> 2.4.1) ast (~> 2.4.1)
racc racc
psych (5.1.2) psych (5.1.2)
stringio stringio
racc (1.7.3) racc (1.8.1)
rack (3.0.8) rack (3.1.7)
rack-session (2.0.0) rack-session (2.0.0)
rack (>= 3.0.0) rack (>= 3.0.0)
rack-test (2.1.0) rack-test (2.1.0)
@@ -107,42 +107,43 @@ GEM
rails-html-sanitizer (1.6.0) rails-html-sanitizer (1.6.0)
loofah (~> 2.21) loofah (~> 2.21)
nokogiri (~> 1.14) nokogiri (~> 1.14)
railties (7.1.2) railties (7.1.3.4)
actionpack (= 7.1.2) actionpack (= 7.1.3.4)
activesupport (= 7.1.2) activesupport (= 7.1.3.4)
irb irb
rackup (>= 1.0.0) rackup (>= 1.0.0)
rake (>= 12.2) rake (>= 12.2)
thor (~> 1.0, >= 1.2.2) thor (~> 1.0, >= 1.2.2)
zeitwerk (~> 2.6) zeitwerk (~> 2.6)
rainbow (3.1.1) rainbow (3.1.1)
rake (13.1.0) rake (13.2.1)
rdoc (6.6.2) rdoc (6.7.0)
psych (>= 4.0.0) psych (>= 4.0.0)
regexp_parser (2.9.0) regexp_parser (2.9.2)
reline (0.4.2) reline (0.5.9)
io-console (~> 0.5) io-console (~> 0.5)
rexml (3.2.6) rexml (3.3.4)
rubocop (1.62.1) strscan
rubocop (1.65.1)
json (~> 2.3) json (~> 2.3)
language_server-protocol (>= 3.17.0) language_server-protocol (>= 3.17.0)
parallel (~> 1.10) parallel (~> 1.10)
parser (>= 3.3.0.2) parser (>= 3.3.0.2)
rainbow (>= 2.2.2, < 4.0) rainbow (>= 2.2.2, < 4.0)
regexp_parser (>= 1.8, < 3.0) regexp_parser (>= 2.4, < 3.0)
rexml (>= 3.2.5, < 4.0) rexml (>= 3.2.5, < 4.0)
rubocop-ast (>= 1.31.1, < 2.0) rubocop-ast (>= 1.31.1, < 2.0)
ruby-progressbar (~> 1.7) ruby-progressbar (~> 1.7)
unicode-display_width (>= 2.4.0, < 3.0) unicode-display_width (>= 2.4.0, < 3.0)
rubocop-ast (1.31.2) rubocop-ast (1.32.0)
parser (>= 3.3.0.4) parser (>= 3.3.1.0)
rubocop-minitest (0.35.0) rubocop-minitest (0.35.1)
rubocop (>= 1.61, < 2.0) rubocop (>= 1.61, < 2.0)
rubocop-ast (>= 1.31.1, < 2.0) rubocop-ast (>= 1.31.1, < 2.0)
rubocop-performance (1.20.2) rubocop-performance (1.21.1)
rubocop (>= 1.48.1, < 2.0) rubocop (>= 1.48.1, < 2.0)
rubocop-ast (>= 1.30.0, < 2.0) rubocop-ast (>= 1.31.1, < 2.0)
rubocop-rails (2.24.0) rubocop-rails (2.25.1)
activesupport (>= 4.2.0) activesupport (>= 4.2.0)
rack (>= 1.1) rack (>= 1.1)
rubocop (>= 1.33.0, < 2.0) rubocop (>= 1.33.0, < 2.0)
@@ -154,20 +155,19 @@ GEM
rubocop-rails rubocop-rails
ruby-progressbar (1.13.0) ruby-progressbar (1.13.0)
ruby2_keywords (0.0.5) ruby2_keywords (0.0.5)
sshkit (1.22.2) sshkit (1.23.0)
base64 base64
mutex_m
net-scp (>= 1.1.2) net-scp (>= 1.1.2)
net-sftp (>= 2.1.2) net-sftp (>= 2.1.2)
net-ssh (>= 2.8.0) net-ssh (>= 2.8.0)
stringio (3.1.0) stringio (3.1.1)
thor (1.3.0) strscan (3.1.0)
thor (1.3.1)
tzinfo (2.0.6) tzinfo (2.0.6)
concurrent-ruby (~> 1.0) concurrent-ruby (~> 1.0)
unicode-display_width (2.5.0) unicode-display_width (2.5.0)
webrick (1.8.1) webrick (1.8.1)
x25519 (1.0.10) zeitwerk (2.6.17)
zeitwerk (2.6.12)
PLATFORMS PLATFORMS
arm64-darwin arm64-darwin

View File

@@ -1,6 +1,6 @@
# Kamal: Deploy web apps anywhere # Kamal: Deploy web apps anywhere
From bare metal to cloud VMs, deploy web apps anywhere with zero downtime. Kamal has the dynamic reverse-proxy Traefik hold requests while a new app container is started and the old one is stopped. Works seamlessly across multiple hosts, using SSHKit to execute commands. Originally built for Rails apps, Kamal will work with any type of web app that can be containerized with Docker. From bare metal to cloud VMs, deploy web apps anywhere with zero downtime. Kamal uses [kamal-proxy](https://github.com/basecamp/kamal-proxy) to seamlessly switch requests between containers. Works seamlessly across multiple servers, using SSHKit to execute commands. Originally built for Rails apps, Kamal will work with any type of web app that can be containerized with Docker.
➡️ See [kamal-deploy.org](https://kamal-deploy.org) for documentation on [installation](https://kamal-deploy.org/docs/installation), [configuration](https://kamal-deploy.org/docs/configuration), and [commands](https://kamal-deploy.org/docs/commands). ➡️ See [kamal-deploy.org](https://kamal-deploy.org) for documentation on [installation](https://kamal-deploy.org/docs/installation), [configuration](https://kamal-deploy.org/docs/configuration), and [commands](https://kamal-deploy.org/docs/commands).

View File

@@ -17,19 +17,20 @@ end
DOCS = { DOCS = {
"accessory" => "Accessories", "accessory" => "Accessories",
"alias" => "Aliases",
"boot" => "Booting", "boot" => "Booting",
"builder" => "Builders", "builder" => "Builders",
"configuration" => "Configuration overview", "configuration" => "Configuration overview",
"env" => "Environment variables", "env" => "Environment variables",
"healthcheck" => "Healthchecks",
"logging" => "Logging", "logging" => "Logging",
"proxy" => "Proxy",
"registry" => "Docker Registry", "registry" => "Docker Registry",
"role" => "Roles", "role" => "Roles",
"servers" => "Servers", "servers" => "Servers",
"ssh" => "SSH", "ssh" => "SSH",
"sshkit" => "SSHKit", "sshkit" => "SSHKit"
"traefik" => "Traefik"
} }
DOCS_PATH = "lib/kamal/configuration/docs"
class DocWriter class DocWriter
attr_reader :from_file, :to_file, :key, :heading, :body, :output, :in_yaml attr_reader :from_file, :to_file, :key, :heading, :body, :output, :in_yaml
@@ -67,38 +68,42 @@ class DocWriter
output.puts output.puts
place = :new_section place = :new_section
elsif line =~ /^ *#/ elsif line =~ /^ *#/
generate_line(line, place: place) generate_line(line, heading: place == :new_section)
place = :in_section
else
output.puts "```yaml"
output.print line
place = :in_yaml
end
when :in_yaml
if line =~ /^ *#/
output.puts "```"
generate_line(line, place: :new_section)
place = :in_section place = :in_section
else else
output.puts output.puts
output.print line output.puts "```yaml"
output.puts line
place = :in_yaml
end
when :in_yaml, :in_empty_line_yaml
if line =~ /^ *#/
output.puts "```"
output.puts
generate_line(line, heading: place == :in_empty_line_yaml)
place = :in_section
elsif line.empty?
place = :in_empty_line_yaml
else
output.puts line
end end
end end
end end
output.puts "\n```" if place == :in_yaml output.puts "```" if place == :in_yaml
end end
def generate_header def generate_header
output.puts "---" output.puts "---"
output.puts "# This file has been generated from the Kamal source, do not edit directly."
output.puts "# Find the source of this file at #{DOCS_PATH}/#{key}.yml in the Kamal repository."
output.puts "title: #{heading[2..-1]}" output.puts "title: #{heading[2..-1]}"
output.puts "---" output.puts "---"
output.puts output.puts
output.puts heading output.puts heading
output.puts
end end
def generate_line(line, place: :in_section) def generate_line(line, heading: false)
line = line.gsub(/^ *#\s?/, "") line = line.gsub(/^ *#\s?/, "")
if line =~ /(.*)kamal docs ([a-z]*)(.*)/ if line =~ /(.*)kamal docs ([a-z]*)(.*)/
@@ -109,7 +114,7 @@ class DocWriter
line = "#{$1}[#{titlify($2.split("/").last)}](#{$2})#{$3}" line = "#{$1}[#{titlify($2.split("/").last)}](#{$2})#{$3}"
end end
if place == :new_section if heading
output.puts "## [#{line}](##{linkify(line)})" output.puts "## [#{line}](##{linkify(line)})"
else else
output.puts line output.puts line
@@ -117,18 +122,20 @@ class DocWriter
end end
def linkify(text) def linkify(text)
if text == "Configuration overview"
"overview"
else
text.downcase.gsub(" ", "-") text.downcase.gsub(" ", "-")
end end
end
def titlify(text) def titlify(text)
text.capitalize.gsub("-", " ") text.capitalize.gsub("-", " ")
end end
end end
from_dir = File.join(File.dirname(__FILE__), "../lib/kamal/configuration/docs") from_dir = File.join(File.dirname(__FILE__), "../#{DOCS_PATH}")
to_dir = File.join(kamal_site_repo, "docs/configuration") to_dir = File.join(kamal_site_repo, "docs/configuration")
Dir.glob("#{from_dir}/*") do |from_file| Dir.glob("#{from_dir}/*") do |from_file|
key = File.basename(from_file, ".yml")
DocWriter.new(from_file, to_dir).write DocWriter.new(from_file, to_dir).write
end end

View File

@@ -1,6 +0,0 @@
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gemspec path: "../"
gem "nokogiri", "~> 1.15.0"

View File

@@ -12,13 +12,12 @@ Gem::Specification.new do |spec|
spec.executables = %w[ kamal ] spec.executables = %w[ kamal ]
spec.add_dependency "activesupport", ">= 7.0" spec.add_dependency "activesupport", ">= 7.0"
spec.add_dependency "sshkit", ">= 1.22.2", "< 2.0" spec.add_dependency "sshkit", ">= 1.23.0", "< 2.0"
spec.add_dependency "net-ssh", "~> 7.0" spec.add_dependency "net-ssh", "~> 7.0"
spec.add_dependency "thor", "~> 1.2" spec.add_dependency "thor", "~> 1.3"
spec.add_dependency "dotenv", "~> 2.8" spec.add_dependency "dotenv", "~> 3.1"
spec.add_dependency "zeitwerk", "~> 2.5" spec.add_dependency "zeitwerk", "~> 2.5"
spec.add_dependency "ed25519", "~> 1.2" spec.add_dependency "ed25519", "~> 1.2"
spec.add_dependency "x25519", "~> 1.0", ">= 1.0.10"
spec.add_dependency "bcrypt_pbkdf", "~> 1.0" spec.add_dependency "bcrypt_pbkdf", "~> 1.0"
spec.add_dependency "concurrent-ruby", "~> 1.2" spec.add_dependency "concurrent-ruby", "~> 1.2"
spec.add_dependency "base64", "~> 0.2" spec.add_dependency "base64", "~> 0.2"

View File

@@ -5,8 +5,10 @@ end
require "active_support" require "active_support"
require "zeitwerk" require "zeitwerk"
require "yaml" require "yaml"
require "tmpdir"
require "pathname"
loader = Zeitwerk::Loader.for_gem loader = Zeitwerk::Loader.for_gem
loader.ignore(File.join(__dir__, "kamal", "sshkit_with_ext.rb")) loader.ignore(File.join(__dir__, "kamal", "sshkit_with_ext.rb"))
loader.setup loader.setup
loader.eager_load # We need all commands loaded. loader.eager_load_namespace(Kamal::Cli) # We need all commands loaded.

View File

@@ -1,4 +1,5 @@
module Kamal::Cli module Kamal::Cli
class BootError < StandardError; end
class HookError < StandardError; end class HookError < StandardError; end
class LockError < StandardError; end class LockError < StandardError; end
end end

View File

@@ -1,17 +1,22 @@
require "active_support/core_ext/array/conversions"
class Kamal::Cli::Accessory < Kamal::Cli::Base class Kamal::Cli::Accessory < Kamal::Cli::Base
desc "boot [NAME]", "Boot new accessory service on host (use NAME=all to boot all accessories)" desc "boot [NAME]", "Boot new accessory service on host (use NAME=all to boot all accessories)"
def boot(name, login: true) def boot(name, prepare: true)
with_lock do with_lock do
if name == "all" if name == "all"
KAMAL.accessory_names.each { |accessory_name| boot(accessory_name) } KAMAL.accessory_names.each { |accessory_name| boot(accessory_name) }
else else
prepare(name) if prepare
with_accessory(name) do |accessory, hosts| with_accessory(name) do |accessory, hosts|
directories(name) directories(name)
upload(name) upload(name)
on(hosts) do on(hosts) do
execute *KAMAL.registry.login if login
execute *KAMAL.auditor.record("Booted #{name} accessory"), verbosity: :debug execute *KAMAL.auditor.record("Booted #{name} accessory"), verbosity: :debug
execute *accessory.ensure_env_directory
upload! accessory.secrets_io, accessory.secrets_path, mode: "0600"
execute *accessory.run execute *accessory.run
end end
end end
@@ -55,15 +60,10 @@ class Kamal::Cli::Accessory < Kamal::Cli::Base
if name == "all" if name == "all"
KAMAL.accessory_names.each { |accessory_name| reboot(accessory_name) } KAMAL.accessory_names.each { |accessory_name| reboot(accessory_name) }
else else
with_accessory(name) do |accessory, hosts| prepare(name)
on(hosts) do
execute *KAMAL.registry.login
end
stop(name) stop(name)
remove_container(name) remove_container(name)
boot(name, login: false) boot(name, prepare: false)
end
end end
end end
end end
@@ -95,12 +95,10 @@ class Kamal::Cli::Accessory < Kamal::Cli::Base
desc "restart [NAME]", "Restart existing accessory container on host" desc "restart [NAME]", "Restart existing accessory container on host"
def restart(name) def restart(name)
with_lock do with_lock do
with_accessory(name) do
stop(name) stop(name)
start(name) start(name)
end end
end end
end
desc "details [NAME]", "Show details about accessory on host (use NAME=all to show all accessories)" desc "details [NAME]", "Show details about accessory on host (use NAME=all to show all accessories)"
def details(name) def details(name)
@@ -151,23 +149,25 @@ class Kamal::Cli::Accessory < Kamal::Cli::Base
option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)" option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)"
option :grep_options, aliases: "-o", desc: "Additional options supplied to grep" option :grep_options, aliases: "-o", desc: "Additional options supplied to grep"
option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)" option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)"
option :skip_timestamps, type: :boolean, aliases: "-T", desc: "Skip appending timestamps to logging output"
def logs(name) def logs(name)
with_accessory(name) do |accessory, hosts| with_accessory(name) do |accessory, hosts|
grep = options[:grep] grep = options[:grep]
grep_options = options[:grep_options] grep_options = options[:grep_options]
timestamps = !options[:skip_timestamps]
if options[:follow] if options[:follow]
run_locally do run_locally do
info "Following logs on #{hosts}..." info "Following logs on #{hosts}..."
info accessory.follow_logs(grep: grep, grep_options: grep_options) info accessory.follow_logs(timestamps: timestamps, grep: grep, grep_options: grep_options)
exec accessory.follow_logs(grep: grep, grep_options: grep_options) exec accessory.follow_logs(timestamps: timestamps, grep: grep, grep_options: grep_options)
end end
else else
since = options[:since] since = options[:since]
lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set
on(hosts) do on(hosts) do
puts capture_with_info(*accessory.logs(since: since, lines: lines, grep: grep, grep_options: grep_options)) puts capture_with_info(*accessory.logs(timestamps: timestamps, since: since, lines: lines, grep: grep, grep_options: grep_options))
end end
end end
end end
@@ -222,6 +222,25 @@ class Kamal::Cli::Accessory < Kamal::Cli::Base
end end
end end
desc "upgrade", "Upgrade accessories from Kamal 1.x to 2.0 (restart them in 'kamal' network)"
option :rolling, type: :boolean, default: false, desc: "Upgrade one host at a time"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def upgrade(name)
confirming "This will restart all accessories" do
with_lock do
host_groups = options[:rolling] ? KAMAL.accessory_hosts : [ KAMAL.accessory_hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
KAMAL.with_specific_hosts(hosts) do
say "Upgrading #{name} accessories on #{host_list}...", :magenta
reboot name
say "Upgraded #{name} accessories on #{host_list}...", :magenta
end
end
end
end
end
private private
def with_accessory(name) def with_accessory(name)
if KAMAL.config.accessory(name) if KAMAL.config.accessory(name)
@@ -249,11 +268,20 @@ class Kamal::Cli::Accessory < Kamal::Cli::Base
end end
def remove_accessory(name) def remove_accessory(name)
with_accessory(name) do
stop(name) stop(name)
remove_container(name) remove_container(name)
remove_image(name) remove_image(name)
remove_service_directory(name) remove_service_directory(name)
end end
def prepare(name)
with_accessory(name) do |accessory, hosts|
on(hosts) do
execute *KAMAL.registry.login
execute *KAMAL.docker.create_network
rescue SSHKit::Command::Failed => e
raise unless e.message.include?("already exists")
end
end
end end
end end

View File

@@ -0,0 +1,9 @@
class Kamal::Cli::Alias::Command < Thor::DynamicCommand
def run(instance, args = [])
if (_alias = KAMAL.config.aliases[name])
Kamal::Cli::Main.start(Shellwords.split(_alias.command) + ARGV[1..-1])
else
super
end
end
end

View File

@@ -4,7 +4,7 @@ class Kamal::Cli::App < Kamal::Cli::Base
with_lock do with_lock do
say "Get most recent version available as an image...", :magenta unless options[:version] say "Get most recent version available as an image...", :magenta unless options[:version]
using_version(version_or_latest) do |version| using_version(version_or_latest) do |version|
say "Start container with version #{version} using a #{KAMAL.config.readiness_delay}s readiness delay (or reboot if already running)...", :magenta say "Start container with version #{version} (or reboot if already running)...", :magenta
# Assets are prepared in a separate step to ensure they are on all hosts before booting # Assets are prepared in a separate step to ensure they are on all hosts before booting
on(KAMAL.hosts) do on(KAMAL.hosts) do
@@ -38,8 +38,17 @@ class Kamal::Cli::App < Kamal::Cli::Base
roles = KAMAL.roles_on(host) roles = KAMAL.roles_on(host)
roles.each do |role| roles.each do |role|
app = KAMAL.app(role: role, host: host)
execute *KAMAL.auditor.record("Started app version #{KAMAL.config.version}"), verbosity: :debug execute *KAMAL.auditor.record("Started app version #{KAMAL.config.version}"), verbosity: :debug
execute *KAMAL.app(role: role, host: host).start, raise_on_non_zero_exit: false execute *app.start, raise_on_non_zero_exit: false
if role.running_proxy?
version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
endpoint = capture_with_info(*app.container_id_for_version(version)).strip
raise Kamal::Cli::BootError, "Failed to get endpoint for #{role} on #{host}, did the container boot?" if endpoint.empty?
execute *app.deploy(target: endpoint)
end
end end
end end
end end
@@ -52,8 +61,18 @@ class Kamal::Cli::App < Kamal::Cli::Base
roles = KAMAL.roles_on(host) roles = KAMAL.roles_on(host)
roles.each do |role| roles.each do |role|
app = KAMAL.app(role: role, host: host)
execute *KAMAL.auditor.record("Stopped app", role: role), verbosity: :debug execute *KAMAL.auditor.record("Stopped app", role: role), verbosity: :debug
execute *KAMAL.app(role: role, host: host).stop, raise_on_non_zero_exit: false
if role.running_proxy?
version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
endpoint = capture_with_info(*app.container_id_for_version(version)).strip
if endpoint.present?
execute *app.remove, raise_on_non_zero_exit: false
end
end
execute *app.stop, raise_on_non_zero_exit: false
end end
end end
end end
@@ -71,11 +90,12 @@ class Kamal::Cli::App < Kamal::Cli::Base
end end
end end
desc "exec [CMD]", "Execute a custom command on servers within the app container (use --help to show options)" desc "exec [CMD...]", "Execute a custom command on servers within the app container (use --help to show options)"
option :interactive, aliases: "-i", type: :boolean, default: false, desc: "Execute command over ssh for an interactive shell (use for console/bash)" option :interactive, aliases: "-i", type: :boolean, default: false, desc: "Execute command over ssh for an interactive shell (use for console/bash)"
option :reuse, type: :boolean, default: false, desc: "Reuse currently running container instead of starting a new one" option :reuse, type: :boolean, default: false, desc: "Reuse currently running container instead of starting a new one"
option :env, aliases: "-e", type: :hash, desc: "Set environment variables for the command" option :env, aliases: "-e", type: :hash, desc: "Set environment variables for the command"
def exec(cmd) def exec(*cmd)
cmd = Kamal::Utils.join_commands(cmd)
env = options[:env] env = options[:env]
case case
when options[:interactive] && options[:reuse] when options[:interactive] && options[:reuse]
@@ -168,12 +188,14 @@ class Kamal::Cli::App < Kamal::Cli::Base
option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)" option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)"
option :grep_options, aliases: "-o", desc: "Additional options supplied to grep" option :grep_options, aliases: "-o", desc: "Additional options supplied to grep"
option :follow, aliases: "-f", desc: "Follow log on primary server (or specific host set by --hosts)" option :follow, aliases: "-f", desc: "Follow log on primary server (or specific host set by --hosts)"
option :skip_timestamps, type: :boolean, aliases: "-T", desc: "Skip appending timestamps to logging output"
def logs def logs
# FIXME: Catch when app containers aren't running # FIXME: Catch when app containers aren't running
grep = options[:grep] grep = options[:grep]
grep_options = options[:grep_options] grep_options = options[:grep_options]
since = options[:since] since = options[:since]
timestamps = !options[:skip_timestamps]
if options[:follow] if options[:follow]
lines = options[:lines].presence || ((since || grep) ? nil : 10) # Default to 10 lines if since or grep isn't set lines = options[:lines].presence || ((since || grep) ? nil : 10) # Default to 10 lines if since or grep isn't set
@@ -181,12 +203,12 @@ class Kamal::Cli::App < Kamal::Cli::Base
run_locally do run_locally do
info "Following logs on #{KAMAL.primary_host}..." info "Following logs on #{KAMAL.primary_host}..."
KAMAL.specific_roles ||= [ "web" ] KAMAL.specific_roles ||= [ KAMAL.primary_role.name ]
role = KAMAL.roles_on(KAMAL.primary_host).first role = KAMAL.roles_on(KAMAL.primary_host).first
app = KAMAL.app(role: role, host: host) app = KAMAL.app(role: role, host: host)
info app.follow_logs(host: KAMAL.primary_host, lines: lines, grep: grep, grep_options: grep_options) info app.follow_logs(host: KAMAL.primary_host, timestamps: timestamps, lines: lines, grep: grep, grep_options: grep_options)
exec app.follow_logs(host: KAMAL.primary_host, lines: lines, grep: grep, grep_options: grep_options) exec app.follow_logs(host: KAMAL.primary_host, timestamps: timestamps, lines: lines, grep: grep, grep_options: grep_options)
end end
else else
lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set
@@ -196,7 +218,7 @@ class Kamal::Cli::App < Kamal::Cli::Base
roles.each do |role| roles.each do |role|
begin begin
puts_by_host host, capture_with_info(*KAMAL.app(role: role, host: host).logs(since: since, lines: lines, grep: grep, grep_options: grep_options)) puts_by_host host, capture_with_info(*KAMAL.app(role: role, host: host).logs(timestamps: timestamps, since: since, lines: lines, grep: grep, grep_options: grep_options))
rescue SSHKit::Command::Failed rescue SSHKit::Command::Failed
puts_by_host host, "Nothing found" puts_by_host host, "Nothing found"
end end
@@ -211,6 +233,7 @@ class Kamal::Cli::App < Kamal::Cli::Base
stop stop
remove_containers remove_containers
remove_images remove_images
remove_app_directory
end end
end end
@@ -252,6 +275,20 @@ class Kamal::Cli::App < Kamal::Cli::Base
end end
end end
desc "remove_app_directory", "Remove the service directory from servers", hide: true
def remove_app_directory
with_lock do
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
execute *KAMAL.auditor.record("Removed #{KAMAL.config.app_directory} on all servers", role: role), verbosity: :debug
execute *KAMAL.server.remove_app_directory, raise_on_non_zero_exit: false
end
end
end
end
desc "version", "Show app version currently running on servers" desc "version", "Show app version currently running on servers"
def version def version
on(KAMAL.hosts) do |host| on(KAMAL.hosts) do |host|

View File

@@ -1,7 +1,7 @@
class Kamal::Cli::App::Boot class Kamal::Cli::App::Boot
attr_reader :host, :role, :version, :barrier, :sshkit attr_reader :host, :role, :version, :barrier, :sshkit
delegate :execute, :capture_with_info, :capture_with_pretty_json, :info, :error, to: :sshkit delegate :execute, :capture_with_info, :capture_with_pretty_json, :info, :error, :upload!, to: :sshkit
delegate :uses_cord?, :assets?, :running_traefik?, to: :role delegate :assets?, :running_proxy?, to: :role
def initialize(host, role, sshkit, version, barrier) def initialize(host, role, sshkit, version, barrier)
@host = host @host = host
@@ -45,28 +45,30 @@ class Kamal::Cli::App::Boot
def start_new_version def start_new_version
audit "Booted app version #{version}" audit "Booted app version #{version}"
execute *app.tie_cord(role.cord_host_file) if uses_cord?
hostname = "#{host.to_s[0...51].gsub(/\.+$/, '')}-#{SecureRandom.hex(6)}" hostname = "#{host.to_s[0...51].gsub(/\.+$/, '')}-#{SecureRandom.hex(6)}"
execute *app.ensure_env_directory
upload! role.secrets_io(host), role.secrets_path, mode: "0600"
execute *app.run(hostname: hostname) execute *app.run(hostname: hostname)
if running_proxy?
endpoint = capture_with_info(*app.container_id_for_version(version)).strip
raise Kamal::Cli::BootError, "Failed to get endpoint for #{role} on #{host}, did the container boot?" if endpoint.empty?
execute *app.deploy(target: endpoint)
else
Kamal::Cli::Healthcheck::Poller.wait_for_healthy(pause_after_ready: true) { capture_with_info(*app.status(version: version)) } Kamal::Cli::Healthcheck::Poller.wait_for_healthy(pause_after_ready: true) { capture_with_info(*app.status(version: version)) }
end end
rescue => e
error "Failed to boot #{role} on #{host}"
raise e
end
def stop_new_version def stop_new_version
execute *app.stop(version: version), raise_on_non_zero_exit: false execute *app.stop(version: version), raise_on_non_zero_exit: false
end end
def stop_old_version(version) def stop_old_version(version)
if uses_cord?
cord = capture_with_info(*app.cord(version: version), raise_on_non_zero_exit: false).strip
if cord.present?
execute *app.cut_cord(cord)
Kamal::Cli::Healthcheck::Poller.wait_for_unhealthy(pause_after_ready: true) { capture_with_info(*app.status(version: version)) }
end
end
execute *app.stop(version: version), raise_on_non_zero_exit: false execute *app.stop(version: version), raise_on_non_zero_exit: false
execute *app.clean_up_assets if assets? execute *app.clean_up_assets if assets?
end end
@@ -88,8 +90,12 @@ class Kamal::Cli::App::Boot
def close_barrier def close_barrier
if barrier.close if barrier.close
info "First #{KAMAL.primary_role} container is unhealthy on #{host}, not booting any other roles" info "First #{KAMAL.primary_role} container is unhealthy on #{host}, not booting any other roles"
begin
error capture_with_info(*app.logs(version: version)) error capture_with_info(*app.logs(version: version))
error capture_with_info(*app.container_health_log(version: version)) error capture_with_info(*app.container_health_log(version: version))
rescue SSHKit::Command::Failed
error "Could not fetch logs for #{version}"
end
end end
end end

View File

@@ -1,12 +1,12 @@
require "thor" require "thor"
require "dotenv"
require "kamal/sshkit_with_ext" require "kamal/sshkit_with_ext"
module Kamal::Cli module Kamal::Cli
class Base < Thor class Base < Thor
include SSHKit::DSL include SSHKit::DSL
def self.exit_on_failure?() true end def self.exit_on_failure?() false end
def self.dynamic_command_class() Kamal::Cli::Alias::Command end
class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging" class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging"
class_option :quiet, type: :boolean, aliases: "-q", desc: "Minimal logging" class_option :quiet, type: :boolean, aliases: "-q", desc: "Minimal logging"
@@ -22,33 +22,23 @@ module Kamal::Cli
class_option :skip_hooks, aliases: "-H", type: :boolean, default: false, desc: "Don't run hooks" class_option :skip_hooks, aliases: "-H", type: :boolean, default: false, desc: "Don't run hooks"
def initialize(*) def initialize(args = [], local_options = {}, config = {})
if config[:current_command].is_a?(Kamal::Cli::Alias::Command)
# When Thor generates a dynamic command, it doesn't attempt to parse the arguments.
# For our purposes, it means the arguments are passed in args rather than local_options.
super([], args, config)
else
super super
@original_env = ENV.to_h.dup end
load_envs initialize_commander unless KAMAL.configured?
initialize_commander(options_with_subcommand_class_options)
end end
private private
def load_envs
if destination = options[:destination]
Dotenv.load(".env.#{destination}", ".env")
else
Dotenv.load(".env")
end
end
def reload_envs
ENV.clear
ENV.update(@original_env)
load_envs
end
def options_with_subcommand_class_options def options_with_subcommand_class_options
options.merge(@_initializer.last[:class_options] || {}) options.merge(@_initializer.last[:class_options] || {})
end end
def initialize_commander(options) def initialize_commander
KAMAL.tap do |commander| KAMAL.tap do |commander|
if options[:verbose] if options[:verbose]
ENV["VERBOSE"] = "1" # For backtraces via cli/start ENV["VERBOSE"] = "1" # For backtraces via cli/start
@@ -83,8 +73,6 @@ module Kamal::Cli
if KAMAL.holding_lock? if KAMAL.holding_lock?
yield yield
else else
ensure_run_and_locks_directory
acquire_lock acquire_lock
begin begin
@@ -113,6 +101,8 @@ module Kamal::Cli
end end
def acquire_lock def acquire_lock
ensure_run_directory
raise_if_locked do raise_if_locked do
say "Acquiring the deploy lock...", :magenta say "Acquiring the deploy lock...", :magenta
on(KAMAL.primary_host) { execute *KAMAL.lock.acquire("Automatic deploy lock", KAMAL.config.version), verbosity: :debug } on(KAMAL.primary_host) { execute *KAMAL.lock.acquire("Automatic deploy lock", KAMAL.config.version), verbosity: :debug }
@@ -145,8 +135,10 @@ module Kamal::Cli
details = { hosts: KAMAL.hosts.join(","), command: command, subcommand: subcommand } details = { hosts: KAMAL.hosts.join(","), command: command, subcommand: subcommand }
say "Running the #{hook} hook...", :magenta say "Running the #{hook} hook...", :magenta
with_env KAMAL.hook.env(**details, **extra_details) do
run_locally do run_locally do
execute *KAMAL.hook.run(hook, **details, **extra_details) execute *KAMAL.hook.run(hook)
end
rescue SSHKit::Command::Failed => e rescue SSHKit::Command::Failed => e
raise HookError.new("Hook `#{hook}` failed:\n#{e.message}") raise HookError.new("Hook `#{hook}` failed:\n#{e.message}")
end end
@@ -184,14 +176,23 @@ module Kamal::Cli
instance_variable_get("@_invocations").first instance_variable_get("@_invocations").first
end end
def ensure_run_and_locks_directory def reset_invocation(cli_class)
instance_variable_get("@_invocations")[cli_class].pop
end
def ensure_run_directory
on(KAMAL.hosts) do on(KAMAL.hosts) do
execute(*KAMAL.server.ensure_run_directory) execute(*KAMAL.server.ensure_run_directory)
end end
end
on(KAMAL.primary_host) do def with_env(env)
execute(*KAMAL.lock.ensure_locks_directory) current_env = ENV.to_h.dup
end ENV.update(env)
yield
ensure
ENV.clear
ENV.update(current_env)
end end
end end
end end

View File

@@ -30,46 +30,50 @@ class Kamal::Cli::Build < Kamal::Cli::Base
say "Building with uncommitted changes:\n #{uncommitted_changes}", :yellow say "Building with uncommitted changes:\n #{uncommitted_changes}", :yellow
end end
# Get the command here to ensure the Dir.chdir doesn't interfere with it with_env(KAMAL.config.builder.secrets) do
push = KAMAL.builder.push
run_locally do run_locally do
begin begin
context_hosts = capture_with_info(*KAMAL.builder.context_hosts).split("\n") execute *KAMAL.builder.inspect_builder
if context_hosts != KAMAL.builder.config_context_hosts
warn "Context hosts have changed, so re-creating builder, was: #{context_hosts.join(", ")}], now: #{KAMAL.builder.config_context_hosts.join(", ")}"
cli.remove
cli.create
end
rescue SSHKit::Command::Failed => e rescue SSHKit::Command::Failed => e
if e.message =~ /(context not found|no builder|does not exist)/ if e.message =~ /(context not found|no builder|no compatible builder|does not exist)/
warn "Missing compatible builder, so creating a new one first" warn "Missing compatible builder, so creating a new one first"
begin
cli.remove
rescue SSHKit::Command::Failed
raise unless e.message =~ /(context not found|no builder|does not exist)/
end
cli.create cli.create
else else
raise raise
end end
end end
# Get the command here to ensure the Dir.chdir doesn't interfere with it
push = KAMAL.builder.push
KAMAL.with_verbosity(:debug) do KAMAL.with_verbosity(:debug) do
Dir.chdir(KAMAL.config.builder.build_directory) { execute *push } Dir.chdir(KAMAL.config.builder.build_directory) { execute *push }
end end
end end
end end
end
desc "pull", "Pull app image from registry onto servers" desc "pull", "Pull app image from registry onto servers"
def pull def pull
on(KAMAL.hosts) do if (first_hosts = mirror_hosts).any?
execute *KAMAL.auditor.record("Pulled image with version #{KAMAL.config.version}"), verbosity: :debug #  Pull on a single host per mirror first to seed them
execute *KAMAL.builder.clean, raise_on_non_zero_exit: false say "Pulling image on #{first_hosts.join(", ")} to seed the #{"mirror".pluralize(first_hosts.count)}...", :magenta
execute *KAMAL.builder.pull pull_on_hosts(first_hosts)
execute *KAMAL.builder.validate_image say "Pulling image on remaining hosts...", :magenta
pull_on_hosts(KAMAL.hosts - first_hosts)
else
pull_on_hosts(KAMAL.hosts)
end end
end end
desc "create", "Create a build setup" desc "create", "Create a build setup"
def create def create
if (remote_host = KAMAL.config.builder.remote_host) if (remote_host = KAMAL.config.builder.remote)
connect_to_remote_host(remote_host) connect_to_remote_host(remote_host)
end end
@@ -131,4 +135,28 @@ class Kamal::Cli::Build < Kamal::Cli::Base
end end
end end
end end
def mirror_hosts
if KAMAL.hosts.many?
mirror_hosts = Concurrent::Hash.new
on(KAMAL.hosts) do |host|
first_mirror = capture_with_info(*KAMAL.builder.first_mirror).strip.presence
mirror_hosts[first_mirror] ||= host.to_s if first_mirror
rescue SSHKit::Command::Failed => e
raise unless e.message =~ /error calling index: reflect: slice index out of range/
end
mirror_hosts.values
else
[]
end
end
def pull_on_hosts(hosts)
on(hosts) do
execute *KAMAL.auditor.record("Pulled image with version #{KAMAL.config.version}"), verbosity: :debug
execute *KAMAL.builder.clean, raise_on_non_zero_exit: false
execute *KAMAL.builder.pull
execute *KAMAL.builder.validate_image
end
end
end end

View File

@@ -1,54 +0,0 @@
require "tempfile"
class Kamal::Cli::Env < Kamal::Cli::Base
desc "push", "Push the env file to the remote hosts"
def push
with_lock do
on(KAMAL.hosts) do
execute *KAMAL.auditor.record("Pushed env files"), verbosity: :debug
KAMAL.roles_on(host).each do |role|
execute *KAMAL.app(role: role, host: host).make_env_directory
upload! role.env(host).secrets_io, role.env(host).secrets_file, mode: 400
end
end
on(KAMAL.traefik_hosts) do
execute *KAMAL.traefik.make_env_directory
upload! KAMAL.traefik.env.secrets_io, KAMAL.traefik.env.secrets_file, mode: 400
end
on(KAMAL.accessory_hosts) do
KAMAL.accessories_on(host).each do |accessory|
accessory_config = KAMAL.config.accessory(accessory)
execute *KAMAL.accessory(accessory).make_env_directory
upload! accessory_config.env.secrets_io, accessory_config.env.secrets_file, mode: 400
end
end
end
end
desc "delete", "Delete the env file from the remote hosts"
def delete
with_lock do
on(KAMAL.hosts) do
execute *KAMAL.auditor.record("Deleted env files"), verbosity: :debug
KAMAL.roles_on(host).each do |role|
execute *KAMAL.app(role: role, host: host).remove_env_file
end
end
on(KAMAL.traefik_hosts) do
execute *KAMAL.traefik.remove_env_file
end
on(KAMAL.accessory_hosts) do
KAMAL.accessories_on(host).each do |accessory|
accessory_config = KAMAL.config.accessory(accessory)
execute *KAMAL.accessory(accessory).remove_env_file
end
end
end
end
end

View File

@@ -1,3 +1,5 @@
require "concurrent/ivar"
class Kamal::Cli::Healthcheck::Barrier class Kamal::Cli::Healthcheck::Barrier
def initialize def initialize
@ivar = Concurrent::IVar.new @ivar = Concurrent::IVar.new

View File

@@ -1,26 +1,30 @@
module Kamal::Cli::Healthcheck::Poller module Kamal::Cli::Healthcheck::Poller
extend self extend self
TRAEFIK_UPDATE_DELAY = 5 def wait_for_healthy(role, &block)
def wait_for_healthy(pause_after_ready: false, &block)
attempt = 1 attempt = 1
max_attempts = KAMAL.config.healthcheck.max_attempts timeout_at = Time.now + KAMAL.config.deploy_timeout
readiness_delay = KAMAL.config.readiness_delay
begin begin
case status = block.call status = block.call
when "healthy"
sleep TRAEFIK_UPDATE_DELAY if pause_after_ready if status == "running"
when "running" # No health check configured # Wait for the readiness delay and confirm it is still running
sleep KAMAL.config.readiness_delay if pause_after_ready if readiness_delay > 0
else info "Container is running, waiting for readiness delay of #{readiness_delay} seconds"
raise Kamal::Cli::Healthcheck::Error, "container not ready (#{status})" sleep readiness_delay
status = block.call
end
end
unless %w[ running healthy ].include?(status)
raise Kamal::Cli::Healthcheck::Error, "container not ready after #{KAMAL.config.deploy_timeout} seconds (#{status})"
end end
rescue Kamal::Cli::Healthcheck::Error => e rescue Kamal::Cli::Healthcheck::Error => e
if attempt <= max_attempts time_left = timeout_at - Time.now
info "#{e.message}, retrying in #{attempt}s (attempt #{attempt}/#{max_attempts})..." if time_left > 0
sleep attempt sleep [ attempt, time_left ].min
attempt += 1 attempt += 1
retry retry
else else
@@ -31,31 +35,6 @@ module Kamal::Cli::Healthcheck::Poller
info "Container is healthy!" info "Container is healthy!"
end end
def wait_for_unhealthy(pause_after_ready: false, &block)
attempt = 1
max_attempts = KAMAL.config.healthcheck.max_attempts
begin
case status = block.call
when "unhealthy"
sleep TRAEFIK_UPDATE_DELAY if pause_after_ready
else
raise Kamal::Cli::Healthcheck::Error, "container not unhealthy (#{status})"
end
rescue Kamal::Cli::Healthcheck::Error => e
if attempt <= max_attempts
info "#{e.message}, retrying in #{attempt}s (attempt #{attempt}/#{max_attempts})..."
sleep attempt
attempt += 1
retry
else
raise
end
end
info "Container is unhealthy!"
end
private private
def info(message) def info(message)
SSHKit.config.output.info(message) SSHKit.config.output.info(message)

View File

@@ -3,7 +3,6 @@ class Kamal::Cli::Lock < Kamal::Cli::Base
def status def status
handle_missing_lock do handle_missing_lock do
on(KAMAL.primary_host) do on(KAMAL.primary_host) do
execute *KAMAL.server.ensure_run_directory
puts capture_with_debug(*KAMAL.lock.status) puts capture_with_debug(*KAMAL.lock.status)
end end
end end
@@ -13,9 +12,10 @@ class Kamal::Cli::Lock < Kamal::Cli::Base
option :message, aliases: "-m", type: :string, desc: "A lock message", required: true option :message, aliases: "-m", type: :string, desc: "A lock message", required: true
def acquire def acquire
message = options[:message] message = options[:message]
ensure_run_directory
raise_if_locked do raise_if_locked do
on(KAMAL.primary_host) do on(KAMAL.primary_host) do
execute *KAMAL.server.ensure_run_directory
execute *KAMAL.lock.acquire(message, KAMAL.config.version), verbosity: :debug execute *KAMAL.lock.acquire(message, KAMAL.config.version), verbosity: :debug
end end
say "Acquired the deploy lock" say "Acquired the deploy lock"
@@ -26,7 +26,6 @@ class Kamal::Cli::Lock < Kamal::Cli::Base
def release def release
handle_missing_lock do handle_missing_lock do
on(KAMAL.primary_host) do on(KAMAL.primary_host) do
execute *KAMAL.server.ensure_run_directory
execute *KAMAL.lock.release, verbosity: :debug execute *KAMAL.lock.release, verbosity: :debug
end end
say "Released the deploy lock" say "Released the deploy lock"

View File

@@ -9,10 +9,6 @@ class Kamal::Cli::Main < Kamal::Cli::Base
say "Ensure Docker is installed...", :magenta say "Ensure Docker is installed...", :magenta
invoke "kamal:cli:server:bootstrap", [], invoke_options invoke "kamal:cli:server:bootstrap", [], invoke_options
say "Evaluate and push env files...", :magenta
invoke "kamal:cli:main:envify", [], invoke_options
invoke "kamal:cli:env:push", [], invoke_options
invoke "kamal:cli:accessory:boot", [ "all" ], invoke_options invoke "kamal:cli:accessory:boot", [ "all" ], invoke_options
deploy deploy
end end
@@ -37,10 +33,10 @@ class Kamal::Cli::Main < Kamal::Cli::Base
end end
with_lock do with_lock do
run_hook "pre-deploy" run_hook "pre-deploy", secrets: true
say "Ensure Traefik is running...", :magenta say "Ensure kamal-proxy is running...", :magenta
invoke "kamal:cli:traefik:boot", [], invoke_options invoke "kamal:cli:proxy:boot", [], invoke_options
say "Detect stale containers...", :magenta say "Detect stale containers...", :magenta
invoke "kamal:cli:app:stale_containers", [], invoke_options.merge(stop: true) invoke "kamal:cli:app:stale_containers", [], invoke_options.merge(stop: true)
@@ -52,10 +48,10 @@ class Kamal::Cli::Main < Kamal::Cli::Base
end end
end end
run_hook "post-deploy", runtime: runtime.round run_hook "post-deploy", secrets: true, runtime: runtime.round.to_s
end end
desc "redeploy", "Deploy app to servers without bootstrapping servers, starting Traefik, pruning, and registry login" desc "redeploy", "Deploy app to servers without bootstrapping servers, starting kamal-proxy, pruning, and registry login"
option :skip_push, aliases: "-P", type: :boolean, default: false, desc: "Skip image build and push" option :skip_push, aliases: "-P", type: :boolean, default: false, desc: "Skip image build and push"
def redeploy def redeploy
runtime = print_runtime do runtime = print_runtime do
@@ -70,7 +66,7 @@ class Kamal::Cli::Main < Kamal::Cli::Base
end end
with_lock do with_lock do
run_hook "pre-deploy" run_hook "pre-deploy", secrets: true
say "Detect stale containers...", :magenta say "Detect stale containers...", :magenta
invoke "kamal:cli:app:stale_containers", [], invoke_options.merge(stop: true) invoke "kamal:cli:app:stale_containers", [], invoke_options.merge(stop: true)
@@ -79,7 +75,7 @@ class Kamal::Cli::Main < Kamal::Cli::Base
end end
end end
run_hook "post-deploy", runtime: runtime.round run_hook "post-deploy", secrets: true, runtime: runtime.round.to_s
end end
desc "rollback [VERSION]", "Rollback app to VERSION" desc "rollback [VERSION]", "Rollback app to VERSION"
@@ -93,7 +89,7 @@ class Kamal::Cli::Main < Kamal::Cli::Base
old_version = nil old_version = nil
if container_available?(version) if container_available?(version)
run_hook "pre-deploy" run_hook "pre-deploy", secrets: true
invoke "kamal:cli:app:boot", [], invoke_options.merge(version: version) invoke "kamal:cli:app:boot", [], invoke_options.merge(version: version)
rolled_back = true rolled_back = true
@@ -103,12 +99,12 @@ class Kamal::Cli::Main < Kamal::Cli::Base
end end
end end
run_hook "post-deploy", runtime: runtime.round if rolled_back run_hook "post-deploy", secrets: true, runtime: runtime.round.to_s if rolled_back
end end
desc "details", "Show details about all containers" desc "details", "Show details about all containers"
def details def details
invoke "kamal:cli:traefik:details" invoke "kamal:cli:proxy:details"
invoke "kamal:cli:app:details" invoke "kamal:cli:app:details"
invoke "kamal:cli:accessory:details", [ "all" ] invoke "kamal:cli:accessory:details", [ "all" ]
end end
@@ -127,7 +123,7 @@ class Kamal::Cli::Main < Kamal::Cli::Base
end end
end end
desc "docs", "Show Kamal documentation for configuration setting" desc "docs [SECTION]", "Show Kamal configuration documentation"
def docs(section = nil) def docs(section = nil)
case section case section
when NilClass when NilClass
@@ -139,7 +135,7 @@ class Kamal::Cli::Main < Kamal::Cli::Base
puts "No documentation found for #{section}" puts "No documentation found for #{section}"
end end
desc "init", "Create config stub in config/deploy.yml and env stub in .env" desc "init", "Create config stub in config/deploy.yml and secrets stub in .kamal"
option :bundle, type: :boolean, default: false, desc: "Add Kamal to the Gemfile and create a bin/kamal binstub" option :bundle, type: :boolean, default: false, desc: "Add Kamal to the Gemfile and create a bin/kamal binstub"
def init def init
require "fileutils" require "fileutils"
@@ -152,9 +148,10 @@ class Kamal::Cli::Main < Kamal::Cli::Base
puts "Created configuration file in config/deploy.yml" puts "Created configuration file in config/deploy.yml"
end end
unless (deploy_file = Pathname.new(File.expand_path(".env"))).exist? unless (secrets_file = Pathname.new(File.expand_path(".kamal/secrets"))).exist?
FileUtils.cp_r Pathname.new(File.expand_path("templates/template.env", __dir__)), deploy_file FileUtils.mkdir_p secrets_file.dirname
puts "Created .env file" FileUtils.cp_r Pathname.new(File.expand_path("templates/secrets", __dir__)), secrets_file
puts "Created .kamal/secrets file"
end end
unless (hooks_dir = Pathname.new(File.expand_path(".kamal/hooks"))).exist? unless (hooks_dir = Pathname.new(File.expand_path(".kamal/hooks"))).exist?
@@ -179,42 +176,50 @@ class Kamal::Cli::Main < Kamal::Cli::Base
end end
end end
desc "envify", "Create .env by evaluating .env.erb (or .env.staging.erb -> .env.staging when using -d staging)" desc "remove", "Remove kamal-proxy, app, accessories, and registry session from servers"
option :skip_push, aliases: "-P", type: :boolean, default: false, desc: "Skip .env file push"
def envify
if destination = options[:destination]
env_template_path = ".env.#{destination}.erb"
env_path = ".env.#{destination}"
else
env_template_path = ".env.erb"
env_path = ".env"
end
if Pathname.new(File.expand_path(env_template_path)).exist?
File.write(env_path, ERB.new(File.read(env_template_path), trim_mode: "-").result, perm: 0600)
unless options[:skip_push]
reload_envs
invoke "kamal:cli:env:push", options
end
else
puts "Skipping envify (no #{env_template_path} exist)"
end
end
desc "remove", "Remove Traefik, app, accessories, and registry session from servers"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question" option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def remove def remove
confirming "This will remove all containers and images. Are you sure?" do confirming "This will remove all containers and images. Are you sure?" do
with_lock do with_lock do
invoke "kamal:cli:traefik:remove", [], options.without(:confirmed)
invoke "kamal:cli:app:remove", [], options.without(:confirmed) invoke "kamal:cli:app:remove", [], options.without(:confirmed)
invoke "kamal:cli:proxy:remove", [], options.without(:confirmed)
invoke "kamal:cli:accessory:remove", [ "all" ], options invoke "kamal:cli:accessory:remove", [ "all" ], options
invoke "kamal:cli:registry:logout", [], options.without(:confirmed).merge(skip_local: true) invoke "kamal:cli:registry:logout", [], options.without(:confirmed).merge(skip_local: true)
end end
end end
end end
desc "upgrade", "Upgrade from Kamal 1.x to 2.0"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
option :rolling, type: :boolean, default: false, desc: "Upgrade one host at a time"
def upgrade
confirming "This will replace Traefik with kamal-proxy and restart all accessories" do
with_lock do
if options[:rolling]
(KAMAL.hosts | KAMAL.accessory_hosts).each do |host|
KAMAL.with_specific_hosts(host) do
say "Upgrading #{host}...", :magenta
if KAMAL.hosts.include?(host)
invoke "kamal:cli:proxy:upgrade", [], options.merge(confirmed: true, rolling: false)
reset_invocation(Kamal::Cli::Proxy)
end
if KAMAL.accessory_hosts.include?(host)
invoke "kamal:cli:accessory:upgrade", [ "all" ], options.merge(confirmed: true, rolling: false)
reset_invocation(Kamal::Cli::Accessory)
end
say "Upgraded #{host}", :magenta
end
end
else
say "Upgrading all hosts...", :magenta
invoke "kamal:cli:proxy:upgrade", [], options.merge(confirmed: true)
invoke "kamal:cli:accessory:upgrade", [ "all" ], options.merge(confirmed: true)
say "Upgraded all hosts", :magenta
end
end
end
end
desc "version", "Show Kamal version" desc "version", "Show Kamal version"
def version def version
puts Kamal::VERSION puts Kamal::VERSION
@@ -229,24 +234,24 @@ class Kamal::Cli::Main < Kamal::Cli::Base
desc "build", "Build application image" desc "build", "Build application image"
subcommand "build", Kamal::Cli::Build subcommand "build", Kamal::Cli::Build
desc "env", "Manage environment files"
subcommand "env", Kamal::Cli::Env
desc "lock", "Manage the deploy lock" desc "lock", "Manage the deploy lock"
subcommand "lock", Kamal::Cli::Lock subcommand "lock", Kamal::Cli::Lock
desc "proxy", "Manage kamal-proxy"
subcommand "proxy", Kamal::Cli::Proxy
desc "prune", "Prune old application images and containers" desc "prune", "Prune old application images and containers"
subcommand "prune", Kamal::Cli::Prune subcommand "prune", Kamal::Cli::Prune
desc "registry", "Login and -out of the image registry" desc "registry", "Login and -out of the image registry"
subcommand "registry", Kamal::Cli::Registry subcommand "registry", Kamal::Cli::Registry
desc "secrets", "Helpers for extracting secrets"
subcommand "secrets", Kamal::Cli::Secrets
desc "server", "Bootstrap servers with curl and Docker" desc "server", "Bootstrap servers with curl and Docker"
subcommand "server", Kamal::Cli::Server subcommand "server", Kamal::Cli::Server
desc "traefik", "Manage Traefik load balancer"
subcommand "traefik", Kamal::Cli::Traefik
private private
def container_available?(version) def container_available?(version)
begin begin

257
lib/kamal/cli/proxy.rb Normal file
View File

@@ -0,0 +1,257 @@
class Kamal::Cli::Proxy < Kamal::Cli::Base
desc "boot", "Boot proxy on servers"
def boot
with_lock do
on(KAMAL.hosts) do |host|
execute *KAMAL.docker.create_network
rescue SSHKit::Command::Failed => e
raise unless e.message.include?("already exists")
end
on(KAMAL.proxy_hosts) do |host|
execute *KAMAL.registry.login
version = capture_with_info(*KAMAL.proxy.version).strip.presence
if version && Kamal::Utils.older_version?(version, Kamal::Configuration::PROXY_MINIMUM_VERSION)
raise "kamal-proxy version #{version} is too old, please reboot to update to at least #{Kamal::Configuration::PROXY_MINIMUM_VERSION}"
end
execute *KAMAL.proxy.start_or_run
end
end
end
desc "boot_config <set|get|reset>", "Mange kamal-proxy boot configuration"
option :publish, type: :boolean, default: true, desc: "Publish the proxy ports on the host"
option :http_port, type: :numeric, default: Kamal::Configuration::PROXY_HTTP_PORT, desc: "HTTP port to publish on the host"
option :https_port, type: :numeric, default: Kamal::Configuration::PROXY_HTTPS_PORT, desc: "HTTPS port to publish on the host"
option :log_max_size, type: :string, default: Kamal::Configuration::PROXY_LOG_MAX_SIZE, desc: "Max size of proxy logs"
option :docker_options, type: :array, default: [], desc: "Docker options to pass to the proxy container", banner: "option=value option2=value2"
def boot_config(subcommand)
case subcommand
when "set"
boot_options = [
*(KAMAL.config.proxy_publish_args(options[:http_port], options[:https_port]) if options[:publish]),
*(KAMAL.config.proxy_logging_args(options[:log_max_size])),
*options[:docker_options].map { |option| "--#{option}" }
]
on(KAMAL.proxy_hosts) do |host|
execute(*KAMAL.proxy.ensure_proxy_directory)
upload! StringIO.new(boot_options.join(" ")), KAMAL.config.proxy_options_file
end
when "get"
on(KAMAL.proxy_hosts) do |host|
puts "Host #{host}: #{capture_with_info(*KAMAL.proxy.get_boot_options)}"
end
when "reset"
on(KAMAL.proxy_hosts) do |host|
execute *KAMAL.proxy.reset_boot_options
end
else
raise ArgumentError, "Unknown boot_config subcommand #{subcommand}"
end
end
desc "reboot", "Reboot proxy on servers (stop container, remove container, start new container)"
option :rolling, type: :boolean, default: false, desc: "Reboot proxy on hosts in sequence, rather than in parallel"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def reboot
confirming "This will cause a brief outage on each host. Are you sure?" do
with_lock do
host_groups = options[:rolling] ? KAMAL.proxy_hosts : [ KAMAL.proxy_hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
run_hook "pre-proxy-reboot", hosts: host_list
on(hosts) do |host|
execute *KAMAL.auditor.record("Rebooted proxy"), verbosity: :debug
execute *KAMAL.registry.login
"Stopping and removing Traefik on #{host}, if running..."
execute *KAMAL.proxy.cleanup_traefik
"Stopping and removing kamal-proxy on #{host}, if running..."
execute *KAMAL.proxy.stop, raise_on_non_zero_exit: false
execute *KAMAL.proxy.remove_container
execute *KAMAL.proxy.run
KAMAL.roles_on(host).select(&:running_proxy?).each do |role|
app = KAMAL.app(role: role, host: host)
version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
endpoint = capture_with_info(*app.container_id_for_version(version)).strip
if endpoint.present?
info "Deploying #{endpoint} for role `#{role}` on #{host}..."
execute *app.deploy(target: endpoint)
end
end
end
run_hook "post-proxy-reboot", hosts: host_list
end
end
end
end
desc "upgrade", "Upgrade to kamal-proxy on servers (stop container, remove container, start new container, reboot app)", hide: true
option :rolling, type: :boolean, default: false, desc: "Reboot proxy on hosts in sequence, rather than in parallel"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def upgrade
invoke_options = { "version" => KAMAL.config.latest_tag }.merge(options)
confirming "This will cause a brief outage on each host. Are you sure?" do
host_groups = options[:rolling] ? KAMAL.hosts : [ KAMAL.hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
say "Upgrading proxy on #{host_list}...", :magenta
run_hook "pre-proxy-reboot", hosts: host_list
on(hosts) do |host|
execute *KAMAL.auditor.record("Rebooted proxy"), verbosity: :debug
execute *KAMAL.registry.login
"Stopping and removing Traefik on #{host}, if running..."
execute *KAMAL.proxy.cleanup_traefik
"Stopping and removing kamal-proxy on #{host}, if running..."
execute *KAMAL.proxy.stop, raise_on_non_zero_exit: false
execute *KAMAL.proxy.remove_container
execute *KAMAL.proxy.remove_image
end
KAMAL.with_specific_hosts(hosts) do
invoke "kamal:cli:proxy:boot", [], invoke_options
reset_invocation(Kamal::Cli::Proxy)
invoke "kamal:cli:app:boot", [], invoke_options
reset_invocation(Kamal::Cli::App)
invoke "kamal:cli:prune:all", [], invoke_options
reset_invocation(Kamal::Cli::Prune)
end
run_hook "post-proxy-reboot", hosts: host_list
say "Upgraded proxy on #{host_list}", :magenta
end
end
end
desc "start", "Start existing proxy container on servers"
def start
with_lock do
on(KAMAL.proxy_hosts) do |host|
execute *KAMAL.auditor.record("Started proxy"), verbosity: :debug
execute *KAMAL.proxy.start
end
end
end
desc "stop", "Stop existing proxy container on servers"
def stop
with_lock do
on(KAMAL.proxy_hosts) do |host|
execute *KAMAL.auditor.record("Stopped proxy"), verbosity: :debug
execute *KAMAL.proxy.stop, raise_on_non_zero_exit: false
end
end
end
desc "restart", "Restart existing proxy container on servers"
def restart
with_lock do
stop
start
end
end
desc "details", "Show details about proxy container from servers"
def details
on(KAMAL.proxy_hosts) { |host| puts_by_host host, capture_with_info(*KAMAL.proxy.info), type: "Proxy" }
end
desc "logs", "Show log lines from proxy on servers"
option :since, aliases: "-s", desc: "Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)"
option :lines, type: :numeric, aliases: "-n", desc: "Number of log lines to pull from each server"
option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)"
option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)"
option :skip_timestamps, type: :boolean, aliases: "-T", desc: "Skip appending timestamps to logging output"
def logs
grep = options[:grep]
timestamps = !options[:skip_timestamps]
if options[:follow]
run_locally do
info "Following logs on #{KAMAL.primary_host}..."
info KAMAL.proxy.follow_logs(host: KAMAL.primary_host, timestamps: timestamps, grep: grep)
exec KAMAL.proxy.follow_logs(host: KAMAL.primary_host, timestamps: timestamps, grep: grep)
end
else
since = options[:since]
lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set
on(KAMAL.proxy_hosts) do |host|
puts_by_host host, capture(*KAMAL.proxy.logs(timestamps: timestamps, since: since, lines: lines, grep: grep)), type: "Proxy"
end
end
end
desc "remove", "Remove proxy container and image from servers"
option :force, type: :boolean, default: false, desc: "Force removing proxy when apps are still installed"
def remove
with_lock do
if removal_allowed?(options[:force])
stop
remove_container
remove_image
remove_proxy_directory
end
end
end
desc "remove_container", "Remove proxy container from servers", hide: true
def remove_container
with_lock do
on(KAMAL.proxy_hosts) do
execute *KAMAL.auditor.record("Removed proxy container"), verbosity: :debug
execute *KAMAL.proxy.remove_container
end
end
end
desc "remove_image", "Remove proxy image from servers", hide: true
def remove_image
with_lock do
on(KAMAL.proxy_hosts) do
execute *KAMAL.auditor.record("Removed proxy image"), verbosity: :debug
execute *KAMAL.proxy.remove_image
end
end
end
desc "remove_proxy_directory", "Remove the proxy directory from servers", hide: true
def remove_proxy_directory
with_lock do
on(KAMAL.proxy_hosts) do
execute *KAMAL.proxy.remove_proxy_directory, raise_on_non_zero_exit: false
end
end
end
private
def removal_allowed?(force)
on(KAMAL.proxy_hosts) do |host|
app_count = capture_with_info(*KAMAL.server.app_directory_count).chomp.to_i
raise "The are other applications installed on #{host}" if app_count > 0
end
true
rescue SSHKit::Runner::ExecuteError => e
raise unless e.message.include?("The are other applications installed on")
if force
say "Forcing, so removing the proxy, even though other apps are installed", :magenta
else
say "Not removing the proxy, as other apps are installed, ignore this check with kamal proxy remove --force", :magenta
end
force
end
end

View File

@@ -28,7 +28,6 @@ class Kamal::Cli::Prune < Kamal::Cli::Base
on(KAMAL.hosts) do on(KAMAL.hosts) do
execute *KAMAL.auditor.record("Pruned containers"), verbosity: :debug execute *KAMAL.auditor.record("Pruned containers"), verbosity: :debug
execute *KAMAL.prune.app_containers(retain: retain) execute *KAMAL.prune.app_containers(retain: retain)
execute *KAMAL.prune.healthcheck_containers
end end
end end
end end

43
lib/kamal/cli/secrets.rb Normal file
View File

@@ -0,0 +1,43 @@
class Kamal::Cli::Secrets < Kamal::Cli::Base
desc "fetch [SECRETS...]", "Fetch secrets from a vault"
option :adapter, type: :string, aliases: "-a", required: true, desc: "Which vault adapter to use"
option :account, type: :string, required: true, desc: "The account identifier or username"
option :from, type: :string, required: false, desc: "A vault or folder to fetch the secrets from"
option :inline, type: :boolean, required: false, hidden: true
def fetch(*secrets)
results = adapter(options[:adapter]).fetch(secrets, **options.slice(:account, :from).symbolize_keys)
return_or_puts JSON.dump(results).shellescape, inline: options[:inline]
end
desc "extract", "Extract a single secret from the results of a fetch call"
option :inline, type: :boolean, required: false, hidden: true
def extract(name, secrets)
parsed_secrets = JSON.parse(secrets)
value = parsed_secrets[name] || parsed_secrets.find { |k, v| k.end_with?("/#{name}") }&.last
raise "Could not find secret #{name}" if value.nil?
return_or_puts value, inline: options[:inline]
end
desc "print", "Print the secrets (for debugging)"
def print
KAMAL.config.secrets.to_h.each do |key, value|
puts "#{key}=#{value}"
end
end
private
def adapter(adapter)
Kamal::Secrets::Adapters.lookup(adapter)
end
def return_or_puts(value, inline: nil)
if inline
value
else
puts value
end
end
end

View File

@@ -1,7 +1,8 @@
class Kamal::Cli::Server < Kamal::Cli::Base class Kamal::Cli::Server < Kamal::Cli::Base
desc "exec", "Run a custom command on the server (use --help to show options)" desc "exec", "Run a custom command on the server (use --help to show options)"
option :interactive, type: :boolean, aliases: "-i", default: false, desc: "Run the command interactively (use for console/bash)" option :interactive, type: :boolean, aliases: "-i", default: false, desc: "Run the command interactively (use for console/bash)"
def exec(cmd) def exec(*cmd)
cmd = Kamal::Utils.join_commands(cmd)
hosts = KAMAL.hosts | KAMAL.accessory_hosts hosts = KAMAL.hosts | KAMAL.accessory_hosts
case case
@@ -35,8 +36,6 @@ class Kamal::Cli::Server < Kamal::Cli::Base
missing << host missing << host
end end
end end
execute(*KAMAL.server.ensure_run_directory)
end end
if missing.any? if missing.any?

View File

@@ -2,11 +2,25 @@
service: my-app service: my-app
# Name of the container image. # Name of the container image.
image: user/my-app image: my-user/my-app
# Deploy to these servers. # Deploy to these servers.
servers: servers:
web:
- 192.168.0.1 - 192.168.0.1
# job:
# hosts:
# - 192.168.0.1
# cmd: bin/jobs
# Enable SSL auto certification via Let's Encrypt (and allow for multiple apps on one server).
# If using something like Cloudflare, it is recommended to set encryption mode
# in Cloudflare's SSL/TLS setting to "Full" to enable end-to-end encryption.
proxy:
ssl: true
host: app.example.com
# kamal-proxy connects to your container over port 80, use `app_port` to specify a different port.
# app_port: 3000
# Credentials for your image host. # Credentials for your image host.
registry: registry:
@@ -14,33 +28,52 @@ registry:
# server: registry.digitalocean.com / ghcr.io / ... # server: registry.digitalocean.com / ghcr.io / ...
username: my-user username: my-user
# Always use an access token rather than real password when possible. # Always use an access token rather than real password (pulled from .kamal/secrets).
password: password:
- KAMAL_REGISTRY_PASSWORD - KAMAL_REGISTRY_PASSWORD
# Inject ENV variables into containers (secrets come from .env). # Configure builder setup.
# Remember to run `kamal env push` after making changes! builder:
arch: amd64
# Inject ENV variables into containers (secrets come from .kamal/secrets).
#
# env: # env:
# clear: # clear:
# DB_HOST: 192.168.0.2 # DB_HOST: 192.168.0.2
# secret: # secret:
# - RAILS_MASTER_KEY # - RAILS_MASTER_KEY
# Aliases are triggered with "bin/kamal <alias>". You can overwrite arguments on invocation:
# "bin/kamal logs -r job" will tail logs from the first server in the job section.
#
# aliases:
# shell: app exec --interactive --reuse "bash"
# Use a different ssh user than root # Use a different ssh user than root
#
# ssh: # ssh:
# user: app # user: app
# Configure builder setup. # Use a persistent storage volume.
# builder: #
# args: # volumes:
# RUBY_VERSION: 3.2.0 # - "app_storage:/app/storage"
# secrets:
# - GITHUB_TOKEN
# remote:
# arch: amd64
# host: ssh://app@192.168.0.1
# Use accessory services (secrets come from .env). # Bridge fingerprinted assets, like JS and CSS, between versions to avoid
# hitting 404 on in-flight requests. Combines all files from new and old
# version inside the asset_path.
#
# asset_path: /app/public/assets
# Configure rolling deploys by setting a wait time between batches of restarts.
#
# boot:
# limit: 10 # Can also specify as a percentage of total hosts, such as "25%"
# wait: 2
# Use accessory services (secrets come from .kamal/secrets).
#
# accessories: # accessories:
# db: # db:
# image: mysql:8.0 # image: mysql:8.0
@@ -62,40 +95,3 @@ registry:
# port: 6379 # port: 6379
# directories: # directories:
# - data:/data # - data:/data
# Configure custom arguments for Traefik. Be sure to reboot traefik when you modify it.
# traefik:
# args:
# accesslog: true
# accesslog.format: json
# Configure a custom healthcheck (default is /up on port 3000)
# healthcheck:
# path: /healthz
# port: 4000
# Bridge fingerprinted assets, like JS and CSS, between versions to avoid
# hitting 404 on in-flight requests. Combines all files from new and old
# version inside the asset_path.
#
# If your app is using the Sprockets gem, ensure it sets `config.assets.manifest`.
# See https://github.com/basecamp/kamal/issues/626 for details
#
# asset_path: /rails/public/assets
# Configure rolling deploys by setting a wait time between batches of restarts.
# boot:
# limit: 10 # Can also specify as a percentage of total hosts, such as "25%"
# wait: 2
# Configure the role used to determine the primary_host. This host takes
# deploy locks, runs health checks during the deploy, and follow logs, etc.
#
# Caution: there's no support for role renaming yet, so be careful to cleanup
# the previous role on the deployed hosts.
# primary_role: web
# Controls if we abort when see a role with no hosts. Disabling this may be
# useful for more complex deploy configurations.
#
# allow_empty_roles: false

View File

@@ -1,13 +1,3 @@
#!/usr/bin/env ruby #!/bin/sh
# A sample docker-setup hook echo "Docker set up on $KAMAL_HOSTS..."
#
# Sets up a Docker network on defined hosts which can then be used by the applications containers
hosts = ENV["KAMAL_HOSTS"].split(",")
hosts.each do |ip|
destination = "root@#{ip}"
puts "Creating a Docker network \"kamal\" on #{destination}"
`ssh #{destination} docker network create kamal`
end

View File

@@ -0,0 +1,3 @@
#!/bin/sh
echo "Rebooted kamal-proxy on $KAMAL_HOSTS"

View File

@@ -1,3 +0,0 @@
#!/bin/sh
echo "Rebooted Traefik on $KAMAL_HOSTS"

View File

@@ -0,0 +1,3 @@
#!/bin/sh
echo "Rebooting kamal-proxy on $KAMAL_HOSTS..."

View File

@@ -1,3 +0,0 @@
#!/bin/sh
echo "Rebooting Traefik on $KAMAL_HOSTS..."

View File

@@ -0,0 +1,17 @@
# Secrets defined here are available for reference under registry/password, env/secret, builder/secrets,
# and accessories/*/env/secret in config/deploy.yml. All secrets should be pulled from either
# password manager, ENV, or a file. DO NOT ENTER RAW CREDENTIALS HERE! This file needs to be safe for git.
# Option 1: Read secrets from the environment
KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD
# Option 2: Read secrets via a command
# RAILS_MASTER_KEY=$(cat config/master.key)
# Option 3: Read secrets via kamal secrets helpers
# These will handle logging in and fetching the secrets in as few calls as possible
# There are adapters for 1Password, LastPass + Bitwarden
#
# SECRETS=$(kamal secrets fetch --adapter 1password --account my-account --from MyVault/MyItem KAMAL_REGISTRY_PASSWORD RAILS_MASTER_KEY)
# KAMAL_REGISTRY_PASSWORD=$(kamal secrets extract KAMAL_REGISTRY_PASSWORD $SECRETS)
# RAILS_MASTER_KEY=$(kamal secrets extract RAILS_MASTER_KEY $SECRETS)

View File

@@ -1,2 +0,0 @@
KAMAL_REGISTRY_PASSWORD=change-this
RAILS_MASTER_KEY=another-env

View File

@@ -1,122 +0,0 @@
class Kamal::Cli::Traefik < Kamal::Cli::Base
desc "boot", "Boot Traefik on servers"
def boot
with_lock do
on(KAMAL.traefik_hosts) do
execute *KAMAL.registry.login
execute *KAMAL.traefik.start_or_run
end
end
end
desc "reboot", "Reboot Traefik on servers (stop container, remove container, start new container)"
option :rolling, type: :boolean, default: false, desc: "Reboot traefik on hosts in sequence, rather than in parallel"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def reboot
confirming "This will cause a brief outage on each host. Are you sure?" do
with_lock do
host_groups = options[:rolling] ? KAMAL.traefik_hosts : [ KAMAL.traefik_hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
run_hook "pre-traefik-reboot", hosts: host_list
on(hosts) do
execute *KAMAL.auditor.record("Rebooted traefik"), verbosity: :debug
execute *KAMAL.registry.login
execute *KAMAL.traefik.stop, raise_on_non_zero_exit: false
execute *KAMAL.traefik.remove_container
execute *KAMAL.traefik.run
end
run_hook "post-traefik-reboot", hosts: host_list
end
end
end
end
desc "start", "Start existing Traefik container on servers"
def start
with_lock do
on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Started traefik"), verbosity: :debug
execute *KAMAL.traefik.start
end
end
end
desc "stop", "Stop existing Traefik container on servers"
def stop
with_lock do
on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Stopped traefik"), verbosity: :debug
execute *KAMAL.traefik.stop, raise_on_non_zero_exit: false
end
end
end
desc "restart", "Restart existing Traefik container on servers"
def restart
with_lock do
stop
start
end
end
desc "details", "Show details about Traefik container from servers"
def details
on(KAMAL.traefik_hosts) { |host| puts_by_host host, capture_with_info(*KAMAL.traefik.info), type: "Traefik" }
end
desc "logs", "Show log lines from Traefik on servers"
option :since, aliases: "-s", desc: "Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)"
option :lines, type: :numeric, aliases: "-n", desc: "Number of log lines to pull from each server"
option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)"
option :grep_options, aliases: "-o", desc: "Additional options supplied to grep"
option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)"
def logs
grep = options[:grep]
grep_options = options[:grep_options]
if options[:follow]
run_locally do
info "Following logs on #{KAMAL.primary_host}..."
info KAMAL.traefik.follow_logs(host: KAMAL.primary_host, grep: grep, grep_options: grep_options)
exec KAMAL.traefik.follow_logs(host: KAMAL.primary_host, grep: grep, grep_options: grep_options)
end
else
since = options[:since]
lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set
on(KAMAL.traefik_hosts) do |host|
puts_by_host host, capture(*KAMAL.traefik.logs(since: since, lines: lines, grep: grep, grep_options: grep_options)), type: "Traefik"
end
end
end
desc "remove", "Remove Traefik container and image from servers"
def remove
with_lock do
stop
remove_container
remove_image
end
end
desc "remove_container", "Remove Traefik container from servers", hide: true
def remove_container
with_lock do
on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Removed traefik container"), verbosity: :debug
execute *KAMAL.traefik.remove_container
end
end
end
desc "remove_image", "Remove Traefik image from servers", hide: true
def remove_image
with_lock do
on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Removed traefik image"), verbosity: :debug
execute *KAMAL.traefik.remove_image
end
end
end
end

View File

@@ -1,9 +1,10 @@
require "active_support/core_ext/enumerable" require "active_support/core_ext/enumerable"
require "active_support/core_ext/module/delegation" require "active_support/core_ext/module/delegation"
require "active_support/core_ext/object/blank"
class Kamal::Commander class Kamal::Commander
attr_accessor :verbosity, :holding_lock, :connected attr_accessor :verbosity, :holding_lock, :connected
delegate :hosts, :roles, :primary_host, :primary_role, :roles_on, :traefik_hosts, :accessory_hosts, to: :specifics delegate :hosts, :roles, :primary_host, :primary_role, :roles_on, :proxy_hosts, :accessory_hosts, to: :specifics
def initialize def initialize
self.verbosity = :info self.verbosity = :info
@@ -23,12 +24,20 @@ class Kamal::Commander
@config, @config_kwargs = nil, kwargs @config, @config_kwargs = nil, kwargs
end end
def configured?
@config || @config_kwargs
end
attr_reader :specific_roles, :specific_hosts attr_reader :specific_roles, :specific_hosts
def specific_primary! def specific_primary!
@specifics = nil @specifics = nil
if specific_roles.present?
self.specific_hosts = [ specific_roles.first.primary_host ]
else
self.specific_hosts = [ config.primary_host ] self.specific_hosts = [ config.primary_host ]
end end
end
def specific_roles=(role_names) def specific_roles=(role_names)
@specifics = nil @specifics = nil
@@ -56,6 +65,13 @@ class Kamal::Commander
end end
end end
def with_specific_hosts(hosts)
original_hosts, self.specific_hosts = specific_hosts, hosts
yield
ensure
self.specific_hosts = original_hosts
end
def accessory_names def accessory_names
config.accessories&.collect(&:name) || [] config.accessories&.collect(&:name) || []
end end
@@ -85,10 +101,6 @@ class Kamal::Commander
@docker ||= Kamal::Commands::Docker.new(config) @docker ||= Kamal::Commands::Docker.new(config)
end end
def healthcheck
@healthcheck ||= Kamal::Commands::Healthcheck.new(config)
end
def hook def hook
@hook ||= Kamal::Commands::Hook.new(config) @hook ||= Kamal::Commands::Hook.new(config)
end end
@@ -97,6 +109,10 @@ class Kamal::Commander
@lock ||= Kamal::Commands::Lock.new(config) @lock ||= Kamal::Commands::Lock.new(config)
end end
def proxy
@proxy ||= Kamal::Commands::Proxy.new(config)
end
def prune def prune
@prune ||= Kamal::Commands::Prune.new(config) @prune ||= Kamal::Commands::Prune.new(config)
end end
@@ -109,8 +125,8 @@ class Kamal::Commander
@server ||= Kamal::Commands::Server.new(config) @server ||= Kamal::Commands::Server.new(config)
end end
def traefik def alias(name)
@traefik ||= Kamal::Commands::Traefik.new(config) config.aliases[name]
end end

View File

@@ -18,12 +18,12 @@ class Kamal::Commander::Specifics
roles.select { |role| role.hosts.include?(host.to_s) } roles.select { |role| role.hosts.include?(host.to_s) }
end end
def traefik_hosts def proxy_hosts
config.traefik_hosts & specified_hosts config.proxy_hosts & specified_hosts
end end
def accessory_hosts def accessory_hosts
specific_hosts || config.accessories.flat_map(&:hosts) config.accessories.flat_map(&:hosts) & specified_hosts
end end
private private
@@ -43,7 +43,12 @@ class Kamal::Commander::Specifics
end end
def specified_hosts def specified_hosts
(specific_hosts || config.all_hosts) \ specified_hosts = specific_hosts || config.all_hosts
.select { |host| (specific_roles || config.roles).flat_map(&:hosts).include?(host) }
if (specific_role_hosts = specific_roles&.flat_map(&:hosts)).present?
specified_hosts.select { |host| specific_role_hosts.include?(host) }
else
specified_hosts
end
end end
end end

View File

@@ -1,7 +1,9 @@
class Kamal::Commands::Accessory < Kamal::Commands::Base class Kamal::Commands::Accessory < Kamal::Commands::Base
attr_reader :accessory_config attr_reader :accessory_config
delegate :service_name, :image, :hosts, :port, :files, :directories, :cmd, delegate :service_name, :image, :hosts, :port, :files, :directories, :cmd,
:publish_args, :env_args, :volume_args, :label_args, :option_args, to: :accessory_config :publish_args, :env_args, :volume_args, :label_args, :option_args,
:secrets_io, :secrets_path, :env_directory,
to: :accessory_config
def initialize(config, name:) def initialize(config, name:)
super(config) super(config)
@@ -13,6 +15,7 @@ class Kamal::Commands::Accessory < Kamal::Commands::Base
"--name", service_name, "--name", service_name,
"--detach", "--detach",
"--restart", "unless-stopped", "--restart", "unless-stopped",
"--network", "kamal",
*config.logging_args, *config.logging_args,
*publish_args, *publish_args,
*env_args, *env_args,
@@ -36,16 +39,16 @@ class Kamal::Commands::Accessory < Kamal::Commands::Base
end end
def logs(since: nil, lines: nil, grep: nil, grep_options: nil) def logs(timestamps: true, since: nil, lines: nil, grep: nil, grep_options: nil)
pipe \ pipe \
docker(:logs, service_name, (" --since #{since}" if since), (" --tail #{lines}" if lines), "--timestamps", "2>&1"), docker(:logs, service_name, (" --since #{since}" if since), (" --tail #{lines}" if lines), ("--timestamps" if timestamps), "2>&1"),
("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep) ("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep)
end end
def follow_logs(grep: nil, grep_options: nil) def follow_logs(timestamps: true, grep: nil, grep_options: nil)
run_over_ssh \ run_over_ssh \
pipe \ pipe \
docker(:logs, service_name, "--timestamps", "--tail", "10", "--follow", "2>&1"), docker(:logs, service_name, ("--timestamps" if timestamps), "--tail", "10", "--follow", "2>&1"),
(%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep) (%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep)
end end
@@ -61,6 +64,7 @@ class Kamal::Commands::Accessory < Kamal::Commands::Base
docker :run, docker :run,
("-it" if interactive), ("-it" if interactive),
"--rm", "--rm",
"--network", "kamal",
*env_args, *env_args,
*volume_args, *volume_args,
image, image,
@@ -98,12 +102,8 @@ class Kamal::Commands::Accessory < Kamal::Commands::Base
docker :image, :rm, "--force", image docker :image, :rm, "--force", image
end end
def make_env_directory def ensure_env_directory
make_directory accessory_config.env.secrets_directory make_directory env_directory
end
def remove_env_file
[ :rm, "-f", accessory_config.env.secrets_file ]
end end
private private

View File

@@ -1,10 +1,12 @@
class Kamal::Commands::App < Kamal::Commands::Base class Kamal::Commands::App < Kamal::Commands::Base
include Assets, Containers, Cord, Execution, Images, Logging include Assets, Containers, Execution, Images, Logging, Proxy
ACTIVE_DOCKER_STATUSES = [ :running, :restarting ] ACTIVE_DOCKER_STATUSES = [ :running, :restarting ]
attr_reader :role, :host attr_reader :role, :host
delegate :container_name, to: :role
def initialize(config, role: nil, host: nil) def initialize(config, role: nil, host: nil)
super(config) super(config)
@role = role @role = role
@@ -16,11 +18,11 @@ class Kamal::Commands::App < Kamal::Commands::Base
"--detach", "--detach",
"--restart unless-stopped", "--restart unless-stopped",
"--name", container_name, "--name", container_name,
"--network", "kamal",
*([ "--hostname", hostname ] if hostname), *([ "--hostname", hostname ] if hostname),
"-e", "KAMAL_CONTAINER_NAME=\"#{container_name}\"", "-e", "KAMAL_CONTAINER_NAME=\"#{container_name}\"",
"-e", "KAMAL_VERSION=\"#{config.version}\"", "-e", "KAMAL_VERSION=\"#{config.version}\"",
*role.env_args(host), *role.env_args(host),
*role.health_check_args,
*role.logging_args, *role.logging_args,
*config.volume_args, *config.volume_args,
*role.asset_volume_args, *role.asset_volume_args,
@@ -41,7 +43,7 @@ class Kamal::Commands::App < Kamal::Commands::Base
def stop(version: nil) def stop(version: nil)
pipe \ pipe \
version ? container_id_for_version(version) : current_running_container_id, version ? container_id_for_version(version) : current_running_container_id,
xargs(config.stop_wait_time ? docker(:stop, "-t", config.stop_wait_time) : docker(:stop)) xargs(docker(:stop, *role.stop_args))
end end
def info def info
@@ -69,21 +71,11 @@ class Kamal::Commands::App < Kamal::Commands::Base
extract_version_from_name extract_version_from_name
end end
def ensure_env_directory
def make_env_directory make_directory role.env_directory
make_directory role.env(host).secrets_directory
end end
def remove_env_file
[ :rm, "-f", role.env(host).secrets_file ]
end
private private
def container_name(version = nil)
[ role.container_prefix, version || config.version ].compact.join("-")
end
def latest_image_id def latest_image_id
docker :image, :ls, *argumentize("--filter", "reference=#{config.latest_image}"), "--format", "'{{.ID}}'" docker :image, :ls, *argumentize("--filter", "reference=#{config.latest_image}"), "--format", "'{{.ID}}'"
end end

View File

@@ -3,18 +3,18 @@ module Kamal::Commands::App::Assets
asset_container = "#{role.container_prefix}-assets" asset_container = "#{role.container_prefix}-assets"
combine \ combine \
make_directory(role.asset_extracted_path), make_directory(role.asset_extracted_directory),
[ *docker(:stop, "-t 1", asset_container, "2> /dev/null"), "|| true" ], [ *docker(:stop, "-t 1", asset_container, "2> /dev/null"), "|| true" ],
docker(:run, "--name", asset_container, "--detach", "--rm", config.absolute_image, "sleep 1000000"), docker(:run, "--name", asset_container, "--detach", "--rm", "--entrypoint", "sleep", config.absolute_image, "1000000"),
docker(:cp, "-L", "#{asset_container}:#{role.asset_path}/.", role.asset_extracted_path), docker(:cp, "-L", "#{asset_container}:#{role.asset_path}/.", role.asset_extracted_directory),
docker(:stop, "-t 1", asset_container), docker(:stop, "-t 1", asset_container),
by: "&&" by: "&&"
end end
def sync_asset_volumes(old_version: nil) def sync_asset_volumes(old_version: nil)
new_extracted_path, new_volume_path = role.asset_extracted_path(config.version), role.asset_volume.host_path new_extracted_path, new_volume_path = role.asset_extracted_directory(config.version), role.asset_volume.host_path
if old_version.present? if old_version.present?
old_extracted_path, old_volume_path = role.asset_extracted_path(old_version), role.asset_volume(old_version).host_path old_extracted_path, old_volume_path = role.asset_extracted_directory(old_version), role.asset_volume(old_version).host_path
end end
commands = [ make_directory(new_volume_path), copy_contents(new_extracted_path, new_volume_path) ] commands = [ make_directory(new_volume_path), copy_contents(new_extracted_path, new_volume_path) ]
@@ -29,8 +29,8 @@ module Kamal::Commands::App::Assets
def clean_up_assets def clean_up_assets
chain \ chain \
find_and_remove_older_siblings(role.asset_extracted_path), find_and_remove_older_siblings(role.asset_extracted_directory),
find_and_remove_older_siblings(role.asset_volume_path) find_and_remove_older_siblings(role.asset_volume_directory)
end end
private private
@@ -39,7 +39,7 @@ module Kamal::Commands::App::Assets
:find, :find,
Pathname.new(path).dirname.to_s, Pathname.new(path).dirname.to_s,
"-maxdepth 1", "-maxdepth 1",
"-name", "'#{role.container_prefix}-*'", "-name", "'#{role.name}-*'",
"!", "-name", Pathname.new(path).basename.to_s, "!", "-name", Pathname.new(path).basename.to_s,
"-exec rm -rf \"{}\" +" "-exec rm -rf \"{}\" +"
] ]

View File

@@ -1,22 +0,0 @@
module Kamal::Commands::App::Cord
def cord(version:)
pipe \
docker(:inspect, "-f '{{ range .Mounts }}{{printf \"%s %s\\n\" .Source .Destination}}{{ end }}'", container_name(version)),
[ :awk, "'$2 == \"#{role.cord_volume.container_path}\" {print $1}'" ]
end
def tie_cord(cord)
create_empty_file(cord)
end
def cut_cord(cord)
remove_directory(cord)
end
private
def create_empty_file(file)
chain \
make_directory_for(file),
[ :touch, file ]
end
end

View File

@@ -11,6 +11,7 @@ module Kamal::Commands::App::Execution
docker :run, docker :run,
("-it" if interactive), ("-it" if interactive),
"--rm", "--rm",
"--network", "kamal",
*role&.env_args(host), *role&.env_args(host),
*argumentize("--env", env), *argumentize("--env", env),
*config.volume_args, *config.volume_args,

View File

@@ -1,16 +1,16 @@
module Kamal::Commands::App::Logging module Kamal::Commands::App::Logging
def logs(version: nil, since: nil, lines: nil, grep: nil, grep_options: nil) def logs(version: nil, timestamps: true, since: nil, lines: nil, grep: nil, grep_options: nil)
pipe \ pipe \
version ? container_id_for_version(version) : current_running_container_id, version ? container_id_for_version(version) : current_running_container_id,
"xargs docker logs#{" --since #{since}" if since}#{" --tail #{lines}" if lines} 2>&1", "xargs docker logs#{" --timestamps" if timestamps}#{" --since #{since}" if since}#{" --tail #{lines}" if lines} 2>&1",
("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep) ("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep)
end end
def follow_logs(host:, lines: nil, grep: nil, grep_options: nil) def follow_logs(host:, timestamps: true, lines: nil, grep: nil, grep_options: nil)
run_over_ssh \ run_over_ssh \
pipe( pipe(
current_running_container_id, current_running_container_id,
"xargs docker logs --timestamps#{" --tail #{lines}" if lines} --follow 2>&1", "xargs docker logs#{" --timestamps" if timestamps}#{" --tail #{lines}" if lines} --follow 2>&1",
(%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep) (%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep)
), ),
host: host host: host

View File

@@ -0,0 +1,16 @@
module Kamal::Commands::App::Proxy
delegate :proxy_container_name, to: :config
def deploy(target:)
proxy_exec :deploy, role.container_prefix, *role.proxy.deploy_command_args(target: target)
end
def remove
proxy_exec :remove, role.container_prefix
end
private
def proxy_exec(*command)
docker :exec, proxy_container_name, "kamal-proxy", *command
end
end

View File

@@ -8,9 +8,12 @@ class Kamal::Commands::Auditor < Kamal::Commands::Base
# Runs remotely # Runs remotely
def record(line, **details) def record(line, **details)
append \ combine \
[ :echo, audit_tags(**details).except(:version, :service_version).to_s, line ], [ :mkdir, "-p", config.run_directory ],
append(
[ :echo, audit_tags(**details).except(:version, :service_version, :service).to_s, line ],
audit_log_file audit_log_file
)
end end
def reveal def reveal

View File

@@ -37,6 +37,10 @@ module Kamal::Commands
[ :rm, "-r", path ] [ :rm, "-r", path ]
end end
def remove_file(path)
[ :rm, path ]
end
private private
def combine(*commands, by: "&&") def combine(*commands, by: "&&")
commands commands
@@ -81,6 +85,10 @@ module Kamal::Commands
[ :git, *([ "-C", path ] if path), *args.compact ] [ :git, *([ "-C", path ] if path), *args.compact ]
end end
def grep(*args)
args.compact.unshift :grep
end
def tags(**details) def tags(**details)
Kamal::Tags.from_config(config, **details) Kamal::Tags.from_config(config, **details)
end end

View File

@@ -1,8 +1,8 @@
require "active_support/core_ext/string/filters" require "active_support/core_ext/string/filters"
class Kamal::Commands::Builder < Kamal::Commands::Base class Kamal::Commands::Builder < Kamal::Commands::Base
delegate :create, :remove, :push, :clean, :pull, :info, :context_hosts, :config_context_hosts, :validate_image, delegate :create, :remove, :push, :clean, :pull, :info, :inspect_builder, :validate_image, :first_mirror, to: :target
to: :target delegate :local?, :remote?, to: "config.builder"
include Clone include Clone
@@ -11,43 +11,27 @@ class Kamal::Commands::Builder < Kamal::Commands::Base
end end
def target def target
if config.builder.multiarch? if remote?
if config.builder.remote? if local?
if config.builder.local? hybrid
multiarch_remote
else else
native_remote remote
end end
else else
multiarch local
end
else
if config.builder.cached?
native_cached
else
native
end
end end
end end
def native def remote
@native ||= Kamal::Commands::Builder::Native.new(config) @remote ||= Kamal::Commands::Builder::Remote.new(config)
end end
def native_cached def local
@native ||= Kamal::Commands::Builder::Native::Cached.new(config) @local ||= Kamal::Commands::Builder::Local.new(config)
end end
def native_remote def hybrid
@native ||= Kamal::Commands::Builder::Native::Remote.new(config) @hybrid ||= Kamal::Commands::Builder::Hybrid.new(config)
end
def multiarch
@multiarch ||= Kamal::Commands::Builder::Multiarch.new(config)
end
def multiarch_remote
@multiarch_remote ||= Kamal::Commands::Builder::Multiarch::Remote.new(config)
end end

View File

@@ -1,20 +1,41 @@
class Kamal::Commands::Builder::Base < Kamal::Commands::Base class Kamal::Commands::Builder::Base < Kamal::Commands::Base
class BuilderError < StandardError; end class BuilderError < StandardError; end
ENDPOINT_DOCKER_HOST_INSPECT = "'{{.Endpoints.docker.Host}}'" ENDPOINT_DOCKER_HOST_INSPECT = "'{{.Endpoints.docker.Host}}'"
delegate :argumentize, to: Kamal::Utils delegate :argumentize, to: Kamal::Utils
delegate :args, :secrets, :dockerfile, :target, :local_arch, :local_host, :remote_arch, :remote_host, :cache_from, :cache_to, :ssh, to: :builder_config delegate \
:args, :secrets, :dockerfile, :target, :arches, :local_arches, :remote_arches, :remote,
:cache_from, :cache_to, :ssh, :driver, :docker_driver?,
to: :builder_config
def clean def clean
docker :image, :rm, "--force", config.absolute_image docker :image, :rm, "--force", config.absolute_image
end end
def push
docker :buildx, :build,
"--push",
*platform_options(arches),
*([ "--builder", builder_name ] unless docker_driver?),
*build_options,
build_context
end
def pull def pull
docker :pull, config.absolute_image docker :pull, config.absolute_image
end end
def info
combine \
docker(:context, :ls),
docker(:buildx, :ls)
end
def inspect_builder
docker :buildx, :inspect, builder_name unless docker_driver?
end
def build_options def build_options
[ *build_tags, *build_cache, *build_labels, *build_args, *build_secrets, *build_dockerfile, *build_target, *build_ssh ] [ *build_tags, *build_cache, *build_labels, *build_args, *build_secrets, *build_dockerfile, *build_target, *build_ssh ]
end end
@@ -32,12 +53,8 @@ class Kamal::Commands::Builder::Base < Kamal::Commands::Base
) )
end end
def context_hosts def first_mirror
:true docker(:info, "--format '{{index .RegistryConfig.Mirrors 0}}'")
end
def config_context_hosts
[]
end end
private private
@@ -61,7 +78,7 @@ class Kamal::Commands::Builder::Base < Kamal::Commands::Base
end end
def build_secrets def build_secrets
argumentize "--secret", secrets.collect { |secret| [ "id", secret ] } argumentize "--secret", secrets.keys.collect { |secret| [ "id", secret ] }
end end
def build_dockerfile def build_dockerfile
@@ -84,7 +101,7 @@ class Kamal::Commands::Builder::Base < Kamal::Commands::Base
config.builder config.builder
end end
def context_host(builder_name) def platform_options(arches)
docker :context, :inspect, builder_name, "--format", ENDPOINT_DOCKER_HOST_INSPECT argumentize "--platform", arches.map { |arch| "linux/#{arch}" }.join(",") if arches.any?
end end
end end

View File

@@ -6,7 +6,7 @@ module Kamal::Commands::Builder::Clone
end end
def clone def clone
git :clone, Kamal::Git.root, path: clone_directory git :clone, Kamal::Git.root, "--recurse-submodules", path: clone_directory
end end
def clone_reset_steps def clone_reset_steps
@@ -14,7 +14,8 @@ module Kamal::Commands::Builder::Clone
git(:remote, "set-url", :origin, Kamal::Git.root, path: build_directory), git(:remote, "set-url", :origin, Kamal::Git.root, path: build_directory),
git(:fetch, :origin, path: build_directory), git(:fetch, :origin, path: build_directory),
git(:reset, "--hard", Kamal::Git.revision, path: build_directory), git(:reset, "--hard", Kamal::Git.revision, path: build_directory),
git(:clean, "-fdx", path: build_directory) git(:clean, "-fdx", path: build_directory),
git(:submodule, :update, "--init", path: build_directory)
] ]
end end

View File

@@ -0,0 +1,21 @@
class Kamal::Commands::Builder::Hybrid < Kamal::Commands::Builder::Remote
def create
combine \
create_local_buildx,
create_remote_context,
append_remote_buildx
end
private
def builder_name
"kamal-hybrid-#{driver}-#{remote.gsub(/[^a-z0-9_-]/, "-")}"
end
def create_local_buildx
docker :buildx, :create, *platform_options(local_arches), "--name", builder_name, "--driver=#{driver}"
end
def append_remote_buildx
docker :buildx, :create, *platform_options(remote_arches), "--append", "--name", builder_name, remote_context_name
end
end

View File

@@ -0,0 +1,14 @@
class Kamal::Commands::Builder::Local < Kamal::Commands::Builder::Base
def create
docker :buildx, :create, "--name", builder_name, "--driver=#{driver}" unless docker_driver?
end
def remove
docker :buildx, :rm, builder_name unless docker_driver?
end
private
def builder_name
"kamal-local-#{driver}"
end
end

View File

@@ -1,41 +0,0 @@
class Kamal::Commands::Builder::Multiarch < Kamal::Commands::Builder::Base
def create
docker :buildx, :create, "--use", "--name", builder_name
end
def remove
docker :buildx, :rm, builder_name
end
def info
combine \
docker(:context, :ls),
docker(:buildx, :ls)
end
def push
docker :buildx, :build,
"--push",
"--platform", platform_names,
"--builder", builder_name,
*build_options,
build_context
end
def context_hosts
docker :buildx, :inspect, builder_name, "> /dev/null"
end
private
def builder_name
"kamal-#{config.service}-multiarch"
end
def platform_names
if local_arch
"linux/#{local_arch}"
else
"linux/amd64,linux/arm64"
end
end
end

View File

@@ -1,61 +0,0 @@
class Kamal::Commands::Builder::Multiarch::Remote < Kamal::Commands::Builder::Multiarch
def create
combine \
create_contexts,
create_local_buildx,
append_remote_buildx
end
def remove
combine \
remove_contexts,
super
end
def context_hosts
chain \
context_host(builder_name_with_arch(local_arch)),
context_host(builder_name_with_arch(remote_arch))
end
def config_context_hosts
[ local_host, remote_host ].compact
end
private
def builder_name
super + "-remote"
end
def builder_name_with_arch(arch)
"#{builder_name}-#{arch}"
end
def create_local_buildx
docker :buildx, :create, "--name", builder_name, builder_name_with_arch(local_arch), "--platform", "linux/#{local_arch}"
end
def append_remote_buildx
docker :buildx, :create, "--append", "--name", builder_name, builder_name_with_arch(remote_arch), "--platform", "linux/#{remote_arch}"
end
def create_contexts
combine \
create_context(local_arch, local_host),
create_context(remote_arch, remote_host)
end
def create_context(arch, host)
docker :context, :create, builder_name_with_arch(arch), "--description", "'#{builder_name} #{arch} native host'", "--docker", "'host=#{host}'"
end
def remove_contexts
combine \
remove_context(local_arch),
remove_context(remote_arch)
end
def remove_context(arch)
docker :context, :rm, builder_name_with_arch(arch)
end
end

View File

@@ -1,20 +0,0 @@
class Kamal::Commands::Builder::Native < Kamal::Commands::Builder::Base
def create
# No-op on native without cache
end
def remove
# No-op on native without cache
end
def info
# No-op on native
end
def push
combine \
docker(:build, *build_options, build_context),
docker(:push, config.absolute_image),
docker(:push, config.latest_image)
end
end

View File

@@ -1,25 +0,0 @@
class Kamal::Commands::Builder::Native::Cached < Kamal::Commands::Builder::Native
def create
docker :buildx, :create, "--name", builder_name, "--use", "--driver=docker-container"
end
def remove
docker :buildx, :rm, builder_name
end
def push
docker :buildx, :build,
"--push",
*build_options,
build_context
end
def context_hosts
docker :buildx, :inspect, builder_name, "> /dev/null"
end
private
def builder_name
"kamal-#{config.service}-native-cached"
end
end

View File

@@ -1,67 +0,0 @@
class Kamal::Commands::Builder::Native::Remote < Kamal::Commands::Builder::Native
def create
chain \
create_context,
create_buildx
end
def remove
chain \
remove_context,
remove_buildx
end
def info
chain \
docker(:context, :ls),
docker(:buildx, :ls)
end
def push
docker :buildx, :build,
"--push",
"--platform", platform,
"--builder", builder_name,
*build_options,
build_context
end
def context_hosts
context_host(builder_name_with_arch)
end
def config_context_hosts
[ remote_host ]
end
private
def builder_name
"kamal-#{config.service}-native-remote"
end
def builder_name_with_arch
"#{builder_name}-#{remote_arch}"
end
def platform
"linux/#{remote_arch}"
end
def create_context
docker :context, :create,
builder_name_with_arch, "--description", "'#{builder_name} #{remote_arch} native host'", "--docker", "'host=#{remote_host}'"
end
def remove_context
docker :context, :rm, builder_name_with_arch
end
def create_buildx
docker :buildx, :create, "--name", builder_name, builder_name_with_arch, "--platform", platform
end
def remove_buildx
docker :buildx, :rm, builder_name
end
end

View File

@@ -0,0 +1,63 @@
class Kamal::Commands::Builder::Remote < Kamal::Commands::Builder::Base
def create
chain \
create_remote_context,
create_buildx
end
def remove
chain \
remove_remote_context,
remove_buildx
end
def info
chain \
docker(:context, :ls),
docker(:buildx, :ls)
end
def inspect_builder
combine \
combine inspect_buildx, inspect_remote_context,
[ "(echo no compatible builder && exit 1)" ],
by: "||"
end
private
def builder_name
"kamal-remote-#{remote.gsub(/[^a-z0-9_-]/, "-")}"
end
def remote_context_name
"#{builder_name}-context"
end
def inspect_buildx
pipe \
docker(:buildx, :inspect, builder_name),
grep("-q", "Endpoint:.*#{remote_context_name}")
end
def inspect_remote_context
pipe \
docker(:context, :inspect, remote_context_name, "--format", ENDPOINT_DOCKER_HOST_INSPECT),
grep("-xq", remote)
end
def create_remote_context
docker :context, :create, remote_context_name, "--description", "'#{builder_name} host'", "--docker", "'host=#{remote}'"
end
def remove_remote_context
docker :context, :rm, remote_context_name
end
def create_buildx
docker :buildx, :create, "--name", builder_name, remote_context_name
end
def remove_buildx
docker :buildx, :rm, builder_name
end
end

View File

@@ -19,6 +19,10 @@ class Kamal::Commands::Docker < Kamal::Commands::Base
[ '[ "${EUID:-$(id -u)}" -eq 0 ] || command -v sudo >/dev/null || command -v su >/dev/null' ] [ '[ "${EUID:-$(id -u)}" -eq 0 ] || command -v sudo >/dev/null || command -v su >/dev/null' ]
end end
def create_network
docker :network, :create, :kamal
end
private private
def get_docker def get_docker
shell \ shell \

View File

@@ -1,6 +1,12 @@
class Kamal::Commands::Hook < Kamal::Commands::Base class Kamal::Commands::Hook < Kamal::Commands::Base
def run(hook, **details) def run(hook)
[ hook_file(hook), env: tags(**details).env ] [ hook_file(hook) ]
end
def env(secrets: false, **details)
tags(**details).env.tap do |env|
env.merge!(config.secrets.to_h) if secrets
end
end end
def hook_exists?(hook) def hook_exists?(hook)

View File

@@ -44,14 +44,10 @@ class Kamal::Commands::Lock < Kamal::Commands::Base
"/dev/null" "/dev/null"
end end
def locks_dir
File.join(config.run_directory, "locks")
end
def lock_dir def lock_dir
dir_name = [ config.service, config.destination ].compact.join("-") dir_name = [ "lock", config.service, config.destination ].compact.join("-")
File.join(locks_dir, dir_name) File.join(config.run_directory, dir_name)
end end
def lock_details_file def lock_details_file

View File

@@ -0,0 +1,87 @@
class Kamal::Commands::Proxy < Kamal::Commands::Base
delegate :argumentize, :optionize, to: Kamal::Utils
def run
docker :run,
"--name", container_name,
"--network", "kamal",
"--detach",
"--restart", "unless-stopped",
"--volume", "kamal-proxy-config:/home/kamal-proxy/.config/kamal-proxy",
"\$\(#{get_boot_options.join(" ")}\)",
config.proxy_image
end
def start
docker :container, :start, container_name
end
def stop(name: container_name)
docker :container, :stop, name
end
def start_or_run
combine start, run, by: "||"
end
def info
docker :ps, "--filter", "name=^#{container_name}$"
end
def version
pipe \
docker(:inspect, container_name, "--format '{{.Config.Image}}'"),
[ :cut, "-d:", "-f2" ]
end
def logs(timestamps: true, since: nil, lines: nil, grep: nil, grep_options: nil)
pipe \
docker(:logs, container_name, ("--since #{since}" if since), ("--tail #{lines}" if lines), ("--timestamps" if timestamps), "2>&1"),
("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep)
end
def follow_logs(host:, timestamps: true, grep: nil, grep_options: nil)
run_over_ssh pipe(
docker(:logs, container_name, ("--timestamps" if timestamps), "--tail", "10", "--follow", "2>&1"),
(%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep)
).join(" "), host: host
end
def remove_container
docker :container, :prune, "--force", "--filter", "label=org.opencontainers.image.title=kamal-proxy"
end
def remove_image
docker :image, :prune, "--all", "--force", "--filter", "label=org.opencontainers.image.title=kamal-proxy"
end
def cleanup_traefik
chain \
docker(:container, :stop, "traefik"),
combine(
docker(:container, :prune, "--force", "--filter", "label=org.opencontainers.image.title=Traefik"),
docker(:image, :prune, "--all", "--force", "--filter", "label=org.opencontainers.image.title=Traefik")
)
end
def ensure_proxy_directory
make_directory config.proxy_directory
end
def remove_proxy_directory
remove_directory config.proxy_directory
end
def get_boot_options
combine [ :cat, config.proxy_options_file ], [ :echo, "\"#{config.proxy_options_default.join(" ")}\"" ], by: "||"
end
def reset_boot_options
remove_file config.proxy_options_file
end
private
def container_name
config.proxy_container_name
end
end

View File

@@ -9,7 +9,7 @@ class Kamal::Commands::Prune < Kamal::Commands::Base
def tagged_images def tagged_images
pipe \ pipe \
docker(:image, :ls, *service_filter, "--format", "'{{.ID}} {{.Repository}}:{{.Tag}}'"), docker(:image, :ls, *service_filter, "--format", "'{{.ID}} {{.Repository}}:{{.Tag}}'"),
"grep -v -w \"#{active_image_list}\"", grep("-v -w \"#{active_image_list}\""),
"while read image tag; do docker rmi $tag; done" "while read image tag; do docker rmi $tag; done"
end end
@@ -20,10 +20,6 @@ class Kamal::Commands::Prune < Kamal::Commands::Base
"while read container_id; do docker rm $container_id; done" "while read container_id; do docker rm $container_id; done"
end end
def healthcheck_containers
docker :container, :prune, "--force", *healthcheck_service_filter
end
private private
def stopped_containers_filters def stopped_containers_filters
[ "created", "exited", "dead" ].flat_map { |status| [ "--filter", "status=#{status}" ] } [ "created", "exited", "dead" ].flat_map { |status| [ "--filter", "status=#{status}" ] }
@@ -39,8 +35,4 @@ class Kamal::Commands::Prune < Kamal::Commands::Base
def service_filter def service_filter
[ "--filter", "label=service=#{config.service}" ] [ "--filter", "label=service=#{config.service}" ]
end end
def healthcheck_service_filter
[ "--filter", "label=service=#{config.healthcheck_service}" ]
end
end end

View File

@@ -1,5 +1,15 @@
class Kamal::Commands::Server < Kamal::Commands::Base class Kamal::Commands::Server < Kamal::Commands::Base
def ensure_run_directory def ensure_run_directory
[ :mkdir, "-p", config.run_directory ] make_directory config.run_directory
end
def remove_app_directory
remove_directory config.app_directory
end
def app_directory_count
pipe \
[ :ls, config.apps_directory ],
[ :wc, "-l" ]
end end
end end

View File

@@ -1,85 +0,0 @@
class Kamal::Commands::Traefik < Kamal::Commands::Base
delegate :argumentize, :optionize, to: Kamal::Utils
delegate :port, :publish?, :labels, :env, :image, :options, :args, to: :"config.traefik"
def run
docker :run, "--name traefik",
"--detach",
"--restart", "unless-stopped",
*publish_args,
"--volume", "/var/run/docker.sock:/var/run/docker.sock",
*env_args,
*config.logging_args,
*label_args,
*docker_options_args,
image,
"--providers.docker",
*cmd_option_args
end
def start
docker :container, :start, "traefik"
end
def stop
docker :container, :stop, "traefik"
end
def start_or_run
any start, run
end
def info
docker :ps, "--filter", "name=^traefik$"
end
def logs(since: nil, lines: nil, grep: nil, grep_options: nil)
pipe \
docker(:logs, "traefik", (" --since #{since}" if since), (" --tail #{lines}" if lines), "--timestamps", "2>&1"),
("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep)
end
def follow_logs(host:, grep: nil, grep_options: nil)
run_over_ssh pipe(
docker(:logs, "traefik", "--timestamps", "--tail", "10", "--follow", "2>&1"),
(%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep)
).join(" "), host: host
end
def remove_container
docker :container, :prune, "--force", "--filter", "label=org.opencontainers.image.title=Traefik"
end
def remove_image
docker :image, :prune, "--all", "--force", "--filter", "label=org.opencontainers.image.title=Traefik"
end
def make_env_directory
make_directory(env.secrets_directory)
end
def remove_env_file
[ :rm, "-f", env.secrets_file ]
end
private
def publish_args
argumentize "--publish", port if publish?
end
def label_args
argumentize "--label", labels
end
def env_args
env.args
end
def docker_options_args
optionize(options)
end
def cmd_option_args
optionize args, with: "="
end
end

View File

@@ -2,21 +2,27 @@ require "active_support/ordered_options"
require "active_support/core_ext/string/inquiry" require "active_support/core_ext/string/inquiry"
require "active_support/core_ext/module/delegation" require "active_support/core_ext/module/delegation"
require "active_support/core_ext/hash/keys" require "active_support/core_ext/hash/keys"
require "pathname"
require "erb" require "erb"
require "net/ssh/proxy/jump" require "net/ssh/proxy/jump"
class Kamal::Configuration class Kamal::Configuration
delegate :service, :image, :labels, :stop_wait_time, :hooks_path, to: :raw_config, allow_nil: true delegate :service, :image, :labels, :hooks_path, to: :raw_config, allow_nil: true
delegate :argumentize, :optionize, to: Kamal::Utils delegate :argumentize, :optionize, to: Kamal::Utils
attr_reader :destination, :raw_config attr_reader :destination, :raw_config, :secrets
attr_reader :accessories, :boot, :builder, :env, :healthcheck, :logging, :traefik, :servers, :ssh, :sshkit, :registry attr_reader :accessories, :aliases, :boot, :builder, :env, :logging, :proxy, :servers, :ssh, :sshkit, :registry
include Validation include Validation
PROXY_MINIMUM_VERSION = "v0.8.1"
PROXY_HTTP_PORT = 80
PROXY_HTTPS_PORT = 443
PROXY_LOG_MAX_SIZE = "10m"
class << self class << self
def create_from(config_file:, destination: nil, version: nil) def create_from(config_file:, destination: nil, version: nil)
ENV["KAMAL_DESTINATION"] = destination
raw_config = load_config_files(config_file, *destination_config_file(config_file, destination)) raw_config = load_config_files(config_file, *destination_config_file(config_file, destination))
new raw_config, destination: destination, version: version new raw_config, destination: destination, version: version
@@ -47,20 +53,22 @@ class Kamal::Configuration
@destination = destination @destination = destination
@declared_version = version @declared_version = version
validate! raw_config, example: validation_yml.symbolize_keys, context: "" validate! raw_config, example: validation_yml.symbolize_keys, context: "", with: Kamal::Configuration::Validator::Configuration
@secrets = Kamal::Secrets.new(destination: destination)
# Eager load config to validate it, these are first as they have dependencies later on # Eager load config to validate it, these are first as they have dependencies later on
@servers = Servers.new(config: self) @servers = Servers.new(config: self)
@registry = Registry.new(config: self) @registry = Registry.new(config: self)
@accessories = @raw_config.accessories&.keys&.collect { |name| Accessory.new(name, config: self) } || [] @accessories = @raw_config.accessories&.keys&.collect { |name| Accessory.new(name, config: self) } || []
@aliases = @raw_config.aliases&.keys&.to_h { |name| [ name, Alias.new(name, config: self) ] } || {}
@boot = Boot.new(config: self) @boot = Boot.new(config: self)
@builder = Builder.new(config: self) @builder = Builder.new(config: self)
@env = Env.new(config: @raw_config.env || {}) @env = Env.new(config: @raw_config.env || {}, secrets: secrets)
@healthcheck = Healthcheck.new(healthcheck_config: @raw_config.healthcheck)
@logging = Logging.new(logging_config: @raw_config.logging) @logging = Logging.new(logging_config: @raw_config.logging)
@traefik = Traefik.new(config: self) @proxy = Proxy.new(config: self, proxy_config: @raw_config.proxy || {})
@ssh = Ssh.new(config: self) @ssh = Ssh.new(config: self)
@sshkit = Sshkit.new(config: self) @sshkit = Sshkit.new(config: self)
@@ -69,6 +77,9 @@ class Kamal::Configuration
ensure_valid_kamal_version ensure_valid_kamal_version
ensure_retain_containers_valid ensure_retain_containers_valid
ensure_valid_service_name ensure_valid_service_name
ensure_no_traefik_reboot_hooks
ensure_one_host_for_ssl_roles
ensure_unique_hosts_for_ssl_roles
end end
@@ -129,16 +140,16 @@ class Kamal::Configuration
raw_config.allow_empty_roles raw_config.allow_empty_roles
end end
def traefik_roles def proxy_roles
roles.select(&:running_traefik?) roles.select(&:running_proxy?)
end end
def traefik_role_names def proxy_role_names
traefik_roles.flat_map(&:name) proxy_roles.flat_map(&:name)
end end
def traefik_hosts def proxy_hosts
traefik_roles.flat_map(&:hosts).uniq proxy_roles.flat_map(&:hosts).uniq
end end
def repository def repository
@@ -183,31 +194,40 @@ class Kamal::Configuration
end end
def healthcheck_service
[ "healthcheck", service, destination ].compact.join("-")
end
def readiness_delay def readiness_delay
raw_config.readiness_delay || 7 raw_config.readiness_delay || 7
end end
def run_id def deploy_timeout
@run_id ||= SecureRandom.hex(16) raw_config.deploy_timeout || 30
end
def drain_timeout
raw_config.drain_timeout || 30
end end
def run_directory def run_directory
raw_config.run_directory || ".kamal" ".kamal"
end end
def run_directory_as_docker_volume def apps_directory
if Pathname.new(run_directory).absolute? File.join run_directory, "apps"
run_directory
else
File.join "$(pwd)", run_directory
end end
def app_directory
File.join apps_directory, [ service, destination ].compact.join("-")
end end
def env_directory
File.join app_directory, "env"
end
def assets_directory
File.join app_directory, "assets"
end
def hooks_path def hooks_path
raw_config.hooks_path || ".kamal/hooks" raw_config.hooks_path || ".kamal/hooks"
end end
@@ -217,13 +237,9 @@ class Kamal::Configuration
end end
def host_env_directory
File.join(run_directory, "env")
end
def env_tags def env_tags
@env_tags ||= if (tags = raw_config.env["tags"]) @env_tags ||= if (tags = raw_config.env["tags"])
tags.collect { |name, config| Env::Tag.new(name, config: config) } tags.collect { |name, config| Env::Tag.new(name, config: config, secrets: secrets) }
else else
[] []
end end
@@ -233,6 +249,34 @@ class Kamal::Configuration
env_tags.detect { |t| t.name == name.to_s } env_tags.detect { |t| t.name == name.to_s }
end end
def proxy_publish_args(http_port, https_port)
argumentize "--publish", [ "#{http_port}:#{PROXY_HTTP_PORT}", "#{https_port}:#{PROXY_HTTPS_PORT}" ]
end
def proxy_logging_args(max_size)
argumentize "--log-opt", "max-size=#{max_size}"
end
def proxy_options_default
[ *proxy_publish_args(PROXY_HTTP_PORT, PROXY_HTTPS_PORT), *proxy_logging_args(PROXY_LOG_MAX_SIZE) ]
end
def proxy_image
"basecamp/kamal-proxy:#{PROXY_MINIMUM_VERSION}"
end
def proxy_container_name
"kamal-proxy"
end
def proxy_directory
File.join run_directory, "proxy"
end
def proxy_options_file
File.join proxy_directory, "options"
end
def to_h def to_h
{ {
@@ -248,8 +292,7 @@ class Kamal::Configuration
sshkit: sshkit.to_h, sshkit: sshkit.to_h,
builder: builder.to_h, builder: builder.to_h,
accessories: raw_config.accessories, accessories: raw_config.accessories,
logging: logging_args, logging: logging_args
healthcheck: healthcheck.to_h
}.compact }.compact
end end
@@ -307,6 +350,30 @@ class Kamal::Configuration
true true
end end
def ensure_no_traefik_reboot_hooks
hooks = %w[ pre-traefik-reboot post-traefik-reboot ].select { |hook_file| File.exist?(File.join(hooks_path, hook_file)) }
if hooks.any?
raise Kamal::ConfigurationError, "Found #{hooks.join(", ")}, these should be renamed to (pre|post)-proxy-reboot"
end
true
end
def ensure_one_host_for_ssl_roles
roles.each(&:ensure_one_host_for_ssl)
true
end
def ensure_unique_hosts_for_ssl_roles
hosts = roles.select(&:ssl?).flat_map { |role| role.proxy.hosts }
duplicates = hosts.tally.filter_map { |host, count| host if count > 1 }
raise Kamal::ConfigurationError, "Different roles can't share the same host for SSL: #{duplicates.join(", ")}" if duplicates.any?
true
end
def role_names def role_names
raw_config.servers.is_a?(Array) ? [ "web" ] : raw_config.servers.keys.sort raw_config.servers.is_a?(Array) ? [ "web" ] : raw_config.servers.keys.sort

View File

@@ -16,7 +16,7 @@ class Kamal::Configuration::Accessory
@env = Kamal::Configuration::Env.new \ @env = Kamal::Configuration::Env.new \
config: accessory_config.fetch("env", {}), config: accessory_config.fetch("env", {}),
secrets_file: File.join(config.host_env_directory, "accessories", "#{service_name}.env"), secrets: config.secrets,
context: "accessories/#{name}/env" context: "accessories/#{name}/env"
end end
@@ -51,7 +51,19 @@ class Kamal::Configuration::Accessory
end end
def env_args def env_args
env.args [ *env.clear_args, *argumentize("--env-file", secrets_path) ]
end
def env_directory
File.join(config.env_directory, "accessories")
end
def secrets_io
env.secrets_io
end
def secrets_path
File.join(config.env_directory, "accessories", "#{name}.env")
end end
def files def files

View File

@@ -0,0 +1,15 @@
class Kamal::Configuration::Alias
include Kamal::Configuration::Validation
attr_reader :name, :command
def initialize(name, config:)
@name, @command = name.inquiry, config.raw_config["aliases"][name]
validate! \
command,
example: validation_yml["aliases"]["uname"],
context: "aliases/#{name}",
with: Kamal::Configuration::Validator::Alias
end
end

View File

@@ -19,16 +19,38 @@ class Kamal::Configuration::Builder
builder_config builder_config
end end
def multiarch? def remote
builder_config["multiarch"] != false builder_config["remote"]
end end
def local? def arches
!!builder_config["local"] Array(builder_config.fetch("arch", default_arch))
end
def local_arches
@local_arches ||= if local_disabled?
[]
elsif remote
arches & [ Kamal::Utils.docker_arch ]
else
arches
end
end
def remote_arches
@remote_arches ||= if remote
arches - local_arches
else
[]
end
end end
def remote? def remote?
!!builder_config["remote"] remote_arches.any?
end
def local?
!local_disabled? && (arches.empty? || local_arches.any?)
end end
def cached? def cached?
@@ -40,7 +62,7 @@ class Kamal::Configuration::Builder
end end
def secrets def secrets
builder_config["secrets"] || [] (builder_config["secrets"] || []).to_h { |key| [ key, config.secrets[key] ] }
end end
def dockerfile def dockerfile
@@ -55,20 +77,12 @@ class Kamal::Configuration::Builder
builder_config["context"] || "." builder_config["context"] || "."
end end
def local_arch def driver
builder_config["local"]["arch"] if local? builder_config.fetch("driver", "docker-container")
end end
def local_host def local_disabled?
builder_config["local"]["host"] if local? builder_config["local"] == false
end
def remote_arch
builder_config["remote"]["arch"] if remote?
end
def remote_host
builder_config["remote"]["host"] if remote?
end end
def cache_from def cache_from
@@ -114,7 +128,23 @@ class Kamal::Configuration::Builder
end end
end end
def docker_driver?
driver == "docker"
end
private private
def valid?
if docker_driver?
raise ArgumentError, "Invalid builder configuration: the `docker` driver does not not support remote builders" if remote
raise ArgumentError, "Invalid builder configuration: the `docker` driver does not not support caching" if cached?
raise ArgumentError, "Invalid builder configuration: the `docker` driver does not not support multiple arches" if arches.many?
end
if @options["cache"] && @options["cache"]["type"]
raise ArgumentError, "Invalid cache type: #{@options["cache"]["type"]}" unless [ "gha", "registry" ].include?(@options["cache"]["type"])
end
end
def cache_image def cache_image
builder_config["cache"]&.fetch("image", nil) || "#{image}-build-cache" builder_config["cache"]&.fetch("image", nil) || "#{image}-build-cache"
end end
@@ -150,4 +180,8 @@ class Kamal::Configuration::Builder
def pwd_sha def pwd_sha
Digest::SHA256.hexdigest(Dir.pwd)[0..12] Digest::SHA256.hexdigest(Dir.pwd)[0..12]
end end
def default_arch
docker_driver? ? [] : [ "amd64", "arm64" ]
end
end end

View File

@@ -3,32 +3,32 @@
# Accessories can be booted on a single host, a list of hosts, or on specific roles. # Accessories can be booted on a single host, a list of hosts, or on specific roles.
# The hosts do not need to be defined in the Kamal servers configuration. # The hosts do not need to be defined in the Kamal servers configuration.
# #
# Accessories are managed separately from the main service - they are not updated # Accessories are managed separately from the main service they are not updated
# when you deploy and they do not have zero-downtime deployments. # when you deploy, and they do not have zero-downtime deployments.
# #
# Run `kamal accessory boot <accessory>` to boot an accessory. # Run `kamal accessory boot <accessory>` to boot an accessory.
# See `kamal accessory --help` for more information. # See `kamal accessory --help` for more information.
# Configuring accessories # Configuring accessories
# #
# First define the accessory in the `accessories` # First, define the accessory in the `accessories`:
accessories: accessories:
mysql: mysql:
# Service name # Service name
# #
# This is used in the service label and defaults to `<service>-<accessory>` # This is used in the service label and defaults to `<service>-<accessory>`,
# where `<service>` is the main service name from the root configuration # where `<service>` is the main service name from the root configuration:
service: mysql service: mysql
# Image # Image
# #
# The Docker image to use, prefix with a registry if not using Docker hub # The Docker image to use, prefix it with a registry if not using Docker Hub:
image: mysql:8.0 image: mysql:8.0
# Accessory hosts # Accessory hosts
# #
# Specify one of `host`, `hosts` or `roles` # Specify one of `host`, `hosts`, or `roles`:
host: mysql-db1 host: mysql-db1
hosts: hosts:
- mysql-db1 - mysql-db1
@@ -38,12 +38,12 @@ accessories:
# Custom command # Custom command
# #
# You can set a custom command to run in the container, if you do not want to use the default # You can set a custom command to run in the container if you do not want to use the default:
cmd: "bin/mysqld" cmd: "bin/mysqld"
# Port mappings # Port mappings
# #
# See https://docs.docker.com/network/, especially note the warning about the security # See https://docs.docker.com/network/, and especially note the warning about the security
# implications of exposing ports publicly. # implications of exposing ports publicly.
port: "127.0.0.1:3306:3306" port: "127.0.0.1:3306:3306"
@@ -52,20 +52,22 @@ accessories:
app: myapp app: myapp
# Options # Options
# These are passed to the Docker run command in the form `--<name> <value>` #
# These are passed to the Docker run command in the form `--<name> <value>`:
options: options:
restart: always restart: always
cpus: 2 cpus: 2
# Environment variables # Environment variables
# See kamal docs env for more information #
# See kamal docs env for more information:
env: env:
... ...
# Copying files # Copying files
# #
# You can specify files to mount into the container. # You can specify files to mount into the container.
# The format is `local:remote` where `local` is the path to the file on the local machine # The format is `local:remote`, where `local` is the path to the file on the local machine
# and `remote` is the path to the file in the container. # and `remote` is the path to the file in the container.
# #
# They will be uploaded from the local repo to the host and then mounted. # They will be uploaded from the local repo to the host and then mounted.
@@ -78,13 +80,13 @@ accessories:
# Directories # Directories
# #
# You can specify directories to mount into the container. They will be created on the host # You can specify directories to mount into the container. They will be created on the host
# before being mounted # before being mounted:
directories: directories:
- mysql-logs:/var/log/mysql - mysql-logs:/var/log/mysql
# Volumes # Volumes
# #
# Any other volumes to mount, in addition to the files and directories. # Any other volumes to mount, in addition to the files and directories.
# They are not created or copied before mounting # They are not created or copied before mounting:
volumes: volumes:
- /path/to/mysql-logs:/var/log/mysql - /path/to/mysql-logs:/var/log/mysql

View File

@@ -0,0 +1,26 @@
# Aliases
#
# Aliases are shortcuts for Kamal commands.
#
# For example, for a Rails app, you might open a console with:
#
# ```shell
# kamal app exec -i -r console "rails console"
# ```
#
# By defining an alias, like this:
aliases:
console: app exec -r console -i "rails console"
# You can now open the console with:
#
# ```shell
# kamal console
# ```
# Configuring aliases
#
# Aliases are defined in the root config under the alias key.
#
# Each alias is named and can only contain lowercase letters, numbers, dashes, and underscores:
aliases:
uname: app exec -p -q -r web "uname -a"

View File

@@ -2,18 +2,18 @@
# #
# When deploying to large numbers of hosts, you might prefer not to restart your services on every host at the same time. # When deploying to large numbers of hosts, you might prefer not to restart your services on every host at the same time.
# #
# Kamals default is to boot new containers on all hosts in parallel. But you can control this with the boot configuration. # Kamals default is to boot new containers on all hosts in parallel. However, you can control this with the boot configuration.
# Fixed group sizes # Fixed group sizes
# #
# Here we boot 2 hosts at a time with a 10 second gap between each group. # Here, we boot 2 hosts at a time with a 10-second gap between each group:
boot: boot:
limit: 2 limit: 2
wait: 10 wait: 10
# Percentage of hosts # Percentage of hosts
# #
# Here we boot 25% of the hosts at a time with a 2 second gap between each group. # Here, we boot 25% of the hosts at a time with a 2-second gap between each group:
boot: boot:
limit: 25% limit: 25%
wait: 2 wait: 2

View File

@@ -1,47 +1,41 @@
# Builder # Builder
# #
# The builder configuration controls how the application is built with `docker build` or `docker buildx build` # The builder configuration controls how the application is built with `docker build`.
# #
# If no configuration is specified, Kamal will: # See https://kamal-deploy.org/docs/configuration/builder-examples/ for more information.
# 1. Create a buildx context called `kamal-<service>-multiarch`
# 2. Use `docker buildx build` to build a multiarch image for linux/amd64,linux/arm64 with that context
#
# See https://kamal-deploy.org/docs/configuration/builder-examples/ for more information
# Builder options # Builder options
# #
# Options go under the builder key in the root configuration. # Options go under the builder key in the root configuration.
builder: builder:
# Multiarch # Arch
# #
# Enables multiarch builds, defaults to `true` # The architectures to build for — you can set an array or just a single value.
multiarch: false #
# Allowed values are `amd64` and `arm64`:
arch:
- amd64
# Local configuration # Remote
# #
# The build configuration for local builds, only used if multiarch is enabled (the default) # The connection string for a remote builder. If supplied, Kamal will use this
# # for builds that do not match the local architecture of the deployment host.
# If there is no remote configuration, by default we build for amd64 and arm64. remote: ssh://docker@docker-builder
# If you only want to build for one architecture, you can specify it here.
# The docker socket is optional and uses the default docker host socket when not specified
local:
arch: amd64
host: /var/run/docker.sock
# Remote configuration # Local
# #
# The build configuration for remote builds, also only used if multiarch is enabled. # If set to false, Kamal will always use the remote builder even when building
# The arch is required and can be either amd64 or arm64. # the local architecture.
remote: #
arch: arm64 # Defaults to true:
host: ssh://docker@docker-builder local: true
# Builder cache # Builder cache
# #
# The type must be either 'gha' or 'registry' # The type must be either 'gha' or 'registry'.
# #
# The image is only used for registry cache # The image is only used for registry cache and is not compatible with the Docker driver:
cache: cache:
type: registry type: registry
options: mode=max options: mode=max
@@ -49,25 +43,25 @@ builder:
# Build context # Build context
# #
# If this is not set, then a local git clone of the repo is used. # If this is not set, then a local Git clone of the repo is used.
# This ensures a clean build with no uncommitted changes. # This ensures a clean build with no uncommitted changes.
# #
# To use the local checkout instead you can set the context to `.`, or a path to another directory. # To use the local checkout instead, you can set the context to `.`, or a path to another directory.
context: . context: .
# Dockerfile # Dockerfile
# #
# The Dockerfile to use for building, defaults to `Dockerfile` # The Dockerfile to use for building, defaults to `Dockerfile`:
dockerfile: Dockerfile.production dockerfile: Dockerfile.production
# Build target # Build target
# #
# If not set, then the default target is used # If not set, then the default target is used:
target: production target: production
# Build Arguments # Build arguments
# #
# Any additional build arguments, passed to `docker build` with `--build-arg <key>=<value>` # Any additional build arguments, passed to `docker build` with `--build-arg <key>=<value>`:
args: args:
ENVIRONMENT: production ENVIRONMENT: production
@@ -80,28 +74,31 @@ builder:
# Build secrets # Build secrets
# #
# Values are read from the environment. # Values are read from `.kamal/secrets`:
#
secrets: secrets:
- SECRET1 - SECRET1
- SECRET2 - SECRET2
# Referencing Build Secrets # Referencing build secrets
# #
# ```shell # ```shell
# # Copy Gemfiles # # Copy Gemfiles
# COPY Gemfile Gemfile.lock ./ # COPY Gemfile Gemfile.lock ./
# #
# # Install dependencies, including private repositories via access token # # Install dependencies, including private repositories via access token
# # Then remove bundle cache with exposed GITHUB_TOKEN) # # Then remove bundle cache with exposed GITHUB_TOKEN
# RUN --mount=type=secret,id=GITHUB_TOKEN \ # RUN --mount=type=secret,id=GITHUB_TOKEN \
# BUNDLE_GITHUB__COM=x-access-token:$(cat /run/secrets/GITHUB_TOKEN) \ # BUNDLE_GITHUB__COM=x-access-token:$(cat /run/secrets/GITHUB_TOKEN) \
# bundle install && \ # bundle install && \
# rm -rf /usr/local/bundle/cache # rm -rf /usr/local/bundle/cache
# ``` # ```
# SSH # SSH
# #
# SSH agent socket or keys to expose to the build # SSH agent socket or keys to expose to the build:
ssh: default=$SSH_AUTH_SOCK ssh: default=$SSH_AUTH_SOCK
# Driver
#
# The build driver to use, defaults to `docker-container`:
driver: docker

View File

@@ -1,16 +1,27 @@
# Kamal Configuration # Kamal Configuration
# #
# Configuration is read from the `config/deploy.yml` # Configuration is read from the `config/deploy.yml`.
# Destinations
# #
# When running commands, you can specify a destination with the `-d` flag, # When running commands, you can specify a destination with the `-d` flag,
# e.g. `kamal deploy -d staging` # e.g., `kamal deploy -d staging`.
# #
# In this case the configuration will also be read from `config/deploy.staging.yml` # In this case, the configuration will also be read from `config/deploy.staging.yml`
# and merged with the base configuration. # and merged with the base configuration.
# Extensions
# #
# The available configuration options are explained below. # Kamal will not accept unrecognized keys in the configuration file.
#
# However, you might want to declare a configuration block using YAML anchors
# and aliases to avoid repetition.
#
# You can prefix a configuration section with `x-` to indicate that it is an
# extension. Kamal will ignore the extension and not raise an error.
# The service name # The service name
#
# This is a required value. It is used as the container name prefix. # This is a required value. It is used as the container name prefix.
service: myapp service: myapp
@@ -21,137 +32,147 @@ image: my-image
# Labels # Labels
# #
# Additional labels to add to the container # Additional labels to add to the container:
labels: labels:
my-label: my-value my-label: my-value
# Additional volumes to mount into the container # Volumes
#
# Additional volumes to mount into the container:
volumes: volumes:
- /path/on/host:/path/in/container:ro - /path/on/host:/path/in/container:ro
# Registry # Registry
# #
# The Docker registry configuration, see kamal docs registry # The Docker registry configuration, see kamal docs registry:
registry: registry:
... ...
# Servers # Servers
# #
# The servers to deploy to, optionally with custom roles, see kamal docs servers # The servers to deploy to, optionally with custom roles, see kamal docs servers:
servers: servers:
... ...
# Environment variables # Environment variables
# #
# See kamal docs env # See kamal docs env:
env: env:
... ...
# Asset Bridging # Asset path
# #
# Used for asset bridging across deployments, default to `nil` # Used for asset bridging across deployments, default to `nil`.
# #
# If there are changes to CSS or JS files, we may get requests # If there are changes to CSS or JS files, we may get requests
# for the old versions on the new container and vice-versa. # for the old versions on the new container, and vice versa.
# #
# To avoid 404s we can specify an asset path. # To avoid 404s, we can specify an asset path.
# Kamal will replace that path in the container with a mapped # Kamal will replace that path in the container with a mapped
# volume containing both sets of files. # volume containing both sets of files.
# This requires that file names change when the contents change # This requires that file names change when the contents change
# (e.g. by including a hash of the contents in the name). # (e.g., by including a hash of the contents in the name).
#
# To configure this, set the path to the assets: # To configure this, set the path to the assets:
asset_path: /path/to/assets asset_path: /path/to/assets
# Path to hooks, defaults to `.kamal/hooks` # Hooks path
# See https://kamal-deploy.org/docs/hooks for more information #
# Path to hooks, defaults to `.kamal/hooks`.
# See https://kamal-deploy.org/docs/hooks for more information:
hooks_path: /user_home/kamal/hooks hooks_path: /user_home/kamal/hooks
# Require destinations # Require destinations
# #
# Whether deployments require a destination to be specified, defaults to `false` # Whether deployments require a destination to be specified, defaults to `false`:
require_destination: true require_destination: true
# The primary role # Primary role
# #
# This defaults to `web`, but if you have no web role, you can change this # This defaults to `web`, but if you have no web role, you can change this:
primary_role: workers primary_role: workers
# Allowing empty roles # Allowing empty roles
# #
# Whether roles with no servers are allowed. Defaults to `false`. # Whether roles with no servers are allowed. Defaults to `false`:
allow_empty_roles: false allow_empty_roles: false
# Stop wait time
#
# How long we wait for a container to stop before killing it, defaults to 30 seconds
stop_wait_time: 60
# Retain containers # Retain containers
# #
# How many old containers and images we retain, defaults to 5 # How many old containers and images we retain, defaults to 5:
retain_containers: 3 retain_containers: 3
# Minimum version # Minimum version
# #
# The minimum version of Kamal required to deploy this configuration, defaults to nil # The minimum version of Kamal required to deploy this configuration, defaults to `nil`:
minimum_version: 1.3.0 minimum_version: 1.3.0
# Readiness delay # Readiness delay
# #
# Seconds to wait for a container to boot after is running, default 7 # Seconds to wait for a container to boot after it is running, default 7.
# This only applies to containers that do not specify a healthcheck #
# This only applies to containers that do not run a proxy or specify a healthcheck:
readiness_delay: 4 readiness_delay: 4
# Deploy timeout
#
# How long to wait for a container to become ready, default 30:
deploy_timeout: 10
# Drain timeout
#
# How long to wait for a container to drain, default 30:
drain_timeout: 10
# Run directory # Run directory
# #
# Directory to store kamal runtime files in on the host, default `.kamal` # Directory to store kamal runtime files in on the host, default `.kamal`:
run_directory: /etc/kamal run_directory: /etc/kamal
# SSH options # SSH options
# #
# See kamal docs ssh # See kamal docs ssh:
ssh: ssh:
... ...
# Builder options # Builder options
# #
# See kamal docs builder # See kamal docs builder:
builder: builder:
... ...
# Accessories # Accessories
# #
# Additionals services to run in Docker, see kamal docs accessory # Additional services to run in Docker, see kamal docs accessory:
accessories: accessories:
... ...
# Traefik # Proxy
# #
# The Traefik proxy is used for zero-downtime deployments, see kamal docs traefik # Configuration for kamal-proxy, see kamal docs proxy:
traefik: proxy:
... ...
# SSHKit # SSHKit
# #
# See kamal docs sshkit # See kamal docs sshkit:
sshkit: sshkit:
... ...
# Boot options # Boot options
# #
# See kamal docs boot # See kamal docs boot:
boot: boot:
... ...
# Healthcheck
#
# Configuring healthcheck commands, intervals and timeouts, see kamal docs healthcheck
healthcheck:
...
# Logging # Logging
# #
# Docker logging configuration, see kamal docs logging # Docker logging configuration, see kamal docs logging:
logging: logging:
... ...
# Aliases
#
# Alias configuration, see kamal docs alias:
aliases:
...

View File

@@ -1,37 +1,50 @@
# Environment variables # Environment variables
# #
# Environment variables can be set directory in the Kamal configuration or # Environment variables can be set directly in the Kamal configuration or
# for loaded from a .env file, for secrets that should not be checked into Git. # read from `.kamal/secrets`.
# Reading environment variables from the configuration # Reading environment variables from the configuration
# #
# Environment variables can be set directly in the configuration file. # Environment variables can be set directly in the configuration file.
# #
# These are passed to the docker run command when deploying. # These are passed to the `docker run` command when deploying.
env: env:
DATABASE_HOST: mysql-db1 DATABASE_HOST: mysql-db1
DATABASE_PORT: 3306 DATABASE_PORT: 3306
# Using .env file to load required environment variables # Secrets
# #
# Kamal uses dotenv to automatically load environment variables set in the .env file present # Kamal uses dotenv to automatically load environment variables set in the `.kamal/secrets` file.
# in the application root.
# #
# This file can be used to set variables like KAMAL_REGISTRY_PASSWORD or database passwords. # If you are using destinations, secrets will instead be read from `.kamal/secrets.<DESTINATION>` if
# But for this reason you must ensure that .env files are not checked into Git or included # it exists.
# in your Dockerfile! The format is just key-value like: #
# Common secrets across all destinations can be set in `.kamal/secrets-common`.
#
# This file can be used to set variables like `KAMAL_REGISTRY_PASSWORD` or database passwords.
# You can use variable or command substitution in the secrets file.
#
# ```shell
# KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD
# RAILS_MASTER_KEY=$(cat config/master.key)
# ``` # ```
# KAMAL_REGISTRY_PASSWORD=pw
# DB_PASSWORD=secret123
# ```
# See https://kamal-deploy.org/docs/commands/envify/ for how to use generated .env files.
# #
# To pass the secrets you should list them under the `secret` key. When you do this the # You can also use [secret helpers](../../commands/secrets) for some common password managers.
#
# ```shell
# SECRETS=$(kamal secrets fetch ...)
#
# REGISTRY_PASSWORD=$(kamal secrets extract REGISTRY_PASSWORD $SECRETS)
# DB_PASSWORD=$(kamal secrets extract DB_PASSWORD $SECRETS)
# ```
#
# If you store secrets directly in `.kamal/secrets`, ensure that it is not checked into version control.
#
# To pass the secrets, you should list them under the `secret` key. When you do this, the
# other variables need to be moved under the `clear` key. # other variables need to be moved under the `clear` key.
# #
# Unlike clear valies, secrets are not passed directly to the container, # Unlike clear values, secrets are not passed directly to the container
# but are stored in an env file on the host # but are stored in an env file on the host:
# The file is not updated when deploying, only when running `kamal envify` or `kamal env push`.
env: env:
clear: clear:
DB_USER: app DB_USER: app
@@ -43,7 +56,7 @@ env:
# Tags are used to add extra env variables to specific hosts. # Tags are used to add extra env variables to specific hosts.
# See kamal docs servers for how to tag hosts. # See kamal docs servers for how to tag hosts.
# #
# Tags are only allowed in the top level env configuration (i.e not under a role specific env). # Tags are only allowed in the top-level env configuration (i.e., not under a role-specific env).
# #
# The env variables can be specified with secret and clear values as explained above. # The env variables can be specified with secret and clear values as explained above.
env: env:

View File

@@ -1,59 +0,0 @@
# Healthcheck configuration
#
# On roles that are running Traefik, Kamal will supply a default healthcheck to `docker run`.
# For other roles, by default no healthcheck is supplied.
#
# If no healthcheck is supplied and the image does not define one, they we wait for the container
# to reach a running state and then pause for the readiness delay.
#
# The default healthcheck is `curl -f http://localhost:<port>/<path>`, so it assumes that `curl`
# is available within the container.
# Healthcheck options
#
# These go under the `healthcheck` key in the root or role configuration.
healthcheck:
# Command
#
# The command to run, defaults to `curl -f http://localhost:<port>/<path>` on roles running Traefik
cmd: "curl -f http://localhost"
# Interval
#
# The Docker healthcheck interval, defaults to `1s`
interval: 10s
# Max attempts
#
# The maximum number of times we poll the container to see if it is healthy, defaults to `7`
# Each check is separated by an increasing interval starting with 1 second.
max_attempts: 3
# Port
#
# The port to use in the healthcheck, defaults to `3000`
port: "80"
# Path
#
# The path to use in the healthcheck, defaults to `/up`
path: /health
# Cords for zero-downtime deployments
#
# The cord file is used for zero-downtime deployments. The healthcheck is augmented with a check
# for the existance of the file. This allows us to delete the file and force the container to
# become unhealthy, causing Traefik to stop routing traffic to it.
#
# Kamal mounts a volume at this location and creates the file before starting the container.
# You can set the value to `false` to disable the cord file, but this loses the zero-downtime
# guarantee.
#
# The default value is `/tmp/kamal-cord`
cord: /cord
# Log lines
#
# Number of lines to log from the container when the healthcheck fails, defaults to `50`
log_lines: 100

View File

@@ -6,16 +6,16 @@
# #
# These go under the logging key in the configuration file. # These go under the logging key in the configuration file.
# #
# This can be specified in the root level or for a specific role. # This can be specified at the root level or for a specific role.
logging: logging:
# Driver # Driver
# #
# The logging driver to use, passed to Docker via `--log-driver` # The logging driver to use, passed to Docker via `--log-driver`:
driver: json-file driver: json-file
# Options # Options
# #
# Any logging options to pass to the driver, passed to Docker via `--log-opt` # Any logging options to pass to the driver, passed to Docker via `--log-opt`:
options: options:
max-size: 100m max-size: 100m

View File

@@ -0,0 +1,105 @@
# Proxy
#
# Kamal uses [kamal-proxy](https://github.com/basecamp/kamal-proxy) to provide
# gapless deployments. It runs on ports 80 and 443 and forwards requests to the
# application container.
#
# The proxy is configured in the root configuration under `proxy`. These are
# options that are set when deploying the application, not when booting the proxy.
#
# They are application-specific, so they are not shared when multiple applications
# run on the same proxy.
#
# The proxy is enabled by default on the primary role but can be disabled by
# setting `proxy: false`.
#
# It is disabled by default on all other roles but can be enabled by setting
# `proxy: true` or providing a proxy configuration.
proxy:
# Hosts
#
# The hosts that will be used to serve the app. The proxy will only route requests
# to this host to your app.
#
# If no hosts are set, then all requests will be forwarded, except for matching
# requests for other apps deployed on that server that do have a host set.
#
# Specify one of `host` or `hosts`.
host: foo.example.com
hosts:
- foo.example.com
- bar.example.com
# App port
#
# The port the application container is exposed on.
#
# Defaults to 80:
app_port: 3000
# SSL
#
# kamal-proxy can provide automatic HTTPS for your application via Let's Encrypt.
#
# This requires that we are deploying to one server and the host option is set.
# The host value must point to the server we are deploying to, and port 443 must be
# open for the Let's Encrypt challenge to succeed.
#
# Defaults to `false`:
ssl: true
# Response timeout
#
# How long to wait for requests to complete before timing out, defaults to 30 seconds:
response_timeout: 10
# Healthcheck
#
# When deploying, the proxy will by default hit `/up` once every second until we hit
# the deploy timeout, with a 5-second timeout for each request.
#
# Once the app is up, the proxy will stop hitting the healthcheck endpoint.
healthcheck:
interval: 3
path: /health
timeout: 3
# Buffering
#
# Whether to buffer request and response bodies in the proxy.
#
# By default, buffering is enabled with a max request body size of 1GB and no limit
# for response size.
#
# You can also set the memory limit for buffering, which defaults to 1MB; anything
# larger than that is written to disk.
buffering:
requests: true
responses: true
max_request_body: 40_000_000
max_response_body: 0
memory: 2_000_000
# Logging
#
# Configure request logging for the proxy.
# You can specify request and response headers to log.
# By default, `Cache-Control`, `Last-Modified`, and `User-Agent` request headers are logged:
logging:
request_headers:
- Cache-Control
- X-Forwarded-Proto
response_headers:
- X-Request-ID
- X-Request-Start
# Forward headers
#
# Whether to forward the `X-Forwarded-For` and `X-Forwarded-Proto` headers.
#
# If you are behind a trusted proxy, you can set this to `true` to forward the headers.
#
# By default, kamal-proxy will not forward the headers if the `ssl` option is set to `true`, and
# will forward them if it is set to `false`.
forward_headers: true

View File

@@ -1,10 +1,9 @@
# Registry # Registry
# #
# The default registry is Docker Hub, but you can change it using registry/server: # The default registry is Docker Hub, but you can change it using `registry/server`.
# #
# A reference to secret (in this case DOCKER_REGISTRY_TOKEN) will look up the secret # A reference to a secret (in this case, `DOCKER_REGISTRY_TOKEN`) will look up the secret
# in the local environment. # in the local environment:
registry: registry:
server: registry.digitalocean.com server: registry.digitalocean.com
username: username:
@@ -13,28 +12,31 @@ registry:
- DOCKER_REGISTRY_TOKEN - DOCKER_REGISTRY_TOKEN
# Using AWS ECR as the container registry # Using AWS ECR as the container registry
# You will need to have the aws CLI installed locally for this to work. #
# AWS ECRs access token is only valid for 12hrs. In order to not have to manually regenerate the token every time, you can use ERB in the deploy.yml file to shell out to the aws cli command, and obtain the token: # You will need to have the AWS CLI installed locally for this to work.
# AWS ECRs access token is only valid for 12 hours. In order to avoid having to manually regenerate the token every time, you can use ERB in the `deploy.yml` file to shell out to the AWS CLI command and obtain the token:
registry: registry:
server: <your aws account id>.dkr.ecr.<your aws region id>.amazonaws.com server: <your aws account id>.dkr.ecr.<your aws region id>.amazonaws.com
username: AWS username: AWS
password: <%= %x(aws ecr get-login-password) %> password: <%= %x(aws ecr get-login-password) %>
# Using GCP Artifact Registry as the container registry # Using GCP Artifact Registry as the container registry
# To sign into Artifact Registry, you would need to #
# To sign into Artifact Registry, you need to
# [create a service account](https://cloud.google.com/iam/docs/service-accounts-create#creating) # [create a service account](https://cloud.google.com/iam/docs/service-accounts-create#creating)
# and [set up roles and permissions](https://cloud.google.com/artifact-registry/docs/access-control#permissions). # and [set up roles and permissions](https://cloud.google.com/artifact-registry/docs/access-control#permissions).
# Normally, assigning a roles/artifactregistry.writer role should be sufficient. # Normally, assigning the `roles/artifactregistry.writer` role should be sufficient.
# #
# Once the service account is ready, you need to generate and download a JSON key, base64 encode it and add to .env: # Once the service account is ready, you need to generate and download a JSON key and base64 encode it:
# #
# ```shell # ```shell
# echo "KAMAL_REGISTRY_PASSWORD=$(base64 -i /path/to/key.json)" | tr -d "\\n" >> .env # base64 -i /path/to/key.json | tr -d "\\n"
# ``` # ```
# Use the env variable as password along with _json_key_base64 as username. #
# You'll then need to set the `KAMAL_REGISTRY_PASSWORD` secret to that value.
#
# Use the environment variable as the password along with `_json_key_base64` as the username.
# Heres the final configuration: # Heres the final configuration:
registry: registry:
server: <your registry region>-docker.pkg.dev server: <your registry region>-docker.pkg.dev
username: _json_key_base64 username: _json_key_base64
@@ -44,6 +46,7 @@ registry:
# Validating the configuration # Validating the configuration
# #
# You can validate the configuration by running: # You can validate the configuration by running:
#
# ```shell # ```shell
# kamal registry login # kamal registry login
# ``` # ```

View File

@@ -1,22 +1,21 @@
# Roles # Roles
# #
# Roles are used to configure different types of servers in the deployment. # Roles are used to configure different types of servers in the deployment.
# The most common use for this is to run a web servers and job servers. # The most common use for this is to run web servers and job servers.
# #
# Kamal expects there to be a `web` role, unless you set a different `primary_role` # Kamal expects there to be a `web` role, unless you set a different `primary_role`
# in the root configuration. # in the root configuration.
# Role configuration # Role configuration
# #
# Roles are specified under the servers key # Roles are specified under the servers key:
servers: servers:
# Simple role configuration # Simple role configuration
# #
# This can be a list of hosts if you don't need custom configuration for the role.
# #
# This can be a list of hosts, if you don't need custom configuration for the role. # You can set tags on the hosts for custom env variables (see kamal docs env):
#
# You can set tags on the hosts for custom env variables (see kamal docs env)
web: web:
- 172.1.0.1 - 172.1.0.1
- 172.1.0.2: experiment1 - 172.1.0.2: experiment1
@@ -24,29 +23,31 @@ servers:
# Custom role configuration # Custom role configuration
# #
# When there are other options to set, the list of hosts goes under the `hosts` key # When there are other options to set, the list of hosts goes under the `hosts` key.
# #
# By default only the primary role uses Traefik, but you can set `traefik` to change # By default, only the primary role uses a proxy.
# it.
# #
# You can also set a custom cmd to run in the container, and overwrite other settings # For other roles, you can set it to `proxy: true` to enable it and inherit the root proxy
# configuration or provide a map of options to override the root configuration.
#
# For the primary role, you can set `proxy: false` to disable the proxy.
#
# You can also set a custom `cmd` to run in the container and overwrite other settings
# from the root configuration. # from the root configuration.
workers: workers:
hosts: hosts:
- 172.1.0.3 - 172.1.0.3
- 172.1.0.4: experiment1 - 172.1.0.4: experiment1
traefik: true
cmd: "bin/jobs" cmd: "bin/jobs"
options: options:
memory: 2g memory: 2g
cpus: 4 cpus: 4
healthcheck:
...
logging: logging:
... ...
proxy:
...
labels: labels:
my-label: workers my-label: workers
env: env:
... ...
asset_path: /public asset_path: /public

View File

@@ -2,7 +2,7 @@
# #
# Servers are split into different roles, with each role having its own configuration. # Servers are split into different roles, with each role having its own configuration.
# #
# For simpler deployments though where all servers are identical, you can just specify a list of servers # For simpler deployments, though, where all servers are identical, you can just specify a list of servers.
# They will be implicitly assigned to the `web` role. # They will be implicitly assigned to the `web` role.
servers: servers:
- 172.0.0.1 - 172.0.0.1
@@ -19,7 +19,7 @@ servers:
# Roles # Roles
# #
# For more complex deployments (e.g. if you are running job hosts), you can specify roles, and configure each separately (see kamal docs role) # For more complex deployments (e.g., if you are running job hosts), you can specify roles and configure each separately (see kamal docs role):
servers: servers:
web: web:
... ...

View File

@@ -1,9 +1,9 @@
# SSH configuration # SSH configuration
# #
# Kamal uses SSH to connect run commands on your hosts. # Kamal uses SSH to connect and run commands on your hosts.
# By default it will attempt to connect to the root user on port 22 # By default, it will attempt to connect to the root user on port 22.
# #
# If you are using non-root user, you may need to bootstrap your servers manually, before using them with Kamal. On Ubuntu, youd do: # If you are using a non-root user, you may need to bootstrap your servers manually before using them with Kamal. On Ubuntu, youd do:
# #
# ```shell # ```shell
# sudo apt update # sudo apt update
@@ -12,7 +12,6 @@
# sudo usermod -a -G docker app # sudo usermod -a -G docker app
# ``` # ```
# SSH options # SSH options
# #
# The options are specified under the ssh key in the configuration file. # The options are specified under the ssh key in the configuration file.
@@ -20,27 +19,52 @@ ssh:
# The SSH user # The SSH user
# #
# Defaults to `root` # Defaults to `root`:
#
user: app user: app
# The SSH port # The SSH port
# #
# Defaults to 22 # Defaults to 22:
port: "2222" port: "2222"
# Proxy host # Proxy host
# #
# Specified in the form <host> or <user>@<host> # Specified in the form <host> or <user>@<host>:
proxy: root@proxy-host proxy: root@proxy-host
# Proxy command # Proxy command
# #
# A custom proxy command, required for older versions of SSH # A custom proxy command, required for older versions of SSH:
proxy_command: "ssh -W %h:%p user@proxy" proxy_command: "ssh -W %h:%p user@proxy"
# Log level # Log level
# #
# Defaults to `fatal`. Set this to debug if you are having # Defaults to `fatal`. Set this to `debug` if you are having SSH connection issues.
# SSH connection issues.
log_level: debug log_level: debug
# Keys only
#
# Set to `true` to use only private keys from the `keys` and `key_data` parameters,
# even if ssh-agent offers more identities. This option is intended for
# situations where ssh-agent offers many different identities or you
# need to overwrite all identities and force a single one.
keys_only: false
# Keys
#
# An array of file names of private keys to use for public key
# and host-based authentication:
keys: [ "~/.ssh/id.pem" ]
# Key data
#
# An array of strings, with each element of the array being
# a raw private key in PEM format.
key_data: [ "-----BEGIN OPENSSH PRIVATE KEY-----" ]
# Config
#
# Set to true to load the default OpenSSH config files (~/.ssh/config,
# /etc/ssh_config), to false ignore config files, or to a file path
# (or array of paths) to load specific configuration. Defaults to true.
config: true

View File

@@ -2,8 +2,8 @@
# #
# [SSHKit](https://github.com/capistrano/sshkit) is the SSH toolkit used by Kamal. # [SSHKit](https://github.com/capistrano/sshkit) is the SSH toolkit used by Kamal.
# #
# The default settings should be sufficient for most use cases, but # The default, settings should be sufficient for most use cases, but
# when connecting to a large number of hosts you may need to adjust # when connecting to a large number of hosts, you may need to adjust.
# SSHKit options # SSHKit options
# #
@@ -13,11 +13,11 @@ sshkit:
# Max concurrent starts # Max concurrent starts
# #
# Creating SSH connections concurrently can be an issue when deploying to many servers. # Creating SSH connections concurrently can be an issue when deploying to many servers.
# By default Kamal will limit concurrent connection starts to 30 at a time. # By default, Kamal will limit concurrent connection starts to 30 at a time.
max_concurrent_starts: 10 max_concurrent_starts: 10
# Pool idle timeout # Pool idle timeout
# #
# Kamal sets a long idle timeout of 900 seconds on connections to try to avoid # Kamal sets a long idle timeout of 900 seconds on connections to try to avoid
# re-connection storms after an idle period, like building an image or waiting for CI. # re-connection storms after an idle period, such as building an image or waiting for CI.
pool_idle_timeout: 300 pool_idle_timeout: 300

View File

@@ -1,62 +0,0 @@
# Traefik
#
# Traefik is a reverse proxy, used by Kamal for zero-downtime deployments.
#
# We start an instance on the hosts in it's own container.
#
# During a deployment:
# 1. We start a new container which Traefik automatically detects due to the labels we have applied
# 2. Traefik starts routing traffic to the new container
# 3. We force the old container to fail it's healthcheck, causing Traefik to stop routing traffic to it
# 4. We stop the old container
# Traefik settings
#
# Traekik is configured in the root configuration under `traefik`.
traefik:
# Image
#
# The Traefik image to use, defaults to `traefik:v2.10`
image: traefik:v2.9
# Host port
#
# The host port to publish the Traefik container on, defaults to `80`
host_port: "8080"
# Disabling publishing
#
# To avoid publishing the Traefik container, set this to `false`
publish: false
# Labels
#
# Additional labels to apply to the Traefik container
labels:
traefik.http.routers.catchall.entryPoints: http
traefik.http.routers.catchall.rule: PathPrefix(`/`)
traefik.http.routers.catchall.service: unavailable
traefik.http.routers.catchall.priority: "1"
traefik.http.services.unavailable.loadbalancer.server.port: "0"
# Arguments
#
# Additional arguments to pass to the Traefik container
args:
entryPoints.http.address: ":80"
entryPoints.http.forwardedHeaders.insecure: true
accesslog: true
accesslog.format: json
# Options
#
# Additional options to pass to `docker run`
options:
cpus: 2
# Environment variables
#
# See kamal docs env
env:
...

View File

@@ -1,36 +1,29 @@
class Kamal::Configuration::Env class Kamal::Configuration::Env
include Kamal::Configuration::Validation include Kamal::Configuration::Validation
attr_reader :secrets_keys, :clear, :secrets_file, :context attr_reader :context, :secrets
attr_reader :clear, :secret_keys
delegate :argumentize, to: Kamal::Utils delegate :argumentize, to: Kamal::Utils
def initialize(config:, secrets_file: nil, context: "env") def initialize(config:, secrets:, context: "env")
@clear = config.fetch("clear", config.key?("secret") || config.key?("tags") ? {} : config) @clear = config.fetch("clear", config.key?("secret") || config.key?("tags") ? {} : config)
@secrets_keys = config.fetch("secret", []) @secrets = secrets
@secrets_file = secrets_file @secret_keys = config.fetch("secret", [])
@context = context @context = context
validate! config, context: context, with: Kamal::Configuration::Validator::Env validate! config, context: context, with: Kamal::Configuration::Validator::Env
end end
def args def clear_args
[ "--env-file", secrets_file, *argumentize("--env", clear) ] argumentize("--env", clear)
end end
def secrets_io def secrets_io
StringIO.new(Kamal::EnvFile.new(secrets).to_s) Kamal::EnvFile.new(secret_keys.to_h { |key| [ key, secrets[key] ] }).to_io
end
def secrets
@secrets ||= secrets_keys.to_h { |key| [ key, ENV.fetch(key) ] }
end
def secrets_directory
File.dirname(secrets_file)
end end
def merge(other) def merge(other)
self.class.new \ self.class.new \
config: { "clear" => clear.merge(other.clear), "secret" => secrets_keys | other.secrets_keys }, config: { "clear" => clear.merge(other.clear), "secret" => secret_keys | other.secret_keys },
secrets_file: secrets_file || other.secrets_file secrets: secrets
end end
end end

View File

@@ -1,12 +1,13 @@
class Kamal::Configuration::Env::Tag class Kamal::Configuration::Env::Tag
attr_reader :name, :config attr_reader :name, :config, :secrets
def initialize(name, config:) def initialize(name, config:, secrets:)
@name = name @name = name
@config = config @config = config
@secrets = secrets
end end
def env def env
Kamal::Configuration::Env.new(config: config) Kamal::Configuration::Env.new(config: config, secrets: secrets)
end end
end end

View File

@@ -1,63 +0,0 @@
class Kamal::Configuration::Healthcheck
include Kamal::Configuration::Validation
attr_reader :healthcheck_config
def initialize(healthcheck_config:, context: "healthcheck")
@healthcheck_config = healthcheck_config || {}
validate! @healthcheck_config, context: context
end
def merge(other)
self.class.new healthcheck_config: healthcheck_config.deep_merge(other.healthcheck_config)
end
def cmd
healthcheck_config.fetch("cmd", http_health_check)
end
def port
healthcheck_config.fetch("port", 3000)
end
def path
healthcheck_config.fetch("path", "/up")
end
def max_attempts
healthcheck_config.fetch("max_attempts", 7)
end
def interval
healthcheck_config.fetch("interval", "1s")
end
def cord
healthcheck_config.fetch("cord", "/tmp/kamal-cord")
end
def log_lines
healthcheck_config.fetch("log_lines", 50)
end
def set_port_or_path?
healthcheck_config["port"].present? || healthcheck_config["path"].present?
end
def to_h
{
"cmd" => cmd,
"interval" => interval,
"max_attempts" => max_attempts,
"port" => port,
"path" => path,
"cord" => cord,
"log_lines" => log_lines
}
end
private
def http_health_check
"curl -f #{URI.join("http://localhost:#{port}", path)} || exit 1" if path.present? || port.present?
end
end

View File

@@ -0,0 +1,62 @@
class Kamal::Configuration::Proxy
include Kamal::Configuration::Validation
DEFAULT_LOG_REQUEST_HEADERS = [ "Cache-Control", "Last-Modified", "User-Agent" ]
CONTAINER_NAME = "kamal-proxy"
delegate :argumentize, :optionize, to: Kamal::Utils
attr_reader :config, :proxy_config
def initialize(config:, proxy_config:, context: "proxy")
@config = config
@proxy_config = proxy_config
validate! @proxy_config, with: Kamal::Configuration::Validator::Proxy, context: context
end
def app_port
proxy_config.fetch("app_port", 80)
end
def ssl?
proxy_config.fetch("ssl", false)
end
def hosts
proxy_config["hosts"] || proxy_config["host"]&.split(",") || []
end
def deploy_options
{
host: hosts,
tls: proxy_config["ssl"].presence,
"deploy-timeout": seconds_duration(config.deploy_timeout),
"drain-timeout": seconds_duration(config.drain_timeout),
"health-check-interval": seconds_duration(proxy_config.dig("healthcheck", "interval")),
"health-check-timeout": seconds_duration(proxy_config.dig("healthcheck", "timeout")),
"health-check-path": proxy_config.dig("healthcheck", "path"),
"target-timeout": seconds_duration(proxy_config["response_timeout"]),
"buffer-requests": proxy_config.fetch("buffering", { "requests": true }).fetch("requests", true),
"buffer-responses": proxy_config.fetch("buffering", { "responses": true }).fetch("responses", true),
"buffer-memory": proxy_config.dig("buffering", "memory"),
"max-request-body": proxy_config.dig("buffering", "max_request_body"),
"max-response-body": proxy_config.dig("buffering", "max_response_body"),
"forward-headers": proxy_config.dig("forward_headers"),
"log-request-header": proxy_config.dig("logging", "request_headers") || DEFAULT_LOG_REQUEST_HEADERS,
"log-response-header": proxy_config.dig("logging", "response_headers")
}.compact
end
def deploy_command_args(target:)
optionize ({ target: "#{target}:#{app_port}" }).merge(deploy_options), with: "="
end
def merge(other)
self.class.new config: config, proxy_config: proxy_config.deep_merge(other.proxy_config)
end
private
def seconds_duration(value)
value ? "#{value}s" : nil
end
end

View File

@@ -1,10 +1,11 @@
class Kamal::Configuration::Registry class Kamal::Configuration::Registry
include Kamal::Configuration::Validation include Kamal::Configuration::Validation
attr_reader :registry_config attr_reader :registry_config, :secrets
def initialize(config:) def initialize(config:)
@registry_config = config.raw_config.registry || {} @registry_config = config.raw_config.registry || {}
@secrets = config.secrets
validate! registry_config, with: Kamal::Configuration::Validator::Registry validate! registry_config, with: Kamal::Configuration::Validator::Registry
end end
@@ -23,7 +24,7 @@ class Kamal::Configuration::Registry
private private
def lookup(key) def lookup(key)
if registry_config[key].is_a?(Array) if registry_config[key].is_a?(Array)
ENV.fetch(registry_config[key].first).dup secrets[registry_config[key].first]
else else
registry_config[key] registry_config[key]
end end

View File

@@ -1,10 +1,9 @@
class Kamal::Configuration::Role class Kamal::Configuration::Role
include Kamal::Configuration::Validation include Kamal::Configuration::Validation
CORD_FILE = "cord"
delegate :argumentize, :optionize, to: Kamal::Utils delegate :argumentize, :optionize, to: Kamal::Utils
attr_reader :name, :config, :specialized_env, :specialized_logging, :specialized_healthcheck attr_reader :name, :config, :specialized_env, :specialized_logging, :specialized_proxy
alias to_s name alias to_s name
@@ -18,16 +17,14 @@ class Kamal::Configuration::Role
@specialized_env = Kamal::Configuration::Env.new \ @specialized_env = Kamal::Configuration::Env.new \
config: specializations.fetch("env", {}), config: specializations.fetch("env", {}),
secrets_file: File.join(config.host_env_directory, "roles", "#{container_prefix}.env"), secrets: config.secrets,
context: "servers/#{name}/env" context: "servers/#{name}/env"
@specialized_logging = Kamal::Configuration::Logging.new \ @specialized_logging = Kamal::Configuration::Logging.new \
logging_config: specializations.fetch("logging", {}), logging_config: specializations.fetch("logging", {}),
context: "servers/#{name}/logging" context: "servers/#{name}/logging"
@specialized_healthcheck = Kamal::Configuration::Healthcheck.new \ initialize_specialized_proxy
healthcheck_config: specializations.fetch("healthcheck", {}),
context: "servers/#{name}/healthcheck"
end end
def primary_host def primary_host
@@ -55,7 +52,7 @@ class Kamal::Configuration::Role
end end
def labels def labels
default_labels.merge(traefik_labels).merge(custom_labels) default_labels.merge(custom_labels)
end end
def label_args def label_args
@@ -70,6 +67,24 @@ class Kamal::Configuration::Role
@logging ||= config.logging.merge(specialized_logging) @logging ||= config.logging.merge(specialized_logging)
end end
def proxy
@proxy ||= config.proxy.merge(specialized_proxy) if running_proxy?
end
def running_proxy?
@running_proxy
end
def ssl?
running_proxy? && proxy.ssl?
end
def stop_args
# When deploying with the proxy, kamal-proxy will drain request before returning so we don't need to wait.
timeout = running_proxy? ? nil : config.drain_timeout
[ *argumentize("-t", timeout) ]
end
def env(host) def env(host)
@envs ||= {} @envs ||= {}
@@ -77,7 +92,19 @@ class Kamal::Configuration::Role
end end
def env_args(host) def env_args(host)
env(host).args [ *env(host).clear_args, *argumentize("--env-file", secrets_path) ]
end
def env_directory
File.join(config.env_directory, "roles")
end
def secrets_io(host)
env(host).secrets_io
end
def secrets_path
File.join(config.env_directory, "roles", "#{name}.env")
end end
def asset_volume_args def asset_volume_args
@@ -85,72 +112,8 @@ class Kamal::Configuration::Role
end end
def health_check_args(cord: true)
if running_traefik? || healthcheck.set_port_or_path?
if cord && uses_cord?
optionize({ "health-cmd" => health_check_cmd_with_cord, "health-interval" => healthcheck.interval })
.concat(cord_volume.docker_args)
else
optionize({ "health-cmd" => healthcheck.cmd, "health-interval" => healthcheck.interval })
end
else
[]
end
end
def healthcheck
@healthcheck ||=
if running_traefik?
config.healthcheck.merge(specialized_healthcheck)
else
specialized_healthcheck
end
end
def health_check_cmd_with_cord
"(#{healthcheck.cmd}) && (stat #{cord_container_file} > /dev/null || exit 1)"
end
def running_traefik?
if specializations["traefik"].nil?
primary?
else
specializations["traefik"]
end
end
def primary? def primary?
self == @config.primary_role name == @config.primary_role_name
end
def uses_cord?
running_traefik? && cord_volume && healthcheck.cmd.present?
end
def cord_host_directory
File.join config.run_directory_as_docker_volume, "cords", [ container_prefix, config.run_id ].join("-")
end
def cord_volume
if (cord = healthcheck.cord)
@cord_volume ||= Kamal::Configuration::Volume.new \
host_path: File.join(config.run_directory, "cords", [ container_prefix, config.run_id ].join("-")),
container_path: cord
end
end
def cord_host_file
File.join cord_volume.host_path, CORD_FILE
end
def cord_container_directory
health_check_options.fetch("cord", nil)
end
def cord_container_file
File.join cord_volume.container_path, CORD_FILE
end end
@@ -168,25 +131,52 @@ class Kamal::Configuration::Role
end end
def assets? def assets?
asset_path.present? && running_traefik? asset_path.present? && running_proxy?
end end
def asset_volume(version = nil) def asset_volume(version = config.version)
if assets? if assets?
Kamal::Configuration::Volume.new \ Kamal::Configuration::Volume.new \
host_path: asset_volume_path(version), container_path: asset_path host_path: asset_volume_directory(version), container_path: asset_path
end end
end end
def asset_extracted_path(version = nil) def asset_extracted_directory(version = config.version)
File.join config.run_directory, "assets", "extracted", container_name(version) File.join config.assets_directory, "extracted", [ name, version ].join("-")
end end
def asset_volume_path(version = nil) def asset_volume_directory(version = config.version)
File.join config.run_directory, "assets", "volumes", container_name(version) File.join config.assets_directory, "volumes", [ name, version ].join("-")
end
def ensure_one_host_for_ssl
if running_proxy? && proxy.ssl? && hosts.size > 1
raise Kamal::ConfigurationError, "SSL is only supported on a single server, found #{hosts.size} servers for role #{name}"
end
end end
private private
def initialize_specialized_proxy
proxy_specializations = specializations["proxy"]
if primary?
# only false means no proxy for non-primary roles
@running_proxy = proxy_specializations != false
else
# false and nil both mean no proxy for non-primary roles
@running_proxy = !!proxy_specializations
end
if running_proxy?
proxy_config = proxy_specializations == true || proxy_specializations.nil? ? {} : proxy_specializations
@specialized_proxy = Kamal::Configuration::Proxy.new \
config: config,
proxy_config: proxy_config,
context: "servers/#{name}/proxy"
end
end
def tagged_hosts def tagged_hosts
{}.tap do |tagged_hosts| {}.tap do |tagged_hosts|
extract_hosts_from_config.map do |host_config| extract_hosts_from_config.map do |host_config|
@@ -221,27 +211,6 @@ class Kamal::Configuration::Role
end end
end end
def traefik_labels
if running_traefik?
{
# Setting a service property ensures that the generated service name will be consistent between versions
"traefik.http.services.#{traefik_service}.loadbalancer.server.scheme" => "http",
"traefik.http.routers.#{traefik_service}.rule" => "PathPrefix(`/`)",
"traefik.http.routers.#{traefik_service}.priority" => "2",
"traefik.http.middlewares.#{traefik_service}-retry.retry.attempts" => "5",
"traefik.http.middlewares.#{traefik_service}-retry.retry.initialinterval" => "500ms",
"traefik.http.routers.#{traefik_service}.middlewares" => "#{traefik_service}-retry@docker"
}
else
{}
end
end
def traefik_service
container_prefix
end
def custom_labels def custom_labels
Hash.new.tap do |labels| Hash.new.tap do |labels|
labels.merge!(config.labels) if config.labels.present? labels.merge!(config.labels) if config.labels.present?

View File

@@ -26,8 +26,20 @@ class Kamal::Configuration::Ssh
end end
end end
def keys_only
ssh_config["keys_only"]
end
def keys
ssh_config["keys"]
end
def key_data
ssh_config["key_data"]
end
def options def options
{ user: user, port: port, proxy: proxy, logger: logger, keepalive: true, keepalive_interval: 30 }.compact { user: user, port: port, proxy: proxy, logger: logger, keepalive: true, keepalive_interval: 30, keys_only: keys_only, keys: keys, key_data: key_data }.compact
end end
def to_h def to_h

View File

@@ -1,60 +0,0 @@
class Kamal::Configuration::Traefik
DEFAULT_IMAGE = "traefik:v2.10"
CONTAINER_PORT = 80
DEFAULT_ARGS = {
"log.level" => "DEBUG"
}
DEFAULT_LABELS = {
# These ensure we serve a 502 rather than a 404 if no containers are available
"traefik.http.routers.catchall.entryPoints" => "http",
"traefik.http.routers.catchall.rule" => "PathPrefix(`/`)",
"traefik.http.routers.catchall.service" => "unavailable",
"traefik.http.routers.catchall.priority" => 1,
"traefik.http.services.unavailable.loadbalancer.server.port" => "0"
}
include Kamal::Configuration::Validation
attr_reader :config, :traefik_config
def initialize(config:)
@config = config
@traefik_config = config.raw_config.traefik || {}
validate! traefik_config
end
def publish?
traefik_config["publish"] != false
end
def labels
DEFAULT_LABELS.merge(traefik_config["labels"] || {})
end
def env
Kamal::Configuration::Env.new \
config: traefik_config.fetch("env", {}),
secrets_file: File.join(config.host_env_directory, "traefik", "traefik.env"),
context: "traefik/env"
end
def host_port
traefik_config.fetch("host_port", CONTAINER_PORT)
end
def options
traefik_config.fetch("options", {})
end
def port
"#{host_port}:#{CONTAINER_PORT}"
end
def args
DEFAULT_ARGS.merge(traefik_config.fetch("args", {}))
end
def image
traefik_config.fetch("image", DEFAULT_IMAGE)
end
end

View File

@@ -13,22 +13,28 @@ class Kamal::Configuration::Validator
private private
def validate_against_example!(validation_config, example) def validate_against_example!(validation_config, example)
validate_type! validation_config, Hash validate_type! validation_config, example.class
if (unknown_keys = validation_config.keys - example.keys).any? if example.class == Hash
unknown_keys_error unknown_keys check_unknown_keys! validation_config, example
end
validation_config.each do |key, value| validation_config.each do |key, value|
next if extension?(key)
with_context(key) do with_context(key) do
example_value = example[key] example_value = example[key]
if example_value == "..." if example_value == "..."
unless key.to_s == "proxy" && boolean?(value.class)
validate_type! value, *(Array if key == :servers), Hash validate_type! value, *(Array if key == :servers), Hash
end
elsif key == "hosts" elsif key == "hosts"
validate_servers! value validate_servers! value
elsif example_value.is_a?(Array) elsif example_value.is_a?(Array)
if key == "arch"
validate_array_of_or_type! value, example_value.first.class
else
validate_array_of! value, example_value.first.class validate_array_of! value, example_value.first.class
end
elsif example_value.is_a?(Hash) elsif example_value.is_a?(Hash)
case key.to_s case key.to_s
when "options", "args" when "options", "args"
@@ -44,6 +50,7 @@ class Kamal::Configuration::Validator
end end
end end
end end
end
def valid_type?(value, type) def valid_type?(value, type)
@@ -70,6 +77,16 @@ class Kamal::Configuration::Validator
value.is_a?(String) || value.is_a?(Symbol) || value.is_a?(Numeric) || value.is_a?(TrueClass) || value.is_a?(FalseClass) value.is_a?(String) || value.is_a?(Symbol) || value.is_a?(Numeric) || value.is_a?(TrueClass) || value.is_a?(FalseClass)
end end
def validate_array_of_or_type!(value, type)
if value.is_a?(Array)
validate_array_of! value, type
else
validate_type! value, type
end
rescue Kamal::ConfigurationError
type_error(Array, type)
end
def validate_array_of!(array, type) def validate_array_of!(array, type)
validate_type! array, Array validate_type! array, Array
@@ -137,4 +154,18 @@ class Kamal::Configuration::Validator
ensure ensure
@context = old_context @context = old_context
end end
def allow_extensions?
false
end
def extension?(key)
key.to_s.start_with?("x-")
end
def check_unknown_keys!(config, example)
unknown_keys = config.keys - example.keys
unknown_keys.reject! { |key| extension?(key) } if allow_extensions?
unknown_keys_error unknown_keys if unknown_keys.present?
end
end end

View File

@@ -0,0 +1,15 @@
class Kamal::Configuration::Validator::Alias < Kamal::Configuration::Validator
def validate!
super
name = context.delete_prefix("aliases/")
if name !~ /\A[a-z0-9_-]+\z/
error "Invalid alias name: '#{name}'. Must only contain lowercase letters, alphanumeric, hyphens and underscores."
end
if Kamal::Cli::Main.commands.include?(name)
error "Alias '#{name}' conflicts with a built-in command."
end
end
end

View File

@@ -5,5 +5,9 @@ class Kamal::Configuration::Validator::Builder < Kamal::Configuration::Validator
if config["cache"] && config["cache"]["type"] if config["cache"] && config["cache"]["type"]
error "Invalid cache type: #{config["cache"]["type"]}" unless [ "gha", "registry" ].include?(config["cache"]["type"]) error "Invalid cache type: #{config["cache"]["type"]}" unless [ "gha", "registry" ].include?(config["cache"]["type"])
end end
error "Builder arch not set" unless config["arch"].present?
error "Cannot disable local builds, no remote is set" if config["local"] == false && config["remote"].blank?
end end
end end

View File

@@ -0,0 +1,6 @@
class Kamal::Configuration::Validator::Configuration < Kamal::Configuration::Validator
private
def allow_extensions?
true
end
end

View File

@@ -0,0 +1,15 @@
class Kamal::Configuration::Validator::Proxy < Kamal::Configuration::Validator
def validate!
unless config.nil?
super
if config["host"].blank? && config["hosts"].blank? && config["ssl"]
error "Must set a host to enable automatic SSL"
end
if (config.keys & [ "host", "hosts" ]).size > 1
error "Specify one of 'host' or 'hosts', not both"
end
end
end
end

View File

@@ -15,6 +15,10 @@ class Kamal::EnvFile
env_file.presence || "\n" env_file.presence || "\n"
end end
def to_io
StringIO.new(to_s)
end
alias to_str to_s alias to_str to_s
private private

View File

@@ -9,6 +9,10 @@ module Kamal::Git
`git config user.name`.strip `git config user.name`.strip
end end
def email
`git config user.email`.strip
end
def revision def revision
`git rev-parse HEAD`.strip `git rev-parse HEAD`.strip
end end

37
lib/kamal/secrets.rb Normal file
View File

@@ -0,0 +1,37 @@
require "dotenv"
class Kamal::Secrets
attr_reader :secrets_files
Kamal::Secrets::Dotenv::InlineCommandSubstitution.install!
def initialize(destination: nil)
@secrets_files = \
[ ".kamal/secrets-common", ".kamal/secrets#{(".#{destination}" if destination)}" ].select { |f| File.exist?(f) }
@mutex = Mutex.new
end
def [](key)
# Fetching secrets may ask the user for input, so ensure only one thread does that
@mutex.synchronize do
secrets.fetch(key)
end
rescue KeyError
if secrets_files
raise Kamal::ConfigurationError, "Secret '#{key}' not found in #{secrets_files.join(", ")}"
else
raise Kamal::ConfigurationError, "Secret '#{key}' not found, no secret files provided"
end
end
def to_h
secrets
end
private
def secrets
@secrets ||= secrets_files.inject({}) do |secrets, secrets_file|
secrets.merge!(::Dotenv.parse(secrets_file))
end
end
end

View File

@@ -0,0 +1,14 @@
require "active_support/core_ext/string/inflections"
module Kamal::Secrets::Adapters
def self.lookup(name)
name = "one_password" if name.downcase == "1password"
name = "last_pass" if name.downcase == "lastpass"
adapter_class(name)
end
def self.adapter_class(name)
Object.const_get("Kamal::Secrets::Adapters::#{name.camelize}").new
rescue NameError => e
raise RuntimeError, "Unknown secrets adapter: #{name}"
end
end

View File

@@ -0,0 +1,18 @@
class Kamal::Secrets::Adapters::Base
delegate :optionize, to: Kamal::Utils
def fetch(secrets, account:, from: nil)
session = login(account)
full_secrets = secrets.map { |secret| [ from, secret ].compact.join("/") }
fetch_secrets(full_secrets, account: account, session: session)
end
private
def login(...)
raise NotImplementedError
end
def fetch_secrets(...)
raise NotImplementedError
end
end

View File

@@ -0,0 +1,66 @@
class Kamal::Secrets::Adapters::Bitwarden < Kamal::Secrets::Adapters::Base
private
def login(account)
status = run_command("status")
if status["status"] == "unauthenticated"
run_command("login #{account.shellescape}", raw: true)
status = run_command("status")
end
if status["status"] == "locked"
session = run_command("unlock --raw", raw: true).presence
status = run_command("status", session: session)
end
raise RuntimeError, "Failed to login to and unlock Bitwarden" unless status["status"] == "unlocked"
run_command("sync", session: session, raw: true)
raise RuntimeError, "Failed to sync Bitwarden" unless $?.success?
session
end
def fetch_secrets(secrets, account:, session:)
{}.tap do |results|
items_fields(secrets).each do |item, fields|
item_json = run_command("get item #{item.shellescape}", session: session, raw: true)
raise RuntimeError, "Could not read #{secret} from Bitwarden" unless $?.success?
item_json = JSON.parse(item_json)
if fields.any?
fields.each do |field|
item_field = item_json["fields"].find { |f| f["name"] == field }
raise RuntimeError, "Could not find field #{field} in item #{item} in Bitwarden" unless item_field
value = item_field["value"]
results["#{item}/#{field}"] = value
end
elsif item_json.dig("login", "password")
results[item] = item_json.dig("login", "password")
else
raise RuntimeError, "Item #{item} is not a login type item and no fields were specified"
end
end
end
end
def items_fields(secrets)
{}.tap do |items|
secrets.each do |secret|
item, field = secret.split("/")
items[item] ||= []
items[item] << field
end
end
end
def signedin?(account)
run_command("status")["status"] != "unauthenticated"
end
def run_command(command, session: nil, raw: false)
full_command = [ *("BW_SESSION=#{session.shellescape}" if session), "bw", command ].join(" ")
result = `#{full_command}`.strip
raw ? result : JSON.parse(result)
end
end

Some files were not shown because too many files have changed in this diff Show More