Compare commits

...

990 Commits

Author SHA1 Message Date
Donal McBreen
559bb3667b WIP 2025-03-03 15:52:50 +00:00
Donal McBreen
f87bcf5bc6 Merge pull request #1413 from basecamp/dependabot/bundler/bundler-fd41ac4d62
Bump rack from 3.1.8 to 3.1.10 in the bundler group across 1 directory
2025-03-03 14:49:37 +00:00
Donal McBreen
62dfa45ee6 Bump version for 2.5.3 2025-02-27 10:27:27 +00:00
Donal McBreen
c13ee578df Merge pull request #1420 from smartygus/docker-build-fix
Fix Docker Build under Ruby 3.4
2025-02-27 09:42:57 +00:00
Michael Smart
f01238112e Update nokogiri to 1.18.3
See: https://github.com/sparklemotion/nokogiri/releases/tag/v1.18.0

- required for ruby 3.4 compatibility
- add more platforms to lockfile to support
  docker build process, due to changes
  in the nokogiri native gem setup where
  -musl and -gnu linux platforms are no longer
  interchangeable
- bundler >= 2.5.6 required according to Nokogiri release
  notes, so updated to current latest version (2.6.5)
2025-02-21 13:23:08 +01:00
dependabot[bot]
32ab72089a Bump rack from 3.1.8 to 3.1.10 in the bundler group across 1 directory
Bumps the bundler group with 1 update in the / directory: [rack](https://github.com/rack/rack).


Updates `rack` from 3.1.8 to 3.1.10
- [Release notes](https://github.com/rack/rack/releases)
- [Changelog](https://github.com/rack/rack/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rack/rack/compare/v3.1.8...v3.1.10)

---
updated-dependencies:
- dependency-name: rack
  dependency-type: indirect
  dependency-group: bundler
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-02-12 19:26:36 +00:00
Donal McBreen
6f29d4e78b Bump version for 2.5.2 2025-02-07 15:27:30 +00:00
Donal McBreen
9d2dda0d77 Merge pull request #1402 from basecamp/fix-docker-build
Fix the docker build
2025-02-06 14:59:14 +00:00
Donal McBreen
b130bc0321 Fix the docker build
Somewhere along the way the docker build broke, it now needs yaml-dev
to be installed. Maybe the underlying image changed?

Update to 3.4-alpine while we are here.
2025-02-06 14:45:37 +00:00
Donal McBreen
28be0300b9 Bump version for 2.5.1 2025-02-06 13:54:46 +00:00
Donal McBreen
3c2163ab78 Merge pull request #1401 from basecamp/avoid-docker-check-with-skipping-local
Only check for docker when logging in locally
2025-02-06 11:35:49 +00:00
Donal McBreen
fdf7e6927a Only check for docker when logging in locally
If we are skipping a local registry login, we don't need docker
installed locally.

Fixes: https://github.com/basecamp/kamal/issues/1400
2025-02-06 11:12:56 +00:00
Donal McBreen
45197e46f6 Bump version for 2.5.0 2025-02-04 11:19:44 +00:00
Donal McBreen
6b40a64b9a Merge pull request #1398 from basecamp/kamal-2.5-doc-changes
Doc changes for 2.5
2025-02-04 11:18:40 +00:00
Donal McBreen
9af2425fbd Doc changes for 2.5
Sync up the accessory.yml file with the latest changes in the
kamal-site repo.
2025-02-04 11:05:25 +00:00
Donal McBreen
854dd925ba Merge pull request #1397 from basecamp/prep-for-2.5
Prep for 2.5
2025-02-04 09:18:30 +00:00
Donal McBreen
8775d202bd Prep for 2.5
- Reset KAMAL on alias command, rather than relying on checking
  "invoked_via_subcommand"
- Validate the accessories roles when loading the configuration
  not later on when trying to access them
2025-02-04 09:08:57 +00:00
Donal McBreen
bae7c56e74 Merge pull request #1392 from neiljohari/feature/allow-omitting-aws-account
Allow omitting AWS account parameter while fetching secrets
2025-02-04 08:45:06 +00:00
Neil Johari
07d05ad58a Run rubocop auto correct 2025-02-03 09:44:39 -08:00
Neil Johari
e69611efb6 Add final newline 2025-02-03 08:56:06 -08:00
Donal McBreen
ba6dd6ff14 Merge pull request #1396 from basecamp/allow-accessory-roles-with-no-hosts
Allow accessory roles with no hosts
2025-02-03 16:54:28 +00:00
Donal McBreen
04a96aa5be Allow accessory roles with no hosts
Only raise an exception if the role is not found, not it it has no
hosts.
2025-02-03 16:44:01 +00:00
Donal McBreen
dba3a115bd Merge pull request #1395 from basecamp/app-boot-hooks
Add pre and post app boot hooks
2025-02-03 16:04:14 +00:00
Donal McBreen
cd73cea850 Add pre and post app boot hooks
Add two new hooks pre-app-boot and post-app-boot. They are analagous
to the pre/post proxy reboot hooks.

If the boot strategy deploys in groups, then the hooks are called once
per group of hosts and `KAMAL_HOSTS` contains a comma delimited list of
the hosts in that group.

If all hosts are deployed to at once, then they are called once with
`KAMAL_HOSTS` containing all the hosts.

It is possible to have pauses between groups of hosts in the boot config,
where this is the case the pause happens after the post-app-boot hook is
called.
2025-02-03 15:54:45 +00:00
Donal McBreen
09d020e9bb Merge pull request #1357 from flavorjones/flavorjones-kamal-build-local
Introduce a "build dev" command
2025-02-03 09:07:00 +00:00
Neil Johari
ff3538f81d Undo accidental line deletion 2025-02-02 23:54:53 -08:00
Neil Johari
c7d1711e30 Remove unnecessary var 2025-02-02 23:46:09 -08:00
Neil Johari
d710b5a22b Allow ommitting AWS account while fetching secrets 2025-02-02 23:43:51 -08:00
Mike Dalessio
214d4fd321 The build dev command warns about untracked and uncommitted files
so there's a complete picture of what is being packaged that's not in git.
2025-02-01 22:19:05 -05:00
Donal McBreen
5ddaa3810d Merge pull request #1370 from flavorjones/flavorjones-ci-matrix
ci: use `fail-fast: false` instead of `continue-on-error: true`
2025-01-22 09:28:42 +00:00
Mike Dalessio
3c01dc75fd ci: use fail-fast: false instead of continue-on-error: true
This will give us proper red/green signal on the test suite while
still running the entire matrix.
2025-01-20 18:59:11 -05:00
Mike Dalessio
2127f1708a feat: Introduce a build dev command
which will build a "dirty" image using the working directory.

This command is different from `build push` in two important ways:

- the image tags will have a suffix of `-dirty`
- the export action is "docker", pushing to the local docker image store

The command also supports the `--output` option just added to `build
push` to override that default.

This command is intended to allow developers to quickly iterate on a
docker image built from their local working directory while avoiding
any confusion with a pristine image built from a git clone, and
keeping those images on the local dev system by default.
2025-01-20 18:52:21 -05:00
Mike Dalessio
24e4347c45 feat: Introduce a build push --output option
which controls where the build result is exported.

The default value is "registry" to reflect the current behavior of
`build push`.

Any value provided to this option will be passed to the `buildx build`
command as a `--output=type=<VALUE>` flag.

For example, the following command will push to the local docker image
store:

    kamal build push --output=docker

squash
2025-01-20 18:37:15 -05:00
Donal McBreen
5f04e4266b Merge pull request #1369 from basecamp/dont-cleanup-traefik-on-reboot
Don't cleanup traefik on reboot
2025-01-20 15:23:49 +00:00
Donal McBreen
35a29cc538 Merge pull request #1331 from guillaumebriday/patch-1
Fixing log command on role in example
2025-01-20 15:23:31 +00:00
Donal McBreen
f187080db5 Don't cleanup traefik on reboot
This was designed to help with upgrading from Kamal 1 to Kamal 2
but it causes issues if you have a traefik container you don't want
to be shut down.
2025-01-20 15:06:06 +00:00
Donal McBreen
080fa49fdf Merge pull request #1368 from basecamp/error-free-ruby-version-comment
Don't read a file in sample deploy.yml
2025-01-20 14:38:34 +00:00
Donal McBreen
34050f1036 Don't read a file in sample deploy.yml
The ERB runs first so it does matter if it in a comment. If the file
doesn't exist (e.g. if not using Ruby, you'll get an error).

We'll change the example to match the Rails deploy.yml template won't
have than problem.
2025-01-20 14:26:43 +00:00
Donal McBreen
459c7366ec Merge pull request #1367 from brettabamonte/fix_lastpass_err_msg_typo
Fix LastPass error message typo
2025-01-20 11:43:33 +00:00
Donal McBreen
f8db5de5eb Merge pull request #1354 from matthewbjones/feature/docker-build-cloud
Adds support for Docker Build Cloud
2025-01-20 08:24:27 +00:00
brettabamonte
4d67a1671a Change LassPass to LastPass 2025-01-18 19:11:34 -05:00
Donal McBreen
2c9bba3f88 Merge branch 'main' into feature/docker-build-cloud 2025-01-17 15:49:28 +00:00
Donal McBreen
a388937de8 Merge pull request #1363 from basecamp/check-for-docker-locally
Check for docker locally before registry login
2025-01-17 15:45:18 +00:00
Donal McBreen
9ef6c2f893 Merge pull request #1361 from basecamp/ruby-3.4
Update to Ruby 3.4 from the preview version
2025-01-17 15:27:34 +00:00
Donal McBreen
eee9d67691 Merge pull request #1319 from ShPakvel/fix_bug_in_role_validate_servers
Fix bugs for role validate servers
2025-01-17 15:19:25 +00:00
Donal McBreen
5bd9bc8576 Merge pull request #1320 from ShPakvel/add_optional_accessory_registry
[Feature] Registry for accessory
2025-01-17 15:18:50 +00:00
Donal McBreen
a5b9c69838 Update to Ruby 3.4 from the preview version 2025-01-17 15:17:37 +00:00
Donal McBreen
dc9a95db2c Check for docker locally before registry login
We were checking before `kamal build push`, but not `kamal registry login`.
Since `kamal registry login` is called first by a deploy we don't
get the nice error message.
2025-01-17 15:17:22 +00:00
Donal McBreen
0174b872bf Merge pull request #1362 from basecamp/boot-accessories-after-pre-deploy-hook
Boot accessories after pre-deploy hook
2025-01-17 15:13:04 +00:00
Donal McBreen
1db44c402c Boot accessories after pre-deploy hook
That allows you to set proxy config in the hook before booting
the proxy.
2025-01-17 15:04:16 +00:00
Matthew Jones
b420b2613d Adds support for Docker Build Cloud 2025-01-17 07:14:31 -07:00
Donal McBreen
4ffa772201 Don't boot proxy twice when setting up 2025-01-17 13:04:37 +00:00
Donal McBreen
e081414849 Merge pull request #1308 from pokonski/proxy-accessory-fix
Boot proxy on server setup
2025-01-17 13:04:07 +00:00
Donal McBreen
85c1c47c2f Merge pull request #1360 from basecamp/secret-adapter-tidy
Secret adapter tidy
2025-01-17 13:01:21 +00:00
Donal McBreen
9f1688da7a Fix test 2025-01-17 12:52:23 +00:00
Donal McBreen
2bd716ece4 Drop the TestOptionalAccount adapter
It's included in the gem lib which is best to avoid and we can infer
that it works account optional adapters.
2025-01-17 12:37:12 +00:00
Donal McBreen
f9a78f4fcb gcloud login tidy
Use unless instead of if !, don't suggest running gcloud auth login,
we've just tried that.
2025-01-17 12:34:38 +00:00
Donal McBreen
10dafc058a Extract secrets_get_flags 2025-01-17 12:31:24 +00:00
Donal McBreen
5e2678dece Ensure external input is shell escaped 2025-01-17 12:28:59 +00:00
Donal McBreen
a1708f687f Prefix secrets in fetch_secrets
This allows us to remove the custom fetch method for enpass.
2025-01-17 12:24:46 +00:00
Donal McBreen
db7556ed99 Fix enpass adapter
There were changes in main that meant the tests failed after merging.

Adding the new `requires_account?` method to the enpass adapter fixed it.
2025-01-17 12:07:56 +00:00
Donal McBreen
93133cd7a9 Merge pull request #1236 from andrelaszlo/gcp_secret_manager_adapter
Add GCP Secret Manager adapter
2025-01-17 12:07:33 +00:00
Donal McBreen
a7b2ef56c7 Merge pull request #1189 from egze/enpass
Add support for Enpass - a password manager for secrets
2025-01-17 12:01:24 +00:00
Donal McBreen
06f2cb223e Merge branch 'main' into gcp_secret_manager_adapter 2025-01-17 11:57:52 +00:00
Donal McBreen
ea7e72d75f Merge pull request #1186 from oandalib/bitwarden-secrets-manager
feat: add Bitwarden Secrets Manager adapter
2025-01-17 11:43:19 +00:00
Donal McBreen
9035bd0d88 Merge pull request #1359 from basecamp/add-env-precedence-tests
Add tests for env/secret file precedence
2025-01-17 11:19:32 +00:00
Donal McBreen
dd8cadf743 Add tests for env/secret file precedence 2025-01-17 11:06:29 +00:00
Donal McBreen
f1a9a09929 Merge pull request #1265 from phoozle/proxy-bind-ip
Add proxy boot_config --publish-ip argument
2025-01-17 08:49:17 +00:00
Donal McBreen
620b132138 Merge pull request #1313 from emmceemoore/patch-1
Configure the CLI to exit non-zero on failures.
2025-01-17 08:31:58 +00:00
Donal McBreen
2e7d0ddc44 Merge pull request #1358 from basecamp/dont-run-assets-container
Create but don't run the assets container
2025-01-17 08:09:01 +00:00
Donal McBreen
ab8396fbb2 Merge pull request #1032 from basecamp/set-config-file-and-deploy-in-aliases
Allow destination and config-file in aliases
2025-01-17 08:07:06 +00:00
Donal McBreen
2cdca4596c Create but don't run the assets container
We don't need to run the assets container to copy the assets out,
instead we can just create, copy and remove.
2025-01-16 16:28:02 +00:00
Donal McBreen
78fcc3d88f Allow destination and config-file in aliases
We only loaded the configuration once, which meant that aliases always
used the initial configuration file and destination.

We don't want to load the configuration in subcommands as it is not
passed all the options we need. But just checking if we are in a
subcommand is enough - the alias reloads and the subcommand does not.

One thing to note is that anything passed on the command line overrides
what is in the alias, so if an alias says
`other_config: config -c config/deploy2.yml` and you run
`kamal other_config -c config/deploy.yml`, it won't switch.
2025-01-16 15:51:18 +00:00
Guillaume Briday
2b9d5c2b19 Fixing log command on role 2025-01-02 22:51:01 +01:00
Pavel Shpak
d59c274208 Fix typo in configuration initializer method. 2024-12-22 04:37:15 +02:00
Pavel Shpak
bd8689c185 Fix bug in role validate_servers.
There were typo-bug during `validate_servers!` invocation for role.
It wasn't discovered, because it never met condition. Because role_config wasn't correctly extracted for validation.

Also remove not used anymore `accessories_on`. Leftover from previous changes.
2024-12-22 03:28:12 +02:00
Pavel Shpak
b5aee11a40 [Feature] Add optional accessory registry.
Add test cases to cover new option.
2024-12-22 02:50:53 +02:00
Mike Moore
2943c4a301 Use the newer option name. 2024-12-20 08:45:47 -07:00
Mike Moore
32e1b6504d Re-trigger GitHub actions. 2024-12-20 08:26:14 -07:00
Mike Moore
39e2c4f848 Trying the new method for setting proxy boot config. 2024-12-19 12:14:00 -07:00
Mike Moore
89db5025a0 Configure Thor to "exit on failure". 2024-12-19 09:28:37 -07:00
Piotrek O
c56edba4a9 Boot proxy on server setup 2024-12-18 11:35:57 +01:00
Donal McBreen
1547089da0 Bump version for 2.4.0 2024-12-13 12:38:26 +00:00
Donal McBreen
ae7a4f3411 Update yml files to match doc site changes 2024-12-13 12:27:22 +00:00
Donal McBreen
77c202ebaf Highlight ssl/forward_headers behaviour
Pulled in from: https://github.com/basecamp/kamal-site/pull/141
2024-12-13 12:20:05 +00:00
Donal McBreen
063dfd9edd Merge pull request #1296 from basecamp/fix-for-dotenv-1.3.5
Fix for Dotenv 3.1.5
2024-12-13 10:54:16 +00:00
Donal McBreen
3e4a190173 Fix for Dotenv 3.1.5
In Dotenv 3.1.5, `Dotenv.parse` no longer returns values that are
already in the environment.

See https://github.com/bkeepers/dotenv/issues/518

We can get the values though by setting overwrite: true, which works
with both 3.1.4 and 3.1.5.
2024-12-13 10:42:02 +00:00
Donal McBreen
d9c25533e4 Merge pull request #1292 from nickhammond/aws-secrets-manager-simple
Aws secrets manager simple strings and error checking
2024-12-13 09:45:52 +00:00
Donal McBreen
d5ec0e6ad2 Merge pull request #1291 from nickhammond/remove-grep-options-alias
Remove the alias for grep_options, issues processing with thor
2024-12-13 09:44:06 +00:00
Nick Hammond
725da6aa68 Rubocop, Rubocop 2024-12-12 05:29:15 -07:00
Nick Hammond
84a874e63b Update secrets manager spec to render multiple errors 2024-12-12 05:15:52 -07:00
Nick Hammond
ba567e0474 Just map the secrets returned from AWS 2024-12-12 05:09:12 -07:00
Nick Hammond
e464177349 Check for errors from AWS secrets manager 2024-12-12 04:58:53 -07:00
Nick Hammond
68e6f82b30 Grab from secret2 for assertion 2024-12-12 04:17:03 -07:00
Nick Hammond
55983c6431 AWS secrets manager value can be a string 2024-12-12 04:10:48 -07:00
Nick Hammond
b2cf3f33a7 Remove the alias for grep_options, issues processing with thor 2024-12-12 03:47:25 -07:00
Jeremy Daer
16fb3adacb No need for IO.read for basic file paths
References 3cad095, e1d5182
2024-12-10 16:08:58 -08:00
Jeremy Daer
407c8b834e Simplify hostname trimming. References #762. 2024-12-10 15:57:30 -08:00
dependabot[bot]
3468b45014 Bump actionpack in the bundler group across 1 directory (#1283)
Bumps the bundler group with 1 update in the / directory: [actionpack](https://github.com/rails/rails).


Updates `actionpack` from 7.1.4.1 to 7.1.5.1
- [Release notes](https://github.com/rails/rails/releases)
- [Changelog](https://github.com/rails/rails/blob/v8.0.0.1/actionpack/CHANGELOG.md)
- [Commits](https://github.com/rails/rails/compare/v7.1.4.1...v7.1.5.1)

---
updated-dependencies:
- dependency-name: actionpack
  dependency-type: indirect
  dependency-group: bundler
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-10 15:46:48 -08:00
André Laszlo
8103d68688 Shellescape all interpolated strings in commands 2024-12-06 17:43:47 +01:00
André Laszlo
eb82b4a753 Keep the 'default' prefix for secret items 2024-12-06 17:40:08 +01:00
André Laszlo
19b4359b17 Use a nil session 2024-12-06 17:32:31 +01:00
André Laszlo
dc64aaa0de Add gcloud auth login invocation to test 2024-12-06 17:32:01 +01:00
André Laszlo
ea170fbe5e Run gcloud auth login if user is not authenticated 2024-12-06 17:22:03 +01:00
André Laszlo
18f2aae936 Simplify parsing by changing account separators 2024-12-06 17:15:22 +01:00
André Laszlo
e314f38bdc Merge remote-tracking branch 'origin/main' into gcp_secret_manager_adapter 2024-12-06 17:08:26 +01:00
Matthew Croall
1c8a56b8cf Change invalid publish ip exception class 2024-12-04 10:44:16 +10:30
Matthew Croall
e597ae6155 Add support for multiple publish ip addresses 2024-12-04 10:42:50 +10:30
Donal McBreen
495b3cd95f Merge pull request #1270 from basecamp/kamal-proxy-0.8.4
Update to proxy version 0.8.4
2024-12-03 12:15:00 +00:00
Donal McBreen
b04e8cd8d7 Merge pull request #1272 from basecamp/dependabot/bundler/bundler-7ec3bf8405
Bump rails-html-sanitizer from 1.6.0 to 1.6.1 in the bundler group across 1 directory
2024-12-03 12:14:47 +00:00
Omid Andalib
aa9fe4c525 feat: add Bitwarden Secrets Manager adapter 2024-12-03 00:41:16 -08:00
dependabot[bot]
f5391d7fe4 Bump rails-html-sanitizer in the bundler group across 1 directory
Bumps the bundler group with 1 update in the / directory: [rails-html-sanitizer](https://github.com/rails/rails-html-sanitizer).


Updates `rails-html-sanitizer` from 1.6.0 to 1.6.1
- [Release notes](https://github.com/rails/rails-html-sanitizer/releases)
- [Changelog](https://github.com/rails/rails-html-sanitizer/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rails/rails-html-sanitizer/compare/v1.6.0...v1.6.1)

---
updated-dependencies:
- dependency-name: rails-html-sanitizer
  dependency-type: indirect
  dependency-group: bundler
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-12-02 22:06:59 +00:00
Matthew Croall
0bafa02e7d Rename proxy bind cli argument to publish_host_ip 2024-12-03 08:13:20 +10:30
Matthew Croall
ffe1ac3483 Refactor proxy_publish_args argument concatenation 2024-12-03 08:11:19 +10:30
Donal McBreen
2386c903ca Update to proxy version 0.8.4
Release: https://github.com/basecamp/kamal-proxy/releases/tag/v0.8.4

- Silence late healthcheck requests
2024-12-02 10:37:07 +00:00
Donal McBreen
fbc4515888 Merge pull request #906 from aliismayilov/detached-run
Allow running detached app commands and follow logs by container ID
2024-12-02 10:22:45 +00:00
Donal McBreen
99829092b3 Merge pull request #1229 from matjack1/use-ssh-keys-when-executing-commands
[FIX] - Make kamal use ssh keys from config when performing commands
2024-12-02 09:59:03 +00:00
Donal McBreen
084d1d4a1d Merge pull request #1253 from AxelTheGerman/proxy-0.8.3
Bump proxy minimum version to 0.8.3
2024-12-02 09:54:33 +00:00
Matthew Croall
11e4f37409 Add proxy boot_config --publish-ip argument 2024-11-30 11:10:49 +10:30
André Laszlo
b87bcae6a3 Merge remote-tracking branch 'origin/main' into gcp_secret_manager_adapter 2024-11-27 13:42:21 +01:00
André Laszlo
0c9a367efc Remove overly generic 'secret_manager' alias 2024-11-27 13:33:04 +01:00
Axel Gustav
a1596af815 Bump proxy minimum version to 0.8.3 2024-11-26 09:55:21 -04:00
Donal McBreen
69867e2650 Merge pull request #981 from igor-alexandrov/proxy-accessories
Proxy for accessories
2024-11-26 09:51:22 +00:00
Igor Alexandrov
eee47d10ee Added an integration test for proxied accessory using Busybox and netcat 2024-11-26 13:34:51 +04:00
Ali Ismayilov
8a7843cb35 Allow running the CI manually 2024-11-23 21:45:51 +01:00
Ali Ismayilov
1cc5406b00 Pipe app container id 2024-11-23 21:37:58 +01:00
Matteo Giaccone
e31b98539c Avoid string mutation
For Ruby 3.4
2024-11-22 09:57:45 +01:00
Igor Alexandrov
f367ca8ea5 Replaced Kamal::Commands::Proxy::Exec with Kamal::Commands::App::Proxy and Kamal::Commands::Accessory::Proxy 2024-11-21 23:08:03 +04:00
Igor Alexandrov
14068b32b1 Added alias to accessories proxy configuration example 2024-11-21 22:38:06 +04:00
Igor Alexandrov
f52826b2d6 Updated accessory proxy to support hosts option 2024-11-21 22:23:56 +04:00
Igor Alexandrov
9204624752 Removed duplicated method 2024-11-21 22:23:56 +04:00
Igor Alexandrov
006fa0de17 Extracted proxy commands to a module 2024-11-21 22:23:56 +04:00
Igor Alexandrov
4d8241ebab Fixed kamal-proxy remove command 2024-11-21 22:23:56 +04:00
Igor Alexandrov
86657b0172 Fixed kamal-proxy remove command 2024-11-21 22:23:56 +04:00
Igor Alexandrov
aa2906086a Added host to the expected accessory deploy command result 2024-11-21 22:23:56 +04:00
Igor Alexandrov
f4b7c886fb Added tests for accessory deploy and remove commands 2024-11-21 22:23:56 +04:00
Igor Alexandrov
4c778de2d9 Added tests for accessory configuration with proxy 2024-11-21 22:23:56 +04:00
Igor Alexandrov
70d2c71734 Added commands to deploy accessory to kamal-proxy 2024-11-21 22:23:56 +04:00
Ali Ismayilov
ac90ee068f Prefer dasherized notation 2024-11-21 18:54:34 +01:00
Ali Ismayilov
75b44cd328 Capture logs for specific container_id 2024-11-21 18:54:34 +01:00
Ali Ismayilov
183fe9e06e Follow logs of a specific container 2024-11-21 18:05:56 +01:00
Ali Ismayilov
1da882bb01 Enable logging on app exec new containers 2024-11-21 18:05:55 +01:00
Ali Ismayilov
c662b8d578 Make --detach incompatible with reuse or interactive 2024-11-21 18:05:55 +01:00
Ali Ismayilov
dbe0c3a7f8 Allow running detached app commands
this is useful for long running rake tasks or scripts
that can be run without having to keep open connection to the server.

Example:
```
kamal app exec 'bin/rails db:backfill_task' --detach
```
2024-11-21 18:05:55 +01:00
Donal McBreen
b9804a07aa Merge pull request #1239 from matjack1/output-accessory
Add support for exec output in accessories
2024-11-21 16:57:17 +00:00
Donal McBreen
f4d98bb67a Merge pull request #1225 from matthewbjones/feature/sbom-attestations
Adds support for SBOM attestations
2024-11-21 16:21:41 +00:00
Donal McBreen
42c3425411 Merge pull request #1235 from basecamp/support-line-filtering
Support line filtering when running tests
2024-11-21 15:26:25 +00:00
Donal McBreen
57e48a33bb Merge pull request #1141 from justindell/feat-add-aws-secrets-manager-adapter
feat: add secrets adapter for aws secrets manager
2024-11-21 15:03:54 +00:00
Donal McBreen
4acb78fff6 Merge pull request #1099 from mrbongiolo/feat-secrets-add-doppler-adapter
feat(secrets): add Doppler adapter
2024-11-21 15:03:36 +00:00
Donal McBreen
1a86b3ae6e Merge pull request #1196 from tiramizoo/role-accessories-msg
Improve error on unknown role in accessories config.
2024-11-21 15:02:48 +00:00
Donal McBreen
a4ab34d8d9 Merge pull request #1170 from davidstosik/sto/spaces
Remove trailing spaces from deploy.yml template
2024-11-21 14:52:35 +00:00
Donal McBreen
24d03fd60e Merge pull request #1105 from igor-alexandrov/ruby-version-example
Updated deploy.yml template to fetch the Ruby version automatically
2024-11-21 14:50:11 +00:00
Matteo Giaccone
83fd2a051d Add support for exec output in accessories
When running accessory exec now you get the output from the hosts.

Also you can pass commands with arguments and it will work
e.g.: cat yourfilename
2024-11-21 11:06:36 +01:00
André Laszlo
a07ef64fad Fix --account documentation 2024-11-20 15:27:51 +01:00
André Laszlo
3793bdc2c3 Add GCP Secret Manager adapter 2024-11-20 14:10:20 +01:00
Lewis Buckley
72f30774ba Support line filtering when running tests 2024-11-20 11:56:58 +00:00
Federico
3fa9cd5a41 Make kamal use ssh keys from config when performing commands 2024-11-19 11:38:42 +01:00
Matthew Jones
c970ceebe3 Adds support for SBOM attestations 2024-11-18 13:01:53 -07:00
Aleksandr Lossenko
79bc7584ca make --account optional and pass Enpass vault in --from 2024-11-14 09:16:23 +01:00
Aleksandr Lossenko
c9dec8c79a no need for open3 anymore 2024-11-14 09:16:23 +01:00
Aleksandr Lossenko
8d7a6403b5 enpass-cli now has JSON support 2024-11-14 09:16:23 +01:00
Aleksandr Lossenko
b356b08069 improve password parsing 2024-11-14 09:16:23 +01:00
Aleksandr Lossenko
4d09f3c242 add more docs 2024-11-14 09:16:23 +01:00
Aleksandr Lossenko
d6c4411e97 add support to enpass 2024-11-14 09:16:23 +01:00
Ralf Schmitz Bongiolo
8dd864af89 refactor(secrets): adapter/test_optional_account inherit from adapter/test 2024-11-05 14:14:18 -04:00
Wojciech Wnętrzak
e4ab2a0d24 Improve error on unknown role in accessories config.
Previously when unknown role (or with typo) was placed in accessories.roles,
this error was thrown: `ERROR (NoMethodError): undefined method `hosts' for nil`.
2024-11-05 14:42:17 +01:00
Ralf Schmitz Bongiolo
3069552315 feat(secrets): update doppler adapter to use --from option and DOPPLER_TOKEN env 2024-11-04 19:00:38 -04:00
Ralf Schmitz Bongiolo
77cd29f5ad feat(cli): update secrets --account flag as optional depending on adapter 2024-11-04 18:59:37 -04:00
Ralf Schmitz Bongiolo
d0d9dfcba9 Merge branch 'basecamp:main' into main 2024-11-04 16:26:12 -04:00
Justin Dell
b4d395cec9 shell escape account name in cli command 2024-11-04 09:46:45 -06:00
Justin Dell
e266945413 implement check_dependencies! 2024-11-04 09:18:56 -06:00
Justin Dell
c9fff3cb40 rename secretsmanager to secrets manager 2024-11-04 09:14:47 -06:00
Justin Dell
cef1e53f84 Merge branch 'basecamp:main' into feat-add-aws-secrets-manager-adapter 2024-11-04 09:06:04 -06:00
Donal McBreen
9cf8da64c4 Merge pull request #1193 from basecamp/filter-by-no-destination
Filter correctly for empty destinations
2024-11-04 11:14:42 +00:00
Donal McBreen
e9ba92386c Filter correctly for empty destinations
An empty destination should only filter container with empty
destination, not pick up all containers.

Fixes: https://github.com/basecamp/kamal/issues/1184
2024-11-04 11:05:24 +00:00
Donal McBreen
685312c9f8 Bump version for 2.3.0 2024-10-31 09:14:29 +00:00
Donal McBreen
ca5e53404b Merge pull request #1175 from basecamp/proxy-0.8.2
Bump proxy minimum version to 0.8.2
2024-10-31 08:13:18 +00:00
Donal McBreen
2c14f48300 Bump proxy minimum version to 0.8.2
Detect event-stream content type properly

See: https://github.com/basecamp/kamal-proxy/releases/tag/v0.8.2
2024-10-30 08:06:52 +00:00
dependabot[bot]
cd4e183213 Bump rexml from 3.3.6 to 3.3.9 in the bundler group across 1 directory (#1173)
Bumps the bundler group with 1 update in the / directory: [rexml](https://github.com/ruby/rexml).


Updates `rexml` from 3.3.6 to 3.3.9
- [Release notes](https://github.com/ruby/rexml/releases)
- [Changelog](https://github.com/ruby/rexml/blob/master/NEWS.md)
- [Commits](https://github.com/ruby/rexml/compare/v3.3.6...v3.3.9)

---
updated-dependencies:
- dependency-name: rexml
  dependency-type: indirect
  dependency-group: bundler
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-29 01:39:47 -07:00
David Stosik
7e8a8eb6e5 Remove trailing spaces from deploy.yml template
Just a minor cleanup, nothing important.
`git` highlighted these spaces in red in my commit so I thought I'd
remove them.
2024-10-27 23:38:45 +09:00
Donal McBreen
2465681408 Merge pull request #1151 from basecamp/net-ssh-7.3.0
Ensure using at least net-ssh 7.3.0
2024-10-25 16:16:18 +01:00
Donal McBreen
b917d7cd40 Merge pull request #1152 from basecamp/skip-log-max-size
Allow log max size to not be set
2024-10-25 08:55:18 +01:00
Donal McBreen
1980a79e73 Update lib/kamal/cli/proxy.rb
Co-authored-by: Sijawusz Pur Rahnama <sija@sija.pl>
2024-10-25 08:10:25 +01:00
Donal McBreen
347eb69350 Merge pull request #994 from honzasterba/bw_fetch_all_fields
[bitwarden] ability to fetch all fields from an item
2024-10-23 16:32:39 +01:00
Donal McBreen
9a8a45015b Allow log max size to not be set
The max-size log opt is not valid for all logging drivers, such as
syslog. Allow the option to be removed from the boot config with:

```
kamal proxy boot_config set --log-max-size=
or
kamal proxy boot_config set --log-max-size=""
```
2024-10-23 15:21:06 +01:00
Donal McBreen
8d0f4903ae Ensure using at least net-ssh 7.3.0
This has support for aes(128|256)gcm ciphers and some fixes for
Ruby 3.3.
2024-10-23 14:58:36 +01:00
Donal McBreen
57d582e3bc Merge pull request #972 from kohkimakimoto/dev-provenance-flag
Add provenance option
2024-10-23 14:07:19 +01:00
Donal McBreen
bf8779cef4 Merge pull request #950 from admtnnr/fix-registry-cache-options
builder/cache/options: fix order of build args when using registry
2024-10-23 13:59:59 +01:00
Jan Sterba
7142534e77 [bitwarden] ability to fetch all fields from an item
Sometimes a projects has a lot of secrets (more than 10). And its
cumbersome to write $(kama secrets fetch ...) with a lot of field
names.

I want to be able to just fetch all the fields from a given item
 and then just use these with $(kamal extract NAME)
2024-10-23 13:28:37 +01:00
Donal McBreen
0f97e0b056 Merge pull request #1114 from alanoliveira/main
prevent escape '#' when generating env_file string
2024-10-23 12:35:07 +01:00
Donal McBreen
bd8c35b194 Merge pull request #1020 from igor-alexandrov/network-args
Allow to override network
2024-10-23 12:22:36 +01:00
Donal McBreen
35075e2e4d Merge pull request #1136 from aidanharan/secrets-files-not-found-message
Updated secrets error message if secrets files do not exist
2024-10-23 11:14:41 +01:00
Donal McBreen
53dad5f54f Merge pull request #1121 from kylerippey/adapter-cli-installation-checks
Raise meaningful error messages when secret adapter CLIs are not installed
2024-10-23 11:12:50 +01:00
Donal McBreen
66f6e8b576 Merge pull request #1045 from junket/allow-false-env-var-value
Allow false env var value
2024-10-23 11:11:43 +01:00
Alan Oliveira
a3f5830728 improve test legibility 2024-10-23 08:06:27 +09:00
Donal McBreen
a3e5505bb2 Merge pull request #1117 from pardeyke/add_proxy_reboot_info
Added `kamal proxy reboot` to raised error message
2024-10-22 17:26:36 +01:00
Donal McBreen
fdf8ef1343 Merge pull request #1144 from basecamp/zeitwerk-2.7
Require zeitwerk 2.6.18
2024-10-22 15:03:45 +01:00
Donal McBreen
3ee45d7b30 Require zeitwerk 2.6.12
We were requiring Zeitwerk 2.5+, but calling eager_load_namespace
which was added in 2.6.2.

Fixes: https://github.com/basecamp/kamal/issues/1109
2024-10-22 13:01:36 +01:00
Justin Dell
6856742eca add secrets adapter for aws secrets manager 2024-10-21 09:19:06 -05:00
Aidan Haran
c320343bb2 Updated secrets error message if secrets files do not exist 2024-10-19 20:01:00 +01:00
dependabot[bot]
74a06b0ccd Bump actionpack in the bundler group across 1 directory (#1127)
Bumps the bundler group with 1 update in the / directory: [actionpack](https://github.com/rails/rails).


Updates `actionpack` from 7.1.3.4 to 7.1.4.1
- [Release notes](https://github.com/rails/rails/releases)
- [Changelog](https://github.com/rails/rails/blob/v7.2.1.1/actionpack/CHANGELOG.md)
- [Commits](https://github.com/rails/rails/compare/v7.1.3.4...v7.1.4.1)

---
updated-dependencies:
- dependency-name: actionpack
  dependency-type: indirect
  dependency-group: bundler
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-16 11:30:14 -07:00
dependabot[bot]
c0ca5e6dbb Bump rexml from 3.3.4 to 3.3.6 in the bundler group across 1 directory (#1126)
Bumps the bundler group with 1 update in the / directory: [rexml](https://github.com/ruby/rexml).


Updates `rexml` from 3.3.4 to 3.3.6
- [Release notes](https://github.com/ruby/rexml/releases)
- [Changelog](https://github.com/ruby/rexml/blob/master/NEWS.md)
- [Commits](https://github.com/ruby/rexml/compare/v3.3.4...v3.3.6)

---
updated-dependencies:
- dependency-name: rexml
  dependency-type: indirect
  dependency-group: bundler
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-16 11:29:59 -07:00
dependabot[bot]
6f08750c3e Bump webrick from 1.8.1 to 1.8.2 in the bundler group across 1 directory (#1125)
Bumps the bundler group with 1 update in the / directory: [webrick](https://github.com/ruby/webrick).


Updates `webrick` from 1.8.1 to 1.8.2
- [Release notes](https://github.com/ruby/webrick/releases)
- [Commits](https://github.com/ruby/webrick/compare/v1.8.1...v1.8.2)

---
updated-dependencies:
- dependency-name: webrick
  dependency-type: indirect
  dependency-group: bundler
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-16 11:29:41 -07:00
Jonas Pardeyke
e362b0106a changed text 2024-10-16 09:08:30 +02:00
Kyle Rippey
8cec17dd05 Made secret adapters raise a meaningful error if the required CLI is not installed 2024-10-15 23:20:18 -07:00
Jonas Pardeyke
0f3786781b added kamal proxy reboot to raised error 2024-10-15 22:47:08 +02:00
Alan Oliveira
844e3acf50 prevent escape '#' when generating env_file string 2024-10-15 14:57:53 +09:00
David Heinemeier Hansson
607368121e Merge pull request #1079 from jjatinggoyal/valkey
Switch Redis to Valkey in deploy template and tests
2024-10-13 19:31:12 +02:00
Puru
0f16ba1995 Upgrade Ruby base image from 3.2.0 to 3.3.x (#1107)
* Upgrade ruby base image to fix HIGH and CRITICAL CVEs
* Float on latest 3.3.x

---------

Co-authored-by: Jeremy Daer <jeremydaer@gmail.com>
2024-10-13 10:07:09 -07:00
Jatin Goyal
f3b8a59133 Use valkey for redis image in deploy template 2024-10-13 22:04:03 +05:30
Igor Alexandrov
b4df51b8b4 Added example how to read the Ruby version from the .ruby-version file. 2024-10-12 21:27:56 +04:00
David Heinemeier Hansson
bf79c7192f Clearer still 2024-10-11 10:40:37 -07:00
David Heinemeier Hansson
cb82767d0f Clarify proxy settings 2024-10-11 10:39:58 -07:00
Ralf Schmitz Bongiolo
3c91a83942 feat(secrets): add Doppler adapter 2024-10-10 21:41:09 -04:00
Donal McBreen
5cb9fb787b Bump version for 2.2.2 2024-10-10 13:29:38 -04:00
Donal McBreen
493c5690f1 Merge pull request #1066 from davidstosik/v2.1.1/path-space-fix
Support spaces in git repository path
2024-10-10 10:17:00 -04:00
Donal McBreen
5de55a22ff Merge pull request #1088 from emmceemoore/patch-1
Typo fix.
2024-10-10 10:16:01 -04:00
Nick Pezza
a1e40f9fec Update to be able to run on 3.4 with frozen strings (#1080)
* Update to be able to run on 3.4 with frozen strings

---------

Co-authored-by: Jeremy Daer <jeremydaer@gmail.com>
Co-authored-by: Sijawusz Pur Rahnama <sija@sija.pl>
2024-10-09 21:11:06 -04:00
Mike Moore
7ddf3bcb02 Typo fix. 2024-10-09 17:34:42 -06:00
Donal McBreen
3654a7e1be Bump version for 2.2.1 2024-10-09 14:46:44 -04:00
Donal McBreen
6a7783c979 Merge pull request #1086 from basecamp/proxy-1.8.1
Bump proxy to version 0.8.1
2024-10-09 14:46:09 -04:00
Donal McBreen
7dc2609b77 Merge pull request #1082 from graysonchen/patch-1
ERROR (ArgumentError): Unknown boot_config subcommand clear
2024-10-09 14:03:07 -04:00
Donal McBreen
74960499c0 Bump proxy to version 0.8.1
Fixes issue where incorrect status code may be returned when buffering
responses.

https://github.com/basecamp/kamal-proxy/releases/tag/v0.8.1
2024-10-09 14:00:38 -04:00
Igor Alexandrov
69b13ebc6a Renamed NETWORK to DEFAULT_NETWORK 2024-10-09 10:00:57 +04:00
Igor Alexandrov
da2a543cbc Reverted network arguments everywhere except to accessory config 2024-10-09 10:00:49 +04:00
Igor Alexandrov
08dacd2745 Added command tests 2024-10-09 09:53:25 +04:00
Igor Alexandrov
b6a10df56a Added tests for network configuration option 2024-10-09 09:53:25 +04:00
Igor Alexandrov
c917dd82cf Added network_args to proxy configuration 2024-10-09 09:53:25 +04:00
Igor Alexandrov
f04cae529a Added network configuration option to application, proxy and accessory sections 2024-10-09 09:53:17 +04:00
Grayson Chen
50c96e36c0 typo clear change to reset
kamal proxy boot_config clear
 
 ERROR (ArgumentError): Unknown boot_config subcommand clear
2024-10-09 00:11:54 +08:00
Donal McBreen
7b48648bf2 Bump version for 2.2.0 2024-10-08 08:59:23 -04:00
Donal McBreen
91df935d05 Merge pull request #1076 from basecamp/active-support-require-for-to-sentence
Add Active Support require for to_sentence
2024-10-08 07:46:53 -04:00
Donal McBreen
bbfcbfa94b Merge pull request #1064 from basecamp/kamal-proxy-0.8.0
Update to kamal-proxy 0.8.0
2024-10-08 07:42:48 -04:00
Donal McBreen
440044b900 Merge pull request #1072 from basecamp/proxy-default-10m-logs
Default to keeping 10m of proxy logs
2024-10-08 07:35:38 -04:00
Donal McBreen
06419f8749 Add Active Support require for to_sentence
Fixes: https://github.com/basecamp/kamal/issues/1061
2024-10-08 07:33:03 -04:00
David Stosik
8d6d7ffed0 s/refute_match/assert_no_match/ 2024-10-08 07:10:08 +09:00
Donal McBreen
67ce1912f7 Default to keeping 10m of proxy logs
Match the defaults for the application containers of 10m of logs.

Allow them to be altered with the proxy boot_config set command.
2024-10-07 16:20:40 -04:00
David Stosik
f45c754e53 Remove unnecessary method 2024-10-07 15:46:04 +09:00
David Stosik
d40057286d Escape more paths and write a test 2024-10-07 15:46:04 +09:00
David Stosik
0840fdf0dd Support spaces in git repository path
See https://github.com/basecamp/kamal/issues/1036
2024-10-07 15:46:04 +09:00
Donal McBreen
a434b10bfd Update to kamal-proxy 0.8.0
Proxy changes:
- Add option to use custom TLS certificates (#17)
- Don't buffer SSE responses (#36)
- Allow routing to wildcard subdomains (#45)

Custom TLS certificates not supported in Kamal itself yet. Buffering
SSE responses and wildcard subdomains will work without any Kamal
changes.
2024-10-06 13:48:00 -04:00
Donal McBreen
e34031f70c Bump version for 2.1.2 2024-10-06 13:40:53 -04:00
Donal McBreen
23898a5197 Merge pull request #1062 from basecamp/skip-proxy-flag-ssl-false
Skip setting the proxy flag when ssl is false
2024-10-06 18:32:45 +01:00
Donal McBreen
1e9c9e9103 Skip setting the proxy flag when ssl is false
Fixes: https://github.com/basecamp/kamal/issues/1037
2024-10-06 13:22:43 -04:00
David Heinemeier Hansson
4b2c9cdc72 Merge pull request #1026 from ehutzelman/patch-1
Update init description for kamal secrets
2024-10-05 01:54:08 +02:00
David Heinemeier Hansson
80191588c2 Merge pull request #1050 from tiramizoo/template-docker-setup
Update sample template for docker setup hook.
2024-10-05 01:46:09 +02:00
David Heinemeier Hansson
5ca806f4d3 Merge pull request #1054 from tuladhar/cloudflare-ssl
Clarify SSL comment when using Cloudflare
2024-10-04 21:03:23 +02:00
Puru
1d04a6644f Clarify SSL comment when using Cloudflare 2024-10-05 00:45:04 +05:45
Wojciech Wnętrzak
950624d667 Update sample template for docker setup hook.
"kamal" network is already created (in v2.0) so the sample code is
no longer accurate.
2024-10-04 09:27:17 +02:00
David Heinemeier Hansson
81f3508507 Bump version for 2.1.1 2024-10-03 11:39:56 -07:00
David Heinemeier Hansson
9a16873f21 Merge pull request #1035 from basecamp/fix-kamal-setup-on-accessory-hosts
Restore kamal setup to accessory hosts
2024-10-03 19:25:33 +02:00
junket
6d1d7a4c82 Updates argumentize test for false values 2024-10-03 10:05:54 -04:00
junket
ccf32c2c1f Pass false values in env vars to docker 2024-10-03 09:31:30 -04:00
David Heinemeier Hansson
e5ca53db6e Use new deploy config so as not to update all other tests 2024-10-02 17:34:13 -07:00
David Heinemeier Hansson
82a436fa02 Rubocop 2024-10-02 17:07:51 -07:00
David Heinemeier Hansson
7be2e7e0ba Test accessory_hosts with roles and without filtering 2024-10-02 17:03:30 -07:00
David Heinemeier Hansson
4f7ebd73a3 Specifics#accessory_hosts was being filtered out by role host check 2024-10-02 16:30:32 -07:00
Donal McBreen
279bda2770 Bump version for 2.1.0 2024-10-02 11:35:45 +01:00
Donal McBreen
aa15fa532a Merge pull request #1024 from jeromedalbert/follow-primary-role
Follow logs on primary role by default
2024-10-02 09:29:16 +01:00
Donal McBreen
276b469c2b Merge pull request #1019 from nickhammond/ENV/destination
Set KAMAL_DESTINATION when loading config
2024-10-02 09:28:01 +01:00
Donal McBreen
c10b3fb07a Merge pull request #908 from basecamp/ignorable-ssh-config
SSH: allow setting `config: false` to ignore local user `~/.ssh/config`
2024-10-02 08:28:57 +01:00
Eric Hutzelman
0ff1450a74 Update init description for kamal secrets
No longer uses .env stub, replace with secrets stub in .kamal directory.
2024-10-01 18:49:08 -05:00
Jerome Dalbert
f47fd13e5b Follow logs on primary role by default
Do not default `kamal app logs -f` to use the hardcoded “web” role;
instead use the primary role, which can be different from “web”.
2024-10-01 13:51:09 -07:00
Adam Tanner
256933f6f3 builder/cache/options: fix order of build args when using registry 2024-10-01 12:27:45 -04:00
Nick Hammond
1d8c40f5d2 Run RC 2024-10-01 08:20:21 -07:00
Nick Hammond
73c78079bc Set KAMAL_DESTINATION when loading config 2024-10-01 08:16:52 -07:00
Donal McBreen
cd12f95a97 Merge pull request #1018 from basecamp/kamal-proxy-deploy-equals-in-args
Use `=` in kamal-proxy deploy command args
2024-10-01 15:51:43 +01:00
Donal McBreen
641e9056b3 Use = in kamal-proxy deploy command args
`=` is required for boolean values and works for all values.
2024-10-01 15:42:12 +01:00
Donal McBreen
b4bcf35f78 Merge pull request #1000 from kpumuk/hosts
Allow specifying multiple hosts for kamal-proxy via an array
2024-09-30 10:26:15 -04:00
Donal McBreen
7f6095c9eb Merge pull request #1009 from basecamp/secrets-print
Add `kamal secrets print` for secret debugging
2024-09-30 09:55:53 -04:00
Donal McBreen
ef1271df47 Merge pull request #1010 from basecamp/kamal-proxy-0-7-0
Added support for ACME `http-01` challenges
2024-09-30 09:55:34 -04:00
Donal McBreen
df1232d90f Added support for ACME http-01 challenges
Update to kamal-proxy 0.7.0 for ACME `http-01` challenge support.
2024-09-30 14:44:28 +01:00
Dmytro Shteflyuk
e75365c8c6 Simplified deploy options for kamal-proxy as it supports multiple --host arguments 2024-09-30 08:09:05 -04:00
Donal McBreen
e441399255 Add kamal secrets print for secret debugging
Dotenv's variable substitution doesn't work the same way as commands run
in the shell. It needs values to be escaped.

```sh
$ cat /tmp/env
SECRETS=$(cat /tmp/json)
SECRETS2=$(echo $SECRETS | jq)
$ cat /tmp/json
\{\ \"foo\"\ :\ \"bar\" \}
$ SECRETS=$(cat /tmp/json)
$ SECRETS2=$(echo $SECRETS | jq)
jq: parse error: Invalid numeric literal at line 1, column 2
$ ruby -e 'require "dotenv"; puts Dotenv.parse("/tmp/env")["SECRETS2"]'
{
  "foo": "bar"
}
```

Since you then can't use the shell to debug, `kamal secrets print` will
allow you to see what the secrets will be set to.
2024-09-30 12:28:29 +01:00
Donal McBreen
af992ce755 Merge pull request #996 from iximiuz/patch-1
Fix git --add safe.directory command in Dockerfile
2024-09-30 04:36:00 -04:00
Donal McBreen
32caf4b148 Merge pull request #995 from honzasterba/bw_nicer_error_message_on_non_field_fetch
[bitwarden] default fetch raises NoMethodError
2024-09-30 04:35:48 -04:00
Donal McBreen
28a02262df Merge pull request #988 from igor-alexandrov/kamal-proxy-remove-no-target
Fixed kamal-proxy remove command
2024-09-30 04:18:12 -04:00
Donal McBreen
b11fb93a6c Merge pull request #971 from igor-alexandrov/add-app-port-example
Added app_port example to the proxy section
2024-09-30 04:16:34 -04:00
Dmytro Shteflyuk
67ad7662ab Simplified proxy hosts validation and documentation, similar to accessory config 2024-09-29 20:56:23 -04:00
Dmytro Shteflyuk
c63ec39f07 Added a test for colliding hosts passed via hosts array 2024-09-29 20:56:23 -04:00
Dmytro Shteflyuk
8df7d7d92d Do not allow both host and hosts for proxy configuration 2024-09-29 20:43:44 -04:00
Dmytro Shteflyuk
1d48a0fb0a Allow specifying multiple hosts for kamal proxy via an array 2024-09-29 20:43:44 -04:00
David Heinemeier Hansson
f331605efa Merge pull request #1004 from kpumuk/grammar
Backporting changes to the documentation committed directly to kamal-site
2024-09-30 02:22:39 +02:00
Dmytro Shteflyuk
7ea995db91 Added a comment to the front matter of the configuration docs about the generator 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
994a8faf6b Re-applied corrections to configuration YAML files that were merged directly into kamal-site 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
6c75fe40df Removed second newline characters after the section title 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
7567cae964 Added empty lines around YAML code fences 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
ecd842ab9b 'Configuration overview' section was moved to overview.md file 2024-09-29 15:33:06 -04:00
Dmytro Shteflyuk
91ae1dd7b9 Removed unused variable from bin/doc 2024-09-29 15:33:06 -04:00
Ivan Velichko
0f815e17e4 Relax the safe.directory requirement
Co-authored-by: Jeremy Daer <jeremydaer@gmail.com>
2024-09-28 18:00:10 +02:00
Ivan Velichko
a310aa8fef Fix git --add safe.directory command in Dockerfile
Upgrading kamal from `v1.8.3` to `v1.9.0` broke my [kamal playground](https://labs.iximiuz.com/playgrounds/kamal):

```
laborant@dev-machine:~/svc-a$ kamal setup
  INFO [34d0def6] Running /usr/bin/env mkdir -p .kamal on 172.16.0.3
  INFO [c34cf833] Running /usr/bin/env mkdir -p .kamal on 172.16.0.4
  INFO [34d0def6] Finished in 0.147 seconds with exit status 0 (successful).
  INFO [c34cf833] Finished in 0.204 seconds with exit status 0 (successful).
Acquiring the deploy lock...
Ensure Docker is installed...
  INFO [413ee426] Running docker -v on 172.16.0.4
  INFO [f1acacba] Running docker -v on 172.16.0.3
  INFO [413ee426] Finished in 0.036 seconds with exit status 0 (successful).
  INFO [f1acacba] Finished in 0.076 seconds with exit status 0 (successful).
Log into image registry...
  INFO [94cff492] Running docker login registry.iximiuz.com -u [REDACTED] -p [REDACTED] on localhost
  INFO [94cff492] Finished in 0.077 seconds with exit status 0 (successful).
  INFO [605c535f] Running docker login registry.iximiuz.com -u [REDACTED] -p [REDACTED] on 172.16.0.4
  INFO [6002b598] Running docker login registry.iximiuz.com -u [REDACTED] -p [REDACTED] on 172.16.0.3
  INFO [605c535f] Finished in 0.083 seconds with exit status 0 (successful).
  INFO [6002b598] Finished in 0.083 seconds with exit status 0 (successful).
Build and push app image...
  INFO [9d172b1e] Running docker --version && docker buildx version on localhost
  INFO [9d172b1e] Finished in 0.059 seconds with exit status 0 (successful).
  INFO Cloning repo into build directory `/tmp/kamal-clones/svc-a-2f65914456263/workdir/`...
  INFO [26fb1bd3] Running /usr/bin/env git -C /tmp/kamal-clones/svc-a-2f65914456263 clone /workdir --recurse-submodules on localhost
 ERROR Error preparing clone: Failed to clone repo: git exit status: 32768
git stdout: Nothing written
git stderr: Cloning into 'workdir'...
fatal: detected dubious ownership in repository at '/workdir/.git'
To add an exception for this directory, call:

        git config --global --add safe.directory /workdir/.git
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
, deleting and retrying...
  INFO Cloning repo into build directory `/tmp/kamal-clones/svc-a-2f65914456263/workdir/`...
  INFO [fd4aac0c] Running /usr/bin/env git -C /tmp/kamal-clones/svc-a-2f65914456263 clone /workdir --recurse-submodules on localhost
  Finished all in 0.3 seconds
Releasing the deploy lock...
  Finished all in 0.6 seconds
  ERROR (SSHKit::Command::Failed): git exit status: 32768
git stdout: Nothing written
git stderr: Cloning into 'workdir'...
fatal: detected dubious ownership in repository at '/workdir/.git'
To add an exception for this directory, call:

        git config --global --add safe.directory /workdir/.git
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

laborant@dev-machine:~/svc-a$ kamal version
2.0.0
```

I checked the [v1.8.3...v1.9.0](https://github.com/basecamp/kamal/compare/v1.8.3...v1.9.0) diff, and couldn't find anything even remotely related to the above error.

Then I checked the `git` versions in kamal `v1.8.3` and `v1.9.0` images:

```
docker run -it --rm --entrypoint sh ghcr.io/basecamp/kamal:v1.8.3 
/workdir # git --version
git version 2.38.5
```

vs.

```
docker run -it --rm --entrypoint sh ghcr.io/basecamp/kamal:v2.0.0 
/workdir # git --version
git version 2.39.5
```

Apparently, something changed in between `2.38.5` and `2.39.5` git releases (likely yet another CVE fix), and the `git config --global --add safe.directory /workdir` stopped working.

Here is the mitigation I currently use, but it's a bit awkward to do it:

```
docker build -t ghcr.io/basecamp/kamal:v2.0.0 - <<EOF
FROM ghcr.io/basecamp/kamal:v2.0.0

RUN git config --global --add safe.directory /workdir/.git
EOF
```

Hence, this PR.

To repro, you can start a [kamal playground](https://labs.iximiuz.com/playgrounds/kamal), then `docker pull ghcr.io/basecamp/kamal:v2.0.0` to override my patched image, and `cd svc-a && kamal setup`.
2024-09-28 14:23:30 +02:00
Jan Sterba
29b02f5c30 [bitwarden] default fetch raises NoMethodError
When fetched item is not a login, Bitwarden adapter raises NoMethodError
because the returned JSON does not have the login.password value.

Add a nicer error message for that case.
2024-09-28 13:24:14 +02:00
Igor Alexandrov
6d63c4e9c6 Fixed example with the proxy status after the application has been removed 2024-09-27 21:11:51 +04:00
Igor Alexandrov
472d163cc7 Assert 404 after app is stopped 2024-09-27 19:15:42 +04:00
Igor Alexandrov
dadac999d7 Fixed kamal-proxy remove call 2024-09-27 17:45:35 +04:00
Donal McBreen
5036f8843f Merge pull request #986 from basecamp/rails-8-requires-ruby-3.2
Rails 8 doesn't support Ruby 3.1
2024-09-27 08:23:32 -04:00
Donal McBreen
14d0396581 Rails 8 doesn't support Ruby 3.1
Remove it from the build matrix as its no longer a supported combination.
2024-09-27 08:13:24 -04:00
Donal McBreen
8c32e6af07 Bump version for 2.0.0 2024-09-26 15:34:24 -04:00
Donal McBreen
a765c501a3 Bump version for 2.0.0.rc4 2024-09-26 07:06:51 -04:00
Donal McBreen
ae990efd02 Merge pull request #978 from basecamp/ignore-ssl-false
Handle ssl: false in proxy config
2024-09-26 11:56:00 +01:00
Donal McBreen
b3a6921118 Handle ssl: false in proxy config
Fixes: https://github.com/basecamp/kamal/issues/956
2024-09-26 06:17:45 -04:00
Donal McBreen
325bf9a797 Merge pull request #975 from basecamp/kamal-proxy-0.6.0
Bump to latest version of kamal-proxy
2024-09-25 23:03:25 +01:00
Donal McBreen
7bdf6cd2e8 Bump to latest version of kamal-proxy 2024-09-25 17:54:38 -04:00
Kohki Makimoto
92d82dd1a7 test: If the provenance is false, output "--provenance false". 2024-09-26 05:50:51 +09:00
Donal McBreen
7633fe0293 Merge pull request #974 from basecamp/proxy-boot-config
Proxy boot config
2024-09-25 20:28:24 +01:00
Donal McBreen
f6851048a6 Proxy boot config
Add commands for managing proxy boot config. Since the proxy can be
shared by multiple applications, the configuration doesn't belong in
`config/deploy.yml`.

Instead you can set the config with:

```
Usage:
  kamal proxy boot_config <set|get|clear>

Options:
      [--publish], [--no-publish], [--skip-publish]   # Publish the proxy ports on the host
                                                      # Default: true
      [--http-port=N]                                 # HTTP port to publish on the host
                                                      # Default: 80
      [--https-port=N]                                # HTTPS port to publish on the host
                                                      # Default: 443
      [--docker-options=option=value option2=value2]  # Docker options to pass to the proxy container
```

By default we boot the proxy with `--publish 80:80 --publish 443:443`.

You can stop it from publishing ports, specify different ports and pass
other docker options.

The config is stored in `.kamal/proxy/options` as arguments to be passed
verbatim to docker run.

Where someone wants to set the options in their application they can do
that by calling `kamal proxy boot_config set` in a pre-deploy hook.

There's an example in the integration tests showing how to use this to
front kamal-proxy with Traefik, using an accessory.
2024-09-25 15:15:26 -04:00
Donal McBreen
f0d7f786fa Traefik should be kamal-proxy in reboot hooks 2024-09-25 14:51:22 -04:00
Donal McBreen
4d8387b1c9 Merge pull request #973 from eroluysal/main
Fix adapter names
2024-09-25 19:48:40 +01:00
eroluysal
0258ac4297 Fix adapter names. 2024-09-25 21:22:59 +03:00
Kohki Makimoto
c17bdba61c add tests 2024-09-25 23:50:05 +09:00
Kohki Makimoto
13328687d1 support the "provenance" option in the "builder" config 2024-09-25 23:24:52 +09:00
Igor Aleksandrov
2b0810d063 Update lib/kamal/cli/templates/deploy.yml
Co-authored-by: Nick Hammond <nick@nickhammond.com>
2024-09-25 17:19:20 +04:00
Igor Alexandrov
098f1855e2 Added back accidentially removed new line 2024-09-25 12:11:45 +04:00
Igor Alexandrov
88351312bf Added app_port example to the proxy section 2024-09-25 11:25:42 +04:00
David Heinemeier Hansson
4a13803119 Bump version for 2.0.0.rc3 2024-09-23 16:48:07 -07:00
David Heinemeier Hansson
bda252835b Merge pull request #966 from basecamp/cleanup-default-templates
Bring default templates up to par with what Rails generates
2024-09-24 01:46:09 +02:00
David Heinemeier Hansson
0f5dfa204f Rearrange one last time 2024-09-23 16:44:54 -07:00
David Heinemeier Hansson
9dde204480 Rearange 2024-09-23 16:30:16 -07:00
David Heinemeier Hansson
b6cd4f8070 Bring default templates up to par with what Rails generates 2024-09-23 14:41:31 -07:00
David Heinemeier Hansson
e71bfcbadd Bump version for 2.0.0.rc2 2024-09-20 15:41:26 -07:00
David Heinemeier Hansson
567309596a Make the skip of timestamps a boolean 2024-09-20 12:50:46 -07:00
David Heinemeier Hansson
b89ec2bf63 Bump version for 2.0.0.rc1 2024-09-20 11:08:45 -07:00
David Heinemeier Hansson
3172adca30 Merge pull request #958 from basecamp/optional-timestamps
Add option to skip timestamps on logging output
2024-09-20 18:01:14 +02:00
David Heinemeier Hansson
04d21f45bb Fix test 2024-09-20 08:45:40 -07:00
David Heinemeier Hansson
eabd57350c Fix tests 2024-09-20 08:33:14 -07:00
David Heinemeier Hansson
487f6f5f53 Fix excess spacing 2024-09-20 08:31:56 -07:00
David Heinemeier Hansson
d98500982d Update tests 2024-09-20 08:19:38 -07:00
David Heinemeier Hansson
8693e968c1 Timestamps now default on for app logs too 2024-09-20 08:17:19 -07:00
David Heinemeier Hansson
6ab5fc9459 Allow timestamps on/off for app logging too 2024-09-20 08:04:28 -07:00
David Heinemeier Hansson
6fc2915884 Merge branch 'main' into optional-timestamps 2024-09-20 07:58:49 -07:00
David Heinemeier Hansson
afa6898a82 Fix pipe 2024-09-20 07:58:38 -07:00
David Heinemeier Hansson
384b36d158 Add option to skip timestamps on logging output
So it is easier to follow live when you are doing debugging, especially
early days app setup when you are the only user.
2024-09-20 07:42:31 -07:00
Donal McBreen
6df169a4fb Doc updates 2024-09-20 15:27:10 +01:00
Donal McBreen
ab109afc52 Merge pull request #957 from basecamp/numeric-timeouts
Response timeout should be a number
2024-09-20 09:50:05 +01:00
Donal McBreen
a6a48c456c Response timeout should be a number
Kamal will append the `s` for the duration when talking to kamal-proxy
so no need to have it in the config.
2024-09-20 09:26:06 +01:00
David Heinemeier Hansson
a4e5dbe5d4 Bump version for 2.0.0.beta2 2024-09-19 11:37:22 -07:00
Donal McBreen
56e90906b1 Merge pull request #954 from basecamp/two-app-integration-test
Integration test two apps
2024-09-19 16:36:45 +01:00
Donal McBreen
6e65968bdc Integration test two apps
Use localhost for app_with_roles and 127.0.0.1 for app. Confirm we can
deploy both and the respond to requests. Ensure the proxy is removed
once both have been removed.
2024-09-19 16:25:09 +01:00
Donal McBreen
85f1e14b97 Merge pull request #953 from basecamp/inherit-env-for-secrets
Avoid setting env via SSHKit
2024-09-19 15:18:31 +01:00
Donal McBreen
2c829a4824 Avoid setting env via SSHKit
SSHKit puts the env in the command, so leaks them in process listings.
2024-09-19 15:09:17 +01:00
Donal McBreen
45a58f7e15 Merge pull request #952 from basecamp/app-exec-in-kamal-network
Run app exec in the kamal network
2024-09-19 14:46:58 +01:00
Donal McBreen
834b343ded Run app exec in the kamal network
All other containers run in the kamal network, so let's add app exec-ed
containers as well.
2024-09-19 14:29:33 +01:00
Donal McBreen
9fe1821cae Merge pull request #951 from basecamp/proxy-config-ownership
Fix /home/kamal-proxy/.config/kamal-proxy ownership
2024-09-19 12:57:10 +01:00
Donal McBreen
1d7c9fec1d Fix /home/kamal-proxy/.config/kamal-proxy ownership
1. Update to kamal-proxy 0.4.0 which creates and chowns
/home/kamal-proxy/.config/kamal-proxy to kamal-proxy
2. Use a docker volume rather than mapping in a directory, so docker
keeps it owned by the correct user
2024-09-19 12:25:57 +01:00
David Heinemeier Hansson
a6b983de06 Bump version for 2.0.0.beta1 2024-09-18 15:33:21 -07:00
Donal McBreen
3ec4ad2ea5 Merge pull request #949 from basecamp/role-proxy-config
Allow role specific proxy config
2024-09-18 18:27:36 +01:00
Donal McBreen
63f854ea18 Add validations for host/ssl roles
Roles with SSL can only have one server.
Two roles with SSL can't use the same host.
2024-09-18 17:42:45 +01:00
Donal McBreen
fd0cdc1ca1 All role specific proxy configuration
By default only the primary role runs the proxy. To disable the proxy
for that role, you can set `proxy: false` under it.

For other roles they default to not running the proxy, but you can
enable it by setting `proxy: true` for the role, or alternatively
setting a proxy configuration.

The proxy configuration will be merged into the root proxy configuration.
2024-09-18 17:25:35 +01:00
Donal McBreen
d218264b69 Doc output fixes 2024-09-18 15:28:26 +01:00
Donal McBreen
684f7ac148 Merge pull request #948 from basecamp/deploy-drain-timeouts
Simplified deploy/drain timeouts
2024-09-18 15:18:00 +01:00
Donal McBreen
8bcd896242 Simplified deploy/drain timeouts
Remove `stop_wait_time` and `readiness_timeout` from the root config
and remove `deploy_timeout` and `drain_timeout` from the proxy config.

Instead we'll just have `deploy_timeout` and `drain_timeout` in the
root config.

For roles that run the proxy, they are passed to the kamal-proxy deploy
command. Once that returns we can assume the container is ready to
shut down.

For other roles, we'll use the `deploy_timeout` when polling the
container to see if it is ready and the `drain_timeout` when stopping
the container.
2024-09-18 15:08:08 +01:00
Donal McBreen
600bbd77ef Merge pull request #947 from basecamp/doc-updates
Update proxy and docs for Kamal 2.0/kamal-proxy 0.3.0
2024-09-18 14:45:32 +01:00
Donal McBreen
34effef70a Update proxy and docs for Kamal 2.0/kamal-proxy 0.3.0
Update to kamal-proxy 0.3.0 and improve docs making sure they are in
sync with that version.
2024-09-18 14:00:43 +01:00
Donal McBreen
e07ac070aa Merge pull request #945 from basecamp/kamal-2-upgrade
Upgrade on accessory hosts only with correct messages
2024-09-18 10:27:28 +01:00
Donal McBreen
46c0836cd4 Upgrade on accessory hosts only with correct messages 2024-09-18 10:07:07 +01:00
Donal McBreen
bd54c74682 Merge pull request #944 from basecamp/single-secrets-instance
Ensure we don't load the secrets more than once
2024-09-17 14:33:38 +01:00
Donal McBreen
f183419f7a Ensure we don't load the secrets more than once
Secrets were being created twice, which might require logging in twice.
2024-09-17 14:21:43 +01:00
Donal McBreen
190dbd1ea3 Merge pull request #943 from basecamp/response-timeout-string
Need a duration string for the response timeout
2024-09-17 14:14:49 +01:00
Donal McBreen
d6eda3d741 Merge pull request #942 from basecamp/kamal-2-upgrade
Upgrade commands for Kamal 1.x -> 2.0
2024-09-17 13:34:00 +01:00
Donal McBreen
0fe6a17a91 Need a duration string for the response timeout
Add `s` as the timeout is a duration.
2024-09-17 13:23:15 +01:00
Donal McBreen
7f15fd143f Upgrade commands for Kamal 1.x -> 2.0
Adds:
- `kamal upgrade` to upgrade all app hosts and accessory hosts
- `kamal proxy upgrade` to upgrade the proxy on all hosts
- `kamal accessory upgrade [name]` to upgrade accessories on all hosts

Upgrade takes rolling and confirmed options and calls `proxy upgrade`
and `accessory upgrade` in turn.

To just upgrade a single host add -h [host] to the command. But the
upgrade should run on all hosts, not just those running the proxy.

Calling upgrade on a host that has already been upgraded should work ok.

Upgrading hosts causes downtime but you can avoid if you run multiple
hosts by:
1. Implementing the pre-proxy-reboot and post-proxy-reboot hooks to
   remove the host from external load balancers
2. Running the upgrade with the --rolling option

**kamal proxy upgrade**
1. Creates a `kamal` network if required
2. Stops and removes the old proxy (whether Traefik or kamal-proxy)
3. Starts a kamal-proxy container in the `kamal` network
4. Reboots the app containers in the `kamal` network

**kamal accessory upgrade [name]**
1. Creates a `kamal` network if required
2. Reboots the accessory containers in the `kamal` network

A matching `downgrade` command will be added to Kamal 1.9.
2024-09-17 13:02:02 +01:00
Donal McBreen
434490bd0c Merge pull request #940 from basecamp/proxy
Replace Traefik with kamal-proxy
2024-09-17 09:47:16 +01:00
Donal McBreen
267b526438 Switch proxy/hosts to proxy/host
The proxy only supports a single host per app for nowm so make the
config match that.
2024-09-16 20:45:09 +01:00
Donal McBreen
1f721739d6 Use version 0.1.0 of kamal-proxy and add minimum version check 2024-09-16 16:44:58 +01:00
Donal McBreen
6c51e596ae Put locks directories in .kamal so they leave no trace when deleted 2024-09-16 16:44:58 +01:00
Donal McBreen
7f31510aec Set hosts via config rather than options 2024-09-16 16:44:58 +01:00
Donal McBreen
e8ff233e81 Fix default log header tests 2024-09-16 16:44:58 +01:00
Donal McBreen
a316e51eda Add user agent to default headers 2024-09-16 16:44:58 +01:00
Donal McBreen
bf91d6c1ca Fix command description 2024-09-16 16:44:58 +01:00
Donal McBreen
a84ee6315f Rename service -> app directory 2024-09-16 16:44:58 +01:00
Donal McBreen
3c39086613 Not experimental 2024-09-16 16:44:58 +01:00
Donal McBreen
8b965b0a31 Handle polling without the healthcheck config 2024-09-16 16:44:58 +01:00
Donal McBreen
d2672c771e Remove redundant call to env remove 2024-09-16 16:44:58 +01:00
Donal McBreen
24031fefb0 Remove proxy only if no apps are installed 2024-09-16 16:44:58 +01:00
Donal McBreen
35fe9c154d Move audits back to run dir so they survive kamal remove 2024-09-16 16:44:58 +01:00
Donal McBreen
b8972a6833 Remove service directory on kamal remove 2024-09-16 16:44:58 +01:00
Donal McBreen
d7d6fa34b0 Use Volume for kamal proxy config volume 2024-09-16 16:44:58 +01:00
Donal McBreen
c21757f747 Move all files on the host under a common directory
This will make running kamal remove simpler, we can just clean up that
directory.
2024-09-16 16:44:58 +01:00
Donal McBreen
cb73c730f9 No need for run_id 2024-09-16 16:44:58 +01:00
Donal McBreen
109339189a Fix up integration app_test.rb 2024-09-16 16:44:58 +01:00
Donal McBreen
33834a266a Drop sleep after container healthy 2024-09-16 16:44:58 +01:00
Donal McBreen
e1016b2469 No need to wait_for_healthy 2024-09-16 16:44:58 +01:00
Donal McBreen
a40b644145 Check that there's no traefik hooks left behind 2024-09-16 16:44:58 +01:00
Donal McBreen
ccb7424197 Remove stray exit! 2024-09-16 16:44:58 +01:00
Donal McBreen
2125327d54 proxy/host -> proxy/hosts 2024-09-16 16:44:58 +01:00
Donal McBreen
f4d309c5cc Rip out Traefik 2024-09-16 16:44:55 +01:00
Donal McBreen
5bca8015bc Map kamal proxy config into .kamal/proxy/config
This will allow us to share files with the proxy via the host.
2024-09-16 16:44:41 +01:00
Donal McBreen
27a7b339a6 Drop run_directory configuration option
We need to drop to be fixed so multiple applications put the config in
the same place.
2024-09-16 16:44:41 +01:00
Donal McBreen
dcd4778dd9 Port -> app_port 2024-09-16 16:44:41 +01:00
Donal McBreen
6f2eaed398 Work out the host and port for the container
Avoid docker inspect:
1. Use the container ID as the host
2. Configure the port, default to 3000
2024-09-16 16:44:41 +01:00
Donal McBreen
e9d480b514 Add the proxy/ssl config and pass on to kamal-proxy 2024-09-16 16:44:41 +01:00
Donal McBreen
2fdc59a3aa Fix tests 2024-09-16 16:44:41 +01:00
Donal McBreen
b33c999125 Remove envify, make proxy booting work with env files 2024-09-16 16:44:41 +01:00
Donal McBreen
2056351c38 Use kamal network for accessories 2024-09-16 16:44:41 +01:00
Donal McBreen
9c2d5f83f7 Boot latest version when upgrading proxy 2024-09-16 16:44:41 +01:00
Donal McBreen
f347ef7e44 Add proxy upgrade command 2024-09-16 16:44:41 +01:00
Donal McBreen
63ebeda489 Create proxy and app containers in a kamal network 2024-09-16 16:44:41 +01:00
Donal McBreen
13bdf50ceb Fix tests for proxy defaults and required builder arch 2024-09-16 16:44:41 +01:00
Donal McBreen
bd6558630f Fix merge error 2024-09-16 16:44:39 +01:00
Donal McBreen
53903ddcd2 Read buffer not buffering 2024-09-16 16:44:21 +01:00
Donal McBreen
55756fa6f3 Set request and response headers 2024-09-16 16:44:21 +01:00
Donal McBreen
fe0c656de5 Split buffer requests/responses 2024-09-16 16:44:21 +01:00
Donal McBreen
418d8045d8 Add forward headers support 2024-09-16 16:44:21 +01:00
Donal McBreen
d63ff8f251 Set extra fields 2024-09-16 16:44:21 +01:00
Donal McBreen
eab717e0cf Add kamal-proxy in experimental mode
The proxy can be enabled via the config:

```
proxy:
  enabled: true
  hosts:
    - 10.0.0.1
    - 10.0.0.2
```

This will enable the proxy and cause it to be run on the hosts listed
under `hosts`, after running `kamal proxy reboot`.

Enabling the proxy disables `kamal traefik` commands and replaces them
with `kamal proxy` ones. However only the marked hosts will run the
kamal-proxy container, the rest will run Traefik as before.
2024-09-16 16:44:19 +01:00
Donal McBreen
66d5e25834 Merge pull request #939 from basecamp/secrets-mutex
Add a mutex around loading secrets
2024-09-16 16:26:58 +01:00
Donal McBreen
6bbbd81da1 Add a mutex around loading secrets
Loading secrets may ask for use input, so we need to ensure only one
thread does it at a time.
2024-09-16 14:44:39 +01:00
Donal McBreen
876eebc7c5 Merge pull request #936 from basecamp/asset-extraction-entrypoint
Override the entrypoint when extracting assets
2024-09-13 17:46:43 +01:00
Donal McBreen
dc1bbac3c8 Override the entrypoint when extracting assets
When overriding the command, docker will still run the entrypoint. We
want to avoid that here - we just want to get the assets out as quickly
as possible. Otherwise maybe something important is going on when we
stop the container.
2024-09-12 19:31:18 +01:00
Donal McBreen
045aa7d167 Merge pull request #934 from basecamp/secrets-dont-exit
Don't exit from failed secrets commands
2024-09-11 16:27:33 +01:00
Donal McBreen
0660895e75 Don't exit from failed secrets commands
We can let the exception bubble up. We'll still get an error message and
it ensures that any cleanup we need is done (i.e. releasing deploy locks).
2024-09-11 16:10:52 +01:00
Donal McBreen
debdf00cca Merge pull request #933 from basecamp/common-secrets
Add secrets-common for shared secrets
2024-09-11 14:20:01 +01:00
Donal McBreen
9089c41f30 Add secrets-common for shared secrets
Add a shared secrets file used across all destinations. Useful for
things Github tokens or registry passwords.

The secrets are added to a new file called `secrets-common` to highlight
they are shared, and to avoid acciedentally inheriting a secret from the
`secrets` file to `secrets.destination`.
2024-09-11 13:41:36 +01:00
Donal McBreen
c9946808b1 Merge pull request #931 from basecamp/dont-git-ignore-dot-kamal-secrets
Don't git ignore .kamal/secrets
2024-09-11 13:26:07 +01:00
Donal McBreen
deb2a6d298 Merge pull request #930 from basecamp/hide-1password-login-error
Hide the 1password login error
2024-09-11 12:18:09 +01:00
Donal McBreen
0cb69a84f5 Don't git ignore .kamal/secrets
Secrets should be interpolated at runtime so we do want the file in git.

But add a warning at the top to avoid adding secrets or git ignore the
file if you do.

Also provide examples of the three options for interpolating secrets.
2024-09-11 12:16:18 +01:00
Donal McBreen
aa630f156a Hide the 1password login error
Avoid outputting this login error message, it wasn't an error and you
don't need to follow those instructions.

```
[ERROR] 2024/09/11 11:57:08 You are not currently signed in. Please run `op signin --help` for instructions
```
2024-09-11 12:02:53 +01:00
Donal McBreen
63d0b5ddfa Merge pull request #928 from basecamp/kamal-secrets-inline-aware
Make the secrets commands inline aware
2024-09-10 11:08:10 +01:00
Donal McBreen
06f4caa866 Make the secrets commands inline aware
Rather than redirecting the global $stdout, which is not never clever in
a threaded program, we'll make the secrets commands aware they are
being inlined, so they return the value instead of printing it.

Additionally we no longer need to interrupt the parent process on error
as we've inlined the command - exit 1 is enough.
2024-09-10 10:39:44 +01:00
Donal McBreen
5aa3d1aeb0 Merge pull request #927 from basecamp/revert-903-integration-test-insecure-registry
Revert "Integration test insecure registry"
2024-09-10 10:12:57 +01:00
Donal McBreen
a4d668cd39 Revert "Integration test insecure registry" 2024-09-10 10:02:10 +01:00
Donal McBreen
7156c80f34 Merge pull request #924 from basecamp/secrets
Secrets
2024-09-09 15:13:35 +01:00
Donal McBreen
aed2ef99d0 Use env files for secrets
Add env files back in for secrets - hides them from process lists and
allows you to pick up the latest env file when running
`kamal app exec` without reusing.
2024-09-09 14:43:12 +01:00
Donal McBreen
57cbf7cdb5 Inline dotenv kamal secrets calls 2024-09-06 16:56:54 +01:00
Donal McBreen
b99c044327 Update lib/kamal/cli/templates/secrets
Co-authored-by: Sijawusz Pur Rahnama <sija@sija.pl>
2024-09-06 13:25:39 +01:00
Donal McBreen
8ad6a0ed16 Add .kamal/secrets on kamal init 2024-09-06 11:54:12 +01:00
Donal McBreen
8b62e2694a Test non-ascii secret interpolation 2024-09-05 10:01:56 +01:00
Donal McBreen
be1df4356a fetch_from_vault -> fetch_secrets 2024-09-05 09:53:33 +01:00
Donal McBreen
8210e8e768 Drop redundant rescue 2024-09-05 09:53:18 +01:00
Donal McBreen
9b96ef2412 Shellescape command input 2024-09-05 08:37:50 +01:00
Donal McBreen
1522d94ac9 Pass secrets to pre/post deploy hooks 2024-09-04 16:24:10 +01:00
Donal McBreen
a68294c384 Remote test adapter from test_helper.rb 2024-09-04 12:57:25 +01:00
Donal McBreen
31a347c285 Move int parent comment 2024-09-04 12:52:30 +01:00
Donal McBreen
3d502ab12d Add test adapter and interpolate secrets in integration tests 2024-09-04 12:40:27 +01:00
Donal McBreen
5226d52f8a Interrupting parent on error 2024-09-04 12:14:47 +01:00
Donal McBreen
9deb8af4a0 Don't hide command 2024-09-04 09:32:45 +01:00
Donal McBreen
068aaa0bd0 Fix options 2024-09-04 09:32:45 +01:00
Donal McBreen
a726a86a17 Add lastpass, bitwarden adapters 2024-09-04 09:32:45 +01:00
Donal McBreen
b2e1a4d4c1 Secrets test 2024-09-04 09:32:45 +01:00
Donal McBreen
9ade79fc84 OnePassword, LastPass + Bitwarden adapters 2024-09-04 09:32:45 +01:00
Donal McBreen
79731da619 Single fetch command 2024-09-04 09:32:45 +01:00
Donal McBreen
0ae8046905 Add secret tests 2024-09-04 09:32:45 +01:00
Donal McBreen
d5ecca0fd4 Add tests 2024-09-04 09:32:45 +01:00
Donal McBreen
0c6a593554 Remove redundant test 2024-09-04 09:32:45 +01:00
Donal McBreen
3f37fea7c3 Configuration::Secrets -> Secrets 2024-09-04 09:32:45 +01:00
Donal McBreen
7daaabd4d4 One file, no destination env 2024-09-04 09:32:45 +01:00
Donal McBreen
fcdef5fa06 Set KAMAL_DESTINATION for dotenv parsing 2024-09-04 09:32:45 +01:00
Donal McBreen
5480b40ba3 Correct secret files order 2024-09-04 09:32:45 +01:00
Donal McBreen
1d0e81b00a Eager load only CLI for faster commands 2024-09-04 09:32:45 +01:00
Donal McBreen
5910249d02 Add secrets command + 1password integration 2024-09-04 09:32:45 +01:00
Donal McBreen
b464c4fd4a Include dotenv upgrade 2024-09-04 09:32:45 +01:00
Donal McBreen
56754fe40c Lazily load secrets whenever needed 2024-09-04 09:32:45 +01:00
Donal McBreen
6a06efc9d9 Strip out env loading, envify, env push 2024-09-04 09:32:45 +01:00
Donal McBreen
5c4c33e0a8 Replace .env* with .kamal/env*
By default look for the env file in .kamal/env to avoid clashes with
other tools using .env.

For now we'll still load .env and issue a deprecation warning, but in
future we'll stop reading those.
2024-09-04 09:32:45 +01:00
Donal McBreen
0b5506f6f2 Merge pull request #923 from basecamp/disable-local-builder
Allow disabling of local builds
2024-09-03 14:53:23 +01:00
Donal McBreen
a2549b1f60 Allow disabling of local builds
To disable local builds set:
```
builder:
   local: false
   remote: ssh://docker@docker-builder
```
2024-09-03 14:33:25 +01:00
Donal McBreen
9b9e60ec7f Merge pull request #921 from basecamp/remote-hybrid-builders-cleanup
Build and clean remote builders correctly
2024-09-02 15:24:28 +01:00
Donal McBreen
e557eea79c Build and clean remote builders correctly
Check that the builder and context match what we expect, and if not
remove and re-create them.
2024-09-02 15:12:19 +01:00
David Heinemeier Hansson
d7e785cd36 Merge pull request #920 from mblayman/env-docs-typos
Fix typos in "Environment variables" docs.
2024-09-01 14:22:15 -07:00
Matt Layman
5cda3086c4 Found a typo in the healthcheck docs. 2024-08-31 23:38:28 -04:00
Matt Layman
362f5d00f6 Fix typos in "Environment variables" docs. 2024-08-31 23:29:30 -04:00
Donal McBreen
6adf3c117f Merge pull request #905 from basecamp/simplify-builders-config
Simplify builders config
2024-08-29 09:28:51 +01:00
Donal McBreen
9f0b10425c Fix aliases tests 2024-08-29 09:16:07 +01:00
Donal McBreen
5f2384f123 Use docker info to get arch 2024-08-29 08:46:18 +01:00
Donal McBreen
eab7d3adc5 Keep buildx build, in case of old docker versions which don't default to buildkit 2024-08-29 08:45:51 +01:00
Donal McBreen
d2d0223c37 Require an arch to be set, and default to amd64 in the template 2024-08-29 08:45:51 +01:00
Donal McBreen
56268d724d Simplify the builders configuration
1. Add driver as an option, defaulting to `docker-container`. For a
   "native" build you can set it to `docker`
2. Set arch as a array of architectures to build for, defaulting to
   `[ "amd64", "arm64" ]` unless you are using the docker driver in
   which case we default to not setting a platform
3. Remote is now just a connection string for the remote builder
4. If remote is set, we only use it for non-local arches, if we are
   only building for the local arch, we'll ignore it.

Examples:

On arm64, build for arm64 locally, amd64 remotely or
On amd64, build for amd64 locally, arm64 remotely:

```yaml
builder:
  remote: ssh://docker@docker-builder
```

On arm64, build amd64 on remote,
On amd64 build locally:

```yaml
builder:
  arch:
    - amd64
  remote:
    host: ssh://docker@docker-builder
```

Build amd64 on local:

```yaml
builder:
  arch:
    - amd64
```

Use docker driver, building for local arch:

```yaml
builder:
  driver: docker
```
2024-08-29 08:45:48 +01:00
Donal McBreen
cffb6c3d7e Allow the driver to be set 2024-08-29 08:44:11 +01:00
Donal McBreen
bd1726f305 docker buildx build -> docker build 2024-08-29 08:44:11 +01:00
Donal McBreen
7ddb122a22 Get tests passing 2024-08-29 08:44:11 +01:00
Donal McBreen
98c951bbdb Simplfy choosing a builder 2024-08-29 08:44:11 +01:00
Donal McBreen
374c117b79 Validate multiarch configuration
Remote and local are only allowed when multiarch is enabled.
Remote requires a host and arch, local only requires an arch.
2024-08-29 08:44:11 +01:00
Donal McBreen
d6a5cf3c78 Rip out context_hosts checks
The remote host is now encoded in the builder name so we don't need
to check it. We'll just do an inspect to confirm the builder exists.
2024-08-29 08:44:11 +01:00
Donal McBreen
2aeabda455 Move multiarch remote builder to hybrid builder
Include the host name in the builder name, so we can have one builder
per host/arch across all kamal projects.

Inherit from the remote builder. The difference in the hybrid builder
is that we create a local buildx instance and append the remote context
to it.
2024-08-29 08:44:11 +01:00
Donal McBreen
c048c097ed Create a context for local builds
This ensures we use the docker-container driver and not whatever the
local default is.
2024-08-29 08:44:11 +01:00
Donal McBreen
ed148628fb Local build doesn't need a builder 2024-08-29 08:44:11 +01:00
Donal McBreen
d48080c772 Dump native builder
We already ensure that buildx is installed, so let's always use it.
2024-08-29 08:44:11 +01:00
Donal McBreen
3f64338929 Move native remote to just remote
It's just a remote builder, that will build whichever platform is asked
for, so let's remove the "native" part.

We'll also remove the service name from the builder name, so multiple
services can share the same builder.
2024-08-29 08:44:11 +01:00
Donal McBreen
0ab838bc25 Combine multiarch and native/cache builders
Combine the two builders, as they are almost identical. The only
difference was whether the platforms were set.

The native cached builder wasn't using the context it created, so now
we do.

We'll set the driver to `docker-container` - it seems to be the default
but the Docker docs claim it is `docker`.
2024-08-29 08:44:11 +01:00
Donal McBreen
b7382ceeaf Merge pull request #912 from basecamp/alias
Add aliases to Kamal
2024-08-29 08:43:35 +01:00
Donal McBreen
69367fbc6b Merge pull request #917 from basecamp/v2.0-alpha
Switch the version on main to 2.0.0.alpha
2024-08-29 08:43:19 +01:00
Donal McBreen
2515bd705c Switch the version on main to 2.0.0.alpha
All development is now for the 2.0.0 release.
2024-08-29 08:33:21 +01:00
Donal McBreen
579e169be2 Allow multiple arguments for exec commands
If you can have an alias like:

```
aliases:
  rails: app exec -p rails
```

Then `kamal rails db:migrate:status` will execute
`kamal app exec -p rails db:migrate:status`.

So this works, we'll allow multiple arguments `app exec` and
`server exec` to accept multiple arguments.

The arguments are combined by simply joining them with a space. This
means that these are equivalent:

```
kamal app exec -p rails db:migrate:status
kamal app exec -p "rails db:migrate:status"
```

If you want to pass an argument with spaces, you'll need to quote it:

```
kamal app exec -p "git commit -am \"My comment\""
kamal app exec -p git commit -am "\"My comment\""
```
2024-08-28 10:58:25 +01:00
Donal McBreen
d6f5da92be Bump version for 1.8.2 2024-08-28 09:43:06 +01:00
Donal McBreen
9ccfe20b10 Fix up tests 2024-08-26 11:20:26 +01:00
Donal McBreen
e871d347d5 Merge pull request #889 from xiaohui-zhangxh/git-clone-update-submodules
git clone with --recurse-submodules
2024-08-26 11:20:05 +01:00
Donal McBreen
b8af719bb7 Add aliases to Kamal
Aliases are defined in the configuration file under the `aliases` key.

The configuration is a map of alias name to command. When we run the
command the we just do a literal replacement of the alias with the
string.

So if we have:

```yaml
aliases:
  console: app exec -r console -i --reuse "rails console"
```

Then running `kamal console -r workers` will run the command

```sh
$ kamal app exec -r console -i --reuse "rails console" -r workers
```

Because of the order Thor parses the arguments, this allows us to
override the role from the alias command.

There might be cases where we need to munge the command a bit more but
that would involve getting into Thor command parsing internals,
which are complicated and possibly subject to change.

There's a chance that your aliases could conflict with future built-in
commands, but there's not likely to be many of those and if it happens
you'll get a validation error when you upgrade.

Thanks to @dhnaranjo for the idea!
2024-08-26 10:47:43 +01:00
Jeremy Daer
190f4fba28 SSH: allow setting config: false to ignore local user ~/.ssh/config 2024-08-13 12:46:50 -07:00
Donal McBreen
f48987aa03 Merge pull request #903 from basecamp/integration-test-insecure-registry
Integration test insecure registry
2024-08-01 09:57:17 +01:00
Donal McBreen
ef051eca1b Merge pull request #904 from galori/main
Fixed typo in `env.yml`: "valies" --> "values"
2024-08-01 09:57:03 +01:00
Gall Steinitz
173d44ee0a fixed typo in env.yml: valies --> values 2024-07-31 22:12:21 -07:00
Donal McBreen
4e811372f8 Integration test insecure registry
The integrations tests use their own registry so avoid hitting docker
hub rate limits.

This was using a self signed certificate but instead use
`--insecure-registry` to let the docker daemon use HTTP.
2024-07-31 16:54:00 +01:00
Donal McBreen
ec4aa45852 Bump version for 1.8.1 2024-07-29 09:09:57 +01:00
Donal McBreen
5e11a64181 Merge pull request #891 from basecamp/single-pull
Pull once from hosts that warm registry mirrors
2024-07-22 08:18:48 +01:00
Jeremy Daer
57d9ce177a Pull once from hosts that warm registry mirrors 2024-07-18 09:14:22 -07:00
xiaohui
b12de87388 git clone with --recurse-submodules 2024-07-17 10:36:58 +08:00
Donal McBreen
8a98949634 Merge pull request #886 from guoard/patch-2
Remove `--update` flag from `apk add` command
2024-07-16 15:46:37 +01:00
Donal McBreen
0eb9f48082 Merge pull request #887 from basecamp/fix-tests-with-git-config
Fix the tests when you have a git config email set
2024-07-16 13:08:18 +01:00
Donal McBreen
9db6fc0704 Fix the tests when you have a git config email set
The ran ok on CI where we fall back to `whoami`, but failed locally
where there was a git email set.
2024-07-16 12:09:05 +01:00
Donal McBreen
27fede3caa Merge pull request #884 from basecamp/x-config
Add support for configuration extensions
2024-07-16 11:38:28 +01:00
Donal McBreen
29c723f7ec Add support for configuration extensions
Allow blocks prefixed with `x-` in the configuration as a place to
declare reusable blocks with YAML anchors and aliases.

Borrowed from the Docker Compose configuration file format -
https://github.com/compose-spec/compose-spec/blob/main/spec.md#extension

Thanks to @ruyrocha for the suggestion.
2024-07-15 20:47:55 +01:00
Ali Afsharzadeh
2755582c47 Remove --update flag from apk add command 2024-07-15 22:15:25 +03:30
Donal McBreen
fa73d722ea Bump version for 1.8.0 2024-07-15 14:21:23 +01:00
Donal McBreen
c535e4e44f Merge pull request #883 from basecamp/revert-840-main
Revert "Add x25519 gem, support Curve25519"
2024-07-15 13:56:49 +01:00
Donal McBreen
0ea07b1760 Merge pull request #878 from pagbrl/main
feat: Use git email as performer when available
2024-07-15 13:41:17 +01:00
Donal McBreen
03b531f179 Merge pull request #865 from basecamp/clean-envify-env
Ensure envify templates aren't polluted by existing env
2024-07-15 13:41:03 +01:00
Donal McBreen
d8570d1c2c Merge pull request #847 from basecamp/remove-ruby-2.7-from-ci
Remove Ruby 2.7 from CI
2024-07-15 13:40:37 +01:00
Donal McBreen
3fe70b458d Merge pull request #862 from jeromedalbert/bump-sshkit
Bump sshkit to support unbracketed IPv6 addresses
2024-07-15 13:40:18 +01:00
Donal McBreen
ade8b43599 Merge pull request #866 from acidtib/ssh-key-overwrite
Configurable SSH Identity
2024-07-15 13:39:51 +01:00
Donal McBreen
d24fc3ca4e Revert "Add x25519 gem, support Curve25519" 2024-07-15 13:36:50 +01:00
Donal McBreen
7c244bbb98 Merge pull request #879 from basecamp/seed-mirror
Seed docker mirrors by pulling once per mirror first
2024-07-15 13:30:53 +01:00
Donal McBreen
1369c46a83 Seed docker mirrors by pulling once per mirror first
Find the first registry mirror on each host. If we find any, pull the
images on one host per mirror, then do the remainder concurrently.

The initial pulls will seed the mirrors ensuring that we pull the image
from Docker Hub once each.

This works best if there is only one mirror on each host.
2024-07-11 16:20:37 +01:00
Paul Gabriel
deccf1cfaf feat: Use git email as performer when available 2024-07-11 11:19:44 +02:00
Donal McBreen
1573cebadf Merge pull request #868 from nickhammond/env/service
Add ENV['KAMAL_SERVICE'] to hooks
2024-07-10 10:26:59 +01:00
Nick Hammond
85a2926cde Remove the deprecated docker compose version (#869) 2024-06-28 15:00:23 -07:00
Nick Hammond
58a51b079e Add KAMAL_SERVICE to custom hooks and exclude from auditor 2024-06-27 10:52:55 -06:00
Nick Hammond
f1f3fc566f Add ENV['SERVICE'] to hooks 2024-06-27 10:26:11 -06:00
acidtib
44726ff65a overwrite ssh identity 2024-06-26 17:14:13 -06:00
Jerome Dalbert
fd0d4af21f Bump sshkit to support unbracketed IPv6 addresses
Set sshkit minimum version to 1.23.0, which includes an enhancement to
support unbracketed IPv6 addresses.

See https://github.com/capistrano/sshkit/pull/538
2024-06-25 12:17:40 -07:00
Jeremy Daer
13409ada5a Ensure envify templates aren't polluted by existing env
Setting `GITHUB_TOKEN` as in the docs results in reusing the existing
`GITHUB_TOKEN` since `gh` returns that env var if it's set:
```bash
GITHUB_TOKEN=junk gh config get -h github.com oauth_token
junk
```

Using the original env ensures that the templates will be evaluated the
same way regardless of whether envify had been previously invoked.
2024-06-25 11:14:34 -07:00
Donal McBreen
9a1379be6c Bump version for 1.7.3 2024-06-25 15:03:02 +01:00
Donal McBreen
31d6c198da Merge pull request #861 from K4sku/update-docker-setup-sample-hook
Expand on docker-setup.sample hook
2024-06-25 14:44:13 +01:00
Donal McBreen
22afe4de77 Merge pull request #864 from basecamp/allow-arrays-in-args
Allow arrays in args
2024-06-25 14:41:07 +01:00
Donal McBreen
b63982c3a7 Allow arrays in args
Just check that args is a Hash without checking the value types.

Fixes: https://github.com/basecamp/kamal/issues/863
2024-06-25 14:18:23 +01:00
Cezary Kłos
9e12d32cc3 Expand on docker-setup.sample script so it creates docker network "kamal" on each of the defined hosts. 2024-06-24 12:45:56 +02:00
Donal McBreen
ff03891d47 Bump version for 1.7.2 2024-06-24 10:11:27 +01:00
Donal McBreen
f21dc30875 Merge pull request #858 from basecamp/match-does-not-exist
Match a "does not exist" error message
2024-06-24 09:54:25 +01:00
Donal McBreen
69fa7286e2 Match a "does not exist" error message
Only show the warning for building when we are actually going to do that
and match `does not exist` in the error message.

Fixes: https://github.com/basecamp/kamal/issues/851
2024-06-24 08:21:03 +01:00
Donal McBreen
e160852e4d Remove Ruby 2.7 from CI
It's EOL since March 2023.
2024-06-20 08:54:55 +01:00
Donal McBreen
4697f89441 Bump version for 1.7.1 2024-06-20 08:50:37 +01:00
Donal McBreen
dde637ffff Merge pull request #846 from basecamp/always-log-boot-errors
Log on boot errors with one role
2024-06-20 08:50:03 +01:00
Donal McBreen
f8f88af534 Log on boot errors with one role
We didn't log boot errors if there was one role because there was no
barrier and the logging is done by the first host to close the barrier.

Let's always create the barrier to fix this.
2024-06-20 08:28:37 +01:00
Donal McBreen
f6a9698f55 Merge pull request #845 from basecamp/revert-815-envify-already-pushes-env
Revert "Envify already env pushes"
2024-06-20 08:22:06 +01:00
Donal McBreen
3da7fad9ee Revert "Envify already env pushes" 2024-06-20 08:11:18 +01:00
Donal McBreen
1109a864d0 Bump version for 1.7.0 2024-06-18 10:33:02 +01:00
Donal McBreen
da599d90c1 Merge pull request #828 from basecamp/configuration-validation
Configuration validation
2024-06-18 08:31:47 +01:00
Donal McBreen
6bf3f4888a Allow aliases still 2024-06-18 08:20:27 +01:00
Donal McBreen
0a6b0b7133 Merge pull request #840 from HLFH/main
Add x25519 gem, support Curve25519
2024-06-18 08:17:48 +01:00
Gaspard d'Hautefeuille
6d6670a221 Add x25519 gem, support Curve25519
Fixes:

```
  ERROR (Net::SSH::Exception): Exception while executing on host example.com: could not settle on kex algorithm
Server kex preferences: curve25519-sha256@libssh.org,ext-info-s,kex-strict-s-v00@openssh.com
Client kex preferences: ecdh-sha2-nistp521,ecdh-sha2-nistp384,ecdh-sha2-nistp256,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1```

add x25519 in Gemfile.lock
2024-06-15 13:47:10 +02:00
Donal McBreen
10e3229d7c Merge pull request #817 from nickhammond/grep-context
Add grep options to log commands
2024-06-13 14:38:54 +01:00
Nick Hammond
c7bd377fa5 Swap grep context with grep options 2024-06-06 09:26:12 -07:00
Donal McBreen
bdd951b756 Merge pull request #832 from basecamp/registry-skips
Allow registry commands to skip local and remote
2024-06-06 08:12:15 +01:00
Donal McBreen
080897dc4d Merge pull request #831 from basecamp/check-buildx-contexts
Check that we have valid contexts before building
2024-06-06 08:12:05 +01:00
Donal McBreen
d652221100 Merge pull request #818 from fabiosammy/fix-header-docker-setup-template
Fix the header template of the docker-setup hook
2024-06-05 12:18:59 +01:00
Donal McBreen
00e0e5073e Allow registry commands to skip local and remote
- Add local logout to `kamal registry logout`
- Add `skip_local` and `skip_remote` options to `kamal registry` commands
- Skip local login in `kamal deploy` when `--skip-push` is used
2024-06-05 12:10:36 +01:00
Donal McBreen
b52e66814a Check that we have valid contexts before building
Load the hosts from the contexts before trying to build.

If there is no context, we'll create one. If there is one but the hosts
don't match we'll re-create.

Where we just have a local context, there won't be any hosts but we
still inspect the builder to check that it exists.
2024-06-05 11:52:45 +01:00
Donal McBreen
29fbe7a98f Remove redundant Kamal::Configuration:: 2024-06-04 16:45:39 +01:00
Donal McBreen
4f317b8499 Configuration validation
Validate the Kamal configuration giving useful warning on errors.
Each section of the configuration has its own config class and a YAML
file containing documented example configuration.

You can run `kamal docs` to see the example configuration, and
`kamal docs <section>` to see the example configuration for a specific
section.

The validation matches the configuration to the example configuration
checking that there are no unknown keys and that the values are of
matching types.

Where there is more complex validation - e.g for envs and servers, we
have custom validators that implement those rules.

Additonally the configuration examples are used to generate the
configuration documentation in the kamal-site repo.

You generate them by running:

```
bundle exec bin/docs <kamal-site-checkout>
```
2024-06-04 14:19:29 +01:00
Donal McBreen
6e60ab918a Bump version for 1.6.0 2024-06-03 08:34:12 +01:00
Donal McBreen
90ecb6a12a Merge pull request #821 from basecamp/retry-clone
Handle corrupt git clones
2024-05-28 15:23:36 +01:00
Donal McBreen
2c2053558a Handle corrupt git clones
When cloning the git repo:
1. Try to clone
2. If there's already a build directory reset it
3. Check the clone is valid

If anything goes wrong during that process:
1. Delete the clone directory
2. Clone it again
3. Check the clone is valid

Raise any errors after that
2024-05-27 11:17:34 +01:00
fabiosammy
beac539d8c Fix the header template of the docker-setup hook 2024-05-24 13:25:01 -03:00
Nick Hammond
eb79d93139 Run RC 2024-05-24 09:16:14 -07:00
Nick Hammond
89994c8b20 Add grep's context option to show lines before and after a match 2024-05-24 08:59:33 -07:00
Donal McBreen
10b8c826d8 Merge pull request #755 from basecamp/lock-less
Lock less
2024-05-21 12:33:45 +01:00
Donal McBreen
187861fa60 Space not tab 2024-05-21 12:20:19 +01:00
Donal McBreen
5ff1203c80 Always lock before pre-deploy hook 2024-05-21 12:02:16 +01:00
Donal McBreen
0e73f02743 Split lock and connection setup
Allow run the pre-connect hook before the first SSH command is executed,
but only run the locking in `with_lock` blocks.
2024-05-21 12:02:16 +01:00
Donal McBreen
83d0078525 Confirm outside mutating 2024-05-21 12:02:16 +01:00
Donal McBreen
96ef0fbc4d Fix merge error 2024-05-21 12:02:16 +01:00
Donal McBreen
b12654ccd0 Don't lock until confirmed 2024-05-21 12:02:16 +01:00
Donal McBreen
64f5955444 Don't hold lock on error 2024-05-21 12:02:12 +01:00
Donal McBreen
d2a719998a Building doesn't need a deploy lock 2024-05-21 12:01:08 +01:00
Donal McBreen
6a7c90cf4d Only stopping containers locks 2024-05-21 12:01:07 +01:00
Donal McBreen
2c2d94c6d9 Merge pull request #740 from basecamp/remove-healthcheck-step
Remove the healthcheck step
2024-05-21 12:00:25 +01:00
Donal McBreen
c62bd1dc31 Merge pull request #815 from basecamp/envify-already-pushes-env
Envify already env pushes
2024-05-21 11:57:55 +01:00
Donal McBreen
a83df9e135 Merge pull request #620 from mlitwiniuk/allow_custom_ports_for_ssh
Allow custom user and port for builder host
2024-05-21 11:49:06 +01:00
Donal McBreen
7b55f4734e Envify already env pushes
`kamal envify` will do `kamal env push` for us, so no need to call it
ourselves during setup.
2024-05-21 11:47:51 +01:00
Donal McBreen
1e296c4140 Update sshkit_with_ext.rb 2024-05-21 11:38:30 +01:00
Donal McBreen
9700e2b3c4 Merge pull request #646 from nickhammond/server/exec
Add in a server exec command for running ad-hoc commands directly on the server
2024-05-21 11:34:20 +01:00
Donal McBreen
706b82baa1 Simplify messages and remove multiple execute error 2024-05-21 10:40:01 +01:00
Donal McBreen
fa7e941648 Make SSHKit::Runner::Parallel fail slow
Using a different SSHKit runner doesn't work well, because the group
runner uses the Parallel runner internally. So instead we'll patch its
behaviour to fail slow.

We'll also get it to return all the errors so we can report on all the
hosts that failed.
2024-05-21 09:33:22 +01:00
Donal McBreen
78c0a0ba4b Don't start other roles we have a healthy container
If a primary role container is unhealthy, we might take a while to
timeout the health check poller. In the meantime if we have started the
other roles, they'll be running tow containers.

This could be a problem, especially if they read run jobs as that
doubles the worker capacity which could cause exessive load.

We'll wait for the first primary role container to boot successfully
before starting the other containers from other roles.
2024-05-21 08:37:36 +01:00
Donal McBreen
060e5d2027 Update lib/kamal/cli/server.rb
Co-authored-by: Sijawusz Pur Rahnama <sija@sija.pl>
2024-05-21 08:22:20 +01:00
Nick Hammond
8a4f7163bb Apply suggestions from code review
Co-authored-by: Donal McBreen <dmcbreen@gmail.com>
2024-05-20 11:15:14 -07:00
Donal McBreen
ee758d951a Only use barrier when needed, more descriptive info 2024-05-20 12:18:30 +01:00
Donal McBreen
bb2ca81d87 Fix rebase method duplication 2024-05-20 12:18:30 +01:00
Donal McBreen
773ba3a5ab Show container logs and healthcheck status on failure 2024-05-20 12:18:30 +01:00
Donal McBreen
5be6fa3b4e Improve comments 2024-05-20 12:18:30 +01:00
Donal McBreen
07c5658396 Remove redundant method 2024-05-20 12:18:30 +01:00
Donal McBreen
0efb5ccfff Remove the healthcheck step
To speed up deployments, we'll remove the healthcheck step.

This adds some risk to deployments for non-web roles - if they don't
have a Docker healthcheck configured then the only check we do is if
the container is running.

If there is a bad image we might see the container running before it
exits and deploy it. Previously the healthcheck step would have avoided
this by ensuring a web container could boot and serve traffic first.

To mitigate this, we'll add a deployment barrier. Until one of the
primary role containers passes its healthcheck, we'll keep the barrier
up and avoid stopping the containers on the non-primary roles.

It the primary role container fails its healthcheck, we'll close the
barrier and shut down the new containers on the waiting roles.

We also have a new integration test to check we correctly handle a
a broken image. This highlighted that SSHKit's default runner will
stop at the first error it encounters. We'll now have a custom runner
that waits for all threads to finish allowing them to clean up.
2024-05-20 12:18:30 +01:00
Donal McBreen
990f1b4413 Merge pull request #798 from basecamp/git-clone
Build from within a git clone by default
2024-05-20 12:18:07 +01:00
Donal McBreen
da9428f64d Merge pull request #813 from basecamp/handle-no-env-tags
Don't blow up if there are no env tags
2024-05-20 10:58:11 +01:00
Donal McBreen
17dcaccb6a Don't blow up if there are no env tags 2024-05-20 10:50:07 +01:00
Donal McBreen
448349d0e5 Merge pull request #812 from basecamp/sshkit-1.22.2-minimum
Set sshkit minimum version to 1.22.2
2024-05-20 10:39:17 +01:00
Donal McBreen
b6dba57c7d Set sshkit minimum version to 1.22.2
This includes a fix for a bug in the eviction thread that could cause
this error:

```
[ERROR (IOError): Exception while executing on host foo: closed stream]
```

See https://github.com/capistrano/sshkit/pull/534
2024-05-20 10:06:53 +01:00
Donal McBreen
0ea2a2c509 Don't include destination in clone directory
Reusing the clone directory should allow caching of the build context
between deployments to different destinations.
2024-05-20 09:34:42 +01:00
Donal McBreen
307750ff70 Build from within a git clone by default
Docker does not respect the .dockerignore file when building from a tar.

Instead by default we'll make a local clone into a tmp directory and
build from there. Subsequent builds will reset the clone to match the
checkout.

Compared to building directly in the repo, we'll have reproducible
builds.

Compared to using a git archive:
1. .dockerignore is respected
2. We'll have faster builds - docker can be smarter about caching the
build context on subsequent builds from a directory

To build from the repo directly, set the build context to "." in the
config.

If there are uncommitted changes, we'll warn about them either being
included or ignored depending on whether we build from the clone.
2024-05-20 09:30:56 +01:00
Donal McBreen
88947b6a7b Merge pull request #805 from basecamp/env-under-tags
Move env_tags under env key
2024-05-15 11:09:47 +01:00
Donal McBreen
f48c227768 Move env_tags under env key
Instead of:

```
env:
  CLEAR_TAG: untagged
env_tags:
  tag1:
    CLEAR_TAG: tagged
```

We'll have:

```
env:
  clear:
    CLEAR_TAG: untagged
  tags:
    tag1:
      CLEAR_TAG: tagged
```
2024-05-15 10:19:22 +01:00
David Heinemeier Hansson
f98380ef0c Merge pull request #802 from basecamp/envify-during-setup
Envify during setup
2024-05-14 16:00:28 -07:00
David Heinemeier Hansson
0bc27c10cc Fix tests 2024-05-14 11:59:42 -07:00
David Heinemeier Hansson
e58d2f67f2 Fix env template path check and tests 2024-05-14 10:07:31 -07:00
David Heinemeier Hansson
938ac375a1 Only envify if there is a template file available 2024-05-13 17:08:53 -07:00
David Heinemeier Hansson
dc1f707a56 Fix test 2024-05-13 17:01:50 -07:00
David Heinemeier Hansson
033f2a3401 Correct invocation 2024-05-13 16:59:50 -07:00
David Heinemeier Hansson
7cac7e6fb0 Envify during setup 2024-05-13 15:18:11 -07:00
Nick Hammond
fb58fc0ba6 Add in a server exec command for running ad-hoc commands directly on the server 2024-05-13 14:17:06 -07:00
Donal McBreen
12cad5458a Merge pull request #762 from kryachkov/main
Trim long hostnames
2024-05-10 16:05:27 +01:00
Donal McBreen
f8b7f74543 Merge pull request #786 from hundredwatt/add-target-option-to-builder
Add --target option to Builder to support multi-stage Docker builds
2024-05-10 15:15:31 +01:00
Donal McBreen
489d6dbcbb Merge pull request #789 from basecamp/host-tags
Host specific env with tags
2024-05-10 08:08:29 +01:00
Donal McBreen
6d062ce271 Host specific env with tags
Allow hosts to be tagged so we can have host specific env variables.

We might want host specific env variables for things like datacenter
specific tags or testing GC settings on a specific host.

Right now you either need to set up a separate role, or have the app
be host aware.

Now you can define tag env variables and assign those to hosts.

For example:
```
servers:
  - 1.1.1.1
  - 1.1.1.2: tag1
  - 1.1.1.2: tag2
  - 1.1.1.3: [ tag1, tag2 ]
env_tags:
  tag1:
    ENV1: value1
  tag2:
    ENV2: value2
```

The tag env supports the full env format, allowing you to set secret and
clear values.
2024-05-09 16:02:45 +01:00
Jason Nochlin
1e44cc2597 fix rubocop violation 2024-05-08 19:22:25 -06:00
André Falk
63c47eca4c Trim long hostnames
Hostnames longer than 64 characters are not supported by docker
2024-05-07 19:06:39 +02:00
Donal McBreen
3c8428504d Bump version for 1.5.2 2024-05-07 09:44:11 +01:00
Donal McBreen
8e71c48747 Merge pull request #759 from basecamp/details-accessory-host
Output the host when running accessory details
2024-05-02 15:54:08 +01:00
Donal McBreen
67a86e1068 Merge pull request #790 from basecamp/warn-on-missing-builder
Warn on missing builder
2024-05-02 12:50:00 +01:00
Donal McBreen
b67f40bdf7 Warn on missing builder
We are going to try to create a builder if one is missing, so let's warn
rather than report it as an error.
2024-05-02 12:38:20 +01:00
Donal McBreen
375f0283c4 Merge pull request #785 from basecamp/filter-traefik-hosts
Apply --hosts and --roles filters to traefik hosts as well
2024-04-29 14:48:23 +01:00
Jason Nochlin
947be0877f add --target option for builder configuration 2024-04-27 10:24:47 -06:00
Matthew Kent
b8aaddb4c9 Apply --hosts and --roles filters to traefik hosts as well. 2024-04-26 17:08:57 -07:00
Donal McBreen
f48f528043 Bump version for 1.5.1 2024-04-26 14:26:02 +01:00
Donal McBreen
ec0a082542 Merge pull request #779 from basecamp/fix-log-following
Escape single quotes to fix log following
2024-04-26 14:25:27 +01:00
Donal McBreen
6c638a8a77 Merge pull request #778 from basecamp/glob-match-roles-and-hosts
Allow glob matches for roles and hosts
2024-04-26 14:20:17 +01:00
Donal McBreen
1f5b936fa2 Escape single quotes to fix log following
Fixes: https://github.com/basecamp/kamal/issues/777
2024-04-26 14:16:19 +01:00
Donal McBreen
f785451cc7 Allow glob matches for roles and hosts
This lets you do things like:

```
kamal details -h '1.1.1.[1-9]'
kamal details -r 'w{eb,orkers}'
```
2024-04-26 13:43:52 +01:00
Donal McBreen
d475e88dbe Bump version for 1.5.0 2024-04-25 13:39:06 +01:00
Donal McBreen
d551f044d6 Merge pull request #772 from aishek/take-accessory-hosts-into-account
Take accessory hosts into account for --hosts
2024-04-25 11:57:45 +01:00
Donal McBreen
2611179d5e Merge pull request #773 from xiaohui-zhangxh/docker-env-file-keep-non-ascii
don't escape non-ascii characters in docker env file
2024-04-25 11:57:14 +01:00
Donal McBreen
1a013b8d4b Merge pull request #770 from ttilberg/769-ensure-valid-service-name-with-capital-letters
Allow capital letters to match valid service name, such as in MyApp
2024-04-25 11:52:55 +01:00
Maciej Litwiniuk
2f912367ac Allow custom user and port for builder host
When ssh options are set, they overwrite username and password passed as ssh builder uri. Passing part of uri for ssh-kit is fine, as it then properly extracts username and password and forwards it as host.ssh_options (in which case it's no longer empty)
2024-04-17 17:49:50 +02:00
xiaohui
9a9a0914cd don't escape non-ascii characters in docker env file 2024-04-17 17:42:06 +08:00
Alexandr Borisov
12c518097f Take accessory hosts into account 2024-04-17 11:45:33 +03:00
Tim Tilberg
69f90387a8 Allow capital letters to match valid service name, such as in MyApp 2024-04-15 09:09:58 -05:00
Donal McBreen
e6d436f646 Output the host when running accessory details
We already do this for app and Traefik hosts.
2024-04-05 12:46:51 +01:00
Donal McBreen
31669d4dce Merge pull request #758 from basecamp/any-runtime-in-test
Accept any runtime in the hook tests
2024-04-03 16:27:59 +01:00
Donal McBreen
9d20c1466e Merge pull request #757 from basecamp/executable-sample-docker-setup-hook
Make the sample docker setup hook executable
2024-04-03 16:12:46 +01:00
Donal McBreen
ff1dabe7f8 Merge pull request #756 from basecamp/tidy-up-role-host-setup
Tidy up role and host commander setup
2024-04-03 16:09:12 +01:00
Donal McBreen
69aa422890 Accept any runtime in the hook tests
Occasionally in CI things run slowly and it takes more that 1 second
for a cli test to run, so let's allow any value for the runtime in the
hook checks.
2024-04-03 16:06:53 +01:00
Donal McBreen
f8b0883036 Make the sample docker setup hook executable
Fixes https://github.com/basecamp/kamal/issues/754
2024-04-03 15:50:47 +01:00
Donal McBreen
c8100d1f26 Tidy up role and host commander setup
Extract Kamal::Commander::Specifics to deal with host and role setup and
ensure that primary hosts and roles always come first. This means that
in a rolling deploy we deploy to the primary ones first.

This will be important when we remove the healthcheck step as we want
to confirm the primary host can be deployed to before completing a
deployment for other roles.

By setting the hosts and roles all together in one place we can sort
the primary ones to the front without creating infinite loops.
2024-04-03 15:46:30 +01:00
Donal McBreen
3628ecaa44 Merge pull request #753 from basecamp/dump-hook-output-on-failure
Include error message on failure
2024-04-03 10:52:49 +01:00
Donal McBreen
67a2d5e7ca Include error message on failure 2024-04-03 10:43:20 +01:00
Donal McBreen
5e492ecc4d Merge pull request #748 from basecamp/latest-by-tag
Latest by tag
2024-04-03 09:11:03 +01:00
Donal McBreen
77bad291a1 Merge pull request #751 from basecamp/app-exec-env
Set env variables when running kamal app exec
2024-04-02 11:03:38 +01:00
Donal McBreen
a0ce9f66c4 Merge pull request #752 from basecamp/dont-debug-hooks
Use default verbosity for hooks
2024-04-02 11:03:24 +01:00
Donal McBreen
82962c375d Use default verbosity for hooks 2024-04-02 10:47:05 +01:00
Donal McBreen
8a6a51977f Set env variables when running kamal app exec
Allow additional env variable to be set when running `kamal app exec`.
Works for both new and existing containers.
2024-04-01 15:01:32 +01:00
Donal McBreen
2562853ae3 Merge pull request #746 from igor-alexandrov/file-join
Replaced string interpolations with File.join to build paths
2024-03-29 13:47:24 +00:00
Donal McBreen
ed90b99f0d Add tag_latest_image tests 2024-03-29 10:51:57 +00:00
Donal McBreen
ba7a13f895 Only tag after deploying to all hosts 2024-03-29 10:29:58 +00:00
Donal McBreen
05ac808f2a Use image tag to determine stale containers
Use current_running_version to determine the latest version when finding
stale containers.
2024-03-29 10:23:50 +00:00
Donal McBreen
fb7d9077ff Use latest tag for the current destination 2024-03-29 09:48:09 +00:00
Donal McBreen
bade195e93 Redefine what the "latest" container means
Currently the latest container is the one that was created last. But if
we have had a failed deployment that left two containers running that
would not be the one we want. The second container could be in a
restart loop for example.

Instead we want the container that is running the image tagged as
latest. As we now tag as latest after a successful deployment we can
trust that that is a healthy container.

In the case that there is no container running the latest image tag,
we'll fall back to the latest container.

This could happen if the deploy was halted in between the old container
being stopped and the image being tagged as latest.
2024-03-29 08:51:50 +00:00
Donal McBreen
55dd2f49c1 Tag image after booting and include destination
If you are deploying more than one destination to a host, the latest
tags will conflict, so we'll append the destination to the tag.

The latest tag is used when booting the app or exec-ing a new container.

If a deploy doesn't complete on a host for all roles then we should
probably not be using it, so move the tagging to the end of the boot
process.
2024-03-29 08:51:50 +00:00
Igor Alexandrov
511a182539 Replaced string interpolations with to build paths 2024-03-28 20:25:24 +04:00
Donal McBreen
8bb596e216 Merge pull request #741 from igor-alexandrov/destination_in_lock
Added destination to the lock directory
2024-03-28 08:26:57 +00:00
Igor Alexandrov
699bcc0d27 Combined two methods and into one 2024-03-27 20:56:47 +04:00
Donal McBreen
6aacd1f9e2 Merge pull request #745 from basecamp/add-empty-destination-label
Label containers with empty destinations
2024-03-27 15:05:43 +00:00
Donal McBreen
20e71d91c0 Label containers with empty destinations
This will allow us to filter for containers that have no destination in
cases where we deploy an empty + a non empty destination to the same
host.

To note:

```
\# Containers with a destination label
$ docker ps --filter label=destination

\# Containers with an empty destination label
$ docker ps --filter label=destination=
```
2024-03-27 14:48:55 +00:00
Donal McBreen
866303a59b Merge pull request #700 from basecamp/git-archive-build
Build from a git archive
2024-03-27 09:23:00 +00:00
Donal McBreen
53bfefeb2f Make building from a git archive the default
If no context is specified and we are in a git repo, then we'll build
from a git archive by default. This means we don't need a separate
setting and gives us a safer default build.
2024-03-27 08:42:10 +00:00
Donal McBreen
f3b7569032 Build from a git archive
Building directly from a checkout will pull in uncommitted files to or
more sneakily files that are git ignored, but not docker ignored.

To avoid this, we'll add an option to build from a git archive of HEAD
instead. Docker doesn't provide a way to build directly from a git
repo, so instead we create a tarball of the current HEAD with git
archive and pipe it into the build command.

When building from a git archive, we'll still display the warning about
uncommitted changes, but we won't add the `_uncommitted_...` suffix to
the container name as they won't be included in the build.

Perhaps this should be the default, but we'll leave that decision for
now.
2024-03-27 08:38:56 +00:00
Donal McBreen
e5457cf7b4 Merge pull request #736 from tiramizoo/traefik-info
Add tip how to apply changes to traefik by "traefik reboot"
2024-03-27 08:38:15 +00:00
Igor Alexandrov
cee449c269 Put locks in a locks directory. Ensure that locks directory exits on a primary host. 2024-03-27 12:04:39 +04:00
Donal McBreen
786454f2ee Merge pull request #502 from latyshev/main
Fix accessory name checking that is passing to command `kamal accessory`
2024-03-26 13:58:26 +00:00
Donal McBreen
827e18480d Merge pull request #732 from basecamp/always-send-clear-env
Always send the clear env to the container
2024-03-26 11:01:59 +00:00
Donal McBreen
9f9c9ccbde Merge pull request #742 from igor-alexandrov/remove_service_role_dest
Removed unused method from Kamal::Commands::App
2024-03-26 08:10:36 +00:00
Evgeny Latyshev
981d391d4d Fix accessory name check in with_accessory 2024-03-26 09:29:34 +03:00
Igor Alexandrov
900041001a Removed unused method 2024-03-25 22:48:23 +04:00
Igor Alexandrov
43672ec9a5 Added destination to the lock directory 2024-03-25 22:42:22 +04:00
Donal McBreen
5481fbb973 Test that we pull in env host variables
Now that clear env variables specified on the command line we can check
that values specified as `${VAR}` are pulled in from the host.
2024-03-25 12:26:37 +00:00
Donal McBreen
49afdbb09a Always send the clear env to the container
Secret and clear env variables have different lifecycles. The clear ones
are part of the repo, so it makes sense to always deploy them with the
rest of the repo.

The secret ones are external so we can't be sure that they are up to
date, therefore they require an explicit push via `envify` or `env push`.

We'll keep the env file, but now it just contains secrets. The clear
values are passed directly to `docker run`.
2024-03-25 11:42:27 +00:00
Donal McBreen
5f58575b62 Merge pull request #730 from igor-alexandrov/confirming_dialogs
Added -y option to kamal traefik reboot command
2024-03-22 15:14:44 +00:00
Wojciech Wnętrzak
cb49d7dada Add tip how to apply changes to traefik by "traefik reboot"
Running "traefik restart" is not enough to apply changes
2024-03-22 13:50:54 +01:00
Igor Alexandrov
3d26fa8ddd Updated confirmation text for the traefik reboot command 2024-03-22 14:27:18 +04:00
Donal McBreen
ea9f8b488d Merge pull request #735 from basecamp/extract-app-boot-steps
Extract app boot steps
2024-03-22 09:35:04 +00:00
Donal McBreen
83472af32c Merge pull request #734 from basecamp/rubocop-rails-omakase
Switch to rubocop-rails-omakase rubocop rules
2024-03-22 09:33:25 +00:00
Donal McBreen
e99e1955b8 Extract app boot steps
The Kamal::Cli::App#boot has a lot to do, so extract the steps to make
things clearer.
2024-03-22 09:21:52 +00:00
Donal McBreen
30e0c44396 Switch to rubocop-rails-omakase rubocop rules
No code changes required
2024-03-21 13:47:20 +00:00
Donal McBreen
20d6e5365e Merge pull request #733 from basecamp/integration-test-roles
Integration test roles
2024-03-21 13:43:33 +00:00
Donal McBreen
72ace2bf0b Add an integration test for roles
Add an app with roles to the integration tests. We'll deploy two web
containers and one worker. The worker just sleeps, so we are testing
that the container has booted.
2024-03-21 13:30:53 +00:00
Donal McBreen
ba40d026d0 Make integration test app to deploy configurable 2024-03-21 12:09:59 +00:00
Igor Alexandrov
0f13600ba3 Fixed Traefik integration test 2024-03-21 09:25:07 +04:00
Igor Alexandrov
bbf952952d Added -y option to kamal traefik reboot command 2024-03-20 22:00:13 +04:00
Donal McBreen
474b76cf47 Merge pull request #701 from basecamp/rubocop
Add Rubocop
2024-03-20 10:59:35 +00:00
Donal McBreen
3ecfb3744f Add Rubocop
- Pull in the 37signals house style
- Autofix violations
- Add to CI
2024-03-20 10:23:02 +00:00
Donal McBreen
c985fa33d1 Bump version for 1.4.0 2024-03-20 09:27:23 +00:00
Donal McBreen
e8b9f8907f Merge pull request #715 from basecamp/use-role-not-string-in-config
Pass around Roles instead of Strings
2024-03-08 08:55:53 +00:00
Donal McBreen
4966d52919 Pass around Roles instead of Strings
Avoid looking up roles by names everywhere. This avoids the awkward
role/role_config naming as well.
2024-03-08 08:44:35 +00:00
Donal McBreen
52bb40add0 Merge pull request #656 from DanielJackson-Oslo/informative-error-message-on-lock
Informative message on lock error
2024-03-07 11:16:18 +00:00
Donal McBreen
73a9276cdd Fix up app command tests 2024-03-07 11:11:20 +00:00
Donal McBreen
8c0784ed4a Merge pull request #634 from alhafoudh/main
Allow lines option to be configured when following app logs
2024-03-07 11:11:08 +00:00
Donal McBreen
089a2d3bba Merge pull request #710 from basecamp/install-wget-or-curl
Install docker with curl or wget
2024-03-07 11:01:30 +00:00
Donal McBreen
bd76d23916 Merge pull request #593 from CleverFew/role_logging_config
Role specific logging configuration
2024-03-07 10:53:34 +00:00
Donal McBreen
fa37fcd10c Merge pull request #585 from tsvallender/docker-network
Add docker-setup hook
2024-03-07 10:51:08 +00:00
Donal McBreen
f5dc0858b0 Update error message to include wget 2024-03-07 10:49:32 +00:00
Donal McBreen
9dddb140b1 Merge pull request #558 from GeNiuS69/add-skip_push-to-setup
Add --skip_push option to setup
2024-03-07 10:26:41 +00:00
Donal McBreen
26b1d57c90 Install docker with curl or wget
If curl is not available to download the docker install script, try
with wget instead.

If neither is available or both fail, return a simple failing script
so that we don't carry on regardless.

Fixes: https://github.com/basecamp/kamal/issues/395
2024-03-07 10:16:03 +00:00
Donal McBreen
b94199415f Convert combine by: '||' to any 2024-03-07 09:10:49 +00:00
Trevor Vallender
f69c45b7ea Add docker-setup hook
This allows the user to make any necessary configuration changes to
Docker before setting up any containers, allowing those configuration
changes to take effect from the outset.
2024-03-06 19:01:48 +00:00
Donal McBreen
32a2ae5b2c Merge pull request #708 from nickhammond/valid_service_name
Remove warning for valid service name
2024-03-06 16:22:04 +00:00
Nick Hammond
37544a6383 Merge branch 'basecamp:main' into valid_service_name 2024-03-06 09:09:13 -07:00
Nick Hammond
a1bc6d61af Switch the regex ordering for hyphen and underscore for service name to remove warning 2024-03-06 09:08:17 -07:00
Donal McBreen
5c32be10f1 Merge pull request #707 from basecamp/boot-strategy-min-limit-1
Ensure a minimum limit of 1 for % boot strategy
2024-03-06 16:06:35 +00:00
Donal McBreen
dc5af03593 Update tests to match single quotes 2024-03-06 16:04:31 +00:00
Donal McBreen
1abd029ea0 Merge pull request #696 from dorianmariecom/patch-1
Replace \`service\` by 'service' so it doesn't get executed by bash
2024-03-06 16:04:11 +00:00
Donal McBreen
c4d0d3e5eb Merge pull request #704 from basecamp/escape-registry-username-password
Escape the docker registry username and password
2024-03-06 15:58:46 +00:00
Donal McBreen
46e7cf8e78 Merge pull request #706 from basecamp/kamal-remove-noop
Ensure `kamal remove` completes without setup
2024-03-06 15:58:34 +00:00
Donal McBreen
c7cfc074b6 Ensure a minimum limit of 1 for % boot strategy
Fixes: https://github.com/basecamp/kamal/issues/681
2024-03-06 15:51:35 +00:00
Donal McBreen
c10f43e365 Merge pull request #692 from nickhammond/valid_service_name
Add a simple validation to the service name to prevent setup issues
2024-03-06 15:24:39 +00:00
Donal McBreen
8e2184d65e Ensure kamal remove completes without setup
If `kamal setup` has not run or errored out part way through,
`kamal remove` should still complete.

Fixes: https://github.com/basecamp/kamal/issues/629
2024-03-06 14:59:26 +00:00
Donal McBreen
2be397b679 Escape the docker registry username and password
Fixes: https://github.com/basecamp/kamal/issues/278
2024-03-06 11:04:55 +00:00
Donal McBreen
cc8c508556 Merge branch 'main' into valid_service_name 2024-03-05 11:02:33 +00:00
Nick Hammond
3b16e047c5 Add hyphen to the allowed character list for service name 2024-03-04 10:03:22 -07:00
Donal McBreen
6563393d9a Merge pull request #627 from aishek/626-mention-sprockets-config-in-deploy-template
Mention Sprockets config in deploy template
2024-03-04 15:31:41 +00:00
Ahmed Al Hafoudh
91f350fcce Merge branch 'basecamp:main' into main 2024-03-04 16:22:28 +01:00
Nick Lozon
e4e9664049 use double quotes 2024-03-04 10:10:51 -05:00
Nick Lozon
1acef5221f test deep_merge 2024-03-04 10:06:30 -05:00
Nick Lozon
788a57e85e role logging_args method, use in app 2024-03-04 10:06:30 -05:00
Nick Lozon
f9a934a01f configuration logging accessor 2024-03-04 10:06:30 -05:00
Aleksandr Borisov
f286fdc374 Update lib/kamal/cli/templates/deploy.yml
Co-authored-by: Donal McBreen <dmcbreen@gmail.com>
2024-03-04 16:26:11 +03:00
Donal McBreen
828cca322b Merge pull request #650 from basecamp/retained-containers
Config the number of containers to keep
2024-03-04 12:05:35 +00:00
Donal McBreen
cb030e8751 Merge pull request #680 from igor-alexandrov/traefik-2.10
Bump default Traefik image to 2.10
2024-03-04 11:58:37 +00:00
Donal McBreen
6892abb4be Config the number of containers to keep
By default we keep 5 containers around for rollback. The containers
don't take much space, but the images for them can.

Make the number of containers to retain configurable, either in the
config with the `retain_containers` setting on the command line
with the `--retain` option.
2024-03-04 11:55:45 +00:00
Donal McBreen
bcfd0ca88a Merge pull request #645 from juan-apa/fix-missing-netscp-require
require missing net/scp dependency
2024-03-04 11:49:43 +00:00
Donal McBreen
2e8071a5b3 Merge pull request #608 from CleverFew/fix_accessory_cli_host_params
Accessory CLI respects `--hosts`
2024-03-04 11:31:50 +00:00
Donal McBreen
200e2686fd Merge pull request #506 from rience/custom-acc-service-name
Allow for Custom Accessory Service Name
2024-03-04 10:57:10 +00:00
Donal McBreen
db94789dc1 Merge pull request #434 from rience/ssh-agent-support
Supports Passing SSH Agent Socket to Build Options
2024-03-04 10:54:47 +00:00
Dorian Marié
2bffc3bc74 Replace \service\ by 'service' so it doesn't get executed by bash
Fixes #694
2024-03-01 09:54:06 +01:00
Aleksandr Nigomatulin
064ace0598 Rollback passing invoke_options 2024-02-24 21:36:20 +06:00
Nick Hammond
a02af74dda Add a simple validation to the service name to prevent setup issues 2024-02-22 09:47:48 -07:00
Aleksandr Nigomatulin
5ef384d666 Add test 2024-02-17 00:11:03 +06:00
Aleksandr Nigomatulin
b94dfe193b Remove unnecessary code 2024-02-16 12:52:07 +06:00
Aleksandr Nigomatulin
bc6c027315 Upds according remarks 2024-02-16 11:56:58 +06:00
Krzysztof Adamski
1c2a45817a Supports Passing SSH Args to Build Options 2024-02-15 14:20:20 +01:00
Krzysztof Adamski
b411356409 Allow for Custom Accessory Service Name 2024-02-15 11:12:18 +01:00
Igor Alexandrov
77e72e34ce Bumped default Traefik image to 2.10 2024-02-13 16:00:02 +04:00
Daniel Jackson
ad04bb7556 Show context for lock status message on raise_if_locked 2024-01-23 09:17:15 +01:00
Daniel Jackson
1ec69d3764 Tell user about 'kamal lock help' when deploy fails due to a lock 2024-01-23 09:16:09 +01:00
Daniel Jackson
2d1a0dc9ba Informative message on lock error 2024-01-22 09:11:17 +01:00
Juan Aparicio
c984db152f require missing net/scp dependency 2024-01-11 17:00:13 -03:00
David Heinemeier Hansson
aea55480ad Merge pull request #640 from basecamp/local-different-arch
Allow local builds using a different arch than native
2024-01-10 13:28:37 -08:00
dhh
5a09aa12ba Allow local builds using a different arch than native 2024-01-10 13:00:48 -08:00
Donal McBreen
aca7796e9d Bump version for 1.3.1 2024-01-10 08:56:34 +00:00
Donal McBreen
8b6d8306d1 Merge pull request #637 from basecamp/tests-wait-longer-for-health
Be a bit more patient during tests
2024-01-09 16:45:28 +00:00
Donal McBreen
bb50546467 Merge pull request #636 from basecamp/tests-clean-known-hosts
Fix Net::SSH::HostKeyMismatch between bin/test runs
2024-01-09 16:45:12 +00:00
Donal McBreen
acc6b9ad71 Merge pull request #635 from basecamp/missing-base64-require
Add a missing base64 require
2024-01-09 16:44:42 +00:00
Matthew Kent
9c681d4a38 Be a bit more patient during tests.
Seeing reasonably consistent local failures at 20 seconds.
2024-01-09 08:21:45 -08:00
Matthew Kent
2a8924b53c Address Net::SSH::HostKeyMismatch seen locally between bin/test runs. 2024-01-09 08:21:30 -08:00
Matthew Kent
c5ae54d7d4 Add a missing base64 require.
Also, prepare for the moving of base64 from default to a bundled gem in ruby 3.4.
2024-01-09 08:21:10 -08:00
Donal McBreen
4b05068493 Merge pull request #638 from basecamp/rails-7.2-compatible-rubies
Rails 7.2 compatible Rubies
2024-01-09 12:10:29 +00:00
Donal McBreen
68eb549795 Update to actions/checkout@v4 to silence node warning 2024-01-09 11:35:10 +00:00
Donal McBreen
1a3dd52af4 Rails 7.2 compatible Rubies
1. Add Ruby 2.7 specific Gemfile that uses an older version of nokogiri
2. Rails edge doesn't support Ruby 2.7.0, so exclude it.
3. Add Ruby 3.3
4. Update Gemfile.lock to test against Rails 7.1.2 as it's the latest
   version.
5. Remove continue-on-error from the matrix and always set to true
2024-01-09 11:13:11 +00:00
Ahmed Al Hafoudh
0d709a3fdb Allow lines option to be configured when following app logs 2024-01-08 09:34:38 +01:00
Alexandr Borisov
414d29ae4e Mention Sprockets config in deploy template 2024-01-04 09:18:38 +04:00
Nick Lozon
f8d8319c2f better test description 2023-12-12 15:37:12 -05:00
Nick Lozon
f6a9d54902 unit test 2023-12-12 15:07:29 -05:00
Nick Lozon
b2fd5744fb perform intersection on specified hosts 2023-12-12 14:39:33 -05:00
Donal McBreen
457f06da13 Merge pull request #598 from basecamp/fix-duplicate-role-env-vars
Fix duplicate role env vars
2023-11-29 10:09:34 +00:00
Matthew Kent
7fa53d90bd Merge hashes to de-dupe the app and role envs.
This is better then adding them together which confusingly results in
both ENV vars in the same file, though based on the load order, they
worked anyway.
2023-11-28 15:59:03 -08:00
Donal McBreen
a155b7baab Bump version for 1.3.0 2023-11-28 14:06:45 +00:00
Donal McBreen
175e3bc159 Merge pull request #507 from leonvogt/introduce-absolute-accessories-paths
Add option to set an absolute directory path
2023-11-28 10:15:27 +00:00
Donal McBreen
e3d8a2aa82 Merge pull request #594 from basecamp/match-primary-role-in-filters
Try to match primary_role when roles are filtered
2023-11-28 09:15:39 +00:00
Donal McBreen
0e067fb5e1 Merge pull request #595 from basecamp/error-on-filter-miss
Error out when roles or host filters don't match anything
2023-11-27 08:08:19 +00:00
Matthew Kent
63babecba7 Raise an error when either the filtered hosts or roles are empty.
Keeps us confusingly running things on the primary_host when nothing
matches.
2023-11-25 12:47:39 -08:00
Matthew Kent
79baa598fa Make an effort to match the primary_role from a list of specific roles.
This is less surprising than picking the first role and first host.
2023-11-24 17:41:58 -08:00
Donal McBreen
b1dc188841 Remove stray file 2023-11-23 09:22:36 +00:00
Donal McBreen
635876bdb9 Merge pull request #523 from rmacklin/fix-error-message-in-pre-build-sample-hook
Fix duplicate error message in pre-build.sample
2023-11-16 08:51:22 +00:00
Donal McBreen
11521517fa Merge pull request #550 from dmitrytrager/feature-name-all-for-accessory-reboot
feature: add NAME=all option for accessory reboot
2023-11-16 08:50:51 +00:00
Donal McBreen
610d9de3fd Merge pull request #580 from happyscribe/feat/no-web
Allow Kamal to run without traefik
2023-11-16 08:44:45 +00:00
Donal McBreen
bf79df0f72 Bump version for 1.2.0 2023-11-15 14:48:11 +00:00
Donal McBreen
a0959b5afd Merge pull request #573 from basecamp/pre-post-traefik-reboot-hooks
Pre and post Traefik reboot hooks
2023-11-15 14:01:40 +00:00
Yoel Cabo
7472e5dfa6 Merge remote-tracking branch 'origin/main' into feat/no-web 2023-11-14 12:11:18 +01:00
Yoel Cabo
887b7dd46d Do not invoke healthcheck on deploy when no web role 2023-11-14 11:34:32 +01:00
Donal McBreen
77a79b299a Merge pull request #583 from basecamp/wildcard-filters
Add wildcards to roles and hosts filters
2023-11-14 08:19:02 +00:00
Matthew Kent
efcb855db7 Advertise wildcard support. 2023-11-13 23:43:26 -08:00
Matthew Kent
7137850354 Add support for wildcard matches with '*' on roles and hosts.
eg:
  --roles=*_chicago,*_tokyo
  --hosts=app-*

Useful for targeted deploys.
2023-11-13 23:43:23 -08:00
Donal McBreen
8a85840a47 Merge pull request #582 from basecamp/allow-empty-roles
Add allow_empty_roles to control aborting on roles with no hosts.
2023-11-13 09:30:01 +00:00
Donal McBreen
80cc0c23d8 Merge pull request #578 from basecamp/enable-yaml-aliases
Enable yaml aliases
2023-11-13 09:28:40 +00:00
Donal McBreen
14a9129410 Merge pull request #577 from basecamp/set-primary-web-role
Support customizing the primary_web_role
2023-11-13 09:27:18 +00:00
Matthew Kent
60187cc3a4 Add allow_empty_roles to control aborting on roles with no hosts.
This added flexibility allows you to define base roles that might not
necessarily exist in each deploy destination.
2023-11-12 08:54:28 -08:00
Yoel Cabo
87cb8c1f71 fix: allow configurations without web roles 2023-11-12 09:39:07 +01:00
Matthew Kent
ed58ce6e61 Add test coverage with aliases. 2023-11-11 17:25:50 -08:00
Matthew Kent
263b4a4fb8 Enable aliases for more exotic templating situations.
This is super useful for DRY when configuring a number of roles and you
hit the limits of what's reasonable with ERB.
2023-11-11 17:25:50 -08:00
Matthew Kent
073f745677 Test for both undefined roles and missing traefik. 2023-11-11 12:57:52 -08:00
Matthew Kent
a9cc7c73d2 Handle an undefined primary_web_role. 2023-11-11 12:57:31 -08:00
Matthew Kent
6898e8789e Further test the override. 2023-11-10 17:17:16 -08:00
Matthew Kent
d0ac6507e7 Add test coverage. 2023-11-10 16:49:37 -08:00
Matthew Kent
628a47ad88 Background for the new option. 2023-11-10 16:39:06 -08:00
Matthew Kent
47f8725cf3 Support a dynamic primary_web_role instead of assuming it's 'web'.
This allows for more meaningful naming in roles.

The only caution here is that we don't support the renaming of roles, so
any migration is left to the user.
2023-11-10 16:35:25 -08:00
Donal McBreen
5fd4a28bf7 Pre and post Traefik reboot hooks
Provide pre and post reboot hooks for Traefik, that can be used to
remove/add to an external load balancer to prevent requests from being
sent during the reboot.

Works best with the --rolling setting, where each hook is called once
per host.
2023-11-08 15:11:26 +00:00
Donal McBreen
97ba6b746b Merge pull request #564 from basecamp/return-502-if-no-container
Return a 502 when container is down
2023-11-08 14:58:22 +00:00
Donal McBreen
9e25d8a012 Priority 2 for the main app 2023-11-08 14:12:45 +00:00
Donal McBreen
da161445fa Merge pull request #508 from leonvogt/ssh-port-option
Configurable SSH port
2023-11-06 08:48:26 +00:00
Leon
f339626667 Add option to set absolute directory path 2023-11-03 22:48:30 +01:00
Leon
2d86d4f7cc Add SSH port to run_over_ssh 2023-11-03 22:32:37 +01:00
Leon
792aa1dbdf Add SSH port option 2023-11-03 22:32:37 +01:00
Donal McBreen
24a2f51641 Return a 502 when container is down
If the app container is down or not responding then traefik will return
a 404 response code. This is not ideal as it suggests a client rather
than a server problem.

To fix this, we'll define a catch all route that always returns a 502.

This is not ideal as this route would take priority over a shorter route
with priorty 1.

TODO: up the priority of the app route.
2023-11-03 14:20:52 +00:00
Donal McBreen
8f53104d00 Bump version for 1.1.0 2023-11-01 09:20:45 +00:00
dmitrytrager
2d22143a24 feature: add NAME=all option for accessory reboot 2023-10-31 00:13:45 +01:00
Aleksandr Nigomatulin
cbd99306eb Add skip_push option to setup 2023-10-30 23:27:58 +06:00
Donal McBreen
78fc91f2ec Merge pull request #557 from basecamp/envify-reset-env-before-push
Reset the env before pushing
2023-10-30 11:54:00 +00:00
Donal McBreen
dd748fac8c Reset the env before pushing
Calling `load_envs` again does not load updated env variables, because
Dotenv does not overwrite existing values.

To fix this we'll store the original ENV and reset to it before
reloading.

https://github.com/basecamp/kamal/issues/512
2023-10-30 11:31:50 +00:00
Donal McBreen
b732b2dd55 Merge pull request #547 from nickhammond/envify/trim-lines
Enable trim mode with ERB
2023-10-30 08:57:55 +00:00
Donal McBreen
e3254b2aa8 Merge pull request #544 from nickhammond/bugfix-require-sshkit-sensitive-util
Require sshkit within the sshkit util
2023-10-30 08:57:08 +00:00
Donal McBreen
e9269d2ee8 Merge pull request #501 from rience/optional-envify-push
Optionally Skip Push for "envify"
2023-10-30 08:30:21 +00:00
Donal McBreen
d2214b43b7 Merge pull request #499 from basecamp/env-only-needed-for-push
Remove the env check
2023-10-30 08:22:56 +00:00
Donal McBreen
370481921e Merge pull request #498 from basecamp/app-exec-env-file
App exec with env file
2023-10-30 08:22:35 +00:00
Donal McBreen
aa23f26330 Merge pull request #479 from npezza93/main
Loosen superuser check to match docker-installs script check
2023-10-30 08:21:30 +00:00
Donal McBreen
f4933d83bf Merge pull request #477 from clintmiller/patch-1
Pass KAMAL_VERSION env var to container run
2023-10-30 08:19:20 +00:00
Nick Hammond
6c36c82153 Enable trim mode with ERB 2023-10-24 17:09:05 -07:00
Krzysztof Adamski
8ca04032a1 Optionally Skip Push for "envify" 2023-10-23 14:49:39 +02:00
Nick Hammond
2fb22c934b Require sshkit within the sshkit util 2023-10-22 22:34:22 -07:00
Richard Macklin
f96d071222 Fix copy-pasted error message in pre-build.sample
The "No git remote set" error message was appropriate for the previous
block (where it was presumably copy-pasted from), but in this line we
have failed the check that determines if we have a git branch checked
out, so we should output a corresponding error.
2023-10-08 15:14:40 -07:00
Donal McBreen
f6662c7a8f Remove the env check
The env check is not needded anymore as all the commands rely on the
env files having already been created remotely.

The only place the env is needed is when running `kamal env push` and
that will still raise an apropriate error.
2023-09-25 15:23:01 +01:00
Donal McBreen
645f5ab72d App exec with env file
When calling `kamal app exec` for new non interactive containers, run
the command per role on each server and include the role config
including the environment.

Fixes: https://github.com/basecamp/kamal/issues/492
2023-09-25 15:07:05 +01:00
Clint Miller
8dca65f48f Fix commands/app tests 2023-09-20 08:12:27 -05:00
dhh
83a2d52ff4 Bump version for 1.0.0 2023-09-18 17:39:01 -07:00
Nick Pezza
1a2796a7d0 Loosen superuser check to match docker-installs script check 2023-09-18 20:32:59 -04:00
Clint Miller
d80fdf8468 Pass KAMAL_VERSION env var to container run
In lieu of a general purpose mechanism to pass dynamically-evaluated env-vars at container execution time, we can pass the `config.version` as KAMAL_VERSION to avoid having to take apart the container name just to determine the SHA of the deployed version in the entrypoint.
2023-09-18 16:07:36 -05:00
dhh
90fefc419f Point to rolling restarts 2023-09-18 08:31:49 -07:00
dhh
8671963719 Explain asset bridging 2023-09-18 08:16:44 -07:00
Donal McBreen
a03ffd5b92 Merge pull request #476 from basecamp/exec-with-role
Run interactive commands with the correct host
2023-09-18 12:14:13 +01:00
Donal McBreen
0861730e0e Run interactive commands with the correct host
Fixes https://github.com/basecamp/kamal/issues/430
2023-09-18 12:00:36 +01:00
David Heinemeier Hansson
6b0f93a564 Update README.md 2023-09-16 16:02:54 -07:00
David Heinemeier Hansson
e6371faf4f Merge pull request #473 from basecamp/introduce-git-gateway
Extract Kamal::Git as gateway for all git usage
2023-09-16 11:47:18 -07:00
dhh
e95a9b4fa2 Fix tests 2023-09-16 11:35:29 -07:00
dhh
e5886a1a8e Merge branch 'main' into introduce-git-gateway
* main:
  Healthcheck polling is a CLI concern
2023-09-16 11:31:48 -07:00
David Heinemeier Hansson
ec8192b160 Merge pull request #472 from basecamp/move-healthcheck-poller-to-cli
Healthcheck polling is a CLI concern
2023-09-16 11:31:28 -07:00
dhh
2da03a220d Merge branch 'main' into introduce-git-gateway
* main:
  No longer used
  Fix env validation
  Fix tests
  Fix test
  Extract Kamal::EnvFile
2023-09-16 11:31:18 -07:00
dhh
cfbfb37e23 Extract Kamal::Git as gateway for all git usage 2023-09-16 11:30:29 -07:00
David Heinemeier Hansson
ff4d025840 Merge pull request #471 from basecamp/extract-env-writer
Extract Kamal::EnvFile
2023-09-16 11:29:43 -07:00
dhh
59ac59d351 Healthcheck polling is a CLI concern
Also, it has no instance variables, so let's just have it be a module.
2023-09-16 11:19:38 -07:00
dhh
3df87520db No longer used 2023-09-16 11:12:52 -07:00
dhh
85ce65a4ce Merge branch 'main' into extract-env-writer
* main:
  Inline util method only used in one place
2023-09-16 11:12:08 -07:00
dhh
12a82a6c58 Inline util method only used in one place 2023-09-16 11:11:24 -07:00
dhh
b2d2a254d7 Fix env validation 2023-09-16 11:05:47 -07:00
dhh
62cdf31ae2 Fix tests 2023-09-16 11:01:16 -07:00
dhh
0dcebe7d34 Fix test 2023-09-16 10:59:41 -07:00
dhh
32a5c157b9 Merge branch 'main' into extract-env-writer
* main:
  No longer used
2023-09-16 10:56:29 -07:00
dhh
97cea8950d No longer used 2023-09-16 10:56:00 -07:00
dhh
873be0b76b Extract Kamal::EnvFile
Cleaning up the Utils junk drawer.
2023-09-16 10:55:41 -07:00
David Heinemeier Hansson
3a8eb0cf7d Merge pull request #470 from basecamp/extract-app-concerns
Extract app concerns
2023-09-16 10:24:24 -07:00
dhh
e9ef13d06d Group configuration methods in logical sections 2023-09-16 10:20:08 -07:00
dhh
f648fe6c3f Grouping + ordering 2023-09-16 10:14:04 -07:00
dhh
46895d0b08 Better ordering and spacing 2023-09-16 10:11:42 -07:00
dhh
431ca9e809 Remind about env push 2023-09-16 10:09:42 -07:00
dhh
6b5c5f0650 Extract Logging too
Leave only the core essentials in App
2023-09-16 10:03:28 -07:00
dhh
d303fcc621 Extract Containers and Images concerns 2023-09-16 09:58:09 -07:00
dhh
3ae855ef28 Explain method better 2023-09-16 09:53:03 -07:00
dhh
76a3086569 Group related methods with spacing 2023-09-16 09:52:54 -07:00
dhh
07646bc020 Extract Cord, Assets, and Execution concerns from App
It was getting crowded!
2023-09-16 09:51:45 -07:00
dhh
880b8b267a Fix test 2023-09-16 09:38:30 -07:00
dhh
37e5c48a27 Setup run directory on accessory hosts as well
cc @djmb
2023-09-15 11:08:27 -07:00
dhh
deb67386fa No need to suggest use of erb 2023-09-15 10:54:50 -07:00
dhh
81d74e4a9d Record push of env files for audit on app servers 2023-09-15 10:20:31 -07:00
dhh
39c13dcc18 Push env files as part of setup 2023-09-15 10:20:31 -07:00
dhh
e7314a0eea Explain ensuring Docker is installed 2023-09-15 10:20:31 -07:00
Donal McBreen
168c6e2da3 Merge pull request #467 from basecamp/assets-copy-hidden-files
Copy all files into asset volume
2023-09-15 08:46:02 +01:00
Donal McBreen
564765862b Add hidden file check to integration tests 2023-09-15 08:37:41 +01:00
Donal McBreen
3c12d1799c Copy all files into asset volume
Adding -T to the copy command ensures that the files are copied at the
same level into the target directory whether it exists or not.

That allows us to drop the `/*` which was not picking up hidden files.

Fixes: https://github.com/basecamp/kamal/issues/465
2023-09-15 08:07:48 +01:00
Donal McBreen
60835d13a8 Merge pull request #444 from rience/custom-healthcheck-log-lines-count
Configurable Number of Lines in Healthcheck Log Output
2023-09-13 08:57:00 +01:00
Krzysztof Adamski
892cf0e66b Configurable Log Lines Number in Healthcheck Log Output 2023-09-12 21:06:36 +02:00
Krzysztof Adamski
8ddc484ce6 Configurable Lines Number in Healthcheck Log Output 2023-09-12 21:04:18 +02:00
Donal McBreen
0e021e3c57 Merge pull request #461 from basecamp/escape-newline-from-inspect-format
Escape the newline in the inspect query
2023-09-12 19:19:47 +01:00
Donal McBreen
fb0aeec27e Escape the newline in the inspect query 2023-09-12 19:10:39 +01:00
Donal McBreen
a367819a1c Merge pull request #460 from basecamp/traefik-wait-5s-after-unhealthy
Give Traefik 5s to drop old container
2023-09-12 17:12:20 +01:00
Donal McBreen
0afe289a20 Give Traefik 5s to drop old container 2023-09-12 17:03:51 +01:00
Donal McBreen
bf6af46ac3 Merge pull request #459 from basecamp/env-file-escape-newlines
Escape newlines in docker env files
2023-09-12 15:05:38 +01:00
Donal McBreen
df2b76aee1 Escape newlines in docker env files
When env variables were passed via `-e` newlines were escaped. This
updates the env file to do the same thing.
2023-09-12 14:57:19 +01:00
Donal McBreen
70a3c7195a Merge pull request #458 from basecamp/avoid-env-empty-file-warning
Fix empty file warning when uploading env files
2023-09-12 12:05:31 +01:00
Donal McBreen
c651de177f Fix empty file warning when uploading env files 2023-09-12 11:57:28 +01:00
Donal McBreen
7b42daa9fb Merge pull request #457 from basecamp/remove-dangling-image-filter
Remove the `dangling=true` filter
2023-09-12 11:21:50 +01:00
Donal McBreen
9d49b3e391 Merge pull request #456 from basecamp/validate-image
Validate the build image
2023-09-12 11:18:32 +01:00
Donal McBreen
2c5ab054db Remove the dangling=true filter
This has been removed from Docker Engine 24 and `docker image prune`
only deletes dangling images anyway.

Fixes https://github.com/basecamp/kamal/issues/410
2023-09-12 11:09:26 +01:00
Donal McBreen
66291a2aea Validate the build image
Kamal needs images to have the service label so it can track them for
pruning. Images built by Kamal will have the label, but externally built
ones may not.

Without it images will build up over time. The worst case is an outage
if all the hosts disks fill up at the same time.

We'll add a check for the label and halt if it is not there.
2023-09-12 10:45:01 +01:00
Donal McBreen
d96e086945 Merge pull request #452 from basecamp/preconnect-to-build-remote-host
Connect to remote host before creating builder
2023-09-12 09:21:57 +01:00
Donal McBreen
8424458174 Check protocol is SSH before connecting 2023-09-12 09:12:57 +01:00
Donal McBreen
6a3b0249fe Connect to remote host before creating builder
Connecting to the remote host will make any SSH configuration issues
obvious and add the host to known hosts if that is how SSHKit is
configured.
2023-09-12 09:12:57 +01:00
Donal McBreen
dfc2803714 Merge pull request #454 from basecamp/lts-ubuntu
Use LTS version of Ubuntu for integration tests
2023-09-12 09:12:31 +01:00
Donal McBreen
ade90bc051 Use LTS version of Ubuntu for integration tests 2023-09-12 08:59:54 +01:00
Donal McBreen
daa53f5831 Merge pull request #451 from basecamp/require-destinations
Add a require_destination setting
2023-09-12 08:26:36 +01:00
Donal McBreen
50a4f83db6 Merge pull request #450 from basecamp/stop-stale-container-when-deploying
Stop stale containers when deploying
2023-09-12 08:26:16 +01:00
Donal McBreen
00cb7d99d8 Merge pull request #449 from basecamp/asset-path
Asset paths
2023-09-12 08:26:07 +01:00
Donal McBreen
fb74910dc8 Merge pull request #425 from basecamp/prune-healthcheck-containers
Prune healthcheck containers
2023-09-12 08:25:50 +01:00
Donal McBreen
26dcd75423 Add a require_destination setting
If you always want to use a destination, and have a base deploy.yml file
that doesn't specify any hosts, then if you forget to specific the
destination you will get a cryptic error.

Add a "require_destination" setting you can use to avoid this.
2023-09-11 16:57:11 +01:00
Donal McBreen
afb9b0bbe2 Stop stale containers when deploying
An interrupted deployment can leave older containers lying around. To
ensure they are cleaned up subsequently, stop stale containers during
deployments instead of just reporting them.
2023-09-11 14:49:06 +01:00
Donal McBreen
718776eb72 Prune healthcheck containers
If a deployment is interrupted it could leave stale healthcheck
containers around that prevent dependent images from being pruned.
2023-09-11 14:36:25 +01:00
Donal McBreen
9d35793287 Merge pull request #440 from gf3/fix/ssh-auth-methods
fix: do not hardcode Net::SSH auth_methods
2023-09-11 14:32:37 +01:00
Donal McBreen
0b439362da Asset paths
During deployments both the old and new containers will be active for a
small period of time. There also may be lagging requests for older CSS
and JS after the deployment.

This can lead to 404s if a request for old assets hits a new container
or visa-versa.

This PR makes sure that both sets of assets are available throughout the
deployment from before the new version of the app is booted.

This can be configured by setting the asset path:

```yaml
asset_path: "/rails/public/assets"
```

The process is:
1. We extract the assets out of the container, with docker run, docker
cp, docker stop. Docker run sets the container command to "sleep" so
this needs to be available in the container.
2. We create an asset volume directory on the host for the new version
of the app on the host and copy the assets in there.
3. If there is a previous deployment we also copy the new assets into
its asset volume and copy the older assets into the new asset volume.
4. We start the new container mapping the asset volume over the top of
the container's asset path.

This means the both the old and new versions have replaced the asset
path with a volume containing both sets of assets and should be able
to serve any request during the deployment. The older assets will
continue to be available until the next deployment.
2023-09-11 12:18:18 +01:00
Donal McBreen
2962f545b9 Merge pull request #447 from basecamp/output-per-line-mounts
Output one mount per line
2023-09-07 15:30:03 +01:00
Donal McBreen
cd02510d0f Output one mount per line
The go template was concatenating all the mounts into one line. It
happened to work because the mount we are interested was always first.

Fix it to output one mount per line instead.
2023-09-07 15:20:50 +01:00
Donal McBreen
cccf79ed94 Merge branch 'main' into fix/ssh-auth-methods 2023-09-07 10:21:28 +01:00
Donal McBreen
aa9999809c Merge pull request #439 from basecamp/zero-downtime-deploy-file
Zero downtime deployment with cord file
2023-09-07 09:34:40 +01:00
Donal McBreen
6263bf96ba Merge pull request #438 from basecamp/remote-env-file
Copy env files to remote hosts
2023-09-07 09:34:22 +01:00
Gianni Chiappetta
9a539ffc86 chore: update tests to remove hardcoded ssh auth method 2023-09-06 10:59:17 -04:00
Donal McBreen
8a41d15b69 Zero downtime deployment with cord file
When replacing a container currently we:
1. Boot the new container
2. Wait for it to become healthy
3. Stop the old container

Traefik will send requests to the old container until it notices that it
is unhealthy. But it may have stopped serving requests before that point
which can result in errors.

To get round that the new boot process is:

1. Create a directory with a single file on the host
2. Boot the new container, mounting the cord file into /tmp and
including a check for the file in the docker healthcheck
3. Wait for it to become healthy
4. Delete the healthcheck file ("cut the cord") for the old container
5. Wait for it to become unhealthy and give Traefik a couple of seconds
to notice
6. Stop the old container

The extra steps ensure that Traefik stops sending requests before the
old container is shutdown.
2023-09-06 14:35:30 +01:00
Donal McBreen
94bf090657 Copy env files to remote hosts
Setting env variables in the docker arguments requires having them on
the deploy host.

Instead we'll add two new commands `kamal env push` and
`kamal env delete` which will manage copying the environment as .env
files to the remote host.

Docker will pick up the file with `--env-file <path-to-file>`. Env files
will be stored under `<kamal run directory>/env`.

Running `kamal env push` will create env files for each role and
accessory, and traefik if required.

`kamal envify` has been updated to also push the env files.

By avoiding using `kamal envify` and creating the local and remote
secrets manually, you can now avoid accessing secrets needed
for the docker runtime environment locally. You will still need build
secrets.

One thing to note - the Docker doesn't parse the environment variables
in the env file, one result of this is that you can't specify multi-line
values - see https://github.com/moby/moby/issues/12997.

We maybe need to look docker config or docker secrets longer term to get
around this.

Hattip to @kevinmcconnell - this was all his idea.
2023-09-06 14:33:13 +01:00
Donal McBreen
adc7173cf2 Merge pull request #437 from basecamp/kamal-run-directory
Configurable Kamal directory
2023-09-06 14:31:07 +01:00
Donal McBreen
fd6bf5324a Merge pull request #443 from rience/custom-healthcheck-port
Configurable Healthcheck Expose Port
2023-09-06 11:09:48 +01:00
Krzysztof Adamski
c2b2f7ea33 Fixing Tests 2023-09-06 10:16:59 +02:00
Krzysztof Adamski
bbcc90e4d1 Configurable Healthcheck Expose Port 2023-09-05 10:53:32 +02:00
Gianni Chiappetta
84f78cd9f9 fix: do not hardcode Net::SSH auth_methods 2023-09-01 15:11:12 -04:00
Donal McBreen
787688ea08 kamal -> .kamal 2023-08-28 17:13:52 +01:00
Donal McBreen
bcfa1d83e8 Configurable Kamal directory
To avoid polluting the default SSH directory with lots of Kamal config,
we'll default to putting them in a `kamal` sub directory.

But also make the directory configurable with the `run_directory` key,
so for example you can set it as `/var/run/kamal/`

The directory is created during bootstrap or before any command that
will need to access a file.
2023-08-28 16:32:18 +01:00
David Heinemeier Hansson
9363b6a464 Bump version for 0.16.1 2023-08-24 09:16:13 -07:00
David Heinemeier Hansson
338fd4e493 Merge pull request #428 from tbuehlmann/main
Fix picking the first available role on primary_host
2023-08-24 08:36:29 -07:00
David Heinemeier Hansson
eb3cb81a79 Merge pull request #368 from tsvallender/main 2023-08-24 06:12:48 -07:00
Tobias Bühlmann
556f7f5a37 Fix picking the first available role on primary_host 2023-08-24 13:50:24 +02:00
Trevor Vallender
c2ec04f8c1 Allow Traefik to run without publishing port
Adds the `publish` option which, if set to false, does not pass `--publish` to
`docker run` when starting Traefik. This is useful when running Traefik
behind a reverse proxy, for example.
2023-08-24 10:52:10 +01:00
David Heinemeier Hansson
519659b84c Merge pull request #422 from fig/fix-421
require ActiveSupport module to provide String#remove
2023-08-23 13:50:04 -07:00
David Heinemeier Hansson
560d0698ac Merge pull request #426 from northeastprince/fix-site-in-gemspec
Fix site URL in gemspec
2023-08-23 13:47:26 -07:00
fig
f40e8e9af1 Merge branch 'fix-421' of https://github.com/fig/mrsk into fix-421 2023-08-23 15:22:41 +01:00
fig
1ab7405e36 require ActiveSupport module to provide String#remove
fixes #421
2023-08-23 15:17:26 +01:00
Matt Almeida
aeadd7c11f Fix site URL in gemspec 2023-08-23 15:15:51 +02:00
Donal McBreen
d0fbf538d3 Add integration test hooks back in 2023-08-23 07:36:48 +01:00
David Heinemeier Hansson
cfe77934e8 Update README.md
Point all docs to the site so we don't duplicate everything.
2023-08-22 17:11:26 -07:00
David Heinemeier Hansson
3f6ca1648e Update docker-publish.yml
Require setting tag
2023-08-22 15:44:07 -07:00
David Heinemeier Hansson
7c6d302baa Update docker-publish.yml
Allow manual invocation
2023-08-22 15:20:02 -07:00
fig
b8eb50b982 require ActiveSupport module to provide String#remove
fixes #421
2023-08-22 20:58:48 +01:00
David Heinemeier Hansson
d981c3c968 Move hooks 2023-08-22 12:47:00 -07:00
David Heinemeier Hansson
416860d9b0 Update docker-publish.yml
Reflect rename
2023-08-22 12:34:57 -07:00
David Heinemeier Hansson
33d5d7e9a2 Update README.md
Point to name change.
2023-08-22 12:20:24 -07:00
David Heinemeier Hansson
99c1102a3a Update README.md
Will do a new video shortly.
2023-08-22 12:13:54 -07:00
David Heinemeier Hansson
ac11089c7a Bump version for 0.16.0 2023-08-22 11:42:32 -07:00
David Heinemeier Hansson
180ca219df Merge pull request #423 from basecamp/rename
Rename project to Kamal
2023-08-22 11:41:42 -07:00
David Heinemeier Hansson
dc1421a1fc Correct casing 2023-08-22 09:22:32 -07:00
David Heinemeier Hansson
c4a203e648 Rename to Kamal 2023-08-22 08:24:31 -07:00
Donal McBreen
e2c3709d74 Merge pull request #417 from manastyretskyi/main
Fix builder registry cache when using default registry
2023-08-17 14:08:05 +01:00
Liubomyr Manastyretskyi
f68a33465f Fix review comments 2023-08-17 11:58:14 +03:00
Donal McBreen
e7bc74d9ee Merge pull request #418 from mrsked/ssh-logging
Configurable log levels
2023-08-16 07:22:18 +01:00
Donal McBreen
1163c3de07 Configurable log levels
Allow ssh log_level to be set - this will help to debug connection
issues.
2023-08-15 16:51:56 +01:00
Donal McBreen
715cd94bbf Merge pull request #413 from mrsked/extract-version-from-container-name-correctly
Extract versions that contains dashes
2023-08-15 15:11:03 +01:00
Donal McBreen
dda7099b2f Merge pull request #414 from mrsked/traefik-start-stop-run-errors
Don't hide Traefik errors
2023-08-15 15:10:47 +01:00
Donal McBreen
4262fce863 Merge pull request #415 from igor-alexandrov/fix-builder-configuration-validation
Removed validation for remote and local builder params
2023-08-15 15:10:23 +01:00
Liubomyr Manastyretskyi
6774675547 Fix builder registry cache when using default registry 2023-08-13 12:04:03 +03:00
Igor Alexandrov
0c52a1053e Removed not needed configuration test 2023-08-08 19:14:03 +04:00
Igor Alexandrov
c24c7abb79 Fix for https://github.com/mrsked/mrsk/issues/407 2023-08-08 19:04:35 +04:00
Donal McBreen
c2d7fd775f Don't hide Traefik errors
When stopping or starting Traefik, don't hide important errors.

Docker doesn't return an error when starting a started container or
stopping a stopped container.

When rebooting we want to know about errors during run as we've just
stopped and removed the previous container.

When booting, we want to leave the running container if it exists,
restart a stopped container and run a new one if none exists.

We can implement this with `docker start ... || docker run ...`:
- if the container is started, `docker start` will exit with 0
- if the container is stopped, `docker start` will start it and exit with 0
- if the container doesn't exist, `docker start` will return a non zero
exit code and `docker run` will create a new container. Any errors in
`docker run` will be returned.
2023-08-08 15:41:16 +01:00
Donal McBreen
4dd8208290 Extract versions that contains dashes
The version extraction assumed that the version is everything after the
last `-` in the container name. This doesn't work if you deploy a
non-MRSK generated version that contains a `-`.

To fix we'll generate the non version prefix and strip it off. In some
places for this to work we need to make sure to pass the role through.

Fixes: https://github.com/mrsked/mrsk/issues/402
2023-08-08 14:16:32 +01:00
Donal McBreen
aa89ededde Merge pull request #399 from mrsked/manage-ssh-connection-starts
Manage SSH connection starts
2023-08-07 14:37:34 +01:00
David Heinemeier Hansson
299b166db7 Merge pull request #389 from brunoprietog/include-role-options-when-executing-commands
Include role options when executing commands
2023-07-26 14:04:28 +02:00
Donal McBreen
94d6a763a8 Extract ssh and sshkit configuration 2023-07-26 12:26:23 +01:00
Donal McBreen
752ff53458 Merge pull request #396 from igor-alexandrov/track-uncommitted-changes
Log uncommitted changes during deploy
2023-07-25 14:35:44 +01:00
Donal McBreen
eb8c97a417 Document new sshkit settings 2023-07-25 13:09:49 +01:00
Donal McBreen
f64b596907 Prevent SSH connection restarts
Set a high idle timeout on the sshkit connection pool. This will
reduce the incidence of re-connection storms when a deployment has been
idle for a while (e.g. when waiting for a docker build).

The default timeout was 30 seconds, so we'll enable keepalives at a
30s interval to match. This is to help prevent connections from being
killed during long idle periods.
2023-07-25 13:09:46 +01:00
Donal McBreen
b25cfa178b Limit SSH start concurrency
Starting many (90+) SSH connections has caused us some issues such as
failed DNS lookups and hitting process file descriptor limits.

To mitigate this, patch SSHKit::Backend::Netssh to limit concurrency of
connection starts. We'll default to 30 at a time which seems to work
without issue, but can be configured via:

```
sshkit:
  max_concurrent_starts: 10
```
2023-07-25 13:08:44 +01:00
Donal McBreen
edcfc77d95 Bump version for 0.15.1 2023-07-25 13:07:04 +01:00
Donal McBreen
a71e167a03 Merge pull request #400 from mrsked/revert-386-ssh-log-levels
Revert "Configurable SSH log levels"
2023-07-25 13:04:21 +01:00
Donal McBreen
2daaf442fa Revert "Configurable SSH log levels" 2023-07-25 12:53:45 +01:00
Igor Alexandrov
d414253393 Updated uncommitted notification text 2023-07-24 20:12:22 +04:00
Bruno Prieto
cbd180205d Include role options when executing commands 2023-07-24 17:45:24 +02:00
Donal McBreen
61b7dc90f2 Bump version for 0.15.0 2023-07-24 13:43:50 +01:00
David Heinemeier Hansson
f6442513ae Merge pull request #357 from igor-alexandrov/documentation-update
Updated README with info for Rails <7 usage
2023-07-24 14:39:48 +02:00
Igor Alexandrov
ea941f33f9 Moved uncommitted changes message out of run_locally block 2023-07-21 22:45:23 +04:00
Igor Alexandrov
9c2a1dc7cd Removed commented code in tests 2023-07-21 18:44:01 +04:00
Igor Alexandrov
0cfafd1d25 Log uncommitted changes during deploy 2023-07-21 18:37:45 +04:00
Donal McBreen
5e8df58e6b Merge pull request #393 from basecamp/rolling-traefik-restarts
Support a --rolling option for traefik reboots
2023-07-19 16:43:59 +01:00
Lewis Buckley
9d5a6d1321 Document the rolling option for traefik reboots 2023-07-19 15:03:15 +01:00
Lewis Buckley
ecfd258093 Document the rolling reboot option 2023-07-19 14:58:46 +01:00
Lewis Buckley
313f89a108 Merge branch 'main' into rolling-traefik-restarts
* main:
  Removed not needed MRSK.traefik.run command in Traefil reboot
  Updated README with locking directory name
  Include service name to lock details
  Configurable SSH log levels
  Add registry container output to debug
  Minor tweaks to hooks section in readme
  Update README.md
  Updated README.md to make setup examples consistent
  Login to the registry proactively before stoping Accessory and Traefik
2023-07-19 14:46:16 +01:00
Lewis Buckley
9ab448e186 Support a --rolling option for traefik reboots 2023-07-19 14:39:27 +01:00
Donal McBreen
e1433f3895 Merge pull request #349 from igor-alexandrov/login-to-registry-proactively
Login to the registry proactively before stoping Accessory and Traefik
2023-07-19 13:36:00 +01:00
Igor Alexandrov
a29e188c90 Removed not needed MRSK.traefik.run command in Traefil reboot 2023-07-19 15:08:42 +04:00
Donal McBreen
95e3915991 Merge pull request #386 from mrsked/ssh-log-levels
Configurable SSH log levels
2023-07-17 14:09:21 +01:00
Donal McBreen
30d342183d Merge pull request #387 from igor-alexandrov/lock-with-service-name
Include service name to lock details
2023-07-17 14:07:22 +01:00
Igor Alexandrov
83f5f3f053 Updated README with locking directory name 2023-07-17 10:39:50 +04:00
Igor Alexandrov
e6ca270537 Include service name to lock details 2023-07-15 21:50:39 +04:00
Donal McBreen
cd88c49c42 Configurable SSH log levels
Allow ssh log_level to be set to debug connection issues.
2023-07-14 16:08:47 +01:00
Donal McBreen
d03195ce1c Merge pull request #385 from mrsked/integration-test-registry-debug
Add registry container output to debug
2023-07-14 13:50:31 +01:00
Donal McBreen
da1c049829 Add registry container output to debug 2023-07-14 13:41:30 +01:00
Donal McBreen
4095e1853d Merge pull request #356 from helgeblod/fix-readme-username-typo
Update README.md to make setup examples consistent
2023-07-12 13:24:51 +01:00
Donal McBreen
dbc9989730 Merge pull request #364 from nickhammond/patch-2
Update README.md
2023-07-12 13:22:53 +01:00
Donal McBreen
e493369453 Merge pull request #379 from nickhammond/patch-3
Minor tweaks to hooks section in readme
2023-07-12 11:29:36 +01:00
Nick Hammond
e760cfa457 Minor tweaks to hooks section in readme 2023-07-08 10:59:55 -06:00
Nick Hammond
f8d651af0d Update README.md
Add a note about utilizing biometrics with the envify example.
2023-06-28 10:03:27 -06:00
Igor Alexandrov
08172be375 Updated README with info for Rails <7 usage 2023-06-26 16:05:24 +04:00
Jonas Helgemo
a3cc2317e2 Updated README.md to make setup examples consistent
- SSH and apt examples should use same username
2023-06-26 11:57:23 +02:00
Igor Alexandrov
2746a48e88 Login to the registry proactively before stoping Accessory and Traefik 2023-06-22 15:13:47 +04:00
David Heinemeier Hansson
9a501867b4 Bump version for 0.14.0 2023-06-20 17:02:42 +02:00
David Heinemeier Hansson
c5397ff51e Merge pull request #342 from mrsked/only-require-secrets-when-mutating
Only require secrets when mutating
2023-06-20 16:50:14 +02:00
Donal McBreen
4950f61a87 Only require secrets when mutating
Rename `with_lock` to more generic `mutating` and move the env_args
check to that point. This allows read-only actions to be run without
requiring secrets.
2023-06-20 15:39:51 +01:00
David Heinemeier Hansson
08d8790851 Merge pull request #337 from igor-alexandrov/feature/cache
Support for Docker multistage build cache
2023-06-20 11:38:46 +02:00
Igor Alexandrov
02256ac8fe More code style improvements 2023-06-19 18:22:07 +04:00
Igor Alexandrov
dadd8225da Various code style improvements 2023-06-18 23:39:44 +04:00
Igor Alexandrov
aa28ee0f3e Inroduce Native::Cached builder 2023-06-18 22:45:04 +04:00
David Heinemeier Hansson
2007ab475e Merge pull request #327 from bestfriendfinance/fix-run-over-ssh-with-proxy-command
Add support for proxy_command to run_over_ssh
2023-06-18 17:18:50 +02:00
Igor Alexandrov
4df3389d09 Added support for multistage build cache 2023-06-18 19:02:10 +04:00
Matt Robinson
21b13bf8d3 Add support for proxy_command to run_over_ssh 2023-06-16 08:22:10 -03:00
David Heinemeier Hansson
6e6f696717 Merge pull request #335 from mrsked/specify-min-version
Add a minimum version setting
2023-06-16 10:02:40 +02:00
Donal McBreen
98c12a254e Add a minimum version setting
Allow a minimum MRSK version to be specified in the config.
2023-06-15 14:53:03 +01:00
Donal McBreen
f0301d2007 Merge pull request #328 from igor-alexandrov/319-override-default-traefik-args
Added ability to override default Traefik command line arguments
2023-06-15 14:49:56 +01:00
Igor Alexandrov
d3f5e9efe8 Updated Traefik CLI test 2023-06-15 17:11:20 +04:00
Igor Alexandrov
d9b3fac17a Added ability to override default Traefik command line arguments 2023-06-15 15:41:20 +04:00
Donal McBreen
cd5c41ddbe Merge pull request #334 from basecamp/fix-ssh-symlink
Fix ssh symlink
2023-06-15 12:14:30 +01:00
Donal McBreen
a14c6141e5 Dump container logs on failure 2023-06-15 12:05:50 +01:00
Donal McBreen
95d6ee5031 Remove /root/.ssh before symlinking
Ensure the symlinks are created correctly whether or not /root/.ssh
already exists.
2023-06-15 12:02:56 +01:00
David Heinemeier Hansson
80a4ca4f8a Merge pull request #331 from basecamp/fix-sample-pre-deploy-hook
Fix up the sample pre-deploy hook
2023-06-12 14:09:29 +02:00
Donal McBreen
12ca865e71 Fix up the sample pre-deploy hook
- Fix the shebang
- Extract a GithubStatusChecks class
2023-06-12 12:47:51 +01:00
306 changed files with 14415 additions and 5548 deletions

View File

@@ -4,25 +4,47 @@ on:
branches: branches:
- main - main
pull_request: pull_request:
workflow_dispatch:
jobs: jobs:
rubocop:
name: RuboCop
runs-on: ubuntu-latest
env:
BUNDLE_ONLY: rubocop
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Setup Ruby and install gems
uses: ruby/setup-ruby@v1
with:
ruby-version: 3.3.0
bundler-cache: true
- name: Run Rubocop
run: bundle exec rubocop --parallel
tests: tests:
strategy: strategy:
fail-fast: false
matrix: matrix:
ruby-version: ruby-version:
- "2.7"
- "3.1" - "3.1"
- "3.2" - "3.2"
- "3.3"
- "3.4"
gemfile: gemfile:
- Gemfile - Gemfile
- gemfiles/rails_edge.gemfile - gemfiles/rails_edge.gemfile
continue-on-error: [false] exclude:
- ruby-version: "3.1"
gemfile: gemfiles/rails_edge.gemfile
name: ${{ format('Tests (Ruby {0})', matrix.ruby-version) }} name: ${{ format('Tests (Ruby {0})', matrix.ruby-version) }}
runs-on: ubuntu-latest runs-on: ubuntu-latest
continue-on-error: ${{ matrix.continue-on-error }}
env: env:
BUNDLE_GEMFILE: ${{ github.workspace }}/${{ matrix.gemfile }} BUNDLE_GEMFILE: ${{ github.workspace }}/${{ matrix.gemfile }}
steps: steps:
- uses: actions/checkout@v2 - uses: actions/checkout@v4
- name: Remove gemfile.lock
run: rm Gemfile.lock
- name: Install Ruby - name: Install Ruby
uses: ruby/setup-ruby@v1 uses: ruby/setup-ruby@v1
@@ -32,3 +54,5 @@ jobs:
- name: Run tests - name: Run tests
run: bin/test run: bin/test
env:
RUBYOPT: ${{ startsWith(matrix.ruby-version, '3.4.') && '--enable=frozen-string-literal' || '' }}

View File

@@ -1,6 +1,12 @@
name: Docker name: Docker
on: on:
workflow_dispatch:
inputs:
tagInput:
description: 'Tag'
required: true
release: release:
types: [created] types: [created]
tags: tags:
@@ -29,6 +35,14 @@ jobs:
registry: ghcr.io registry: ghcr.io
username: ${{ github.actor }} username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
- name: Determine version tag
id: version-tag
run: |
INPUT_VALUE="${{ github.event.inputs.tagInput }}"
if [ -z "$INPUT_VALUE" ]; then
INPUT_VALUE="${{ github.ref_name }}"
fi
echo "::set-output name=value::$INPUT_VALUE"
- -
name: Build and push name: Build and push
uses: docker/build-push-action@v3 uses: docker/build-push-action@v3
@@ -37,5 +51,5 @@ jobs:
platforms: linux/amd64,linux/arm64 platforms: linux/amd64,linux/arm64
push: true push: true
tags: | tags: |
ghcr.io/mrsked/mrsk:latest ghcr.io/basecamp/kamal:latest
ghcr.io/mrsked/mrsk:${{ github.ref_name }} ghcr.io/basecamp/kamal:${{ steps.version-tag.outputs.value }}

2
.rubocop.yml Normal file
View File

@@ -0,0 +1,2 @@
inherit_gem:
rubocop-rails-omakase: rubocop.yml

View File

@@ -1,10 +1,10 @@
# Contributor Code of Conduct # Contributor Code of Conduct
As contributors and maintainers of the MRSK project, we pledge to create a welcoming and inclusive environment for everyone. We value the participation of each member of our community and want all contributors to feel respected and valued. As contributors and maintainers of the Kamal project, we pledge to create a welcoming and inclusive environment for everyone. We value the participation of each member of our community and want all contributors to feel respected and valued.
We are committed to providing a harassment-free experience for everyone, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age, or religion (or lack thereof). We do not tolerate harassment of participants in any form. We are committed to providing a harassment-free experience for everyone, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age, or religion (or lack thereof). We do not tolerate harassment of participants in any form.
This code of conduct applies to all MRSK project spaces, including but not limited to project code, issue trackers, chat rooms, and mailing lists. Violations of this code of conduct may result in removal from the project community. This code of conduct applies to all Kamal project spaces, including but not limited to project code, issue trackers, chat rooms, and mailing lists. Violations of this code of conduct may result in removal from the project community.
## Our standards ## Our standards

View File

@@ -1,18 +1,18 @@
# Contributing to MRSK development # Contributing to Kamal development
Thank you for considering contributing to MRSK! This document outlines some guidelines for contributing to this open source project. Thank you for considering contributing to Kamal! This document outlines some guidelines for contributing to this open source project.
Please make sure to review our [Code of Conduct](CODE_OF_CONDUCT.md) before contributing to MRSK. Please make sure to review our [Code of Conduct](CODE_OF_CONDUCT.md) before contributing to Kamal.
There are several ways you can contribute to the betterment of the project: There are several ways you can contribute to the betterment of the project:
- **Report an issue?** - If the issue isnt reported, we cant fix it. Please report any bugs, feature, and/or improvement requests on the [MRSK GitHub Issues tracker](https://github.com/mrsked/mrsk/issues). - **Report an issue?** - If the issue isnt reported, we cant fix it. Please report any bugs, feature, and/or improvement requests on the [Kamal GitHub Issues tracker](https://github.com/basecamp/kamal/issues).
- **Submit patches** - Do you have a new feature or a fix you'd like to share? [Submit a pull request](https://github.com/mrsked/mrsk/pulls)! - **Submit patches** - Do you have a new feature or a fix you'd like to share? [Submit a pull request](https://github.com/basecamp/kamal/pulls)!
- **Write blog articles** - Are you using MRSK? We'd love to hear how you're using it with your projects. Write a tutorial and post it on your blog! - **Write blog articles** - Are you using Kamal? We'd love to hear how you're using it with your projects. Write a tutorial and post it on your blog!
## Issues ## Issues
If you encounter any issues with the project, please check the [existing issues](https://github.com/mrsked/mrsk/issues) first to see if the issue has already been reported. If the issue hasn't been reported, please open a new issue with a clear description of the problem and steps to reproduce it. If you encounter any issues with the project, please check the [existing issues](https://github.com/basecamp/kamal/issues) first to see if the issue has already been reported. If the issue hasn't been reported, please open a new issue with a clear description of the problem and steps to reproduce it.
## Pull Requests ## Pull Requests
@@ -33,17 +33,17 @@ A good commit message should describe what changed and why.
## Development ## Development
The `main` branch is regularly built and tested, but it is not guaranteed to be completely stable. Tags are created regularly from release branches to indicate new official, stable release versions of MRSK. The `main` branch is regularly built and tested, but it is not guaranteed to be completely stable. Tags are created regularly from release branches to indicate new official, stable release versions of Kamal.
MRSK is written in Ruby. You should have Ruby 3.2+ installed on your machine in order to work on MRSK. If that's already setup, run `bundle` in the root directory to install all dependencies. Then you can run `bin/test` to run all tests. Kamal is written in Ruby. You should have Ruby 3.2+ installed on your machine in order to work on Kamal. If that's already setup, run `bundle` in the root directory to install all dependencies. Then you can run `bin/test` to run all tests.
1. Fork the project repository. 1. Fork the project repository.
2. Create a new branch for your contribution. 2. Create a new branch for your contribution.
3. Write your code or make the desired changes. 3. Write your code or make the desired changes.
4. **Ensure that your code passes the project's minitests by running ./bin/test.** 4. **Ensure that your code passes the project's minitests by running ./bin/test.**
5. Commit your changes and push them to your forked repository. 5. Commit your changes and push them to your forked repository.
6. [Open a pull request](https://github.com/mrsked/mrsk/pulls) to the main project repository with a detailed description of your changes. 6. [Open a pull request](https://github.com/basecamp/kamal/pulls) to the main project repository with a detailed description of your changes.
## License ## License
MRSK is released under the MIT License. By contributing to this project, you agree to license your contributions under the same license. Kamal is released under the MIT License. By contributing to this project, you agree to license your contributions under the same license.

View File

@@ -1,22 +1,21 @@
# Use the official Ruby 3.2.0 Alpine image as the base image FROM ruby:3.4-alpine
FROM ruby:3.2.0-alpine
# Install docker/buildx-bin # Install docker/buildx-bin
COPY --from=docker/buildx-bin /buildx /usr/libexec/docker/cli-plugins/docker-buildx COPY --from=docker/buildx-bin /buildx /usr/libexec/docker/cli-plugins/docker-buildx
# Set the working directory to /mrsk # Set the working directory to /kamal
WORKDIR /mrsk WORKDIR /kamal
# Copy the Gemfile, Gemfile.lock into the container # Copy the Gemfile, Gemfile.lock into the container
COPY Gemfile Gemfile.lock mrsk.gemspec ./ COPY Gemfile Gemfile.lock kamal.gemspec ./
# Required in mrsk.gemspec # Required in kamal.gemspec
COPY lib/mrsk/version.rb /mrsk/lib/mrsk/version.rb COPY lib/kamal/version.rb /kamal/lib/kamal/version.rb
# Install system dependencies # Install system dependencies
RUN apk add --no-cache --update build-base git docker openrc openssh-client-default \ RUN apk add --no-cache build-base git docker openrc openssh-client-default yaml-dev \
&& rc-update add docker boot \ && rc-update add docker boot \
&& gem install bundler --version=2.4.3 \ && gem install bundler --version=2.6.5 \
&& bundle install && bundle install
# Copy the rest of our application code into the container. # Copy the rest of our application code into the container.
@@ -25,16 +24,16 @@ RUN apk add --no-cache --update build-base git docker openrc openssh-client-defa
COPY . . COPY . .
# Install the gem locally from the project folder # Install the gem locally from the project folder
RUN gem build mrsk.gemspec && \ RUN gem build kamal.gemspec && \
gem install ./mrsk-*.gem --no-document gem install ./kamal-*.gem --no-document
# Set the working directory to /workdir # Set the working directory to /workdir
WORKDIR /workdir WORKDIR /workdir
# Tell git it's safe to access /workdir/.git even if # Tell git it's safe to access /workdir/.git even if
# the directory is owned by a different user # the directory is owned by a different user
RUN git config --global --add safe.directory /workdir RUN git config --global --add safe.directory '*'
# Set the entrypoint to run the installed binary in /workdir # Set the entrypoint to run the installed binary in /workdir
# Example: docker run -it -v "$PWD:/workdir" mrsk init # Example: docker run -it -v "$PWD:/workdir" kamal init
ENTRYPOINT ["mrsk"] ENTRYPOINT ["kamal"]

View File

@@ -1,4 +1,8 @@
source 'https://rubygems.org' source "https://rubygems.org"
git_source(:github) { |repo| "https://github.com/#{repo}.git" } git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gemspec gemspec
group :rubocop do
gem "rubocop-rails-omakase", require: false
end

View File

@@ -1,106 +1,196 @@
PATH PATH
remote: . remote: .
specs: specs:
mrsk (0.13.2) kamal (2.5.3)
activesupport (>= 7.0) activesupport (>= 7.0)
base64 (~> 0.2)
bcrypt_pbkdf (~> 1.0) bcrypt_pbkdf (~> 1.0)
dotenv (~> 2.8) concurrent-ruby (~> 1.2)
dotenv (~> 3.1)
ed25519 (~> 1.2) ed25519 (~> 1.2)
net-ssh (~> 7.0) net-ssh (~> 7.3)
sshkit (~> 1.21) sshkit (>= 1.23.0, < 2.0)
thor (~> 1.2) thor (~> 1.3)
zeitwerk (~> 2.5) zeitwerk (>= 2.6.18, < 3.0)
GEM GEM
remote: https://rubygems.org/ remote: https://rubygems.org/
specs: specs:
actionpack (7.0.4.3) actionpack (8.0.0.1)
actionview (= 7.0.4.3) actionview (= 8.0.0.1)
activesupport (= 7.0.4.3) activesupport (= 8.0.0.1)
rack (~> 2.0, >= 2.2.0) nokogiri (>= 1.8.5)
rack (>= 2.2.4)
rack-session (>= 1.0.1)
rack-test (>= 0.6.3) rack-test (>= 0.6.3)
rails-dom-testing (~> 2.0) rails-dom-testing (~> 2.2)
rails-html-sanitizer (~> 1.0, >= 1.2.0) rails-html-sanitizer (~> 1.6)
actionview (7.0.4.3) useragent (~> 0.16)
activesupport (= 7.0.4.3) actionview (8.0.0.1)
activesupport (= 8.0.0.1)
builder (~> 3.1) builder (~> 3.1)
erubi (~> 1.4) erubi (~> 1.11)
rails-dom-testing (~> 2.0) rails-dom-testing (~> 2.2)
rails-html-sanitizer (~> 1.1, >= 1.2.0) rails-html-sanitizer (~> 1.6)
activesupport (7.0.4.3) activesupport (8.0.0.1)
concurrent-ruby (~> 1.0, >= 1.0.2) base64
benchmark (>= 0.3)
bigdecimal
concurrent-ruby (~> 1.0, >= 1.3.1)
connection_pool (>= 2.2.5)
drb
i18n (>= 1.6, < 2) i18n (>= 1.6, < 2)
logger (>= 1.4.2)
minitest (>= 5.1) minitest (>= 5.1)
tzinfo (~> 2.0) securerandom (>= 0.3)
bcrypt_pbkdf (1.1.0) tzinfo (~> 2.0, >= 2.0.5)
builder (3.2.4) uri (>= 0.13.1)
concurrent-ruby (1.2.2) ast (2.4.2)
base64 (0.2.0)
bcrypt_pbkdf (1.1.1)
benchmark (0.4.0)
bigdecimal (3.1.8)
builder (3.3.0)
concurrent-ruby (1.3.4)
connection_pool (2.4.1)
crass (1.0.6) crass (1.0.6)
debug (1.7.2) date (3.4.1)
irb (>= 1.5.0) debug (1.9.2)
reline (>= 0.3.1) irb (~> 1.10)
dotenv (2.8.1) reline (>= 0.3.8)
dotenv (3.1.5)
drb (2.2.1)
ed25519 (1.3.0) ed25519 (1.3.0)
erubi (1.12.0) erubi (1.13.0)
i18n (1.12.0) i18n (1.14.6)
concurrent-ruby (~> 1.0) concurrent-ruby (~> 1.0)
io-console (0.6.0) io-console (0.8.0)
irb (1.6.3) irb (1.14.2)
reline (>= 0.3.0) rdoc (>= 4.0.0)
loofah (2.20.0) reline (>= 0.4.2)
json (2.9.0)
language_server-protocol (3.17.0.3)
logger (1.6.3)
loofah (2.23.1)
crass (~> 1.0.2) crass (~> 1.0.2)
nokogiri (>= 1.5.9) nokogiri (>= 1.12.0)
method_source (1.0.0) minitest (5.25.4)
minitest (5.18.0) mocha (2.7.1)
mocha (2.0.2)
ruby2_keywords (>= 0.0.5) ruby2_keywords (>= 0.0.5)
net-scp (4.0.0) net-scp (4.0.0)
net-ssh (>= 2.6.5, < 8.0.0) net-ssh (>= 2.6.5, < 8.0.0)
net-ssh (7.1.0) net-sftp (4.0.0)
nokogiri (1.14.2-arm64-darwin) net-ssh (>= 5.0.0, < 8.0.0)
net-ssh (7.3.0)
nokogiri (1.18.3-aarch64-linux-musl)
racc (~> 1.4) racc (~> 1.4)
nokogiri (1.14.2-x86_64-darwin) nokogiri (1.18.3-arm64-darwin)
racc (~> 1.4) racc (~> 1.4)
nokogiri (1.14.2-x86_64-linux) nokogiri (1.18.3-x86_64-darwin)
racc (~> 1.4) racc (~> 1.4)
racc (1.6.2) nokogiri (1.18.3-x86_64-linux-gnu)
rack (2.2.6.4) racc (~> 1.4)
nokogiri (1.18.3-x86_64-linux-musl)
racc (~> 1.4)
ostruct (0.6.1)
parallel (1.26.3)
parser (3.3.6.0)
ast (~> 2.4.1)
racc
psych (5.2.1)
date
stringio
racc (1.8.1)
rack (3.1.10)
rack-session (2.0.0)
rack (>= 3.0.0)
rack-test (2.1.0) rack-test (2.1.0)
rack (>= 1.3) rack (>= 1.3)
rails-dom-testing (2.0.3) rackup (2.2.1)
activesupport (>= 4.2.0) rack (>= 3)
rails-dom-testing (2.2.0)
activesupport (>= 5.0.0)
minitest
nokogiri (>= 1.6) nokogiri (>= 1.6)
rails-html-sanitizer (1.5.0) rails-html-sanitizer (1.6.2)
loofah (~> 2.19, >= 2.19.1) loofah (~> 2.21)
railties (7.0.4.3) nokogiri (>= 1.15.7, != 1.16.7, != 1.16.6, != 1.16.5, != 1.16.4, != 1.16.3, != 1.16.2, != 1.16.1, != 1.16.0.rc1, != 1.16.0)
actionpack (= 7.0.4.3) railties (8.0.0.1)
activesupport (= 7.0.4.3) actionpack (= 8.0.0.1)
method_source activesupport (= 8.0.0.1)
irb (~> 1.13)
rackup (>= 1.0.0)
rake (>= 12.2) rake (>= 12.2)
thor (~> 1.0) thor (~> 1.0, >= 1.2.2)
zeitwerk (~> 2.5) zeitwerk (~> 2.6)
rake (13.0.6) rainbow (3.1.1)
reline (0.3.3) rake (13.2.1)
rdoc (6.8.1)
psych (>= 4.0.0)
regexp_parser (2.9.3)
reline (0.5.12)
io-console (~> 0.5) io-console (~> 0.5)
rubocop (1.69.2)
json (~> 2.3)
language_server-protocol (>= 3.17.0)
parallel (~> 1.10)
parser (>= 3.3.0.2)
rainbow (>= 2.2.2, < 4.0)
regexp_parser (>= 2.9.3, < 3.0)
rubocop-ast (>= 1.36.2, < 2.0)
ruby-progressbar (~> 1.7)
unicode-display_width (>= 2.4.0, < 4.0)
rubocop-ast (1.36.2)
parser (>= 3.3.1.0)
rubocop-minitest (0.36.0)
rubocop (>= 1.61, < 2.0)
rubocop-ast (>= 1.31.1, < 2.0)
rubocop-performance (1.23.0)
rubocop (>= 1.48.1, < 2.0)
rubocop-ast (>= 1.31.1, < 2.0)
rubocop-rails (2.27.0)
activesupport (>= 4.2.0)
rack (>= 1.1)
rubocop (>= 1.52.0, < 2.0)
rubocop-ast (>= 1.31.1, < 2.0)
rubocop-rails-omakase (1.0.0)
rubocop
rubocop-minitest
rubocop-performance
rubocop-rails
ruby-progressbar (1.13.0)
ruby2_keywords (0.0.5) ruby2_keywords (0.0.5)
sshkit (1.21.4) securerandom (0.4.0)
sshkit (1.23.2)
base64
net-scp (>= 1.1.2) net-scp (>= 1.1.2)
net-sftp (>= 2.1.2)
net-ssh (>= 2.8.0) net-ssh (>= 2.8.0)
thor (1.2.1) ostruct
stringio (3.1.2)
thor (1.3.2)
tzinfo (2.0.6) tzinfo (2.0.6)
concurrent-ruby (~> 1.0) concurrent-ruby (~> 1.0)
zeitwerk (2.6.7) unicode-display_width (3.1.2)
unicode-emoji (~> 4.0, >= 4.0.4)
unicode-emoji (4.0.4)
uri (1.0.2)
useragent (0.16.11)
zeitwerk (2.7.1)
PLATFORMS PLATFORMS
aarch64-linux-musl
arm64-darwin arm64-darwin
x86_64-darwin x86_64-darwin
x86_64-linux x86_64-linux
x86_64-linux-musl
DEPENDENCIES DEPENDENCIES
debug debug
kamal!
mocha mocha
mrsk!
railties railties
rubocop-rails-omakase
BUNDLED WITH BUNDLED WITH
2.4.3 2.6.5

928
README.md
View File

@@ -1,929 +1,13 @@
# MRSK # Kamal: Deploy web apps anywhere
MRSK deploys web apps anywhere from bare metal to cloud VMs using Docker with zero downtime. It uses the dynamic reverse-proxy Traefik to hold requests while the new application container is started and the old one is stopped. It works seamlessly across multiple hosts, using SSHKit to execute commands. It was built for Rails applications, but works with any type of web app that can be containerized with Docker. From bare metal to cloud VMs, deploy web apps anywhere with zero downtime. Kamal uses [kamal-proxy](https://github.com/basecamp/kamal-proxy) to seamlessly switch requests between containers. Works seamlessly across multiple servers, using SSHKit to execute commands. Originally built for Rails apps, Kamal will work with any type of web app that can be containerized with Docker.
Watch the screencast: https://www.youtube.com/watch?v=LL1cV2FXZ5I ➡️ See [kamal-deploy.org](https://kamal-deploy.org) for documentation on [installation](https://kamal-deploy.org/docs/installation), [configuration](https://kamal-deploy.org/docs/configuration), and [commands](https://kamal-deploy.org/docs/commands).
Join us on Discord: https://discord.gg/YgHVT7GCXS ## Contributing to the documentation
Ask questions: https://github.com/mrsked/mrsk/discussions Please help us improve Kamal's documentation on the [the basecamp/kamal-site repository](https://github.com/basecamp/kamal-site).
## Installation
If you have a Ruby environment available, you can install MRSK globally with:
```sh
gem install mrsk
```
...otherwise, you can run a dockerized version via an alias (add this to your .bashrc or similar to simplify re-use):
```sh
alias mrsk='docker run --rm -it -v $HOME/.ssh:/root/.ssh -v /var/run/docker.sock:/var/run/docker.sock -v ${PWD}/:/workdir ghcr.io/mrsked/mrsk'
```
Then, inside your app directory, run `mrsk init` (or `mrsk init --bundle` within Rails 7+ apps where you want a bin/mrsk binstub). Now edit the new file `config/deploy.yml`. It could look as simple as this:
```yaml
service: hey
image: 37s/hey
servers:
- 192.168.0.1
- 192.168.0.2
registry:
username: registry-user-name
password:
- MRSK_REGISTRY_PASSWORD
env:
secret:
- RAILS_MASTER_KEY
```
Then edit your `.env` file to add your registry password as `MRSK_REGISTRY_PASSWORD` (and your `RAILS_MASTER_KEY` for production with a Rails app).
Now you're ready to deploy to the servers:
```
mrsk setup
```
This will:
1. Connect to the servers over SSH (using root by default, authenticated by your ssh key)
2. Install Docker and curl on any server that might be missing it (using apt-get): root access is needed via ssh for this.
3. Log into the registry both locally and remotely
4. Build the image using the standard Dockerfile in the root of the application.
5. Push the image to the registry.
6. Pull the image from the registry onto the servers.
7. Ensure Traefik is running and accepting traffic on port 80.
8. Ensure your app responds with `200 OK` to `GET /up` (you must have curl installed inside your app image!).
9. Start a new container with the version of the app that matches the current git version hash.
10. Stop the old container running the previous version of the app.
11. Prune unused images and stopped containers to ensure servers don't fill up.
Voila! All the servers are now serving the app on port 80. If you're just running a single server, you're ready to go. If you're running multiple servers, you need to put a load balancer in front of them. For subsequent deploys, or if your servers already have Docker and curl installed, you can just run `mrsk deploy`.
## Vision
In the past decade+, there's been an explosion in commercial offerings that make deploying web apps easier. Heroku kicked it off with an incredible offering that stayed ahead of the competition seemingly forever. These days we have excellent alternatives like Fly.io and Render. And hosted Kubernetes is making things easier too on AWS, GCP, Digital Ocean, and elsewhere. But these are all offerings that have you renting computers in the cloud at a premium. If you want to run on your own hardware, or even just have a clear migration path to do so in the future, you need to carefully consider how locked in you get to these commercial platforms. Preferably before the bills swallow your business whole!
MRSK seeks to bring the advance in ergonomics pioneered by these commercial offerings to deploying web apps anywhere. Whether that's low-cost cloud options without the managed-service markup from the likes of Digital Ocean, Hetzner, OVH, etc., or it's your own colocated bare metal. To MRSK, it's all the same. Feed the config file a list of IP addresses with vanilla Ubuntu servers that have seen no prep beyond an added SSH key, and you'll be running in literally minutes.
This approach gives you enormous portability. You can have your web app deployed on several clouds at ease like this. Or you can buy the baseline with your own hardware, then deploy to a cloud before a big seasonal spike to get more capacity. When you're not locked into a single provider from a tooling perspective, there are a lot of compelling options available.
Ultimately, MRSK is meant to compress the complexity of going to production using open source tooling that isn't tied to any commercial offering. Not to zero, mind you. You're probably still better off with a fully managed service if basic Linux or Docker is still difficult, but as soon as those concepts are familiar, you'll be ready to go with MRSK.
## Why not just run Capistrano, Kubernetes or Docker Swarm?
MRSK basically is Capistrano for Containers, without the need to carefully prepare servers in advance. No need to ensure that the servers have just the right version of Ruby or other dependencies you need. That all lives in the Docker image now. You can boot a brand new Ubuntu (or whatever) server, add it to the list of servers in MRSK, and it'll be auto-provisioned with Docker, and run right away. Docker's layer caching also speeds up deployments with less mucking about on the server. And the images built for MRSK can be used for CI or later introspection.
Kubernetes is a beast. Running it yourself on your own hardware is not for the faint of heart. It's a fine option if you want to run on someone else's platform, either transparently [like Render](https://thenewstack.io/render-cloud-deployment-with-less-engineering/) or explicitly on AWS/GCP, but if you'd like the freedom to move between cloud and your own hardware, or even mix the two, MRSK is much simpler. You can see everything that's going on, it's just basic Docker commands being called.
Docker Swarm is much simpler than Kubernetes, but it's still built on the same declarative model that uses state reconciliation. MRSK is intentionally designed around imperative commands, like Capistrano.
Ultimately, there are a myriad of ways to deploy web apps, but this is the toolkit we're using at [37signals](https://37signals.com) to bring [HEY](https://www.hey.com) [home from the cloud](https://world.hey.com/dhh/why-we-re-leaving-the-cloud-654b47e0) without losing the advantages of modern containerization tooling.
## Running MRSK from Docker
MRSK is packaged up in a Docker container similarly to [rails/docked](https://github.com/rails/docked). This will allow you to run MRSK (from your application directory) without having to install any dependencies other than Docker. Add the following alias to your profile configuration to make working with the container more convenient:
```bash
alias mrsk="docker run -it --rm -v '${PWD}:/workdir' -v '${SSH_AUTH_SOCK}:/ssh-agent' -v /var/run/docker.sock:/var/run/docker.sock -e 'SSH_AUTH_SOCK=/ssh-agent' ghcr.io/mrsked/mrsk:latest"
```
Since MRSK uses SSH to establish a remote connection, it will need access to your SSH agent. The above command uses a volume mount to make it available inside the container and configures the SSH agent inside the container to make use of it.
## Configuration
### Using .env file to load required environment variables
MRSK uses [dotenv](https://github.com/bkeepers/dotenv) to automatically load environment variables set in the `.env` file present in the application root. This file can be used to set variables like `MRSK_REGISTRY_PASSWORD` or database passwords. But for this reason you must ensure that .env files are not checked into Git or included in your Dockerfile! The format is just key-value like:
```bash
MRSK_REGISTRY_PASSWORD=pw
DB_PASSWORD=secret123
```
### Using a generated .env file
#### 1Password as a secret store
If you're using a centralized secret store, like 1Password, you can create `.env.erb` as a template which looks up the secrets. Example of a .env.erb file:
```erb
<% if (session_token = `op signin --account my-one-password-account --raw`.strip) != "" %># Generated by mrsk envify
GITHUB_TOKEN=<%= `gh config get -h github.com oauth_token`.strip %>
MRSK_REGISTRY_PASSWORD=<%= `op read "op://Vault/Docker Hub/password" -n --session #{session_token}` %>
RAILS_MASTER_KEY=<%= `op read "op://Vault/My App/RAILS_MASTER_SECRET" -n --session #{session_token}` %>
MYSQL_ROOT_PASSWORD=<%= `op read "op://Vault/My App/MYSQL_ROOT_PASSWORD" -n --session #{session_token}` %>
<% else raise ArgumentError, "Session token missing" end %>
```
This template can safely be checked into git. Then everyone deploying the app can run `mrsk envify` when they setup the app for the first time or passwords change to get the correct `.env` file.
If you need separate env variables for different destinations, you can set them with `.env.destination.erb` for the template, which will generate `.env.staging` when run with `mrsk envify -d staging`.
#### Bitwarden as a secret store
If you are using open source secret store like bitwarden, you can create `.env.erb` as a template which looks up the secrets.
You can store `SOME_SECRET` in a secure note in bitwarden vault.
```
$ bw list items --search SOME_SECRET | jq
? Master password: [hidden]
[
{
"object": "item",
"id": "123123123-1232-4224-222f-234234234234",
"organizationId": null,
"folderId": null,
"type": 2,
"reprompt": 0,
"name": "SOME_SECRET",
"notes": "yyy",
"favorite": false,
"secureNote": {
"type": 0
},
"collectionIds": [],
"revisionDate": "2023-02-28T23:54:47.868Z",
"creationDate": "2022-11-07T03:16:05.828Z",
"deletedDate": null
}
]
```
and extract the `id` of `SOME_SECRET` from the `json` above and use in the `erb` below.
Example `.env.erb` file:
```erb
<% if (session_token=`bw unlock --raw`.strip) != "" %># Generated by mrsk envify
SOME_SECRET=<%= `bw get notes 123123123-1232-4224-222f-234234234234 --session #{session_token}` %>
<% else raise ArgumentError, "session_token token missing" end %>
```
Then everyone deploying the app can run `mrsk envify` and mrsk will generate `.env`
### Using another registry than Docker Hub
The default registry is Docker Hub, but you can change it using `registry/server`:
```yaml
registry:
server: registry.digitalocean.com
username:
- DOCKER_REGISTRY_TOKEN
password:
- DOCKER_REGISTRY_TOKEN
```
A reference to secret `DOCKER_REGISTRY_TOKEN` will look for `ENV["DOCKER_REGISTRY_TOKEN"]` on the machine running MRSK.
#### Using AWS ECR as the container registry
AWS ECR's access token is only valid for 12hrs. In order to not have to manually regenerate the token every time, you can use ERB in the `deploy.yml` file to shell out to the `aws` cli command, and obtain the token:
```yaml
registry:
server: <your aws account id>.dkr.ecr.<your aws region id>.amazonaws.com
username: AWS
password: <%= %x(aws ecr get-login-password) %>
```
You will need to have the `aws` CLI installed locally for this to work.
### Using a different SSH user than root
The default SSH user is root, but you can change it using `ssh/user`:
```yaml
ssh:
user: app
```
If you are using non-root user, you need to bootstrap your servers manually, before using them with MRSK. On Ubuntu, you'd do:
```bash
sudo apt update
sudo apt upgrade -y
sudo apt install -y docker.io curl git
sudo usermod -a -G docker ubuntu
```
### Using a proxy SSH host
If you need to connect to server through a proxy host, you can use `ssh/proxy`:
```yaml
ssh:
proxy: "192.168.0.1" # defaults to root as the user
```
Or with specific user:
```yaml
ssh:
proxy: "app@192.168.0.1"
```
Also if you need specific proxy command to connect to the server:
```yaml
ssh:
proxy_command: aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p' --region=us-east-1 ## ssh via aws ssm
```
### Using env variables
You can inject env variables into the app containers using `env`:
```yaml
env:
DATABASE_URL: mysql2://db1/hey_production/
REDIS_URL: redis://redis1:6379/1
```
### Using secret env variables
If you have env variables that are secret, you can divide the `env` block into `clear` and `secret`:
```yaml
env:
clear:
DATABASE_URL: mysql2://db1/hey_production/
REDIS_URL: redis://redis1:6379/1
secret:
- DATABASE_PASSWORD
- REDIS_PASSWORD
```
The list of secret env variables will be expanded at run time from your local machine. So a reference to a secret `DATABASE_PASSWORD` will look for `ENV["DATABASE_PASSWORD"]` on the machine running MRSK. Just like with build secrets.
If the referenced secret ENVs are missing, the configuration will be halted with a `KeyError` exception.
Note: Marking an ENV as secret currently only redacts its value in the output for MRSK. The ENV is still injected in the clear into the container at runtime.
### Using volumes
You can add custom volumes into the app containers using `volumes`:
```yaml
volumes:
- "/local/path:/container/path"
```
### MRSK env variables
The following env variables are set when your container runs:
`MRSK_CONTAINER_NAME` : this contains the current container name and version
### Using different roles for servers
If your application uses separate hosts for running jobs or other roles beyond the default web running, you can specify these hosts in a dedicated role with a new entrypoint command like so:
```yaml
servers:
web:
- 192.168.0.1
- 192.168.0.2
job:
hosts:
- 192.168.0.3
- 192.168.0.4
cmd: bin/jobs
```
Note: Traefik will only by default be installed and run on the servers in the `web` role (and on all servers if no roles are defined). If you need Traefik on hosts in other roles than `web`, add `traefik: true`:
```yaml
servers:
web:
- 192.168.0.1
- 192.168.0.2
web2:
traefik: true
hosts:
- 192.168.0.3
- 192.168.0.4
```
### Using container labels
You can specialize the default Traefik rules by setting labels on the containers that are being started:
```yaml
labels:
traefik.http.routers.hey-web.rule: Host(`app.hey.com`)
```
Traefik rules are in the "service-role-destination" format. The default role will be `web` if no rule is specified. If the destination is not specified, it is not included. To give an example, the above rule would become "traefik.http.routers.hey-web-staging.rule" if it was for the "staging" destination.
Note: The backticks are needed to ensure the rule is passed in correctly and not treated as command substitution by Bash!
This allows you to run multiple applications on the same server sharing the same Traefik instance and port.
See https://doc.traefik.io/traefik/routing/routers/#rule for a full list of available routing rules.
The labels can also be applied on a per-role basis:
```yaml
servers:
web:
- 192.168.0.1
- 192.168.0.2
job:
hosts:
- 192.168.0.3
- 192.168.0.4
cmd: bin/jobs
labels:
my-label: "50"
```
### Using shell expansion
You can use shell expansion to interpolate values from the host machine into labels and env variables with the `${}` syntax.
Anything within the curly braces will be executed on the host machine and the result will be interpolated into the label or env variable.
```yaml
labels:
host-machine: "${cat /etc/hostname}"
env:
HOST_DEPLOYMENT_DIR: "${PWD}"
```
Note: Any other occurrence of `$` will be escaped to prevent unwanted shell expansion!
### Using container options
You can specialize the options used to start containers using the `options` definitions:
```yaml
servers:
web:
- 192.168.0.1
- 192.168.0.2
job:
hosts:
- 192.168.0.3
- 192.168.0.4
cmd: bin/jobs
options:
cap-add: true
cpu-count: 4
```
That'll start the job containers with `docker run ... --cap-add --cpu-count 4 ...`.
### Configuring logging
You can configure the logging driver and options passed to Docker using `logging`:
```yaml
logging:
driver: awslogs
options:
awslogs-region: "eu-central-2"
awslogs-group: "my-app"
```
If nothing is configured, the default option `max-size=10m` is used for all containers. The default logging driver of Docker is `json-file`.
### Using a different stop wait time
On a new deploy, each old running container is gracefully shut down with a `SIGTERM`, and after a grace period of `10` seconds a `SIGKILL` is sent.
You can configure this value via the `stop_wait_time` option:
```yaml
stop_wait_time: 30
```
### Using remote builder for native multi-arch
If you're developing on ARM64 (like Apple Silicon), but you want to deploy on AMD64 (x86 64-bit), you can use multi-architecture images. By default, MRSK will setup a local buildx configuration that does this through QEMU emulation. But this can be quite slow, especially on the first build.
If you want to speed up this process by using a remote AMD64 host to natively build the AMD64 part of the image, while natively building the ARM64 part locally, you can do so using builder options:
```yaml
builder:
local:
arch: arm64
host: unix:///Users/<%= `whoami`.strip %>/.docker/run/docker.sock
remote:
arch: amd64
host: ssh://root@192.168.0.1
```
Note: You must have Docker running on the remote host being used as a builder. This instance should only be shared for builds using the same registry and credentials.
### Using remote builder for single-arch
If you're developing on ARM64 (like Apple Silicon), want to deploy on AMD64 (x86 64-bit), but don't need to run the image locally (or on other ARM64 hosts), you can configure a remote builder that just targets AMD64. This is a bit faster than building with multi-arch, as there's nothing to build locally.
```yaml
builder:
remote:
arch: amd64
host: ssh://root@192.168.0.1
```
### Using native builder when multi-arch isn't needed
If you're developing on the same architecture as the one you're deploying on, you can speed up the build by forgoing both multi-arch and remote building:
```yaml
builder:
multiarch: false
```
This is also a good option if you're running MRSK from a CI server that shares architecture with the deployment servers.
### Using a different Dockerfile or context when building
If you need to pass a different Dockerfile or context to the build command (e.g. if you're using a monorepo or you have
different Dockerfiles), you can do so in the builder options:
```yaml
# Use a different Dockerfile
builder:
dockerfile: Dockerfile.xyz
# Set context
builder:
context: ".."
# Set Dockerfile and context
builder:
dockerfile: "../Dockerfile.xyz"
context: ".."
```
### Using build secrets for new images
Some images need a secret passed in during build time, like a GITHUB_TOKEN, to give access to private gem repositories. This can be done by having the secret in ENV, then referencing it in the builder configuration:
```yaml
builder:
secrets:
- GITHUB_TOKEN
```
This build secret can then be referenced in the Dockerfile:
```dockerfile
# Copy Gemfiles
COPY Gemfile Gemfile.lock ./
# Install dependencies, including private repositories via access token (then remove bundle cache with exposed GITHUB_TOKEN)
RUN --mount=type=secret,id=GITHUB_TOKEN \
BUNDLE_GITHUB__COM=x-access-token:$(cat /run/secrets/GITHUB_TOKEN) \
bundle install && \
rm -rf /usr/local/bundle/cache
```
### Traefik command arguments
Customize the Traefik command line using `args`:
```yaml
traefik:
args:
accesslog: true
accesslog.format: json
```
This starts the Traefik container with `--accesslog=true --accesslog.format=json` arguments.
### Traefik host port binding
Traefik binds to port 80 by default. Specify an alternative port using `host_port`:
```yaml
traefik:
host_port: 8080
```
### Traefik version, upgrades, and custom images
MRSK runs the traefik:v2.9 image to track Traefik 2.9.x releases.
To pin Traefik to a specific version or an image published to your registry,
specify `image`:
```yaml
traefik:
image: traefik:v2.10.0-rc1
```
This is useful for downgrading Traefik if there's an unexpected breaking
change in a minor version release, upgrading Traefik to test forthcoming
releases, or running your own Traefik-derived image.
MRSK has not been tested for compatibility with Traefik 3 betas. Please do!
### Traefik container configuration
Pass additional Docker configuration for the Traefik container using `options`:
```yaml
traefik:
options:
publish:
- 8080:8080
volume:
- /tmp/example.json:/tmp/example.json
memory: 512m
```
This starts the Traefik container with `--volume /tmp/example.json:/tmp/example.json --publish 8080:8080 --memory 512m` arguments to `docker run`.
### Traefik container labels
Add labels to Traefik Docker container.
```yaml
traefik:
labels:
traefik.enable: true
traefik.http.routers.dashboard.rule: Host(`traefik.example.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
traefik.http.routers.dashboard.service: api@internal
traefik.http.routers.dashboard.middlewares: auth
traefik.http.middlewares.auth.basicauth.users: test:$2y$05$H2o72tMaO.TwY1wNQUV1K.fhjRgLHRDWohFvUZOJHBEtUXNKrqUKi # test:password
```
This labels Traefik container with `--label traefik.http.routers.dashboard.middlewares=\"auth\"` and so on.
### Traefik alternate entrypoints
You can configure multiple entrypoints for Traefik like so:
```yaml
service: myservice
labels:
traefik.tcp.routers.other.rule: 'HostSNI(`*`)'
traefik.tcp.routers.other.entrypoints: otherentrypoint
traefik.tcp.services.other.loadbalancer.server.port: 9000
traefik.http.routers.myservice.entrypoints: web
traefik.http.services.myservice.loadbalancer.server.port: 8080
traefik:
options:
publish:
- 9000:9000
args:
entrypoints.web.address: ':80'
entrypoints.otherentrypoint.address: ':9000'
```
### Configuring build args for new images
Build arguments that aren't secret can also be configured:
```yaml
builder:
args:
RUBY_VERSION: 3.2.0
```
This build argument can then be used in the Dockerfile:
```
ARG RUBY_VERSION
FROM ruby:$RUBY_VERSION-slim as base
```
### Using accessories for database, cache, search services
You can manage your accessory services via MRSK as well. Accessories are long-lived services that your app depends on. They are not updated when you deploy.
```yaml
accessories:
mysql:
image: mysql:5.7
host: 1.1.1.3
port: 3306
env:
clear:
MYSQL_ROOT_HOST: '%'
secret:
- MYSQL_ROOT_PASSWORD
volumes:
- /var/lib/mysql:/var/lib/mysql
options:
cpus: 4
memory: "2GB"
redis:
image: redis:latest
roles:
- web
port: "36379:6379"
volumes:
- /var/lib/redis:/data
internal-example:
image: registry.digitalocean.com/user/otherservice:latest
host: 1.1.1.5
port: 44444
```
The hosts that the accessories will run on can be specified by hosts or roles:
```yaml
# Single host
mysql:
host: 1.1.1.1
# Multiple hosts
redis:
hosts:
- 1.1.1.1
- 1.1.1.2
# By role
monitoring:
roles:
- web
- jobs
```
Now run `mrsk accessory start mysql` to start the MySQL server on 1.1.1.3. See `mrsk accessory` for all the commands possible.
Accessory images must be public or tagged in your private registry.
### Using Cron
You can use a specific container to run your Cron jobs:
```yaml
servers:
cron:
hosts:
- 192.168.0.1
cmd:
bash -c "cat config/crontab | crontab - && cron -f"
```
This assumes the Cron settings are stored in `config/crontab`.
### Healthcheck
MRSK uses Docker healthchecks to check the health of your application during deployment. Traefik uses this same healthcheck status to determine when a container is ready to receive traffic.
The healthcheck defaults to testing the HTTP response to the path `/up` on port 3000, up to 7 times. You can tailor this behaviour with the `healthcheck` setting:
```yaml
healthcheck:
path: /healthz
port: 4000
max_attempts: 7
interval: 20s
```
This will ensure your application is configured with a traefik label for the healthcheck against `/healthz` and that the pre-deploy healthcheck that MRSK performs is done against the same path on port 4000.
You can also specify a custom healthcheck command, which is useful for non-HTTP services:
```yaml
healthcheck:
cmd: /bin/check_health
```
The top-level healthcheck configuration applies to all services that use
Traefik, by default. You can also specialize the configuration at the role
level:
```yaml
servers:
job:
hosts: ...
cmd: bin/jobs
healthcheck:
cmd: bin/check
```
The healthcheck allows for an optional `max_attempts` setting, which will attempt the healthcheck up to the specified number of times before failing the deploy. This is useful for applications that take a while to start up. The default is 7.
Note: The HTTP health checks assume that the `curl` command is available inside the container. If that's not the case, use the healthcheck's `cmd` option to specify an alternative check that the container supports.
## Commands
### Running commands on servers
You can execute one-off commands on the servers:
```bash
# Runs command on all servers
mrsk app exec 'ruby -v'
App Host: 192.168.0.1
ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
App Host: 192.168.0.2
ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
# Runs command on primary server
mrsk app exec --primary 'cat .ruby-version'
App Host: 192.168.0.1
3.1.3
# Runs Rails command on all servers
mrsk app exec 'bin/rails about'
App Host: 192.168.0.1
About your application's environment
Rails version 7.1.0.alpha
Ruby version ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
RubyGems version 3.3.26
Rack version 2.2.5
Middleware ActionDispatch::HostAuthorization, Rack::Sendfile, ActionDispatch::Static, ActionDispatch::Executor, Rack::Runtime, Rack::MethodOverride, ActionDispatch::RequestId, ActionDispatch::RemoteIp, Rails::Rack::Logger, ActionDispatch::ShowExceptions, ActionDispatch::DebugExceptions, ActionDispatch::Callbacks, ActionDispatch::Cookies, ActionDispatch::Session::CookieStore, ActionDispatch::Flash, ActionDispatch::ContentSecurityPolicy::Middleware, ActionDispatch::PermissionsPolicy::Middleware, Rack::Head, Rack::ConditionalGet, Rack::ETag, Rack::TempfileReaper
Application root /rails
Environment production
Database adapter sqlite3
Database schema version 20221231233303
App Host: 192.168.0.2
About your application's environment
Rails version 7.1.0.alpha
Ruby version ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
RubyGems version 3.3.26
Rack version 2.2.5
Middleware ActionDispatch::HostAuthorization, Rack::Sendfile, ActionDispatch::Static, ActionDispatch::Executor, Rack::Runtime, Rack::MethodOverride, ActionDispatch::RequestId, ActionDispatch::RemoteIp, Rails::Rack::Logger, ActionDispatch::ShowExceptions, ActionDispatch::DebugExceptions, ActionDispatch::Callbacks, ActionDispatch::Cookies, ActionDispatch::Session::CookieStore, ActionDispatch::Flash, ActionDispatch::ContentSecurityPolicy::Middleware, ActionDispatch::PermissionsPolicy::Middleware, Rack::Head, Rack::ConditionalGet, Rack::ETag, Rack::TempfileReaper
Application root /rails
Environment production
Database adapter sqlite3
Database schema version 20221231233303
# Run Rails runner on primary server
mrsk app exec -p 'bin/rails runner "puts Rails.application.config.time_zone"'
UTC
```
### Running interactive commands over SSH
You can run interactive commands, like a Rails console or a bash session, on a server (default is primary, use `--hosts` to connect to another):
```bash
# Starts a bash session in a new container made from the most recent app image
mrsk app exec -i bash
# Starts a bash session in the currently running container for the app
mrsk app exec -i --reuse bash
# Starts a Rails console in a new container made from the most recent app image
mrsk app exec -i 'bin/rails console'
```
### Running details to show state of containers
You can see the state of your servers by running `mrsk details`:
```
Traefik Host: 192.168.0.1
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6195b2a28c81 traefik "/entrypoint.sh --pr…" 30 minutes ago Up 19 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp traefik
Traefik Host: 192.168.0.2
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
de14a335d152 traefik "/entrypoint.sh --pr…" 30 minutes ago Up 19 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp traefik
App Host: 192.168.0.1
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
badb1aa51db3 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 13 minutes ago Up 13 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123
App Host: 192.168.0.2
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1d3c91ed1f55 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 13 minutes ago Up 13 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123
```
You can also see just info for app containers with `mrsk app details` or just for Traefik with `mrsk traefik details`.
### Running rollback to fix a bad deploy
If you've discovered a bad deploy, you can quickly rollback by reactivating the old, paused container image. You can see what old containers are available for rollback by running `mrsk app containers`. It'll give you a presentation similar to `mrsk app details`, but include all the old containers as well. Showing something like this:
```
App Host: 192.168.0.1
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1d3c91ed1f51 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 19 minutes ago Up 19 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123
539f26b28369 registry.digitalocean.com/user/app:e5d9d7c2b898289dfbc5f7f1334140d984eedae4 "/rails/bin/docker-e…" 31 minutes ago Exited (1) 27 minutes ago chat-e5d9d7c2b898289dfbc5f7f1334140d984eedae4
App Host: 192.168.0.2
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
badb1aa51db4 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 19 minutes ago Up 19 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123
6f170d1172ae registry.digitalocean.com/user/app:e5d9d7c2b898289dfbc5f7f1334140d984eedae4 "/rails/bin/docker-e…" 31 minutes ago Exited (1) 27 minutes ago chat-e5d9d7c2b898289dfbc5f7f1334140d984eedae4
```
From the example above, we can see that `e5d9d7c2b898289dfbc5f7f1334140d984eedae4` was the last version, so it's available as a rollback target. We can perform this rollback by running `mrsk rollback e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. That'll stop `6ef8a6a84c525b123c5245345a8483f86d05a123` and then start `e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. Because the old container is still available, this is very quick. Nothing to download from the registry.
Note that by default old containers are pruned after 3 days when you run `mrsk deploy`.
### Running removal to clean up servers
If you wish to remove the entire application, including Traefik, containers, images, and registry session, you can run `mrsk remove`. This will leave the servers clean.
## Locking
Commands that are unsafe to run concurrently will take a deploy lock while they run. The lock is the `mrsk_lock` directory on the primary server.
You can check the lock status with:
```
mrsk lock status
Locked by: AN Other at 2023-03-24 09:49:03 UTC
Version: 77f45c0686811c68989d6576748475a60bf53fc2
Message: Automatic deploy lock
```
You can also manually acquire and release the lock
```
mrsk lock acquire -m "Doing maintenance"
```
```
mrsk lock release
```
## Rolling deployments
When deploying to large numbers of hosts, you might prefer not to restart your services on every host at the same time.
MRSK's default is to boot new containers on all hosts in parallel. But you can control this by configuring `boot/limit` and `boot/wait` as options:
```yaml
service: myservice
boot:
limit: 10 # Can also specify as a percentage of total hosts, such as "25%"
wait: 2
```
When `limit` is specified, containers will be booted on, at most, `limit` hosts at once. MRSK will pause for `wait` seconds between batches.
These settings only apply when booting containers (using `mrsk deploy`, or `mrsk app boot`). For other commands, MRSK continues to run commands in parallel across all hosts.
## Hooks
You can run custom scripts at specific points with hooks.
Hooks should be stored in the .mrsk/hooks folder. Running mrsk init will build that folder and add some sample scripts.
You can change their location by setting `hooks_path` in the configuration file.
If the script returns a non-zero exit code the command will be aborted.
`MRSK_*` environment variables are available to the hooks command for
fine-grained audit reporting, e.g. for triggering deployment reports or
firing a JSON webhook. These variables include:
- `MRSK_RECORDED_AT` - UTC timestamp in ISO 8601 format, e.g. `2023-04-14T17:07:31Z`
- `MRSK_PERFORMER` - the local user performing the command (from `whoami`)
- `MRSK_SERVICE_VERSION` - an abbreviated service and version for use in messages, e.g. app@150b24f
- `MRSK_VERSION` - an full version being deployed
- `MRSK_HOSTS` - a comma separated list of the hosts targeted by the command
- `MRSK_COMMAND` - The command we are running
- `MRSK_SUBCOMMAND` - optional: The subcommand we are running
- `MRSK_DESTINATION` - optional: destination, e.g. "staging"
- `MRSK_ROLE` - optional: role targeted, e.g. "web"
There are four hooks:
1. pre-connect
Called before taking the deploy lock. For checks that need to run before connecting to remote hosts - e.g. DNS warming.
2. pre-build
Used for pre-build checks - e.g. there are no uncommitted changes or that CI has passed.
3. pre-deploy
For final checks before deploying, e.g. checking CI completed
3. post-deploy - run after a deploy, redeploy or rollback
This hook is also passed a `MRSK_RUNTIME` env variable.
This could be used to broadcast a deployment message, or register the new version with an APM.
The command could look something like:
```bash
#!/usr/bin/env bash
curl -q -d content="[My App] ${MRSK_PERFORMER} Rolled back to version ${MRSK_VERSION}" https://3.basecamp.com/XXXXX/integrations/XXXXX/buckets/XXXXX/chats/XXXXX/lines
```
That'll post a line like follows to a preconfigured chatbot in Basecamp:
```
[My App] [dhh] Rolled back to version d264c4e92470ad1bd18590f04466787262f605de
```
Set `--skip_hooks` to avoid running the hooks.
## Stage of development
This is beta software. Commands may still move around. But we're live in production at [37signals](https://37signals.com).
## License ## License
MRSK is released under the [MIT License](https://opensource.org/licenses/MIT). Kamal is released under the [MIT License](https://opensource.org/licenses/MIT).

141
bin/docs Executable file
View File

@@ -0,0 +1,141 @@
#!/usr/bin/env ruby
require "stringio"
def usage
puts "Usage: #{$0} <kamal_site_repo>"
exit 1
end
usage if ARGV.size != 1
kamal_site_repo = ARGV[0]
if !File.directory?(kamal_site_repo)
puts "Error: #{kamal_site_repo} is not a directory"
exit 1
end
DOCS = {
"accessory" => "Accessories",
"alias" => "Aliases",
"boot" => "Booting",
"builder" => "Builders",
"configuration" => "Configuration overview",
"env" => "Environment variables",
"logging" => "Logging",
"proxy" => "Proxy",
"registry" => "Docker Registry",
"role" => "Roles",
"servers" => "Servers",
"ssh" => "SSH",
"sshkit" => "SSHKit"
}
DOCS_PATH = "lib/kamal/configuration/docs"
class DocWriter
attr_reader :from_file, :to_file, :key, :heading, :body, :output, :in_yaml
def initialize(from_file, to_dir)
@from_file = from_file
@key = File.basename(from_file, ".yml")
@to_file = File.join(to_dir, "#{linkify(DOCS[key])}.md")
@body = File.readlines(from_file)
@heading = body.shift.chomp("\n")
@output = nil
end
def write
puts "Writing #{to_file}"
generate_markdown
File.write(to_file, output.string)
end
private
def generate_markdown
@output = StringIO.new
generate_header
place = :in_section
loop do
line = body.shift&.chomp("\n")
break if line.nil?
case place
when :new_section, :in_section
if line.empty?
output.puts
place = :new_section
elsif line =~ /^ *#/
generate_line(line, heading: place == :new_section)
place = :in_section
else
output.puts
output.puts "```yaml"
output.puts line
place = :in_yaml
end
when :in_yaml, :in_empty_line_yaml
if line =~ /^ *#/
output.puts "```"
output.puts
generate_line(line, heading: place == :in_empty_line_yaml)
place = :in_section
elsif line.empty?
place = :in_empty_line_yaml
else
output.puts line
end
end
end
output.puts "```" if place == :in_yaml
end
def generate_header
output.puts "---"
output.puts "# This file has been generated from the Kamal source, do not edit directly."
output.puts "# Find the source of this file at #{DOCS_PATH}/#{key}.yml in the Kamal repository."
output.puts "title: #{heading[2..-1]}"
output.puts "---"
output.puts
output.puts heading
end
def generate_line(line, heading: false)
line = line.gsub(/^ *#\s?/, "")
if line =~ /(.*)kamal docs ([a-z]*)(.*)/
line = "#{$1}[#{DOCS[$2]}](../#{linkify(DOCS[$2])})#{$3}"
end
if line =~ /(.*)https:\/\/kamal-deploy.org([a-z\/-]*)(.*)/
line = "#{$1}[#{titlify($2.split("/").last)}](#{$2})#{$3}"
end
if heading
output.puts "## [#{line}](##{linkify(line)})"
else
output.puts line
end
end
def linkify(text)
if text == "Configuration overview"
"overview"
else
text.downcase.gsub(" ", "-")
end
end
def titlify(text)
text.capitalize.gsub("-", " ")
end
end
from_dir = File.join(File.dirname(__FILE__), "../#{DOCS_PATH}")
to_dir = File.join(kamal_site_repo, "docs/configuration")
Dir.glob("#{from_dir}/*") do |from_file|
DocWriter.new(from_file, to_dir).write
end

View File

@@ -3,10 +3,10 @@
# Prevent failures from being reported twice. # Prevent failures from being reported twice.
Thread.report_on_exception = false Thread.report_on_exception = false
require "mrsk" require "kamal"
begin begin
Mrsk::Cli::Main.start(ARGV) Kamal::Cli::Main.start(ARGV)
rescue SSHKit::Runner::ExecuteError => e rescue SSHKit::Runner::ExecuteError => e
puts " \e[31mERROR (#{e.cause.class}): #{e.message}\e[0m" puts " \e[31mERROR (#{e.cause.class}): #{e.message}\e[0m"
puts e.cause.backtrace if ENV["VERBOSE"] puts e.cause.backtrace if ENV["VERBOSE"]

View File

@@ -2,13 +2,13 @@
VERSION=$1 VERSION=$1
printf "module Mrsk\n VERSION = \"$VERSION\"\nend\n" > ./lib/mrsk/version.rb printf "module Kamal\n VERSION = \"$VERSION\"\nend\n" > ./lib/kamal/version.rb
bundle bundle
git add Gemfile.lock lib/mrsk/version.rb git add Gemfile.lock lib/kamal/version.rb
git commit -m "Bump version for $VERSION" git commit -m "Bump version for $VERSION"
git push git push
git tag v$VERSION git tag v$VERSION
git push --tags git push --tags
gem build mrsk.gemspec gem build kamal.gemspec
gem push "mrsk-$VERSION.gem" --host https://rubygems.org gem push "kamal-$VERSION.gem" --host https://rubygems.org
rm "mrsk-$VERSION.gem" rm "kamal-$VERSION.gem"

View File

@@ -1,25 +1,26 @@
require_relative "lib/mrsk/version" require_relative "lib/kamal/version"
Gem::Specification.new do |spec| Gem::Specification.new do |spec|
spec.name = "mrsk" spec.name = "kamal"
spec.version = Mrsk::VERSION spec.version = Kamal::VERSION
spec.authors = [ "David Heinemeier Hansson" ] spec.authors = [ "David Heinemeier Hansson" ]
spec.email = "dhh@hey.com" spec.email = "dhh@hey.com"
spec.homepage = "https://github.com/rails/mrsk" spec.homepage = "https://github.com/basecamp/kamal"
spec.summary = "Deploy web apps in containers to servers running Docker with zero downtime." spec.summary = "Deploy web apps in containers to servers running Docker with zero downtime."
spec.license = "MIT" spec.license = "MIT"
spec.files = Dir["lib/**/*", "MIT-LICENSE", "README.md"] spec.files = Dir["lib/**/*", "MIT-LICENSE", "README.md"]
spec.executables = %w[ mrsk ] spec.executables = %w[ kamal ]
spec.add_dependency "activesupport", ">= 7.0" spec.add_dependency "activesupport", ">= 7.0"
spec.add_dependency "sshkit", "~> 1.21" spec.add_dependency "sshkit", ">= 1.23.0", "< 2.0"
spec.add_dependency "net-ssh", "~> 7.0" spec.add_dependency "net-ssh", "~> 7.3"
spec.add_dependency "thor", "~> 1.2" spec.add_dependency "thor", "~> 1.3"
spec.add_dependency "dotenv", "~> 2.8" spec.add_dependency "dotenv", "~> 3.1"
spec.add_dependency "zeitwerk", "~> 2.5" spec.add_dependency "zeitwerk", ">= 2.6.18", "< 3.0"
spec.add_dependency "ed25519", "~> 1.2" spec.add_dependency "ed25519", "~> 1.2"
spec.add_dependency "bcrypt_pbkdf", "~> 1.0" spec.add_dependency "bcrypt_pbkdf", "~> 1.0"
spec.add_dependency "concurrent-ruby", "~> 1.2"
spec.add_dependency "base64", "~> 0.2"
spec.add_development_dependency "debug" spec.add_development_dependency "debug"
spec.add_development_dependency "mocha" spec.add_development_dependency "mocha"

14
lib/kamal.rb Normal file
View File

@@ -0,0 +1,14 @@
module Kamal
class ConfigurationError < StandardError; end
end
require "active_support"
require "zeitwerk"
require "yaml"
require "tmpdir"
require "pathname"
loader = Zeitwerk::Loader.for_gem
loader.ignore(File.join(__dir__, "kamal", "sshkit_with_ext.rb"))
loader.setup
loader.eager_load_namespace(Kamal::Cli) # We need all commands loaded.

View File

@@ -1,7 +1,9 @@
module Mrsk::Cli module Kamal::Cli
class LockError < StandardError; end class BootError < StandardError; end
class HookError < StandardError; end class HookError < StandardError; end
class LockError < StandardError; end
class DependencyError < StandardError; end
end end
# SSHKit uses instance eval, so we need a global const for ergonomics # SSHKit uses instance eval, so we need a global const for ergonomics
MRSK = Mrsk::Commander.new KAMAL = Kamal::Commander.new

302
lib/kamal/cli/accessory.rb Normal file
View File

@@ -0,0 +1,302 @@
require "active_support/core_ext/array/conversions"
class Kamal::Cli::Accessory < Kamal::Cli::Base
desc "boot [NAME]", "Boot new accessory service on host (use NAME=all to boot all accessories)"
def boot(name, prepare: true)
with_lock do
if name == "all"
KAMAL.accessory_names.each { |accessory_name| boot(accessory_name) }
else
prepare(name) if prepare
with_accessory(name) do |accessory, hosts|
directories(name)
upload(name)
on(hosts) do
execute *KAMAL.auditor.record("Booted #{name} accessory"), verbosity: :debug
execute *accessory.ensure_env_directory
upload! accessory.secrets_io, accessory.secrets_path, mode: "0600"
execute *accessory.run
if accessory.running_proxy?
target = capture_with_info(*accessory.container_id_for(container_name: accessory.service_name, only_running: true)).strip
execute *accessory.deploy(target: target)
end
end
end
end
end
end
desc "upload [NAME]", "Upload accessory files to host", hide: true
def upload(name)
with_lock do
with_accessory(name) do |accessory, hosts|
on(hosts) do
accessory.files.each do |(local, remote)|
accessory.ensure_local_file_present(local)
execute *accessory.make_directory_for(remote)
upload! local, remote
execute :chmod, "755", remote
end
end
end
end
end
desc "directories [NAME]", "Create accessory directories on host", hide: true
def directories(name)
with_lock do
with_accessory(name) do |accessory, hosts|
on(hosts) do
accessory.directories.keys.each do |host_path|
execute *accessory.make_directory(host_path)
end
end
end
end
end
desc "reboot [NAME]", "Reboot existing accessory on host (stop container, remove container, start new container; use NAME=all to boot all accessories)"
def reboot(name)
with_lock do
if name == "all"
KAMAL.accessory_names.each { |accessory_name| reboot(accessory_name) }
else
prepare(name)
stop(name)
remove_container(name)
boot(name, prepare: false)
end
end
end
desc "start [NAME]", "Start existing accessory container on host"
def start(name)
with_lock do
with_accessory(name) do |accessory, hosts|
on(hosts) do
execute *KAMAL.auditor.record("Started #{name} accessory"), verbosity: :debug
execute *accessory.start
if accessory.running_proxy?
target = capture_with_info(*accessory.container_id_for(container_name: accessory.service_name, only_running: true)).strip
execute *accessory.deploy(target: target)
end
end
end
end
end
desc "stop [NAME]", "Stop existing accessory container on host"
def stop(name)
with_lock do
with_accessory(name) do |accessory, hosts|
on(hosts) do
execute *KAMAL.auditor.record("Stopped #{name} accessory"), verbosity: :debug
execute *accessory.stop, raise_on_non_zero_exit: false
if accessory.running_proxy?
target = capture_with_info(*accessory.container_id_for(container_name: accessory.service_name, only_running: true)).strip
execute *accessory.remove if target
end
end
end
end
end
desc "restart [NAME]", "Restart existing accessory container on host"
def restart(name)
with_lock do
stop(name)
start(name)
end
end
desc "details [NAME]", "Show details about accessory on host (use NAME=all to show all accessories)"
def details(name)
if name == "all"
KAMAL.accessory_names.each { |accessory_name| details(accessory_name) }
else
type = "Accessory #{name}"
with_accessory(name) do |accessory, hosts|
on(hosts) { puts_by_host host, capture_with_info(*accessory.info), type: type }
end
end
end
desc "exec [NAME] [CMD...]", "Execute a custom command on servers within the accessory container (use --help to show options)"
option :interactive, aliases: "-i", type: :boolean, default: false, desc: "Execute command over ssh for an interactive shell (use for console/bash)"
option :reuse, type: :boolean, default: false, desc: "Reuse currently running container instead of starting a new one"
def exec(name, *cmd)
cmd = Kamal::Utils.join_commands(cmd)
with_accessory(name) do |accessory, hosts|
case
when options[:interactive] && options[:reuse]
say "Launching interactive command via SSH from existing container...", :magenta
run_locally { exec accessory.execute_in_existing_container_over_ssh(cmd) }
when options[:interactive]
say "Launching interactive command via SSH from new container...", :magenta
run_locally { exec accessory.execute_in_new_container_over_ssh(cmd) }
when options[:reuse]
say "Launching command from existing container...", :magenta
on(hosts) do |host|
execute *KAMAL.auditor.record("Executed cmd '#{cmd}' on #{name} accessory"), verbosity: :debug
puts_by_host host, capture_with_info(*accessory.execute_in_existing_container(cmd))
end
else
say "Launching command from new container...", :magenta
on(hosts) do |host|
execute *KAMAL.auditor.record("Executed cmd '#{cmd}' on #{name} accessory"), verbosity: :debug
puts_by_host host, capture_with_info(*accessory.execute_in_new_container(cmd))
end
end
end
end
desc "logs [NAME]", "Show log lines from accessory on host (use --help to show options)"
option :since, aliases: "-s", desc: "Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)"
option :lines, type: :numeric, aliases: "-n", desc: "Number of log lines to pull from each server"
option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)"
option :grep_options, desc: "Additional options supplied to grep"
option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)"
option :skip_timestamps, type: :boolean, aliases: "-T", desc: "Skip appending timestamps to logging output"
def logs(name)
with_accessory(name) do |accessory, hosts|
grep = options[:grep]
grep_options = options[:grep_options]
timestamps = !options[:skip_timestamps]
if options[:follow]
run_locally do
info "Following logs on #{hosts}..."
info accessory.follow_logs(timestamps: timestamps, grep: grep, grep_options: grep_options)
exec accessory.follow_logs(timestamps: timestamps, grep: grep, grep_options: grep_options)
end
else
since = options[:since]
lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set
on(hosts) do
puts capture_with_info(*accessory.logs(timestamps: timestamps, since: since, lines: lines, grep: grep, grep_options: grep_options))
end
end
end
end
desc "remove [NAME]", "Remove accessory container, image and data directory from host (use NAME=all to remove all accessories)"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def remove(name)
confirming "This will remove all containers, images and data directories for #{name}. Are you sure?" do
with_lock do
if name == "all"
KAMAL.accessory_names.each { |accessory_name| remove_accessory(accessory_name) }
else
remove_accessory(name)
end
end
end
end
desc "remove_container [NAME]", "Remove accessory container from host", hide: true
def remove_container(name)
with_lock do
with_accessory(name) do |accessory, hosts|
on(hosts) do
execute *KAMAL.auditor.record("Remove #{name} accessory container"), verbosity: :debug
execute *accessory.remove_container
end
end
end
end
desc "remove_image [NAME]", "Remove accessory image from host", hide: true
def remove_image(name)
with_lock do
with_accessory(name) do |accessory, hosts|
on(hosts) do
execute *KAMAL.auditor.record("Removed #{name} accessory image"), verbosity: :debug
execute *accessory.remove_image
end
end
end
end
desc "remove_service_directory [NAME]", "Remove accessory directory used for uploaded files and data directories from host", hide: true
def remove_service_directory(name)
with_lock do
with_accessory(name) do |accessory, hosts|
on(hosts) do
execute *accessory.remove_service_directory
end
end
end
end
desc "upgrade", "Upgrade accessories from Kamal 1.x to 2.0 (restart them in 'kamal' network)"
option :rolling, type: :boolean, default: false, desc: "Upgrade one host at a time"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def upgrade(name)
confirming "This will restart all accessories" do
with_lock do
host_groups = options[:rolling] ? KAMAL.accessory_hosts : [ KAMAL.accessory_hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
KAMAL.with_specific_hosts(hosts) do
say "Upgrading #{name} accessories on #{host_list}...", :magenta
reboot name
say "Upgraded #{name} accessories on #{host_list}...", :magenta
end
end
end
end
end
private
def with_accessory(name)
if KAMAL.config.accessory(name)
accessory = KAMAL.accessory(name)
yield accessory, accessory_hosts(accessory)
else
error_on_missing_accessory(name)
end
end
def error_on_missing_accessory(name)
options = KAMAL.accessory_names.presence
error \
"No accessory by the name of '#{name}'" +
(options ? " (options: #{options.to_sentence})" : "")
end
def accessory_hosts(accessory)
if KAMAL.specific_hosts&.any?
KAMAL.specific_hosts & accessory.hosts
else
accessory.hosts
end
end
def remove_accessory(name)
stop(name)
remove_container(name)
remove_image(name)
remove_service_directory(name)
end
def prepare(name)
with_accessory(name) do |accessory, hosts|
on(hosts) do
execute *KAMAL.registry.login(registry_config: accessory.registry)
execute *KAMAL.docker.create_network
rescue SSHKit::Command::Failed => e
raise unless e.message.include?("already exists")
end
end
end
end

View File

@@ -0,0 +1,10 @@
class Kamal::Cli::Alias::Command < Thor::DynamicCommand
def run(instance, args = [])
if (_alias = KAMAL.config.aliases[name])
KAMAL.reset
Kamal::Cli::Main.start(Shellwords.split(_alias.command) + ARGV[1..-1])
else
super
end
end
end

355
lib/kamal/cli/app.rb Normal file
View File

@@ -0,0 +1,355 @@
class Kamal::Cli::App < Kamal::Cli::Base
desc "boot", "Boot app on servers (or reboot app if already running)"
def boot
with_lock do
say "Get most recent version available as an image...", :magenta unless options[:version]
using_version(version_or_latest) do |version|
say "Start container with version #{version} (or reboot if already running)...", :magenta
# Assets are prepared in a separate step to ensure they are on all hosts before booting
on(KAMAL.hosts) do
KAMAL.roles_on(host).each do |role|
Kamal::Cli::App::PrepareAssets.new(host, role, self).run
end
end
# Primary hosts and roles are returned first, so they can open the barrier
barrier = Kamal::Cli::Healthcheck::Barrier.new
host_boot_groups.each do |hosts|
host_list = Array(hosts).join(",")
run_hook "pre-app-boot", hosts: host_list
on(hosts) do |host|
KAMAL.roles_on(host).each do |role|
Kamal::Cli::App::Boot.new(host, role, self, version, barrier).run
end
end
run_hook "post-app-boot", hosts: host_list
sleep KAMAL.config.boot.wait if KAMAL.config.boot.wait
end
# Tag once the app booted on all hosts
on(KAMAL.hosts) do |host|
execute *KAMAL.auditor.record("Tagging #{KAMAL.config.absolute_image} as the latest image"), verbosity: :debug
execute *KAMAL.app.tag_latest_image
end
end
end
end
desc "start", "Start existing app container on servers"
def start
with_lock do
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
app = KAMAL.app(role: role, host: host)
execute *KAMAL.auditor.record("Started app version #{KAMAL.config.version}"), verbosity: :debug
execute *app.start, raise_on_non_zero_exit: false
if role.running_proxy?
version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
endpoint = capture_with_info(*app.container_id_for_version(version)).strip
raise Kamal::Cli::BootError, "Failed to get endpoint for #{role} on #{host}, did the container boot?" if endpoint.empty?
execute *app.deploy(target: endpoint)
end
end
end
end
end
desc "stop", "Stop app container on servers"
def stop
with_lock do
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
app = KAMAL.app(role: role, host: host)
execute *KAMAL.auditor.record("Stopped app", role: role), verbosity: :debug
if role.running_proxy?
version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
endpoint = capture_with_info(*app.container_id_for_version(version)).strip
if endpoint.present?
execute *app.remove, raise_on_non_zero_exit: false
end
end
execute *app.stop, raise_on_non_zero_exit: false
end
end
end
end
# FIXME: Drop in favor of just containers?
desc "details", "Show details about app containers"
def details
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
puts_by_host host, capture_with_info(*KAMAL.app(role: role, host: host).info)
end
end
end
desc "exec [CMD...]", "Execute a custom command on servers within the app container (use --help to show options)"
option :interactive, aliases: "-i", type: :boolean, default: false, desc: "Execute command over ssh for an interactive shell (use for console/bash)"
option :reuse, type: :boolean, default: false, desc: "Reuse currently running container instead of starting a new one"
option :env, aliases: "-e", type: :hash, desc: "Set environment variables for the command"
option :detach, type: :boolean, default: false, desc: "Execute command in a detached container"
def exec(*cmd)
if (incompatible_options = [ :interactive, :reuse ].select { |key| options[:detach] && options[key] }.presence)
raise ArgumentError, "Detach is not compatible with #{incompatible_options.join(" or ")}"
end
cmd = Kamal::Utils.join_commands(cmd)
env = options[:env]
detach = options[:detach]
case
when options[:interactive] && options[:reuse]
say "Get current version of running container...", :magenta unless options[:version]
using_version(options[:version] || current_running_version) do |version|
say "Launching interactive command with version #{version} via SSH from existing container on #{KAMAL.primary_host}...", :magenta
run_locally { exec KAMAL.app(role: KAMAL.primary_role, host: KAMAL.primary_host).execute_in_existing_container_over_ssh(cmd, env: env) }
end
when options[:interactive]
say "Get most recent version available as an image...", :magenta unless options[:version]
using_version(version_or_latest) do |version|
say "Launching interactive command with version #{version} via SSH from new container on #{KAMAL.primary_host}...", :magenta
run_locally do
exec KAMAL.app(role: KAMAL.primary_role, host: KAMAL.primary_host).execute_in_new_container_over_ssh(cmd, env: env)
end
end
when options[:reuse]
say "Get current version of running container...", :magenta unless options[:version]
using_version(options[:version] || current_running_version) do |version|
say "Launching command with version #{version} from existing container...", :magenta
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
execute *KAMAL.auditor.record("Executed cmd '#{cmd}' on app version #{version}", role: role), verbosity: :debug
puts_by_host host, capture_with_info(*KAMAL.app(role: role, host: host).execute_in_existing_container(cmd, env: env))
end
end
end
else
say "Get most recent version available as an image...", :magenta unless options[:version]
using_version(version_or_latest) do |version|
say "Launching command with version #{version} from new container...", :magenta
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
execute *KAMAL.auditor.record("Executed cmd '#{cmd}' on app version #{version}"), verbosity: :debug
puts_by_host host, capture_with_info(*KAMAL.app(role: role, host: host).execute_in_new_container(cmd, env: env, detach: detach))
end
end
end
end
end
desc "containers", "Show app containers on servers"
def containers
on(KAMAL.hosts) { |host| puts_by_host host, capture_with_info(*KAMAL.app.list_containers) }
end
desc "stale_containers", "Detect app stale containers"
option :stop, aliases: "-s", type: :boolean, default: false, desc: "Stop the stale containers found"
def stale_containers
stop = options[:stop]
with_lock_if_stopping do
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
app = KAMAL.app(role: role, host: host)
versions = capture_with_info(*app.list_versions, raise_on_non_zero_exit: false).split("\n")
versions -= [ capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip ]
versions.each do |version|
if stop
puts_by_host host, "Stopping stale container for role #{role} with version #{version}"
execute *app.stop(version: version), raise_on_non_zero_exit: false
else
puts_by_host host, "Detected stale container for role #{role} with version #{version} (use `kamal app stale_containers --stop` to stop)"
end
end
end
end
end
end
desc "images", "Show app images on servers"
def images
on(KAMAL.hosts) { |host| puts_by_host host, capture_with_info(*KAMAL.app.list_images) }
end
desc "logs", "Show log lines from app on servers (use --help to show options)"
option :since, aliases: "-s", desc: "Show lines since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)"
option :lines, type: :numeric, aliases: "-n", desc: "Number of lines to show from each server"
option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)"
option :grep_options, desc: "Additional options supplied to grep"
option :follow, aliases: "-f", desc: "Follow log on primary server (or specific host set by --hosts)"
option :skip_timestamps, type: :boolean, aliases: "-T", desc: "Skip appending timestamps to logging output"
option :container_id, desc: "Docker container ID to fetch logs"
def logs
# FIXME: Catch when app containers aren't running
grep = options[:grep]
grep_options = options[:grep_options]
since = options[:since]
container_id = options[:container_id]
timestamps = !options[:skip_timestamps]
if options[:follow]
lines = options[:lines].presence || ((since || grep) ? nil : 10) # Default to 10 lines if since or grep isn't set
run_locally do
info "Following logs on #{KAMAL.primary_host}..."
KAMAL.specific_roles ||= [ KAMAL.primary_role.name ]
role = KAMAL.roles_on(KAMAL.primary_host).first
app = KAMAL.app(role: role, host: host)
info app.follow_logs(host: KAMAL.primary_host, container_id: container_id, timestamps: timestamps, lines: lines, grep: grep, grep_options: grep_options)
exec app.follow_logs(host: KAMAL.primary_host, container_id: container_id, timestamps: timestamps, lines: lines, grep: grep, grep_options: grep_options)
end
else
lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
begin
puts_by_host host, capture_with_info(*KAMAL.app(role: role, host: host).logs(container_id: container_id, timestamps: timestamps, since: since, lines: lines, grep: grep, grep_options: grep_options))
rescue SSHKit::Command::Failed
puts_by_host host, "Nothing found"
end
end
end
end
end
desc "remove", "Remove app containers and images from servers"
def remove
with_lock do
stop
remove_containers
remove_images
remove_app_directory
end
end
desc "remove_container [VERSION]", "Remove app container with given version from servers", hide: true
def remove_container(version)
with_lock do
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
execute *KAMAL.auditor.record("Removed app container with version #{version}", role: role), verbosity: :debug
execute *KAMAL.app(role: role, host: host).remove_container(version: version)
end
end
end
end
desc "remove_containers", "Remove all app containers from servers", hide: true
def remove_containers
with_lock do
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
execute *KAMAL.auditor.record("Removed all app containers", role: role), verbosity: :debug
execute *KAMAL.app(role: role, host: host).remove_containers
end
end
end
end
desc "remove_images", "Remove all app images from servers", hide: true
def remove_images
with_lock do
on(KAMAL.hosts) do
execute *KAMAL.auditor.record("Removed all app images"), verbosity: :debug
execute *KAMAL.app.remove_images
end
end
end
desc "remove_app_directory", "Remove the service directory from servers", hide: true
def remove_app_directory
with_lock do
on(KAMAL.hosts) do |host|
roles = KAMAL.roles_on(host)
roles.each do |role|
execute *KAMAL.auditor.record("Removed #{KAMAL.config.app_directory} on all servers", role: role), verbosity: :debug
execute *KAMAL.server.remove_app_directory, raise_on_non_zero_exit: false
end
end
end
end
desc "version", "Show app version currently running on servers"
def version
on(KAMAL.hosts) do |host|
role = KAMAL.roles_on(host).first
puts_by_host host, capture_with_info(*KAMAL.app(role: role, host: host).current_running_version).strip
end
end
private
def using_version(new_version)
if new_version
begin
old_version = KAMAL.config.version
KAMAL.config.version = new_version
yield new_version
ensure
KAMAL.config.version = old_version
end
else
yield KAMAL.config.version
end
end
def current_running_version(host: KAMAL.primary_host)
version = nil
on(host) do
role = KAMAL.roles_on(host).first
version = capture_with_info(*KAMAL.app(role: role, host: host).current_running_version).strip
end
version.presence
end
def version_or_latest
options[:version] || KAMAL.config.latest_tag
end
def with_lock_if_stopping
if options[:stop]
with_lock { yield }
else
yield
end
end
def host_boot_groups
KAMAL.config.boot.limit ? KAMAL.hosts.each_slice(KAMAL.config.boot.limit).to_a : [ KAMAL.hosts ]
end
end

125
lib/kamal/cli/app/boot.rb Normal file
View File

@@ -0,0 +1,125 @@
class Kamal::Cli::App::Boot
attr_reader :host, :role, :version, :barrier, :sshkit
delegate :execute, :capture_with_info, :capture_with_pretty_json, :info, :error, :upload!, to: :sshkit
delegate :assets?, :running_proxy?, to: :role
def initialize(host, role, sshkit, version, barrier)
@host = host
@role = role
@version = version
@barrier = barrier
@sshkit = sshkit
end
def run
old_version = old_version_renamed_if_clashing
wait_at_barrier if queuer?
begin
start_new_version
rescue => e
close_barrier if gatekeeper?
stop_new_version
raise
end
release_barrier if gatekeeper?
if old_version
stop_old_version(old_version)
end
end
private
def old_version_renamed_if_clashing
if capture_with_info(*app.container_id_for_version(version), raise_on_non_zero_exit: false).present?
renamed_version = "#{version}_replaced_#{SecureRandom.hex(8)}"
info "Renaming container #{version} to #{renamed_version} as already deployed on #{host}"
audit("Renaming container #{version} to #{renamed_version}")
execute *app.rename_container(version: version, new_version: renamed_version)
end
capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip.presence
end
def start_new_version
audit "Booted app version #{version}"
hostname = "#{host.to_s[0...51].chomp(".")}-#{SecureRandom.hex(6)}"
execute *app.ensure_env_directory
upload! role.secrets_io(host), role.secrets_path, mode: "0600"
execute *app.run(hostname: hostname)
if running_proxy?
endpoint = capture_with_info(*app.container_id_for_version(version)).strip
raise Kamal::Cli::BootError, "Failed to get endpoint for #{role} on #{host}, did the container boot?" if endpoint.empty?
execute *app.deploy(target: endpoint)
else
Kamal::Cli::Healthcheck::Poller.wait_for_healthy(pause_after_ready: true) { capture_with_info(*app.status(version: version)) }
end
rescue => e
error "Failed to boot #{role} on #{host}"
raise e
end
def stop_new_version
execute *app.stop(version: version), raise_on_non_zero_exit: false
end
def stop_old_version(version)
execute *app.stop(version: version), raise_on_non_zero_exit: false
execute *app.clean_up_assets if assets?
end
def release_barrier
if barrier.open
info "First #{KAMAL.primary_role} container is healthy on #{host}, booting any other roles"
end
end
def wait_at_barrier
info "Waiting for the first healthy #{KAMAL.primary_role} container before booting #{role} on #{host}..."
barrier.wait
info "First #{KAMAL.primary_role} container is healthy, booting #{role} on #{host}..."
rescue Kamal::Cli::Healthcheck::Error
info "First #{KAMAL.primary_role} container is unhealthy, not booting #{role} on #{host}"
raise
end
def close_barrier
if barrier.close
info "First #{KAMAL.primary_role} container is unhealthy on #{host}, not booting any other roles"
begin
error capture_with_info(*app.logs(container_id: app.container_id_for_version(version)))
error capture_with_info(*app.container_health_log(version: version))
rescue SSHKit::Command::Failed
error "Could not fetch logs for #{version}"
end
end
end
def barrier_role?
role == KAMAL.primary_role
end
def app
@app ||= KAMAL.app(role: role, host: host)
end
def auditor
@auditor = KAMAL.auditor(role: role)
end
def audit(message)
execute *auditor.record(message), verbosity: :debug
end
def gatekeeper?
barrier && barrier_role?
end
def queuer?
barrier && !barrier_role?
end
end

View File

@@ -0,0 +1,24 @@
class Kamal::Cli::App::PrepareAssets
attr_reader :host, :role, :sshkit
delegate :execute, :capture_with_info, :info, to: :sshkit
delegate :assets?, to: :role
def initialize(host, role, sshkit)
@host = host
@role = role
@sshkit = sshkit
end
def run
if assets?
execute *app.extract_assets
old_version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
execute *app.sync_asset_volumes(old_version: old_version)
end
end
private
def app
@app ||= KAMAL.app(role: role, host: host)
end
end

213
lib/kamal/cli/base.rb Normal file
View File

@@ -0,0 +1,213 @@
require "thor"
require "kamal/sshkit_with_ext"
module Kamal::Cli
class Base < Thor
include SSHKit::DSL
def self.exit_on_failure?() true end
def self.dynamic_command_class() Kamal::Cli::Alias::Command end
class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging"
class_option :quiet, type: :boolean, aliases: "-q", desc: "Minimal logging"
class_option :version, desc: "Run commands against a specific app version"
class_option :primary, type: :boolean, aliases: "-p", desc: "Run commands only on primary host instead of all"
class_option :hosts, aliases: "-h", desc: "Run commands on these hosts instead of all (separate by comma, supports wildcards with *)"
class_option :roles, aliases: "-r", desc: "Run commands on these roles instead of all (separate by comma, supports wildcards with *)"
class_option :config_file, aliases: "-c", default: "config/deploy.yml", desc: "Path to config file"
class_option :destination, aliases: "-d", desc: "Specify destination to be used for config file (staging -> deploy.staging.yml)"
class_option :skip_hooks, aliases: "-H", type: :boolean, default: false, desc: "Don't run hooks"
def initialize(args = [], local_options = {}, config = {})
if config[:current_command].is_a?(Kamal::Cli::Alias::Command)
# When Thor generates a dynamic command, it doesn't attempt to parse the arguments.
# For our purposes, it means the arguments are passed in args rather than local_options.
super([], args, config)
else
super
end
initialize_commander unless KAMAL.configured?
end
private
def options_with_subcommand_class_options
options.merge(@_initializer.last[:class_options] || {})
end
def initialize_commander
KAMAL.tap do |commander|
if options[:verbose]
ENV["VERBOSE"] = "1" # For backtraces via cli/start
commander.verbosity = :debug
end
if options[:quiet]
commander.verbosity = :error
end
commander.configure \
config_file: Pathname.new(File.expand_path(options[:config_file])),
destination: options[:destination],
version: options[:version]
commander.specific_hosts = options[:hosts]&.split(",")
commander.specific_roles = options[:roles]&.split(",")
commander.specific_primary! if options[:primary]
end
end
def print_runtime
started_at = Time.now
yield
Time.now - started_at
ensure
runtime = Time.now - started_at
puts " Finished all in #{sprintf("%.1f seconds", runtime)}"
end
def with_lock
if KAMAL.holding_lock?
yield
else
acquire_lock
begin
yield
rescue
begin
release_lock
rescue => e
say "Error releasing the deploy lock: #{e.message}", :red
end
raise
end
release_lock
end
end
def confirming(question)
return yield if options[:confirmed]
if ask(question, limited_to: %w[ y N ], default: "N") == "y"
yield
else
say "Aborted", :red
end
end
def acquire_lock
ensure_run_directory
raise_if_locked do
say "Acquiring the deploy lock...", :magenta
on(KAMAL.primary_host) { execute *KAMAL.lock.acquire("Automatic deploy lock", KAMAL.config.version), verbosity: :debug }
end
KAMAL.holding_lock = true
end
def release_lock
say "Releasing the deploy lock...", :magenta
on(KAMAL.primary_host) { execute *KAMAL.lock.release, verbosity: :debug }
KAMAL.holding_lock = false
end
def raise_if_locked
yield
rescue SSHKit::Runner::ExecuteError => e
if e.message =~ /cannot create directory/
say "Deploy lock already in place!", :red
on(KAMAL.primary_host) { puts capture_with_debug(*KAMAL.lock.status) }
raise LockError, "Deploy lock found. Run 'kamal lock help' for more information"
else
raise e
end
end
def run_hook(hook, **extra_details)
if !options[:skip_hooks] && KAMAL.hook.hook_exists?(hook)
details = { hosts: KAMAL.hosts.join(","), command: command, subcommand: subcommand }
say "Running the #{hook} hook...", :magenta
with_env KAMAL.hook.env(**details, **extra_details) do
run_locally do
execute *KAMAL.hook.run(hook)
end
rescue SSHKit::Command::Failed => e
raise HookError.new("Hook `#{hook}` failed:\n#{e.message}")
end
end
end
def on(*args, &block)
if !KAMAL.connected?
run_hook "pre-connect"
KAMAL.connected = true
end
super
end
def command
@kamal_command ||= begin
invocation_class, invocation_commands = *first_invocation
if invocation_class == Kamal::Cli::Main
invocation_commands[0]
else
Kamal::Cli::Main.subcommand_classes.find { |command, clazz| clazz == invocation_class }[0]
end
end
end
def subcommand
@kamal_subcommand ||= begin
invocation_class, invocation_commands = *first_invocation
invocation_commands[0] if invocation_class != Kamal::Cli::Main
end
end
def first_invocation
instance_variable_get("@_invocations").first
end
def reset_invocation(cli_class)
instance_variable_get("@_invocations")[cli_class].pop
end
def ensure_run_directory
on(KAMAL.hosts) do
execute(*KAMAL.server.ensure_run_directory)
end
end
def with_env(env)
current_env = ENV.to_h.dup
ENV.update(env)
yield
ensure
ENV.clear
ENV.update(current_env)
end
def ensure_docker_installed
run_locally do
begin
execute *KAMAL.builder.ensure_docker_installed
rescue SSHKit::Command::Failed => e
error = e.message =~ /command not found/ ?
"Docker is not installed locally" :
"Docker buildx plugin is not installed locally"
raise DependencyError, error
end
end
end
end
end

184
lib/kamal/cli/build.rb Normal file
View File

@@ -0,0 +1,184 @@
require "uri"
class Kamal::Cli::Build < Kamal::Cli::Base
class BuildError < StandardError; end
desc "deliver", "Build app and push app image to registry then pull image on servers"
def deliver
invoke :push
invoke :pull
end
desc "push", "Build and push app image to registry"
option :output, type: :string, default: "registry", banner: "export_type", desc: "Exported type for the build result, and may be any exported type supported by 'buildx --output'."
def push
cli = self
ensure_docker_installed
run_hook "pre-build"
uncommitted_changes = Kamal::Git.uncommitted_changes
if KAMAL.config.builder.git_clone?
if uncommitted_changes.present?
say "Building from a local git clone, so ignoring these uncommitted changes:\n #{uncommitted_changes}", :yellow
end
run_locally do
Clone.new(self).prepare
end
elsif uncommitted_changes.present?
say "Building with uncommitted changes:\n #{uncommitted_changes}", :yellow
end
with_env(KAMAL.config.builder.secrets) do
run_locally do
begin
execute *KAMAL.builder.inspect_builder
rescue SSHKit::Command::Failed => e
if e.message =~ /(context not found|no builder|no compatible builder|does not exist)/
warn "Missing compatible builder, so creating a new one first"
begin
cli.remove
rescue SSHKit::Command::Failed
raise unless e.message =~ /(context not found|no builder|does not exist)/
end
cli.create
else
raise
end
end
# Get the command here to ensure the Dir.chdir doesn't interfere with it
push = KAMAL.builder.push(cli.options[:output])
KAMAL.with_verbosity(:debug) do
Dir.chdir(KAMAL.config.builder.build_directory) { execute *push }
end
end
end
end
desc "pull", "Pull app image from registry onto servers"
def pull
if (first_hosts = mirror_hosts).any?
#  Pull on a single host per mirror first to seed them
say "Pulling image on #{first_hosts.join(", ")} to seed the #{"mirror".pluralize(first_hosts.count)}...", :magenta
pull_on_hosts(first_hosts)
say "Pulling image on remaining hosts...", :magenta
pull_on_hosts(KAMAL.hosts - first_hosts)
else
pull_on_hosts(KAMAL.hosts)
end
end
desc "create", "Create a build setup"
def create
if (remote_host = KAMAL.config.builder.remote)
connect_to_remote_host(remote_host)
end
run_locally do
begin
debug "Using builder: #{KAMAL.builder.name}"
execute *KAMAL.builder.create
rescue SSHKit::Command::Failed => e
if e.message =~ /stderr=(.*)/
error "Couldn't create remote builder: #{$1}"
false
else
raise
end
end
end
end
desc "remove", "Remove build setup"
def remove
run_locally do
debug "Using builder: #{KAMAL.builder.name}"
execute *KAMAL.builder.remove
end
end
desc "details", "Show build setup"
def details
run_locally do
puts "Builder: #{KAMAL.builder.name}"
puts capture(*KAMAL.builder.info)
end
end
desc "dev", "Build using the working directory, tag it as dirty, and push to local image store."
option :output, type: :string, default: "docker", banner: "export_type", desc: "Exported type for the build result, and may be any exported type supported by 'buildx --output'."
def dev
cli = self
ensure_docker_installed
docker_included_files = Set.new(Kamal::Docker.included_files)
git_uncommitted_files = Set.new(Kamal::Git.uncommitted_files)
git_untracked_files = Set.new(Kamal::Git.untracked_files)
docker_uncommitted_files = docker_included_files & git_uncommitted_files
if docker_uncommitted_files.any?
say "WARNING: Files with uncommitted changes will be present in the dev container:", :yellow
docker_uncommitted_files.sort.each { |f| say " #{f}", :yellow }
say
end
docker_untracked_files = docker_included_files & git_untracked_files
if docker_untracked_files.any?
say "WARNING: Untracked files will be present in the dev container:", :yellow
docker_untracked_files.sort.each { |f| say " #{f}", :yellow }
say
end
with_env(KAMAL.config.builder.secrets) do
run_locally do
build = KAMAL.builder.push(cli.options[:output], tag_as_dirty: true)
KAMAL.with_verbosity(:debug) do
execute(*build)
end
end
end
end
private
def connect_to_remote_host(remote_host)
remote_uri = URI.parse(remote_host)
if remote_uri.scheme == "ssh"
host = SSHKit::Host.new(
hostname: remote_uri.host,
ssh_options: { user: remote_uri.user, port: remote_uri.port }.compact
)
on(host, options) do
execute "true"
end
end
end
def mirror_hosts
if KAMAL.hosts.many?
mirror_hosts = Concurrent::Hash.new
on(KAMAL.hosts) do |host|
first_mirror = capture_with_info(*KAMAL.builder.first_mirror).strip.presence
mirror_hosts[first_mirror] ||= host.to_s if first_mirror
rescue SSHKit::Command::Failed => e
raise unless e.message =~ /error calling index: reflect: slice index out of range/
end
mirror_hosts.values
else
[]
end
end
def pull_on_hosts(hosts)
on(hosts) do
execute *KAMAL.auditor.record("Pulled image with version #{KAMAL.config.version}"), verbosity: :debug
execute *KAMAL.builder.clean, raise_on_non_zero_exit: false
execute *KAMAL.builder.pull
execute *KAMAL.builder.validate_image
end
end
end

View File

@@ -0,0 +1,61 @@
require "uri"
class Kamal::Cli::Build::Clone
attr_reader :sshkit
delegate :info, :error, :execute, :capture_with_info, to: :sshkit
def initialize(sshkit)
@sshkit = sshkit
end
def prepare
begin
clone_repo
rescue SSHKit::Command::Failed => e
if e.message =~ /already exists and is not an empty directory/
reset
else
raise Kamal::Cli::Build::BuildError, "Failed to clone repo: #{e.message}"
end
end
validate!
rescue Kamal::Cli::Build::BuildError => e
error "Error preparing clone: #{e.message}, deleting and retrying..."
FileUtils.rm_rf KAMAL.config.builder.clone_directory
clone_repo
validate!
end
private
def clone_repo
info "Cloning repo into build directory `#{KAMAL.config.builder.build_directory}`..."
FileUtils.mkdir_p KAMAL.config.builder.clone_directory
execute *KAMAL.builder.clone
end
def reset
info "Resetting local clone as `#{KAMAL.config.builder.build_directory}` already exists..."
KAMAL.builder.clone_reset_steps.each { |step| execute *step }
rescue SSHKit::Command::Failed => e
raise Kamal::Cli::Build::BuildError, "Failed to clone repo: #{e.message}"
end
def validate!
status = capture_with_info(*KAMAL.builder.clone_status).strip
unless status.empty?
raise Kamal::Cli::Build::BuildError, "Clone in #{KAMAL.config.builder.build_directory} is dirty, #{status}"
end
revision = capture_with_info(*KAMAL.builder.clone_revision).strip
if revision != Kamal::Git.revision
raise Kamal::Cli::Build::BuildError, "Clone in #{KAMAL.config.builder.build_directory} is not on the correct revision, expected `#{Kamal::Git.revision}` but got `#{revision}`"
end
rescue SSHKit::Command::Failed => e
raise Kamal::Cli::Build::BuildError, "Failed to validate clone: #{e.message}"
end
end

View File

@@ -0,0 +1,33 @@
require "concurrent/ivar"
class Kamal::Cli::Healthcheck::Barrier
def initialize
@ivar = Concurrent::IVar.new
end
def close
set(false)
end
def open
set(true)
end
def wait
unless opened?
raise Kamal::Cli::Healthcheck::Error.new("Halted at barrier")
end
end
private
def opened?
@ivar.value
end
def set(value)
@ivar.set(value)
true
rescue Concurrent::MultipleAssignmentError
false
end
end

View File

@@ -0,0 +1,2 @@
class Kamal::Cli::Healthcheck::Error < StandardError
end

View File

@@ -0,0 +1,42 @@
module Kamal::Cli::Healthcheck::Poller
extend self
def wait_for_healthy(role, &block)
attempt = 1
timeout_at = Time.now + KAMAL.config.deploy_timeout
readiness_delay = KAMAL.config.readiness_delay
begin
status = block.call
if status == "running"
# Wait for the readiness delay and confirm it is still running
if readiness_delay > 0
info "Container is running, waiting for readiness delay of #{readiness_delay} seconds"
sleep readiness_delay
status = block.call
end
end
unless %w[ running healthy ].include?(status)
raise Kamal::Cli::Healthcheck::Error, "container not ready after #{KAMAL.config.deploy_timeout} seconds (#{status})"
end
rescue Kamal::Cli::Healthcheck::Error => e
time_left = timeout_at - Time.now
if time_left > 0
sleep [ attempt, time_left ].min
attempt += 1
retry
else
raise
end
end
info "Container is healthy!"
end
private
def info(message)
SSHKit.config.output.info(message)
end
end

View File

@@ -1,8 +1,10 @@
class Mrsk::Cli::Lock < Mrsk::Cli::Base class Kamal::Cli::Lock < Kamal::Cli::Base
desc "status", "Report lock status" desc "status", "Report lock status"
def status def status
handle_missing_lock do handle_missing_lock do
on(MRSK.primary_host) { puts capture_with_debug(*MRSK.lock.status) } on(KAMAL.primary_host) do
puts capture_with_debug(*KAMAL.lock.status)
end
end end
end end
@@ -10,8 +12,12 @@ class Mrsk::Cli::Lock < Mrsk::Cli::Base
option :message, aliases: "-m", type: :string, desc: "A lock message", required: true option :message, aliases: "-m", type: :string, desc: "A lock message", required: true
def acquire def acquire
message = options[:message] message = options[:message]
ensure_run_directory
raise_if_locked do raise_if_locked do
on(MRSK.primary_host) { execute *MRSK.lock.acquire(message, MRSK.config.version), verbosity: :debug } on(KAMAL.primary_host) do
execute *KAMAL.lock.acquire(message, KAMAL.config.version), verbosity: :debug
end
say "Acquired the deploy lock" say "Acquired the deploy lock"
end end
end end
@@ -19,7 +25,9 @@ class Mrsk::Cli::Lock < Mrsk::Cli::Base
desc "release", "Release the deploy lock" desc "release", "Release the deploy lock"
def release def release
handle_missing_lock do handle_missing_lock do
on(MRSK.primary_host) { execute *MRSK.lock.release, verbosity: :debug } on(KAMAL.primary_host) do
execute *KAMAL.lock.release, verbosity: :debug
end
say "Released the deploy lock" say "Released the deploy lock"
end end
end end

280
lib/kamal/cli/main.rb Normal file
View File

@@ -0,0 +1,280 @@
class Kamal::Cli::Main < Kamal::Cli::Base
desc "setup", "Setup all accessories, push the env, and deploy app to servers"
option :skip_push, aliases: "-P", type: :boolean, default: false, desc: "Skip image build and push"
def setup
print_runtime do
with_lock do
invoke_options = deploy_options
say "Ensure Docker is installed...", :magenta
invoke "kamal:cli:server:bootstrap", [], invoke_options
deploy(boot_accessories: true)
end
end
end
desc "deploy", "Deploy app to servers"
option :skip_push, aliases: "-P", type: :boolean, default: false, desc: "Skip image build and push"
def deploy(boot_accessories: false)
runtime = print_runtime do
invoke_options = deploy_options
say "Log into image registry...", :magenta
invoke "kamal:cli:registry:login", [], invoke_options.merge(skip_local: options[:skip_push])
if options[:skip_push]
say "Pull app image...", :magenta
invoke "kamal:cli:build:pull", [], invoke_options
else
say "Build and push app image...", :magenta
invoke "kamal:cli:build:deliver", [], invoke_options
end
with_lock do
run_hook "pre-deploy", secrets: true
say "Ensure kamal-proxy is running...", :magenta
invoke "kamal:cli:proxy:boot", [], invoke_options
invoke "kamal:cli:accessory:boot", [ "all" ], invoke_options if boot_accessories
say "Detect stale containers...", :magenta
invoke "kamal:cli:app:stale_containers", [], invoke_options.merge(stop: true)
invoke "kamal:cli:app:boot", [], invoke_options
say "Prune old containers and images...", :magenta
invoke "kamal:cli:prune:all", [], invoke_options
end
end
run_hook "post-deploy", secrets: true, runtime: runtime.round.to_s
end
desc "redeploy", "Deploy app to servers without bootstrapping servers, starting kamal-proxy, pruning, and registry login"
option :skip_push, aliases: "-P", type: :boolean, default: false, desc: "Skip image build and push"
def redeploy
runtime = print_runtime do
invoke_options = deploy_options
if options[:skip_push]
say "Pull app image...", :magenta
invoke "kamal:cli:build:pull", [], invoke_options
else
say "Build and push app image...", :magenta
invoke "kamal:cli:build:deliver", [], invoke_options
end
with_lock do
run_hook "pre-deploy", secrets: true
say "Detect stale containers...", :magenta
invoke "kamal:cli:app:stale_containers", [], invoke_options.merge(stop: true)
invoke "kamal:cli:app:boot", [], invoke_options
end
end
run_hook "post-deploy", secrets: true, runtime: runtime.round.to_s
end
desc "rollback [VERSION]", "Rollback app to VERSION"
def rollback(version)
rolled_back = false
runtime = print_runtime do
with_lock do
invoke_options = deploy_options
KAMAL.config.version = version
old_version = nil
if container_available?(version)
run_hook "pre-deploy", secrets: true
invoke "kamal:cli:app:boot", [], invoke_options.merge(version: version)
rolled_back = true
else
say "The app version '#{version}' is not available as a container (use 'kamal app containers' for available versions)", :red
end
end
end
run_hook "post-deploy", secrets: true, runtime: runtime.round.to_s if rolled_back
end
desc "details", "Show details about all containers"
def details
invoke "kamal:cli:proxy:details"
invoke "kamal:cli:app:details"
invoke "kamal:cli:accessory:details", [ "all" ]
end
desc "audit", "Show audit log from servers"
def audit
on(KAMAL.hosts) do |host|
puts_by_host host, capture_with_info(*KAMAL.auditor.reveal)
end
end
desc "config", "Show combined config (including secrets!)"
def config
run_locally do
puts Kamal::Utils.redacted(KAMAL.config.to_h).to_yaml
end
end
desc "docs [SECTION]", "Show Kamal configuration documentation"
def docs(section = nil)
case section
when NilClass
puts Kamal::Configuration.validation_doc
else
puts Kamal::Configuration.const_get(section.titlecase.to_sym).validation_doc
end
rescue NameError
puts "No documentation found for #{section}"
end
desc "init", "Create config stub in config/deploy.yml and secrets stub in .kamal"
option :bundle, type: :boolean, default: false, desc: "Add Kamal to the Gemfile and create a bin/kamal binstub"
def init
require "fileutils"
if (deploy_file = Pathname.new(File.expand_path("config/deploy.yml"))).exist?
puts "Config file already exists in config/deploy.yml (remove first to create a new one)"
else
FileUtils.mkdir_p deploy_file.dirname
FileUtils.cp_r Pathname.new(File.expand_path("templates/deploy.yml", __dir__)), deploy_file
puts "Created configuration file in config/deploy.yml"
end
unless (secrets_file = Pathname.new(File.expand_path(".kamal/secrets"))).exist?
FileUtils.mkdir_p secrets_file.dirname
FileUtils.cp_r Pathname.new(File.expand_path("templates/secrets", __dir__)), secrets_file
puts "Created .kamal/secrets file"
end
unless (hooks_dir = Pathname.new(File.expand_path(".kamal/hooks"))).exist?
hooks_dir.mkpath
Pathname.new(File.expand_path("templates/sample_hooks", __dir__)).each_child do |sample_hook|
FileUtils.cp sample_hook, hooks_dir, preserve: true
end
puts "Created sample hooks in .kamal/hooks"
end
if options[:bundle]
if (binstub = Pathname.new(File.expand_path("bin/kamal"))).exist?
puts "Binstub already exists in bin/kamal (remove first to create a new one)"
else
puts "Adding Kamal to Gemfile and bundle..."
run_locally do
execute :bundle, :add, :kamal
execute :bundle, :binstubs, :kamal
end
puts "Created binstub file in bin/kamal"
end
end
end
desc "remove", "Remove kamal-proxy, app, accessories, and registry session from servers"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def remove
confirming "This will remove all containers and images. Are you sure?" do
with_lock do
invoke "kamal:cli:app:remove", [], options.without(:confirmed)
invoke "kamal:cli:proxy:remove", [], options.without(:confirmed)
invoke "kamal:cli:accessory:remove", [ "all" ], options
invoke "kamal:cli:registry:logout", [], options.without(:confirmed).merge(skip_local: true)
end
end
end
desc "upgrade", "Upgrade from Kamal 1.x to 2.0"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
option :rolling, type: :boolean, default: false, desc: "Upgrade one host at a time"
def upgrade
confirming "This will replace Traefik with kamal-proxy and restart all accessories" do
with_lock do
if options[:rolling]
(KAMAL.hosts | KAMAL.accessory_hosts).each do |host|
KAMAL.with_specific_hosts(host) do
say "Upgrading #{host}...", :magenta
if KAMAL.hosts.include?(host)
invoke "kamal:cli:proxy:upgrade", [], options.merge(confirmed: true, rolling: false)
reset_invocation(Kamal::Cli::Proxy)
end
if KAMAL.accessory_hosts.include?(host)
invoke "kamal:cli:accessory:upgrade", [ "all" ], options.merge(confirmed: true, rolling: false)
reset_invocation(Kamal::Cli::Accessory)
end
say "Upgraded #{host}", :magenta
end
end
else
say "Upgrading all hosts...", :magenta
invoke "kamal:cli:proxy:upgrade", [], options.merge(confirmed: true)
invoke "kamal:cli:accessory:upgrade", [ "all" ], options.merge(confirmed: true)
say "Upgraded all hosts", :magenta
end
end
end
end
desc "version", "Show Kamal version"
def version
puts Kamal::VERSION
end
desc "accessory", "Manage accessories (db/redis/search)"
subcommand "accessory", Kamal::Cli::Accessory
desc "app", "Manage application"
subcommand "app", Kamal::Cli::App
desc "build", "Build application image"
subcommand "build", Kamal::Cli::Build
desc "lock", "Manage the deploy lock"
subcommand "lock", Kamal::Cli::Lock
desc "proxy", "Manage kamal-proxy"
subcommand "proxy", Kamal::Cli::Proxy
desc "prune", "Prune old application images and containers"
subcommand "prune", Kamal::Cli::Prune
desc "registry", "Login and -out of the image registry"
subcommand "registry", Kamal::Cli::Registry
desc "secrets", "Helpers for extracting secrets"
subcommand "secrets", Kamal::Cli::Secrets
desc "server", "Bootstrap servers with curl and Docker"
subcommand "server", Kamal::Cli::Server
private
def container_available?(version)
begin
on(KAMAL.hosts) do
KAMAL.roles_on(host).each do |role|
container_id = capture_with_info(*KAMAL.app(role: role, host: host).container_id_for_version(version))
raise "Container not found" unless container_id.present?
end
end
rescue SSHKit::Runner::ExecuteError, SSHKit::Runner::MultipleExecuteError => e
if e.message =~ /Container not found/
say "Error looking for container version #{version}: #{e.message}"
return false
else
raise
end
end
true
end
def deploy_options
{ "version" => KAMAL.config.version }.merge(options.without("skip_push"))
end
end

255
lib/kamal/cli/proxy.rb Normal file
View File

@@ -0,0 +1,255 @@
class Kamal::Cli::Proxy < Kamal::Cli::Base
desc "boot", "Boot proxy on servers"
def boot
with_lock do
on(KAMAL.hosts) do |host|
execute *KAMAL.docker.create_network
rescue SSHKit::Command::Failed => e
raise unless e.message.include?("already exists")
end
on(KAMAL.proxy_hosts) do |host|
execute *KAMAL.registry.login
version = capture_with_info(*KAMAL.proxy.version).strip.presence
if version && Kamal::Utils.older_version?(version, Kamal::Configuration::PROXY_MINIMUM_VERSION)
raise "kamal-proxy version #{version} is too old, run `kamal proxy reboot` in order to update to at least #{Kamal::Configuration::PROXY_MINIMUM_VERSION}"
end
execute *KAMAL.proxy.start_or_run
end
end
end
desc "boot_config <set|get|reset>", "Manage kamal-proxy boot configuration"
option :publish, type: :boolean, default: true, desc: "Publish the proxy ports on the host"
option :publish_host_ip, type: :string, repeatable: true, default: nil, desc: "Host IP address to bind HTTP/HTTPS traffic to. Defaults to all interfaces"
option :http_port, type: :numeric, default: Kamal::Configuration::PROXY_HTTP_PORT, desc: "HTTP port to publish on the host"
option :https_port, type: :numeric, default: Kamal::Configuration::PROXY_HTTPS_PORT, desc: "HTTPS port to publish on the host"
option :log_max_size, type: :string, default: Kamal::Configuration::PROXY_LOG_MAX_SIZE, desc: "Max size of proxy logs"
option :docker_options, type: :array, default: [], desc: "Docker options to pass to the proxy container", banner: "option=value option2=value2"
def boot_config(subcommand)
case subcommand
when "set"
boot_options = [
*(KAMAL.config.proxy_publish_args(options[:http_port], options[:https_port], options[:publish_host_ip]) if options[:publish]),
*(KAMAL.config.proxy_logging_args(options[:log_max_size])),
*options[:docker_options].map { |option| "--#{option}" }
]
on(KAMAL.proxy_hosts) do |host|
execute(*KAMAL.proxy.ensure_proxy_directory)
upload! StringIO.new(boot_options.join(" ")), KAMAL.config.proxy_options_file
end
when "get"
on(KAMAL.proxy_hosts) do |host|
puts "Host #{host}: #{capture_with_info(*KAMAL.proxy.get_boot_options)}"
end
when "reset"
on(KAMAL.proxy_hosts) do |host|
execute *KAMAL.proxy.reset_boot_options
end
else
raise ArgumentError, "Unknown boot_config subcommand #{subcommand}"
end
end
desc "reboot", "Reboot proxy on servers (stop container, remove container, start new container)"
option :rolling, type: :boolean, default: false, desc: "Reboot proxy on hosts in sequence, rather than in parallel"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def reboot
confirming "This will cause a brief outage on each host. Are you sure?" do
with_lock do
host_groups = options[:rolling] ? KAMAL.proxy_hosts : [ KAMAL.proxy_hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
run_hook "pre-proxy-reboot", hosts: host_list
on(hosts) do |host|
execute *KAMAL.auditor.record("Rebooted proxy"), verbosity: :debug
execute *KAMAL.registry.login
"Stopping and removing kamal-proxy on #{host}, if running..."
execute *KAMAL.proxy.stop, raise_on_non_zero_exit: false
execute *KAMAL.proxy.remove_container
execute *KAMAL.proxy.run
KAMAL.roles_on(host).select(&:running_proxy?).each do |role|
app = KAMAL.app(role: role, host: host)
version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
endpoint = capture_with_info(*app.container_id_for_version(version)).strip
if endpoint.present?
info "Deploying #{endpoint} for role `#{role}` on #{host}..."
execute *app.deploy(target: endpoint)
end
end
end
run_hook "post-proxy-reboot", hosts: host_list
end
end
end
end
desc "upgrade", "Upgrade to kamal-proxy on servers (stop container, remove container, start new container, reboot app)", hide: true
option :rolling, type: :boolean, default: false, desc: "Reboot proxy on hosts in sequence, rather than in parallel"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def upgrade
invoke_options = { "version" => KAMAL.config.latest_tag }.merge(options)
confirming "This will cause a brief outage on each host. Are you sure?" do
host_groups = options[:rolling] ? KAMAL.hosts : [ KAMAL.hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
say "Upgrading proxy on #{host_list}...", :magenta
run_hook "pre-proxy-reboot", hosts: host_list
on(hosts) do |host|
execute *KAMAL.auditor.record("Rebooted proxy"), verbosity: :debug
execute *KAMAL.registry.login
"Stopping and removing Traefik on #{host}, if running..."
execute *KAMAL.proxy.cleanup_traefik
"Stopping and removing kamal-proxy on #{host}, if running..."
execute *KAMAL.proxy.stop, raise_on_non_zero_exit: false
execute *KAMAL.proxy.remove_container
execute *KAMAL.proxy.remove_image
end
KAMAL.with_specific_hosts(hosts) do
invoke "kamal:cli:proxy:boot", [], invoke_options
reset_invocation(Kamal::Cli::Proxy)
invoke "kamal:cli:app:boot", [], invoke_options
reset_invocation(Kamal::Cli::App)
invoke "kamal:cli:prune:all", [], invoke_options
reset_invocation(Kamal::Cli::Prune)
end
run_hook "post-proxy-reboot", hosts: host_list
say "Upgraded proxy on #{host_list}", :magenta
end
end
end
desc "start", "Start existing proxy container on servers"
def start
with_lock do
on(KAMAL.proxy_hosts) do |host|
execute *KAMAL.auditor.record("Started proxy"), verbosity: :debug
execute *KAMAL.proxy.start
end
end
end
desc "stop", "Stop existing proxy container on servers"
def stop
with_lock do
on(KAMAL.proxy_hosts) do |host|
execute *KAMAL.auditor.record("Stopped proxy"), verbosity: :debug
execute *KAMAL.proxy.stop, raise_on_non_zero_exit: false
end
end
end
desc "restart", "Restart existing proxy container on servers"
def restart
with_lock do
stop
start
end
end
desc "details", "Show details about proxy container from servers"
def details
on(KAMAL.proxy_hosts) { |host| puts_by_host host, capture_with_info(*KAMAL.proxy.info), type: "Proxy" }
end
desc "logs", "Show log lines from proxy on servers"
option :since, aliases: "-s", desc: "Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)"
option :lines, type: :numeric, aliases: "-n", desc: "Number of log lines to pull from each server"
option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)"
option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)"
option :skip_timestamps, type: :boolean, aliases: "-T", desc: "Skip appending timestamps to logging output"
def logs
grep = options[:grep]
timestamps = !options[:skip_timestamps]
if options[:follow]
run_locally do
info "Following logs on #{KAMAL.primary_host}..."
info KAMAL.proxy.follow_logs(host: KAMAL.primary_host, timestamps: timestamps, grep: grep)
exec KAMAL.proxy.follow_logs(host: KAMAL.primary_host, timestamps: timestamps, grep: grep)
end
else
since = options[:since]
lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set
on(KAMAL.proxy_hosts) do |host|
puts_by_host host, capture(*KAMAL.proxy.logs(timestamps: timestamps, since: since, lines: lines, grep: grep)), type: "Proxy"
end
end
end
desc "remove", "Remove proxy container and image from servers"
option :force, type: :boolean, default: false, desc: "Force removing proxy when apps are still installed"
def remove
with_lock do
if removal_allowed?(options[:force])
stop
remove_container
remove_image
remove_proxy_directory
end
end
end
desc "remove_container", "Remove proxy container from servers", hide: true
def remove_container
with_lock do
on(KAMAL.proxy_hosts) do
execute *KAMAL.auditor.record("Removed proxy container"), verbosity: :debug
execute *KAMAL.proxy.remove_container
end
end
end
desc "remove_image", "Remove proxy image from servers", hide: true
def remove_image
with_lock do
on(KAMAL.proxy_hosts) do
execute *KAMAL.auditor.record("Removed proxy image"), verbosity: :debug
execute *KAMAL.proxy.remove_image
end
end
end
desc "remove_proxy_directory", "Remove the proxy directory from servers", hide: true
def remove_proxy_directory
with_lock do
on(KAMAL.proxy_hosts) do
execute *KAMAL.proxy.remove_proxy_directory, raise_on_non_zero_exit: false
end
end
end
private
def removal_allowed?(force)
on(KAMAL.proxy_hosts) do |host|
app_count = capture_with_info(*KAMAL.server.app_directory_count).chomp.to_i
raise "The are other applications installed on #{host}" if app_count > 0
end
true
rescue SSHKit::Runner::ExecuteError => e
raise unless e.message.include?("The are other applications installed on")
if force
say "Forcing, so removing the proxy, even though other apps are installed", :magenta
else
say "Not removing the proxy, as other apps are installed, ignore this check with kamal proxy remove --force", :magenta
end
force
end
end

34
lib/kamal/cli/prune.rb Normal file
View File

@@ -0,0 +1,34 @@
class Kamal::Cli::Prune < Kamal::Cli::Base
desc "all", "Prune unused images and stopped containers"
def all
with_lock do
containers
images
end
end
desc "images", "Prune unused images"
def images
with_lock do
on(KAMAL.hosts) do
execute *KAMAL.auditor.record("Pruned images"), verbosity: :debug
execute *KAMAL.prune.dangling_images
execute *KAMAL.prune.tagged_images
end
end
end
desc "containers", "Prune all stopped containers, except the last n (default 5)"
option :retain, type: :numeric, default: nil, desc: "Number of containers to retain"
def containers
retain = options.fetch(:retain, KAMAL.config.retain_containers)
raise "retain must be at least 1" if retain < 1
with_lock do
on(KAMAL.hosts) do
execute *KAMAL.auditor.record("Pruned containers"), verbosity: :debug
execute *KAMAL.prune.app_containers(retain: retain)
end
end
end
end

19
lib/kamal/cli/registry.rb Normal file
View File

@@ -0,0 +1,19 @@
class Kamal::Cli::Registry < Kamal::Cli::Base
desc "login", "Log in to registry locally and remotely"
option :skip_local, aliases: "-L", type: :boolean, default: false, desc: "Skip local login"
option :skip_remote, aliases: "-R", type: :boolean, default: false, desc: "Skip remote login"
def login
ensure_docker_installed unless options[:skip_local]
run_locally { execute *KAMAL.registry.login } unless options[:skip_local]
on(KAMAL.hosts) { execute *KAMAL.registry.login } unless options[:skip_remote]
end
desc "logout", "Log out of registry locally and remotely"
option :skip_local, aliases: "-L", type: :boolean, default: false, desc: "Skip local login"
option :skip_remote, aliases: "-R", type: :boolean, default: false, desc: "Skip remote login"
def logout
run_locally { execute *KAMAL.registry.logout } unless options[:skip_local]
on(KAMAL.hosts) { execute *KAMAL.registry.logout } unless options[:skip_remote]
end
end

49
lib/kamal/cli/secrets.rb Normal file
View File

@@ -0,0 +1,49 @@
class Kamal::Cli::Secrets < Kamal::Cli::Base
desc "fetch [SECRETS...]", "Fetch secrets from a vault"
option :adapter, type: :string, aliases: "-a", required: true, desc: "Which vault adapter to use"
option :account, type: :string, required: false, desc: "The account identifier or username"
option :from, type: :string, required: false, desc: "A vault or folder to fetch the secrets from"
option :inline, type: :boolean, required: false, hidden: true
def fetch(*secrets)
adapter = initialize_adapter(options[:adapter])
if adapter.requires_account? && options[:account].blank?
return puts "No value provided for required options '--account'"
end
results = adapter.fetch(secrets, **options.slice(:account, :from).symbolize_keys)
return_or_puts JSON.dump(results).shellescape, inline: options[:inline]
end
desc "extract", "Extract a single secret from the results of a fetch call"
option :inline, type: :boolean, required: false, hidden: true
def extract(name, secrets)
parsed_secrets = JSON.parse(secrets)
value = parsed_secrets[name] || parsed_secrets.find { |k, v| k.end_with?("/#{name}") }&.last
raise "Could not find secret #{name}" if value.nil?
return_or_puts value, inline: options[:inline]
end
desc "print", "Print the secrets (for debugging)"
def print
KAMAL.config.secrets.to_h.each do |key, value|
puts "#{key}=#{value}"
end
end
private
def initialize_adapter(adapter)
Kamal::Secrets::Adapters.lookup(adapter)
end
def return_or_puts(value, inline: nil)
if inline
value
else
puts value
end
end
end

48
lib/kamal/cli/server.rb Normal file
View File

@@ -0,0 +1,48 @@
class Kamal::Cli::Server < Kamal::Cli::Base
desc "exec", "Run a custom command on the server (use --help to show options)"
option :interactive, type: :boolean, aliases: "-i", default: false, desc: "Run the command interactively (use for console/bash)"
def exec(*cmd)
cmd = Kamal::Utils.join_commands(cmd)
hosts = KAMAL.hosts | KAMAL.accessory_hosts
case
when options[:interactive]
host = KAMAL.primary_host
say "Running '#{cmd}' on #{host} interactively...", :magenta
run_locally { exec KAMAL.server.run_over_ssh(cmd, host: host) }
else
say "Running '#{cmd}' on #{hosts.join(', ')}...", :magenta
on(hosts) do |host|
execute *KAMAL.auditor.record("Executed cmd '#{cmd}' on #{host}"), verbosity: :debug
puts_by_host host, capture_with_info(cmd)
end
end
end
desc "bootstrap", "Set up Docker to run Kamal apps"
def bootstrap
with_lock do
missing = []
on(KAMAL.hosts | KAMAL.accessory_hosts) do |host|
unless execute(*KAMAL.docker.installed?, raise_on_non_zero_exit: false)
if execute(*KAMAL.docker.superuser?, raise_on_non_zero_exit: false)
info "Missing Docker on #{host}. Installing…"
execute *KAMAL.docker.install
else
missing << host
end
end
end
if missing.any?
raise "Docker is not installed on #{missing.join(", ")} and can't be automatically installed without having root access and either `wget` or `curl`. Install Docker manually: https://docs.docker.com/engine/install/"
end
run_hook "docker-setup"
end
end
end

View File

@@ -0,0 +1,101 @@
# Name of your application. Used to uniquely configure containers.
service: my-app
# Name of the container image.
image: my-user/my-app
# Deploy to these servers.
servers:
web:
- 192.168.0.1
# job:
# hosts:
# - 192.168.0.1
# cmd: bin/jobs
# Enable SSL auto certification via Let's Encrypt and allow for multiple apps on a single web server.
# Remove this section when using multiple web servers and ensure you terminate SSL at your load balancer.
#
# Note: If using Cloudflare, set encryption mode in SSL/TLS setting to "Full" to enable CF-to-app encryption.
proxy:
ssl: true
host: app.example.com
# Proxy connects to your container on port 80 by default.
# app_port: 3000
# Credentials for your image host.
registry:
# Specify the registry server, if you're not using Docker Hub
# server: registry.digitalocean.com / ghcr.io / ...
username: my-user
# Always use an access token rather than real password (pulled from .kamal/secrets).
password:
- KAMAL_REGISTRY_PASSWORD
# Configure builder setup.
builder:
arch: amd64
# Pass in additional build args needed for your Dockerfile.
# args:
# RUBY_VERSION: <%= ENV["RBENV_VERSION"] || ENV["rvm_ruby_string"] || "#{RUBY_ENGINE}-#{RUBY_ENGINE_VERSION}" %>
# Inject ENV variables into containers (secrets come from .kamal/secrets).
#
# env:
# clear:
# DB_HOST: 192.168.0.2
# secret:
# - RAILS_MASTER_KEY
# Aliases are triggered with "bin/kamal <alias>". You can overwrite arguments on invocation:
# "bin/kamal app logs -r job" will tail logs from the first server in the job section.
#
# aliases:
# shell: app exec --interactive --reuse "bash"
# Use a different ssh user than root
#
# ssh:
# user: app
# Use a persistent storage volume.
#
# volumes:
# - "app_storage:/app/storage"
# Bridge fingerprinted assets, like JS and CSS, between versions to avoid
# hitting 404 on in-flight requests. Combines all files from new and old
# version inside the asset_path.
#
# asset_path: /app/public/assets
# Configure rolling deploys by setting a wait time between batches of restarts.
#
# boot:
# limit: 10 # Can also specify as a percentage of total hosts, such as "25%"
# wait: 2
# Use accessory services (secrets come from .kamal/secrets).
#
# accessories:
# db:
# image: mysql:8.0
# host: 192.168.0.2
# port: 3306
# env:
# clear:
# MYSQL_ROOT_HOST: '%'
# secret:
# - MYSQL_ROOT_PASSWORD
# files:
# - config/mysql/production.cnf:/etc/mysql/my.cnf
# - db/production.sql:/docker-entrypoint-initdb.d/setup.sql
# directories:
# - data:/var/lib/mysql
# redis:
# image: valkey/valkey:8
# host: 192.168.0.2
# port: 6379
# directories:
# - data:/data

View File

@@ -0,0 +1,3 @@
#!/bin/sh
echo "Docker set up on $KAMAL_HOSTS..."

View File

@@ -0,0 +1,3 @@
#!/bin/sh
echo "Booted app version $KAMAL_VERSION on $KAMAL_HOSTS..."

View File

@@ -0,0 +1,14 @@
#!/bin/sh
# A sample post-deploy hook
#
# These environment variables are available:
# KAMAL_RECORDED_AT
# KAMAL_PERFORMER
# KAMAL_VERSION
# KAMAL_HOSTS
# KAMAL_ROLE (if set)
# KAMAL_DESTINATION (if set)
# KAMAL_RUNTIME
echo "$KAMAL_PERFORMER deployed $KAMAL_VERSION to $KAMAL_DESTINATION in $KAMAL_RUNTIME seconds"

View File

@@ -0,0 +1,3 @@
#!/bin/sh
echo "Rebooted kamal-proxy on $KAMAL_HOSTS"

View File

@@ -0,0 +1,3 @@
#!/bin/sh
echo "Booting app version $KAMAL_VERSION on $KAMAL_HOSTS..."

View File

@@ -9,12 +9,12 @@
# 4. The version we are deploying matches the remote # 4. The version we are deploying matches the remote
# #
# These environment variables are available: # These environment variables are available:
# MRSK_RECORDED_AT # KAMAL_RECORDED_AT
# MRSK_PERFORMER # KAMAL_PERFORMER
# MRSK_VERSION # KAMAL_VERSION
# MRSK_HOSTS # KAMAL_HOSTS
# MRSK_ROLE (if set) # KAMAL_ROLE (if set)
# MRSK_DESTINATION (if set) # KAMAL_DESTINATION (if set)
if [ -n "$(git status --porcelain)" ]; then if [ -n "$(git status --porcelain)" ]; then
echo "Git checkout is not clean, aborting..." >&2 echo "Git checkout is not clean, aborting..." >&2
@@ -32,7 +32,7 @@ fi
current_branch=$(git branch --show-current) current_branch=$(git branch --show-current)
if [ -z "$current_branch" ]; then if [ -z "$current_branch" ]; then
echo "No git remote set, aborting..." >&2 echo "Not on a git branch, aborting..." >&2
exit 1 exit 1
fi fi
@@ -43,8 +43,8 @@ if [ -z "$remote_head" ]; then
exit 1 exit 1
fi fi
if [ "$MRSK_VERSION" != "$remote_head" ]; then if [ "$KAMAL_VERSION" != "$remote_head" ]; then
echo "Version ($MRSK_VERSION) does not match remote HEAD ($remote_head), aborting..." >&2 echo "Version ($KAMAL_VERSION) does not match remote HEAD ($remote_head), aborting..." >&2
exit 1 exit 1
fi fi

View File

@@ -5,15 +5,15 @@
# Warms DNS before connecting to hosts in parallel # Warms DNS before connecting to hosts in parallel
# #
# These environment variables are available: # These environment variables are available:
# MRSK_RECORDED_AT # KAMAL_RECORDED_AT
# MRSK_PERFORMER # KAMAL_PERFORMER
# MRSK_VERSION # KAMAL_VERSION
# MRSK_HOSTS # KAMAL_HOSTS
# MRSK_ROLE (if set) # KAMAL_ROLE (if set)
# MRSK_DESTINATION (if set) # KAMAL_DESTINATION (if set)
# MRSK_RUNTIME # KAMAL_RUNTIME
hosts = ENV["MRSK_HOSTS"].split(",") hosts = ENV["KAMAL_HOSTS"].split(",")
results = nil results = nil
max = 3 max = 3

View File

@@ -0,0 +1,109 @@
#!/usr/bin/env ruby
# A sample pre-deploy hook
#
# Checks the Github status of the build, waiting for a pending build to complete for up to 720 seconds.
#
# Fails unless the combined status is "success"
#
# These environment variables are available:
# KAMAL_RECORDED_AT
# KAMAL_PERFORMER
# KAMAL_VERSION
# KAMAL_HOSTS
# KAMAL_COMMAND
# KAMAL_SUBCOMMAND
# KAMAL_ROLE (if set)
# KAMAL_DESTINATION (if set)
# Only check the build status for production deployments
if ENV["KAMAL_COMMAND"] == "rollback" || ENV["KAMAL_DESTINATION"] != "production"
exit 0
end
require "bundler/inline"
# true = install gems so this is fast on repeat invocations
gemfile(true, quiet: true) do
source "https://rubygems.org"
gem "octokit"
gem "faraday-retry"
end
MAX_ATTEMPTS = 72
ATTEMPTS_GAP = 10
def exit_with_error(message)
$stderr.puts message
exit 1
end
class GithubStatusChecks
attr_reader :remote_url, :git_sha, :github_client, :combined_status
def initialize
@remote_url = `git config --get remote.origin.url`.strip.delete_prefix("https://github.com/")
@git_sha = `git rev-parse HEAD`.strip
@github_client = Octokit::Client.new(access_token: ENV["GITHUB_TOKEN"])
refresh!
end
def refresh!
@combined_status = github_client.combined_status(remote_url, git_sha)
end
def state
combined_status[:state]
end
def first_status_url
first_status = combined_status[:statuses].find { |status| status[:state] == state }
first_status && first_status[:target_url]
end
def complete_count
combined_status[:statuses].count { |status| status[:state] != "pending"}
end
def total_count
combined_status[:statuses].count
end
def current_status
if total_count > 0
"Completed #{complete_count}/#{total_count} checks, see #{first_status_url} ..."
else
"Build not started..."
end
end
end
$stdout.sync = true
puts "Checking build status..."
attempts = 0
checks = GithubStatusChecks.new
begin
loop do
case checks.state
when "success"
puts "Checks passed, see #{checks.first_status_url}"
exit 0
when "failure"
exit_with_error "Checks failed, see #{checks.first_status_url}"
when "pending"
attempts += 1
end
exit_with_error "Checks are still pending, gave up after #{MAX_ATTEMPTS * ATTEMPTS_GAP} seconds" if attempts == MAX_ATTEMPTS
puts checks.current_status
sleep(ATTEMPTS_GAP)
checks.refresh!
end
rescue Octokit::NotFound
exit_with_error "Build status could not be found"
end

View File

@@ -0,0 +1,3 @@
#!/bin/sh
echo "Rebooting kamal-proxy on $KAMAL_HOSTS..."

View File

@@ -0,0 +1,17 @@
# Secrets defined here are available for reference under registry/password, env/secret, builder/secrets,
# and accessories/*/env/secret in config/deploy.yml. All secrets should be pulled from either
# password manager, ENV, or a file. DO NOT ENTER RAW CREDENTIALS HERE! This file needs to be safe for git.
# Option 1: Read secrets from the environment
KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD
# Option 2: Read secrets via a command
# RAILS_MASTER_KEY=$(cat config/master.key)
# Option 3: Read secrets via kamal secrets helpers
# These will handle logging in and fetching the secrets in as few calls as possible
# There are adapters for 1Password, LastPass + Bitwarden
#
# SECRETS=$(kamal secrets fetch --adapter 1password --account my-account --from MyVault/MyItem KAMAL_REGISTRY_PASSWORD RAILS_MASTER_KEY)
# KAMAL_REGISTRY_PASSWORD=$(kamal secrets extract KAMAL_REGISTRY_PASSWORD $SECRETS)
# RAILS_MASTER_KEY=$(kamal secrets extract RAILS_MASTER_KEY $SECRETS)

167
lib/kamal/commander.rb Normal file
View File

@@ -0,0 +1,167 @@
require "active_support/core_ext/enumerable"
require "active_support/core_ext/module/delegation"
require "active_support/core_ext/object/blank"
class Kamal::Commander
attr_accessor :verbosity, :holding_lock, :connected
attr_reader :specific_roles, :specific_hosts
delegate :hosts, :roles, :primary_host, :primary_role, :roles_on, :proxy_hosts, :accessory_hosts, to: :specifics
def initialize
reset
end
def reset
self.verbosity = :info
self.holding_lock = false
self.connected = false
@specifics = @specific_roles = @specific_hosts = nil
@config = @config_kwargs = nil
@commands = {}
end
def config
@config ||= Kamal::Configuration.create_from(**@config_kwargs).tap do |config|
@config_kwargs = nil
configure_sshkit_with(config)
end
end
def configure(**kwargs)
@config, @config_kwargs = nil, kwargs
end
def configured?
@config || @config_kwargs
end
def specific_primary!
@specifics = nil
if specific_roles.present?
self.specific_hosts = [ specific_roles.first.primary_host ]
else
self.specific_hosts = [ config.primary_host ]
end
end
def specific_roles=(role_names)
@specifics = nil
if role_names.present?
@specific_roles = Kamal::Utils.filter_specific_items(role_names, config.roles)
if @specific_roles.empty?
raise ArgumentError, "No --roles match for #{role_names.join(',')}"
end
@specific_roles
end
end
def specific_hosts=(hosts)
@specifics = nil
if hosts.present?
@specific_hosts = Kamal::Utils.filter_specific_items(hosts, config.all_hosts)
if @specific_hosts.empty?
raise ArgumentError, "No --hosts match for #{hosts.join(',')}"
end
@specific_hosts
end
end
def with_specific_hosts(hosts)
original_hosts, self.specific_hosts = specific_hosts, hosts
yield
ensure
self.specific_hosts = original_hosts
end
def accessory_names
config.accessories&.collect(&:name) || []
end
def app(role: nil, host: nil)
Kamal::Commands::App.new(config, role: role, host: host)
end
def accessory(name)
Kamal::Commands::Accessory.new(config, name: name)
end
def auditor(**details)
Kamal::Commands::Auditor.new(config, **details)
end
def builder
@commands[:builder] ||= Kamal::Commands::Builder.new(config)
end
def docker
@commands[:docker] ||= Kamal::Commands::Docker.new(config)
end
def hook
@commands[:hook] ||= Kamal::Commands::Hook.new(config)
end
def lock
@commands[:lock] ||= Kamal::Commands::Lock.new(config)
end
def proxy
@commands[:proxy] ||= Kamal::Commands::Proxy.new(config)
end
def prune
@commands[:prune] ||= Kamal::Commands::Prune.new(config)
end
def registry
@commands[:registry] ||= Kamal::Commands::Registry.new(config)
end
def server
@commands[:server] ||= Kamal::Commands::Server.new(config)
end
def alias(name)
config.aliases[name]
end
def with_verbosity(level)
old_level = self.verbosity
self.verbosity = level
SSHKit.config.output_verbosity = level
yield
ensure
self.verbosity = old_level
SSHKit.config.output_verbosity = old_level
end
def holding_lock?
self.holding_lock
end
def connected?
self.connected
end
private
# Lazy setup of SSHKit
def configure_sshkit_with(config)
SSHKit::Backend::Netssh.pool.idle_timeout = config.sshkit.pool_idle_timeout
SSHKit::Backend::Netssh.configure do |sshkit|
sshkit.max_concurrent_starts = config.sshkit.max_concurrent_starts
sshkit.ssh_options = config.ssh.options
end
SSHKit.config.command_map[:docker] = "docker" # No need to use /usr/bin/env, just clogs up the logs
SSHKit.config.output_verbosity = verbosity
end
def specifics
@specifics ||= Kamal::Commander::Specifics.new(config, specific_hosts, specific_roles)
end
end

View File

@@ -0,0 +1,54 @@
class Kamal::Commander::Specifics
attr_reader :primary_host, :primary_role, :hosts, :roles
delegate :stable_sort!, to: Kamal::Utils
def initialize(config, specific_hosts, specific_roles)
@config, @specific_hosts, @specific_roles = config, specific_hosts, specific_roles
@roles, @hosts = specified_roles, specified_hosts
@primary_host = specific_hosts&.first || primary_specific_role&.primary_host || config.primary_host
@primary_role = primary_or_first_role(roles_on(primary_host))
stable_sort!(roles) { |role| role == primary_role ? 0 : 1 }
stable_sort!(hosts) { |host| roles_on(host).any? { |role| role == primary_role } ? 0 : 1 }
end
def roles_on(host)
roles.select { |role| role.hosts.include?(host.to_s) }
end
def proxy_hosts
config.proxy_hosts & specified_hosts
end
def accessory_hosts
config.accessories.flat_map(&:hosts) & specified_hosts
end
private
attr_reader :config, :specific_hosts, :specific_roles
def primary_specific_role
primary_or_first_role(specific_roles) if specific_roles.present?
end
def primary_or_first_role(roles)
roles.detect { |role| role == config.primary_role } || roles.first
end
def specified_roles
(specific_roles || config.roles) \
.select { |role| ((specific_hosts || config.all_hosts) & role.hosts).any? }
end
def specified_hosts
specified_hosts = specific_hosts || config.all_hosts
if (specific_role_hosts = specific_roles&.flat_map(&:hosts)).present?
specified_hosts.select { |host| specific_role_hosts.include?(host) }
else
specified_hosts
end
end
end

2
lib/kamal/commands.rb Normal file
View File

@@ -0,0 +1,2 @@
module Kamal::Commands
end

View File

@@ -1,7 +1,12 @@
class Mrsk::Commands::Accessory < Mrsk::Commands::Base class Kamal::Commands::Accessory < Kamal::Commands::Base
include Proxy
attr_reader :accessory_config attr_reader :accessory_config
delegate :service_name, :image, :hosts, :port, :files, :directories, :cmd, delegate :service_name, :image, :hosts, :port, :files, :directories, :cmd,
:publish_args, :env_args, :volume_args, :label_args, :option_args, to: :accessory_config :network_args, :publish_args, :env_args, :volume_args, :label_args, :option_args,
:secrets_io, :secrets_path, :env_directory, :proxy, :running_proxy?, :registry,
to: :accessory_config
delegate :proxy_container_name, to: :config
def initialize(config, name:) def initialize(config, name:)
super(config) super(config)
@@ -13,6 +18,7 @@ class Mrsk::Commands::Accessory < Mrsk::Commands::Base
"--name", service_name, "--name", service_name,
"--detach", "--detach",
"--restart", "unless-stopped", "--restart", "unless-stopped",
*network_args,
*config.logging_args, *config.logging_args,
*publish_args, *publish_args,
*env_args, *env_args,
@@ -35,21 +41,19 @@ class Mrsk::Commands::Accessory < Mrsk::Commands::Base
docker :ps, *service_filter docker :ps, *service_filter
end end
def logs(timestamps: true, since: nil, lines: nil, grep: nil, grep_options: nil)
def logs(since: nil, lines: nil, grep: nil)
pipe \ pipe \
docker(:logs, service_name, (" --since #{since}" if since), (" --tail #{lines}" if lines), "--timestamps", "2>&1"), docker(:logs, service_name, (" --since #{since}" if since), (" --tail #{lines}" if lines), ("--timestamps" if timestamps), "2>&1"),
("grep '#{grep}'" if grep) ("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep)
end end
def follow_logs(grep: nil) def follow_logs(timestamps: true, grep: nil, grep_options: nil)
run_over_ssh \ run_over_ssh \
pipe \ pipe \
docker(:logs, service_name, "--timestamps", "--tail", "10", "--follow", "2>&1"), docker(:logs, service_name, ("--timestamps" if timestamps), "--tail", "10", "--follow", "2>&1"),
(%(grep "#{grep}") if grep) (%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep)
end end
def execute_in_existing_container(*command, interactive: false) def execute_in_existing_container(*command, interactive: false)
docker :exec, docker :exec,
("-it" if interactive), ("-it" if interactive),
@@ -61,6 +65,7 @@ class Mrsk::Commands::Accessory < Mrsk::Commands::Base
docker :run, docker :run,
("-it" if interactive), ("-it" if interactive),
"--rm", "--rm",
*network_args,
*env_args, *env_args,
*volume_args, *volume_args,
image, image,
@@ -79,21 +84,12 @@ class Mrsk::Commands::Accessory < Mrsk::Commands::Base
super command, host: hosts.first super command, host: hosts.first
end end
def ensure_local_file_present(local_file) def ensure_local_file_present(local_file)
if !local_file.is_a?(StringIO) && !Pathname.new(local_file).exist? if !local_file.is_a?(StringIO) && !Pathname.new(local_file).exist?
raise "Missing file: #{local_file}" raise "Missing file: #{local_file}"
end end
end end
def make_directory_for(remote_file)
make_directory Pathname.new(remote_file).dirname.to_s
end
def make_directory(path)
[ :mkdir, "-p", path ]
end
def remove_service_directory def remove_service_directory
[ :rm, "-rf", service_name ] [ :rm, "-rf", service_name ]
end end
@@ -106,6 +102,10 @@ class Mrsk::Commands::Accessory < Mrsk::Commands::Base
docker :image, :rm, "--force", image docker :image, :rm, "--force", image
end end
def ensure_env_directory
make_directory env_directory
end
private private
def service_filter def service_filter
[ "--filter", "label=service=#{service_name}" ] [ "--filter", "label=service=#{service_name}" ]

View File

@@ -0,0 +1,16 @@
module Kamal::Commands::Accessory::Proxy
delegate :proxy_container_name, to: :config
def deploy(target:)
proxy_exec :deploy, service_name, *proxy.deploy_command_args(target: target)
end
def remove
proxy_exec :remove, service_name
end
private
def proxy_exec(*command)
docker :exec, proxy_container_name, "kamal-proxy", *command
end
end

123
lib/kamal/commands/app.rb Normal file
View File

@@ -0,0 +1,123 @@
class Kamal::Commands::App < Kamal::Commands::Base
include Assets, Containers, Execution, Images, Logging, Proxy
ACTIVE_DOCKER_STATUSES = [ :running, :restarting ]
attr_reader :role, :host
delegate :container_name, to: :role
def initialize(config, role: nil, host: nil)
super(config)
@role = role
@host = host
end
def run(hostname: nil)
docker :run,
"--detach",
"--restart unless-stopped",
"--name", container_name,
"--network", "kamal",
*([ "--hostname", hostname ] if hostname),
"-e", "KAMAL_CONTAINER_NAME=\"#{container_name}\"",
"-e", "KAMAL_VERSION=\"#{config.version}\"",
*role.env_args(host),
*role.logging_args,
*config.volume_args,
*role.asset_volume_args,
*role.label_args,
*role.option_args,
config.absolute_image,
role.cmd
end
def start
docker :start, container_name
end
def status(version:)
pipe container_id_for_version(version), xargs(docker(:inspect, "--format", DOCKER_HEALTH_STATUS_FORMAT))
end
def stop(version: nil)
pipe \
version ? container_id_for_version(version) : current_running_container_id,
xargs(docker(:stop, *role.stop_args))
end
def info
docker :ps, *container_filter_args
end
def current_running_container_id
current_running_container(format: "--quiet")
end
def container_id_for_version(version, only_running: false)
container_id_for(container_name: container_name(version), only_running: only_running)
end
def current_running_version
pipe \
current_running_container(format: "--format '{{.Names}}'"),
extract_version_from_name
end
def list_versions(*docker_args, statuses: nil)
pipe \
docker(:ps, *container_filter_args(statuses: statuses), *docker_args, "--format", '"{{.Names}}"'),
extract_version_from_name
end
def ensure_env_directory
make_directory role.env_directory
end
private
def latest_image_id
docker :image, :ls, *argumentize("--filter", "reference=#{config.latest_image}"), "--format", "'{{.ID}}'"
end
def current_running_container(format:)
pipe \
shell(chain(latest_image_container(format: format), latest_container(format: format))),
[ :head, "-1" ]
end
def latest_image_container(format:)
latest_container format: format, filters: [ "ancestor=$(#{latest_image_id.join(" ")})" ]
end
def latest_container(format:, filters: nil)
docker :ps, "--latest", *format, *container_filter_args(statuses: ACTIVE_DOCKER_STATUSES), argumentize("--filter", filters)
end
def container_filter_args(statuses: nil)
argumentize "--filter", container_filters(statuses: statuses)
end
def image_filter_args
argumentize "--filter", image_filters
end
def extract_version_from_name
# Extract SHA from "service-role-dest-SHA"
%(while read line; do echo ${line##{role.container_prefix}-}; done)
end
def container_filters(statuses: nil)
[ "label=service=#{config.service}" ].tap do |filters|
filters << "label=destination=#{config.destination}"
filters << "label=role=#{role}" if role
statuses&.each do |status|
filters << "status=#{status}"
end
end
end
def image_filters
[ "label=service=#{config.service}" ]
end
end

View File

@@ -0,0 +1,51 @@
module Kamal::Commands::App::Assets
def extract_assets
asset_container = "#{role.container_prefix}-assets"
combine \
make_directory(role.asset_extracted_directory),
[ *docker(:container, :rm, asset_container, "2> /dev/null"), "|| true" ],
docker(:container, :create, "--name", asset_container, config.absolute_image),
docker(:container, :cp, "-L", "#{asset_container}:#{role.asset_path}/.", role.asset_extracted_directory),
docker(:container, :rm, asset_container),
by: "&&"
end
def sync_asset_volumes(old_version: nil)
new_extracted_path, new_volume_path = role.asset_extracted_directory(config.version), role.asset_volume.host_path
if old_version.present?
old_extracted_path, old_volume_path = role.asset_extracted_directory(old_version), role.asset_volume(old_version).host_path
end
commands = [ make_directory(new_volume_path), copy_contents(new_extracted_path, new_volume_path) ]
if old_version.present?
commands << copy_contents(new_extracted_path, old_volume_path, continue_on_error: true)
commands << copy_contents(old_extracted_path, new_volume_path, continue_on_error: true)
end
chain *commands
end
def clean_up_assets
chain \
find_and_remove_older_siblings(role.asset_extracted_directory),
find_and_remove_older_siblings(role.asset_volume_directory)
end
private
def find_and_remove_older_siblings(path)
[
:find,
Pathname.new(path).dirname.to_s,
"-maxdepth 1",
"-name", "'#{role.name}-*'",
"!", "-name", Pathname.new(path).basename.to_s,
"-exec rm -rf \"{}\" +"
]
end
def copy_contents(source, destination, continue_on_error: false)
[ :cp, "-rnT", "#{source}", destination, *("|| true" if continue_on_error) ]
end
end

View File

@@ -0,0 +1,31 @@
module Kamal::Commands::App::Containers
DOCKER_HEALTH_LOG_FORMAT = "'{{json .State.Health}}'"
def list_containers
docker :container, :ls, "--all", *container_filter_args
end
def list_container_names
[ *list_containers, "--format", "'{{ .Names }}'" ]
end
def remove_container(version:)
pipe \
container_id_for(container_name: container_name(version)),
xargs(docker(:container, :rm))
end
def rename_container(version:, new_version:)
docker :rename, container_name(version), container_name(new_version)
end
def remove_containers
docker :container, :prune, "--force", *container_filter_args
end
def container_health_log(version:)
pipe \
container_id_for(container_name: container_name(version)),
xargs(docker(:inspect, "--format", DOCKER_HEALTH_LOG_FORMAT))
end
end

View File

@@ -0,0 +1,32 @@
module Kamal::Commands::App::Execution
def execute_in_existing_container(*command, interactive: false, env:)
docker :exec,
("-it" if interactive),
*argumentize("--env", env),
container_name,
*command
end
def execute_in_new_container(*command, interactive: false, detach: false, env:)
docker :run,
("-it" if interactive),
("--detach" if detach),
("--rm" unless detach),
"--network", "kamal",
*role&.env_args(host),
*argumentize("--env", env),
*role.logging_args,
*config.volume_args,
*role&.option_args,
config.absolute_image,
*command
end
def execute_in_existing_container_over_ssh(*command, env:)
run_over_ssh execute_in_existing_container(*command, interactive: true, env: env), host: host
end
def execute_in_new_container_over_ssh(*command, env:)
run_over_ssh execute_in_new_container(*command, interactive: true, env: env), host: host
end
end

View File

@@ -0,0 +1,13 @@
module Kamal::Commands::App::Images
def list_images
docker :image, :ls, config.repository
end
def remove_images
docker :image, :prune, "--all", "--force", *image_filter_args
end
def tag_latest_image
docker :tag, config.absolute_image, config.latest_image
end
end

View File

@@ -0,0 +1,28 @@
module Kamal::Commands::App::Logging
def logs(container_id: nil, timestamps: true, since: nil, lines: nil, grep: nil, grep_options: nil)
pipe \
container_id_command(container_id),
"xargs docker logs#{" --timestamps" if timestamps}#{" --since #{since}" if since}#{" --tail #{lines}" if lines} 2>&1",
("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep)
end
def follow_logs(host:, container_id: nil, timestamps: true, lines: nil, grep: nil, grep_options: nil)
run_over_ssh \
pipe(
container_id_command(container_id),
"xargs docker logs#{" --timestamps" if timestamps}#{" --tail #{lines}" if lines} --follow 2>&1",
(%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep)
),
host: host
end
private
def container_id_command(container_id)
case container_id
when Array then container_id
when String, Symbol then "echo #{container_id}"
else current_running_container_id
end
end
end

View File

@@ -0,0 +1,16 @@
module Kamal::Commands::App::Proxy
delegate :proxy_container_name, to: :config
def deploy(target:)
proxy_exec :deploy, role.container_prefix, *role.proxy.deploy_command_args(target: target)
end
def remove
proxy_exec :remove, role.container_prefix
end
private
def proxy_exec(*command)
docker :exec, proxy_container_name, "kamal-proxy", *command
end
end

View File

@@ -0,0 +1,33 @@
class Kamal::Commands::Auditor < Kamal::Commands::Base
attr_reader :details
def initialize(config, **details)
super(config)
@details = details
end
# Runs remotely
def record(line, **details)
combine \
[ :mkdir, "-p", config.run_directory ],
append(
[ :echo, audit_tags(**details).except(:version, :service_version, :service).to_s, line ],
audit_log_file
)
end
def reveal
[ :tail, "-n", 50, audit_log_file ]
end
private
def audit_log_file
file = [ config.service, config.destination, "audit.log" ].compact.join("-")
File.join(config.run_directory, file)
end
def audit_tags(**details)
tags(**self.details, **details)
end
end

122
lib/kamal/commands/base.rb Normal file
View File

@@ -0,0 +1,122 @@
module Kamal::Commands
class Base
delegate :sensitive, :argumentize, to: Kamal::Utils
DOCKER_HEALTH_STATUS_FORMAT = "'{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}'"
attr_accessor :config
def initialize(config)
@config = config
end
def run_over_ssh(*command, host:)
"ssh#{ssh_proxy_args}#{ssh_keys_args} -t #{config.ssh.user}@#{host} -p #{config.ssh.port} '#{command.join(" ").gsub("'", "'\\\\''")}'"
end
def container_id_for(container_name:, only_running: false)
docker :container, :ls, *("--all" unless only_running), "--filter", "name=^#{container_name}$", "--quiet"
end
def make_directory_for(remote_file)
make_directory Pathname.new(remote_file).dirname.to_s
end
def make_directory(path)
[ :mkdir, "-p", path ]
end
def remove_directory(path)
[ :rm, "-r", path ]
end
def remove_file(path)
[ :rm, path ]
end
def ensure_docker_installed
combine \
ensure_local_docker_installed,
ensure_local_buildx_installed
end
private
def combine(*commands, by: "&&")
commands
.compact
.collect { |command| Array(command) + [ by ] }.flatten # Join commands
.tap { |commands| commands.pop } # Remove trailing combiner
end
def chain(*commands)
combine *commands, by: ";"
end
def pipe(*commands)
combine *commands, by: "|"
end
def append(*commands)
combine *commands, by: ">>"
end
def write(*commands)
combine *commands, by: ">"
end
def any(*commands)
combine *commands, by: "||"
end
def xargs(command)
[ :xargs, command ].flatten
end
def shell(command)
[ :sh, "-c", "'#{command.flatten.join(" ").gsub("'", "'\\\\''")}'" ]
end
def docker(*args)
args.compact.unshift :docker
end
def git(*args, path: nil)
[ :git, *([ "-C", path ] if path), *args.compact ]
end
def grep(*args)
args.compact.unshift :grep
end
def tags(**details)
Kamal::Tags.from_config(config, **details)
end
def ssh_proxy_args
case config.ssh.proxy
when Net::SSH::Proxy::Jump
" -J #{config.ssh.proxy.jump_proxies}"
when Net::SSH::Proxy::Command
" -o ProxyCommand='#{config.ssh.proxy.command_line_template}'"
end
end
def ssh_keys_args
"#{ ssh_keys.join("") if ssh_keys}" + "#{" -o IdentitiesOnly=yes" if config.ssh&.keys_only}"
end
def ssh_keys
config.ssh.keys&.map do |key|
" -i #{key}"
end
end
def ensure_local_docker_installed
docker "--version"
end
def ensure_local_buildx_installed
docker :buildx, "version"
end
end
end

View File

@@ -0,0 +1,42 @@
require "active_support/core_ext/string/filters"
class Kamal::Commands::Builder < Kamal::Commands::Base
delegate :create, :remove, :dev, :push, :clean, :pull, :info, :inspect_builder, :validate_image, :first_mirror, to: :target
delegate :local?, :remote?, :cloud?, to: "config.builder"
include Clone
def name
target.class.to_s.remove("Kamal::Commands::Builder::").underscore.inquiry
end
def target
if remote?
if local?
hybrid
else
remote
end
elsif cloud?
cloud
else
local
end
end
def remote
@remote ||= Kamal::Commands::Builder::Remote.new(config)
end
def local
@local ||= Kamal::Commands::Builder::Local.new(config)
end
def hybrid
@hybrid ||= Kamal::Commands::Builder::Hybrid.new(config)
end
def cloud
@cloud ||= Kamal::Commands::Builder::Cloud.new(config)
end
end

View File

@@ -0,0 +1,122 @@
class Kamal::Commands::Builder::Base < Kamal::Commands::Base
class BuilderError < StandardError; end
ENDPOINT_DOCKER_HOST_INSPECT = "'{{.Endpoints.docker.Host}}'"
delegate :argumentize, to: Kamal::Utils
delegate \
:args, :secrets, :dockerfile, :target, :arches, :local_arches, :remote_arches, :remote,
:cache_from, :cache_to, :ssh, :provenance, :sbom, :driver, :docker_driver?,
to: :builder_config
def clean
docker :image, :rm, "--force", config.absolute_image
end
def push(export_action = "registry", tag_as_dirty: false)
docker :buildx, :build,
"--output=type=#{export_action}",
*platform_options(arches),
*([ "--builder", builder_name ] unless docker_driver?),
*build_tag_options(tag_as_dirty: tag_as_dirty),
*build_options,
build_context
end
def pull
docker :pull, config.absolute_image
end
def info
combine \
docker(:context, :ls),
docker(:buildx, :ls)
end
def inspect_builder
docker :buildx, :inspect, builder_name unless docker_driver?
end
def build_options
[ *build_cache, *build_labels, *build_args, *build_secrets, *build_dockerfile, *build_target, *build_ssh, *builder_provenance, *builder_sbom ]
end
def build_context
config.builder.context
end
def validate_image
pipe \
docker(:inspect, "-f", "'{{ .Config.Labels.service }}'", config.absolute_image),
any(
[ :grep, "-x", config.service ],
"(echo \"Image #{config.absolute_image} is missing the 'service' label\" && exit 1)"
)
end
def first_mirror
docker(:info, "--format '{{index .RegistryConfig.Mirrors 0}}'")
end
private
def build_tag_names(tag_as_dirty: false)
tag_names = [ config.absolute_image, config.latest_image ]
tag_names.map! { |t| "#{t}-dirty" } if tag_as_dirty
tag_names
end
def build_tag_options(tag_as_dirty: false)
build_tag_names(tag_as_dirty: tag_as_dirty).flat_map { |name| [ "-t", name ] }
end
def build_cache
if cache_to && cache_from
[ "--cache-to", cache_to,
"--cache-from", cache_from ]
end
end
def build_labels
argumentize "--label", { service: config.service }
end
def build_args
argumentize "--build-arg", args, sensitive: true
end
def build_secrets
argumentize "--secret", secrets.keys.collect { |secret| [ "id", secret ] }
end
def build_dockerfile
if Pathname.new(File.expand_path(dockerfile)).exist?
argumentize "--file", dockerfile
else
raise BuilderError, "Missing #{dockerfile}"
end
end
def build_target
argumentize "--target", target if target.present?
end
def build_ssh
argumentize "--ssh", ssh if ssh.present?
end
def builder_provenance
argumentize "--provenance", provenance unless provenance.nil?
end
def builder_sbom
argumentize "--sbom", sbom unless sbom.nil?
end
def builder_config
config.builder
end
def platform_options(arches)
argumentize "--platform", arches.map { |arch| "linux/#{arch}" }.join(",") if arches.any?
end
end

View File

@@ -0,0 +1,31 @@
module Kamal::Commands::Builder::Clone
def clone
git :clone, escaped_root, "--recurse-submodules", path: config.builder.clone_directory.shellescape
end
def clone_reset_steps
[
git(:remote, "set-url", :origin, escaped_root, path: escaped_build_directory),
git(:fetch, :origin, path: escaped_build_directory),
git(:reset, "--hard", Kamal::Git.revision, path: escaped_build_directory),
git(:clean, "-fdx", path: escaped_build_directory),
git(:submodule, :update, "--init", path: escaped_build_directory)
]
end
def clone_status
git :status, "--porcelain", path: escaped_build_directory
end
def clone_revision
git :"rev-parse", :HEAD, path: escaped_build_directory
end
def escaped_root
Kamal::Git.root.shellescape
end
def escaped_build_directory
config.builder.build_directory.shellescape
end
end

View File

@@ -0,0 +1,22 @@
class Kamal::Commands::Builder::Cloud < Kamal::Commands::Builder::Base
# Expects `driver` to be of format "cloud docker-org-name/builder-name"
def create
docker :buildx, :create, "--driver", driver
end
def remove
docker :buildx, :rm, builder_name
end
private
def builder_name
driver.gsub(/[ \/]/, "-")
end
def inspect_buildx
pipe \
docker(:buildx, :inspect, builder_name),
grep("-q", "Endpoint:.*cloud://.*")
end
end

View File

@@ -0,0 +1,21 @@
class Kamal::Commands::Builder::Hybrid < Kamal::Commands::Builder::Remote
def create
combine \
create_local_buildx,
create_remote_context,
append_remote_buildx
end
private
def builder_name
"kamal-hybrid-#{driver}-#{remote.gsub(/[^a-z0-9_-]/, "-")}"
end
def create_local_buildx
docker :buildx, :create, *platform_options(local_arches), "--name", builder_name, "--driver=#{driver}"
end
def append_remote_buildx
docker :buildx, :create, *platform_options(remote_arches), "--append", "--name", builder_name, remote_context_name
end
end

View File

@@ -0,0 +1,14 @@
class Kamal::Commands::Builder::Local < Kamal::Commands::Builder::Base
def create
docker :buildx, :create, "--name", builder_name, "--driver=#{driver}" unless docker_driver?
end
def remove
docker :buildx, :rm, builder_name unless docker_driver?
end
private
def builder_name
"kamal-local-#{driver}"
end
end

View File

@@ -0,0 +1,63 @@
class Kamal::Commands::Builder::Remote < Kamal::Commands::Builder::Base
def create
chain \
create_remote_context,
create_buildx
end
def remove
chain \
remove_remote_context,
remove_buildx
end
def info
chain \
docker(:context, :ls),
docker(:buildx, :ls)
end
def inspect_builder
combine \
combine inspect_buildx, inspect_remote_context,
[ "(echo no compatible builder && exit 1)" ],
by: "||"
end
private
def builder_name
"kamal-remote-#{remote.gsub(/[^a-z0-9_-]/, "-")}"
end
def remote_context_name
"#{builder_name}-context"
end
def inspect_buildx
pipe \
docker(:buildx, :inspect, builder_name),
grep("-q", "Endpoint:.*#{remote_context_name}")
end
def inspect_remote_context
pipe \
docker(:context, :inspect, remote_context_name, "--format", ENDPOINT_DOCKER_HOST_INSPECT),
grep("-xq", remote)
end
def create_remote_context
docker :context, :create, remote_context_name, "--description", "'#{builder_name} host'", "--docker", "'host=#{remote}'"
end
def remove_remote_context
docker :context, :rm, remote_context_name
end
def create_buildx
docker :buildx, :create, "--name", builder_name, remote_context_name
end
def remove_buildx
docker :buildx, :rm, builder_name
end
end

View File

@@ -0,0 +1,34 @@
class Kamal::Commands::Docker < Kamal::Commands::Base
# Install Docker using the https://github.com/docker/docker-install convenience script.
def install
pipe get_docker, :sh
end
# Checks the Docker client version. Fails if Docker is not installed.
def installed?
docker "-v"
end
# Checks the Docker server version. Fails if Docker is not running.
def running?
docker :version
end
# Do we have superuser access to install Docker and start system services?
def superuser?
[ '[ "${EUID:-$(id -u)}" -eq 0 ] || command -v sudo >/dev/null || command -v su >/dev/null' ]
end
def create_network
docker :network, :create, :kamal
end
private
def get_docker
shell \
any \
[ :curl, "-fsSL", "https://get.docker.com" ],
[ :wget, "-O -", "https://get.docker.com" ],
[ :echo, "\"exit 1\"" ]
end
end

View File

@@ -0,0 +1,20 @@
class Kamal::Commands::Hook < Kamal::Commands::Base
def run(hook)
[ hook_file(hook) ]
end
def env(secrets: false, **details)
tags(**details).env.tap do |env|
env.merge!(config.secrets.to_h) if secrets
end
end
def hook_exists?(hook)
Pathname.new(hook_file(hook)).exist?
end
private
def hook_file(hook)
File.join(config.hooks_path, hook)
end
end

View File

@@ -1,17 +1,18 @@
require "active_support/duration" require "active_support/duration"
require "time" require "time"
require "base64"
class Mrsk::Commands::Lock < Mrsk::Commands::Base class Kamal::Commands::Lock < Kamal::Commands::Base
def acquire(message, version) def acquire(message, version)
combine \ combine \
[:mkdir, lock_dir], [ :mkdir, lock_dir ],
write_lock_details(message, version) write_lock_details(message, version)
end end
def release def release
combine \ combine \
[:rm, lock_details_file], [ :rm, lock_details_file ],
[:rm, "-r", lock_dir] [ :rm, "-r", lock_dir ]
end end
def status def status
@@ -20,31 +21,37 @@ class Mrsk::Commands::Lock < Mrsk::Commands::Base
read_lock_details read_lock_details
end end
def ensure_locks_directory
[ :mkdir, "-p", locks_dir ]
end
private private
def write_lock_details(message, version) def write_lock_details(message, version)
write \ write \
[:echo, "\"#{Base64.encode64(lock_details(message, version))}\""], [ :echo, "\"#{Base64.encode64(lock_details(message, version))}\"" ],
lock_details_file lock_details_file
end end
def read_lock_details def read_lock_details
pipe \ pipe \
[:cat, lock_details_file], [ :cat, lock_details_file ],
[:base64, "-d"] [ :base64, "-d" ]
end end
def stat_lock_dir def stat_lock_dir
write \ write \
[:stat, lock_dir], [ :stat, lock_dir ],
"/dev/null" "/dev/null"
end end
def lock_dir def lock_dir
:mrsk_lock dir_name = [ "lock", config.service, config.destination ].compact.join("-")
File.join(config.run_directory, dir_name)
end end
def lock_details_file def lock_details_file
[lock_dir, :details].join("/") File.join(lock_dir, "details")
end end
def lock_details(message, version) def lock_details(message, version)
@@ -56,7 +63,7 @@ class Mrsk::Commands::Lock < Mrsk::Commands::Base
end end
def locked_by def locked_by
`git config user.name`.strip Kamal::Git.user_name
rescue Errno::ENOENT rescue Errno::ENOENT
"Unknown" "Unknown"
end end

View File

@@ -0,0 +1,87 @@
class Kamal::Commands::Proxy < Kamal::Commands::Base
delegate :argumentize, :optionize, to: Kamal::Utils
def run
docker :run,
"--name", container_name,
"--network", "kamal",
"--detach",
"--restart", "unless-stopped",
"--volume", "kamal-proxy-config:/home/kamal-proxy/.config/kamal-proxy",
"\$\(#{get_boot_options.join(" ")}\)",
config.proxy_image
end
def start
docker :container, :start, container_name
end
def stop(name: container_name)
docker :container, :stop, name
end
def start_or_run
combine start, run, by: "||"
end
def info
docker :ps, "--filter", "name=^#{container_name}$"
end
def version
pipe \
docker(:inspect, container_name, "--format '{{.Config.Image}}'"),
[ :cut, "-d:", "-f2" ]
end
def logs(timestamps: true, since: nil, lines: nil, grep: nil, grep_options: nil)
pipe \
docker(:logs, container_name, ("--since #{since}" if since), ("--tail #{lines}" if lines), ("--timestamps" if timestamps), "2>&1"),
("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep)
end
def follow_logs(host:, timestamps: true, grep: nil, grep_options: nil)
run_over_ssh pipe(
docker(:logs, container_name, ("--timestamps" if timestamps), "--tail", "10", "--follow", "2>&1"),
(%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep)
).join(" "), host: host
end
def remove_container
docker :container, :prune, "--force", "--filter", "label=org.opencontainers.image.title=kamal-proxy"
end
def remove_image
docker :image, :prune, "--all", "--force", "--filter", "label=org.opencontainers.image.title=kamal-proxy"
end
def cleanup_traefik
chain \
docker(:container, :stop, "traefik"),
combine(
docker(:container, :prune, "--force", "--filter", "label=org.opencontainers.image.title=Traefik"),
docker(:image, :prune, "--all", "--force", "--filter", "label=org.opencontainers.image.title=Traefik")
)
end
def ensure_proxy_directory
make_directory config.proxy_directory
end
def remove_proxy_directory
remove_directory config.proxy_directory
end
def get_boot_options
combine [ :cat, config.proxy_options_file ], [ :echo, "\"#{config.proxy_options_default.join(" ")}\"" ], by: "||"
end
def reset_boot_options
remove_file config.proxy_options_file
end
private
def container_name
config.proxy_container_name
end
end

View File

@@ -1,28 +1,28 @@
require "active_support/duration" require "active_support/duration"
require "active_support/core_ext/numeric/time" require "active_support/core_ext/numeric/time"
class Mrsk::Commands::Prune < Mrsk::Commands::Base class Kamal::Commands::Prune < Kamal::Commands::Base
def dangling_images def dangling_images
docker :image, :prune, "--force", "--filter", "label=service=#{config.service}", "--filter", "dangling=true" docker :image, :prune, "--force", "--filter", "label=service=#{config.service}"
end end
def tagged_images def tagged_images
pipe \ pipe \
docker(:image, :ls, *service_filter, "--format", "'{{.ID}} {{.Repository}}:{{.Tag}}'"), docker(:image, :ls, *service_filter, "--format", "'{{.ID}} {{.Repository}}:{{.Tag}}'"),
"grep -v -w \"#{active_image_list}\"", grep("-v -w \"#{active_image_list}\""),
"while read image tag; do docker rmi $tag; done" "while read image tag; do docker rmi $tag; done"
end end
def containers(keep_last: 5) def app_containers(retain:)
pipe \ pipe \
docker(:ps, "-q", "-a", *service_filter, *stopped_containers_filters), docker(:ps, "-q", "-a", *service_filter, *stopped_containers_filters),
"tail -n +#{keep_last + 1}", "tail -n +#{retain + 1}",
"while read container_id; do docker rm $container_id; done" "while read container_id; do docker rm $container_id; done"
end end
private private
def stopped_containers_filters def stopped_containers_filters
[ "created", "exited", "dead" ].flat_map { |status| ["--filter", "status=#{status}"] } [ "created", "exited", "dead" ].flat_map { |status| [ "--filter", "status=#{status}" ] }
end end
def active_image_list def active_image_list

View File

@@ -0,0 +1,16 @@
class Kamal::Commands::Registry < Kamal::Commands::Base
def login(registry_config: nil)
registry_config ||= config.registry
docker :login,
registry_config.server,
"-u", sensitive(Kamal::Utils.escape_shell_value(registry_config.username)),
"-p", sensitive(Kamal::Utils.escape_shell_value(registry_config.password))
end
def logout(registry_config: nil)
registry_config ||= config.registry
docker :logout, registry_config.server
end
end

View File

@@ -0,0 +1,15 @@
class Kamal::Commands::Server < Kamal::Commands::Base
def ensure_run_directory
make_directory config.run_directory
end
def remove_app_directory
remove_directory config.app_directory
end
def app_directory_count
pipe \
[ :ls, config.apps_directory ],
[ :wc, "-l" ]
end
end

410
lib/kamal/configuration.rb Normal file
View File

@@ -0,0 +1,410 @@
require "active_support/ordered_options"
require "active_support/core_ext/string/inquiry"
require "active_support/core_ext/module/delegation"
require "active_support/core_ext/hash/keys"
require "erb"
require "net/ssh/proxy/jump"
class Kamal::Configuration
delegate :service, :image, :labels, :hooks_path, to: :raw_config, allow_nil: true
delegate :argumentize, :optionize, to: Kamal::Utils
attr_reader :destination, :raw_config, :secrets
attr_reader :accessories, :aliases, :boot, :builder, :env, :logging, :proxy, :servers, :ssh, :sshkit, :registry
include Validation
PROXY_MINIMUM_VERSION = "v0.8.4"
PROXY_HTTP_PORT = 80
PROXY_HTTPS_PORT = 443
PROXY_LOG_MAX_SIZE = "10m"
class << self
def create_from(config_file:, destination: nil, version: nil)
ENV["KAMAL_DESTINATION"] = destination
raw_config = load_config_files(config_file, *destination_config_file(config_file, destination))
new raw_config, destination: destination, version: version
end
private
def load_config_files(*files)
files.inject({}) { |config, file| config.deep_merge! load_config_file(file) }
end
def load_config_file(file)
if file.exist?
# Newer Psych doesn't load aliases by default
load_method = YAML.respond_to?(:unsafe_load) ? :unsafe_load : :load
YAML.send(load_method, ERB.new(File.read(file)).result).symbolize_keys
else
raise "Configuration file not found in #{file}"
end
end
def destination_config_file(base_config_file, destination)
base_config_file.sub_ext(".#{destination}.yml") if destination
end
end
def initialize(raw_config, destination: nil, version: nil, validate: true)
@raw_config = ActiveSupport::InheritableOptions.new(raw_config)
@destination = destination
@declared_version = version
validate! raw_config, example: validation_yml.symbolize_keys, context: "", with: Kamal::Configuration::Validator::Configuration
@secrets = Kamal::Secrets.new(destination: destination)
# Eager load config to validate it, these are first as they have dependencies later on
@servers = Servers.new(config: self)
@registry = Registry.new(config: @raw_config, secrets: secrets)
@accessories = @raw_config.accessories&.keys&.collect { |name| Accessory.new(name, config: self) } || []
@aliases = @raw_config.aliases&.keys&.to_h { |name| [ name, Alias.new(name, config: self) ] } || {}
@boot = Boot.new(config: self)
@builder = Builder.new(config: self)
@env = Env.new(config: @raw_config.env || {}, secrets: secrets)
@logging = Logging.new(logging_config: @raw_config.logging)
@proxy = Proxy.new(config: self, proxy_config: @raw_config.proxy || {})
@ssh = Ssh.new(config: self)
@sshkit = Sshkit.new(config: self)
ensure_destination_if_required
ensure_required_keys_present
ensure_valid_kamal_version
ensure_retain_containers_valid
ensure_valid_service_name
ensure_no_traefik_reboot_hooks
ensure_one_host_for_ssl_roles
ensure_unique_hosts_for_ssl_roles
end
def version=(version)
@declared_version = version
end
def version
@declared_version.presence || ENV["VERSION"] || git_version
end
def abbreviated_version
if version
# Don't abbreviate <sha>_uncommitted_<etc>
if version.include?("_")
version
else
version[0...7]
end
end
end
def minimum_version
raw_config.minimum_version
end
def roles
servers.roles
end
def role(name)
roles.detect { |r| r.name == name.to_s }
end
def accessory(name)
accessories.detect { |a| a.name == name.to_s }
end
def all_hosts
(roles + accessories).flat_map(&:hosts).uniq
end
def primary_host
primary_role&.primary_host
end
def primary_role_name
raw_config.primary_role || "web"
end
def primary_role
role(primary_role_name)
end
def allow_empty_roles?
raw_config.allow_empty_roles
end
def proxy_roles
roles.select(&:running_proxy?)
end
def proxy_role_names
proxy_roles.flat_map(&:name)
end
def proxy_hosts
proxy_roles.flat_map(&:hosts).uniq
end
def repository
[ registry.server, image ].compact.join("/")
end
def absolute_image
"#{repository}:#{version}"
end
def latest_image
"#{repository}:#{latest_tag}"
end
def latest_tag
[ "latest", *destination ].join("-")
end
def service_with_version
"#{service}-#{version}"
end
def require_destination?
raw_config.require_destination
end
def retain_containers
raw_config.retain_containers || 5
end
def volume_args
if raw_config.volumes.present?
argumentize "--volume", raw_config.volumes
else
[]
end
end
def logging_args
logging.args
end
def readiness_delay
raw_config.readiness_delay || 7
end
def deploy_timeout
raw_config.deploy_timeout || 30
end
def drain_timeout
raw_config.drain_timeout || 30
end
def run_directory
".kamal"
end
def apps_directory
File.join run_directory, "apps"
end
def app_directory
File.join apps_directory, [ service, destination ].compact.join("-")
end
def env_directory
File.join app_directory, "env"
end
def assets_directory
File.join app_directory, "assets"
end
def hooks_path
raw_config.hooks_path || ".kamal/hooks"
end
def asset_path
raw_config.asset_path
end
def env_tags
@env_tags ||= if (tags = raw_config.env["tags"])
tags.collect { |name, config| Env::Tag.new(name, config: config, secrets: secrets) }
else
[]
end
end
def env_tag(name)
env_tags.detect { |t| t.name == name.to_s }
end
def proxy_publish_args(http_port, https_port, bind_ips = nil)
ensure_valid_bind_ips(bind_ips)
(bind_ips || [ nil ]).map do |bind_ip|
bind_ip = format_bind_ip(bind_ip)
publish_http = [ bind_ip, http_port, PROXY_HTTP_PORT ].compact.join(":")
publish_https = [ bind_ip, https_port, PROXY_HTTPS_PORT ].compact.join(":")
argumentize "--publish", [ publish_http, publish_https ]
end.join(" ")
end
def proxy_logging_args(max_size)
argumentize "--log-opt", "max-size=#{max_size}" if max_size.present?
end
def proxy_options_default
[ *proxy_publish_args(PROXY_HTTP_PORT, PROXY_HTTPS_PORT), *proxy_logging_args(PROXY_LOG_MAX_SIZE) ]
end
def proxy_image
"basecamp/kamal-proxy:#{PROXY_MINIMUM_VERSION}"
end
def proxy_container_name
"kamal-proxy"
end
def proxy_directory
File.join run_directory, "proxy"
end
def proxy_options_file
File.join proxy_directory, "options"
end
def to_h
{
roles: role_names,
hosts: all_hosts,
primary_host: primary_host,
version: version,
repository: repository,
absolute_image: absolute_image,
service_with_version: service_with_version,
volume_args: volume_args,
ssh_options: ssh.to_h,
sshkit: sshkit.to_h,
builder: builder.to_h,
accessories: raw_config.accessories,
logging: logging_args
}.compact
end
private
# Will raise ArgumentError if any required config keys are missing
def ensure_destination_if_required
if require_destination? && destination.nil?
raise ArgumentError, "You must specify a destination"
end
true
end
def ensure_required_keys_present
%i[ service image registry servers ].each do |key|
raise Kamal::ConfigurationError, "Missing required configuration for #{key}" unless raw_config[key].present?
end
unless role(primary_role_name).present?
raise Kamal::ConfigurationError, "The primary_role #{primary_role_name} isn't defined"
end
if primary_role.hosts.empty?
raise Kamal::ConfigurationError, "No servers specified for the #{primary_role.name} primary_role"
end
unless allow_empty_roles?
roles.each do |role|
if role.hosts.empty?
raise Kamal::ConfigurationError, "No servers specified for the #{role.name} role. You can ignore this with allow_empty_roles: true"
end
end
end
true
end
def ensure_valid_service_name
raise Kamal::ConfigurationError, "Service name can only include alphanumeric characters, hyphens, and underscores" unless raw_config[:service] =~ /^[a-z0-9_-]+$/i
true
end
def ensure_valid_kamal_version
if minimum_version && Gem::Version.new(minimum_version) > Gem::Version.new(Kamal::VERSION)
raise Kamal::ConfigurationError, "Current version is #{Kamal::VERSION}, minimum required is #{minimum_version}"
end
true
end
def ensure_valid_bind_ips(bind_ips)
bind_ips.present? && bind_ips.each do |ip|
next if ip =~ Resolv::IPv4::Regex || ip =~ Resolv::IPv6::Regex
raise ArgumentError, "Invalid publish IP address: #{ip}"
end
true
end
def ensure_retain_containers_valid
raise Kamal::ConfigurationError, "Must retain at least 1 container" if retain_containers < 1
true
end
def ensure_no_traefik_reboot_hooks
hooks = %w[ pre-traefik-reboot post-traefik-reboot ].select { |hook_file| File.exist?(File.join(hooks_path, hook_file)) }
if hooks.any?
raise Kamal::ConfigurationError, "Found #{hooks.join(", ")}, these should be renamed to (pre|post)-proxy-reboot"
end
true
end
def ensure_one_host_for_ssl_roles
roles.each(&:ensure_one_host_for_ssl)
true
end
def ensure_unique_hosts_for_ssl_roles
hosts = roles.select(&:ssl?).flat_map { |role| role.proxy.hosts }
duplicates = hosts.tally.filter_map { |host, count| host if count > 1 }
raise Kamal::ConfigurationError, "Different roles can't share the same host for SSL: #{duplicates.join(", ")}" if duplicates.any?
true
end
def format_bind_ip(ip)
# Ensure IPv6 address inside square brackets - e.g. [::1]
if ip =~ Resolv::IPv6::Regex && ip !~ /\[.*\]/
"[#{ip}]"
else
ip
end
end
def role_names
raw_config.servers.is_a?(Array) ? [ "web" ] : raw_config.servers.keys.sort
end
def git_version
@git_version ||=
if Kamal::Git.used?
if Kamal::Git.uncommitted_changes.present? && !builder.git_clone?
uncommitted_suffix = "_uncommitted_#{SecureRandom.hex(8)}"
end
[ Kamal::Git.revision, uncommitted_suffix ].compact.join
else
raise "Can't use commit hash as version, no git repository found in #{Dir.pwd}"
end
end
end

View File

@@ -0,0 +1,218 @@
class Kamal::Configuration::Accessory
include Kamal::Configuration::Validation
DEFAULT_NETWORK = "kamal"
delegate :argumentize, :optionize, to: Kamal::Utils
attr_reader :name, :env, :proxy, :registry
def initialize(name, config:)
@name, @config, @accessory_config = name.inquiry, config, config.raw_config["accessories"][name]
validate! \
accessory_config,
example: validation_yml["accessories"]["mysql"],
context: "accessories/#{name}",
with: Kamal::Configuration::Validator::Accessory
ensure_valid_roles
@env = initialize_env
@proxy = initialize_proxy if running_proxy?
@registry = initialize_registry if accessory_config["registry"].present?
end
def service_name
accessory_config["service"] || "#{config.service}-#{name}"
end
def image
[ registry&.server, accessory_config["image"] ].compact.join("/")
end
def hosts
hosts_from_host || hosts_from_hosts || hosts_from_roles
end
def port
if port = accessory_config["port"]&.to_s
port.include?(":") ? port : "#{port}:#{port}"
end
end
def network_args
argumentize "--network", network
end
def publish_args
argumentize "--publish", port if port
end
def labels
default_labels.merge(accessory_config["labels"] || {})
end
def label_args
argumentize "--label", labels
end
def env_args
[ *env.clear_args, *argumentize("--env-file", secrets_path) ]
end
def env_directory
File.join(config.env_directory, "accessories")
end
def secrets_io
env.secrets_io
end
def secrets_path
File.join(config.env_directory, "accessories", "#{name}.env")
end
def files
accessory_config["files"]&.to_h do |local_to_remote_mapping|
local_file, remote_file = local_to_remote_mapping.split(":")
[ expand_local_file(local_file), expand_remote_file(remote_file) ]
end || {}
end
def directories
accessory_config["directories"]&.to_h do |host_to_container_mapping|
host_path, container_path = host_to_container_mapping.split(":")
[ expand_host_path(host_path), container_path ]
end || {}
end
def volumes
specific_volumes + remote_files_as_volumes + remote_directories_as_volumes
end
def volume_args
argumentize "--volume", volumes
end
def option_args
if args = accessory_config["options"]
optionize args
else
[]
end
end
def cmd
accessory_config["cmd"]
end
def running_proxy?
accessory_config["proxy"].present?
end
private
attr_reader :config, :accessory_config
def initialize_env
Kamal::Configuration::Env.new \
config: accessory_config.fetch("env", {}),
secrets: config.secrets,
context: "accessories/#{name}/env"
end
def initialize_proxy
Kamal::Configuration::Proxy.new \
config: config,
proxy_config: accessory_config["proxy"],
context: "accessories/#{name}/proxy"
end
def initialize_registry
Kamal::Configuration::Registry.new \
config: accessory_config,
secrets: config.secrets,
context: "accessories/#{name}/registry"
end
def default_labels
{ "service" => service_name }
end
def expand_local_file(local_file)
if local_file.end_with?("erb")
with_clear_env_loaded { read_dynamic_file(local_file) }
else
Pathname.new(File.expand_path(local_file)).to_s
end
end
def with_clear_env_loaded
env.clear.each { |k, v| ENV[k] = v }
yield
ensure
env.clear.each { |k, v| ENV.delete(k) }
end
def read_dynamic_file(local_file)
StringIO.new(ERB.new(File.read(local_file)).result)
end
def expand_remote_file(remote_file)
service_name + remote_file
end
def specific_volumes
accessory_config["volumes"] || []
end
def remote_files_as_volumes
accessory_config["files"]&.collect do |local_to_remote_mapping|
_, remote_file = local_to_remote_mapping.split(":")
"#{service_data_directory + remote_file}:#{remote_file}"
end || []
end
def remote_directories_as_volumes
accessory_config["directories"]&.collect do |host_to_container_mapping|
host_path, container_path = host_to_container_mapping.split(":")
[ expand_host_path(host_path), container_path ].join(":")
end || []
end
def expand_host_path(host_path)
absolute_path?(host_path) ? host_path : File.join(service_data_directory, host_path)
end
def absolute_path?(path)
Pathname.new(path).absolute?
end
def service_data_directory
"$PWD/#{service_name}"
end
def hosts_from_host
[ accessory_config["host"] ] if accessory_config.key?("host")
end
def hosts_from_hosts
accessory_config["hosts"] if accessory_config.key?("hosts")
end
def hosts_from_roles
if accessory_config.key?("roles")
accessory_config["roles"].flat_map { |role| config.role(role)&.hosts }
end
end
def network
accessory_config["network"] || DEFAULT_NETWORK
end
def ensure_valid_roles
if accessory_config["roles"] && (missing_roles = accessory_config["roles"] - config.roles.map(&:name)).any?
raise Kamal::ConfigurationError, "accessories/#{name}: unknown roles #{missing_roles.join(", ")}"
end
end
end

View File

@@ -0,0 +1,15 @@
class Kamal::Configuration::Alias
include Kamal::Configuration::Validation
attr_reader :name, :command
def initialize(name, config:)
@name, @command = name.inquiry, config.raw_config["aliases"][name]
validate! \
command,
example: validation_yml["aliases"]["uname"],
context: "aliases/#{name}",
with: Kamal::Configuration::Validator::Alias
end
end

View File

@@ -0,0 +1,25 @@
class Kamal::Configuration::Boot
include Kamal::Configuration::Validation
attr_reader :boot_config, :host_count
def initialize(config:)
@boot_config = config.raw_config.boot || {}
@host_count = config.all_hosts.count
validate! boot_config
end
def limit
limit = boot_config["limit"]
if limit.to_s.end_with?("%")
[ host_count * limit.to_i / 100, 1 ].max
else
limit
end
end
def wait
boot_config["wait"]
end
end

View File

@@ -0,0 +1,199 @@
class Kamal::Configuration::Builder
include Kamal::Configuration::Validation
attr_reader :config, :builder_config
delegate :image, :service, to: :config
delegate :server, to: :"config.registry"
def initialize(config:)
@config = config
@builder_config = config.raw_config.builder || {}
@image = config.image
@server = config.registry.server
@service = config.service
validate! builder_config, with: Kamal::Configuration::Validator::Builder
end
def to_h
builder_config
end
def remote
builder_config["remote"]
end
def arches
Array(builder_config.fetch("arch", default_arch))
end
def local_arches
@local_arches ||= if local_disabled?
[]
elsif remote
arches & [ Kamal::Utils.docker_arch ]
else
arches
end
end
def remote_arches
@remote_arches ||= if remote
arches - local_arches
else
[]
end
end
def remote?
remote_arches.any?
end
def local?
!local_disabled? && (arches.empty? || local_arches.any?)
end
def cloud?
driver.start_with? "cloud"
end
def cached?
!!builder_config["cache"]
end
def args
builder_config["args"] || {}
end
def secrets
(builder_config["secrets"] || []).to_h { |key| [ key, config.secrets[key] ] }
end
def dockerfile
builder_config["dockerfile"] || "Dockerfile"
end
def target
builder_config["target"]
end
def context
builder_config["context"] || "."
end
def driver
builder_config.fetch("driver", "docker-container")
end
def local_disabled?
builder_config["local"] == false
end
def cache_from
if cached?
case builder_config["cache"]["type"]
when "gha"
cache_from_config_for_gha
when "registry"
cache_from_config_for_registry
end
end
end
def cache_to
if cached?
case builder_config["cache"]["type"]
when "gha"
cache_to_config_for_gha
when "registry"
cache_to_config_for_registry
end
end
end
def ssh
builder_config["ssh"]
end
def provenance
builder_config["provenance"]
end
def sbom
builder_config["sbom"]
end
def git_clone?
Kamal::Git.used? && builder_config["context"].nil?
end
def clone_directory
@clone_directory ||= File.join Dir.tmpdir, "kamal-clones", [ service, pwd_sha ].compact.join("-")
end
def build_directory
@build_directory ||=
if git_clone?
File.join clone_directory, repo_basename, repo_relative_pwd
else
"."
end
end
def docker_driver?
driver == "docker"
end
private
def valid?
if docker_driver?
raise ArgumentError, "Invalid builder configuration: the `docker` driver does not not support remote builders" if remote
raise ArgumentError, "Invalid builder configuration: the `docker` driver does not not support caching" if cached?
raise ArgumentError, "Invalid builder configuration: the `docker` driver does not not support multiple arches" if arches.many?
end
if @options["cache"] && @options["cache"]["type"]
raise ArgumentError, "Invalid cache type: #{@options["cache"]["type"]}" unless [ "gha", "registry" ].include?(@options["cache"]["type"])
end
end
def cache_image
builder_config["cache"]&.fetch("image", nil) || "#{image}-build-cache"
end
def cache_image_ref
[ server, cache_image ].compact.join("/")
end
def cache_from_config_for_gha
"type=gha"
end
def cache_from_config_for_registry
[ "type=registry", "ref=#{cache_image_ref}" ].compact.join(",")
end
def cache_to_config_for_gha
[ "type=gha", builder_config["cache"]&.fetch("options", nil) ].compact.join(",")
end
def cache_to_config_for_registry
[ "type=registry", "ref=#{cache_image_ref}", builder_config["cache"]&.fetch("options", nil) ].compact.join(",")
end
def repo_basename
File.basename(Kamal::Git.root)
end
def repo_relative_pwd
Dir.pwd.delete_prefix(Kamal::Git.root)
end
def pwd_sha
Digest::SHA256.hexdigest(Dir.pwd)[0..12]
end
def default_arch
docker_driver? ? [] : [ "amd64", "arm64" ]
end
end

View File

@@ -0,0 +1,123 @@
# Accessories
#
# Accessories can be booted on a single host, a list of hosts, or on specific roles.
# The hosts do not need to be defined in the Kamal servers configuration.
#
# Accessories are managed separately from the main service — they are not updated
# when you deploy, and they do not have zero-downtime deployments.
#
# Run `kamal accessory boot <accessory>` to boot an accessory.
# See `kamal accessory --help` for more information.
# Configuring accessories
#
# First, define the accessory in the `accessories`:
accessories:
mysql:
# Service name
#
# This is used in the service label and defaults to `<service>-<accessory>`,
# where `<service>` is the main service name from the root configuration:
service: mysql
# Image
#
# The Docker image to use.
# Prefix it with its server when using root level registry different from Docker Hub.
# Define registry directly or via anchors when it differs from root level registry.
image: mysql:8.0
# Registry
#
# By default accessories use Docker Hub registry.
# You can specify different registry per accessory with this option.
# Don't prefix image with this registry server.
# Use anchors if you need to set the same specific registry for several accessories.
#
# ```yml
# registry:
# <<: *specific-registry
# ```
#
# See kamal docs registry for more information:
registry:
...
# Accessory hosts
#
# Specify one of `host`, `hosts`, or `roles`:
host: mysql-db1
hosts:
- mysql-db1
- mysql-db2
roles:
- mysql
# Custom command
#
# You can set a custom command to run in the container if you do not want to use the default:
cmd: "bin/mysqld"
# Port mappings
#
# See [https://docs.docker.com/network/](https://docs.docker.com/network/), and
# especially note the warning about the security implications of exposing ports publicly.
port: "127.0.0.1:3306:3306"
# Labels
labels:
app: myapp
# Options
#
# These are passed to the Docker run command in the form `--<name> <value>`:
options:
restart: always
cpus: 2
# Environment variables
#
# See kamal docs env for more information:
env:
...
# Copying files
#
# You can specify files to mount into the container.
# The format is `local:remote`, where `local` is the path to the file on the local machine
# and `remote` is the path to the file in the container.
#
# They will be uploaded from the local repo to the host and then mounted.
#
# ERB files will be evaluated before being copied.
files:
- config/my.cnf.erb:/etc/mysql/my.cnf
- config/myoptions.cnf:/etc/mysql/myoptions.cnf
# Directories
#
# You can specify directories to mount into the container. They will be created on the host
# before being mounted:
directories:
- mysql-logs:/var/log/mysql
# Volumes
#
# Any other volumes to mount, in addition to the files and directories.
# They are not created or copied before mounting:
volumes:
- /path/to/mysql-logs:/var/log/mysql
# Network
#
# The network the accessory will be attached to.
#
# Defaults to kamal:
network: custom
# Proxy
#
# You can run your accessory behind the Kamal proxy. See kamal docs proxy for more information
proxy:
...

View File

@@ -0,0 +1,26 @@
# Aliases
#
# Aliases are shortcuts for Kamal commands.
#
# For example, for a Rails app, you might open a console with:
#
# ```shell
# kamal app exec -i --reuse "bin/rails console"
# ```
#
# By defining an alias, like this:
aliases:
console: app exec -i --reuse "bin/rails console"
# You can now open the console with:
#
# ```shell
# kamal console
# ```
# Configuring aliases
#
# Aliases are defined in the root config under the alias key.
#
# Each alias is named and can only contain lowercase letters, numbers, dashes, and underscores:
aliases:
uname: app exec -p -q -r web "uname -a"

View File

@@ -0,0 +1,19 @@
# Booting
#
# When deploying to large numbers of hosts, you might prefer not to restart your services on every host at the same time.
#
# Kamals default is to boot new containers on all hosts in parallel. However, you can control this with the boot configuration.
# Fixed group sizes
#
# Here, we boot 2 hosts at a time with a 10-second gap between each group:
boot:
limit: 2
wait: 10
# Percentage of hosts
#
# Here, we boot 25% of the hosts at a time with a 2-second gap between each group:
boot:
limit: 25%
wait: 2

View File

@@ -0,0 +1,119 @@
# Builder
#
# The builder configuration controls how the application is built with `docker build`.
#
# See https://kamal-deploy.org/docs/configuration/builder-examples/ for more information.
# Builder options
#
# Options go under the builder key in the root configuration.
builder:
# Arch
#
# The architectures to build for — you can set an array or just a single value.
#
# Allowed values are `amd64` and `arm64`:
arch:
- amd64
# Remote
#
# The connection string for a remote builder. If supplied, Kamal will use this
# for builds that do not match the local architecture of the deployment host.
remote: ssh://docker@docker-builder
# Local
#
# If set to false, Kamal will always use the remote builder even when building
# the local architecture.
#
# Defaults to true:
local: true
# Builder cache
#
# The type must be either 'gha' or 'registry'.
#
# The image is only used for registry cache and is not compatible with the Docker driver:
cache:
type: registry
options: mode=max
image: kamal-app-build-cache
# Build context
#
# If this is not set, then a local Git clone of the repo is used.
# This ensures a clean build with no uncommitted changes.
#
# To use the local checkout instead, you can set the context to `.`, or a path to another directory.
context: .
# Dockerfile
#
# The Dockerfile to use for building, defaults to `Dockerfile`:
dockerfile: Dockerfile.production
# Build target
#
# If not set, then the default target is used:
target: production
# Build arguments
#
# Any additional build arguments, passed to `docker build` with `--build-arg <key>=<value>`:
args:
ENVIRONMENT: production
# Referencing build arguments
#
# ```shell
# ARG RUBY_VERSION
# FROM ruby:$RUBY_VERSION-slim as base
# ```
# Build secrets
#
# Values are read from `.kamal/secrets`:
secrets:
- SECRET1
- SECRET2
# Referencing build secrets
#
# ```shell
# # Copy Gemfiles
# COPY Gemfile Gemfile.lock ./
#
# # Install dependencies, including private repositories via access token
# # Then remove bundle cache with exposed GITHUB_TOKEN
# RUN --mount=type=secret,id=GITHUB_TOKEN \
# BUNDLE_GITHUB__COM=x-access-token:$(cat /run/secrets/GITHUB_TOKEN) \
# bundle install && \
# rm -rf /usr/local/bundle/cache
# ```
# SSH
#
# SSH agent socket or keys to expose to the build:
ssh: default=$SSH_AUTH_SOCK
# Driver
#
# The build driver to use, defaults to `docker-container`:
driver: docker
#
# If you want to use Docker Build Cloud (https://www.docker.com/products/build-cloud/), you can set the driver to:
driver: cloud org-name/builder-name
# Provenance
#
# It is used to configure provenance attestations for the build result.
# The value can also be a boolean to enable or disable provenance attestations.
provenance: mode=max
# SBOM (Software Bill of Materials)
#
# It is used to configure SBOM generation for the build result.
# The value can also be a boolean to enable or disable SBOM generation.
sbom: true

View File

@@ -0,0 +1,178 @@
# Kamal Configuration
#
# Configuration is read from the `config/deploy.yml`.
# Destinations
#
# When running commands, you can specify a destination with the `-d` flag,
# e.g., `kamal deploy -d staging`.
#
# In this case, the configuration will also be read from `config/deploy.staging.yml`
# and merged with the base configuration.
# Extensions
#
# Kamal will not accept unrecognized keys in the configuration file.
#
# However, you might want to declare a configuration block using YAML anchors
# and aliases to avoid repetition.
#
# You can prefix a configuration section with `x-` to indicate that it is an
# extension. Kamal will ignore the extension and not raise an error.
# The service name
#
# This is a required value. It is used as the container name prefix.
service: myapp
# The Docker image name
#
# The image will be pushed to the configured registry.
image: my-image
# Labels
#
# Additional labels to add to the container:
labels:
my-label: my-value
# Volumes
#
# Additional volumes to mount into the container:
volumes:
- /path/on/host:/path/in/container:ro
# Registry
#
# The Docker registry configuration, see kamal docs registry:
registry:
...
# Servers
#
# The servers to deploy to, optionally with custom roles, see kamal docs servers:
servers:
...
# Environment variables
#
# See kamal docs env:
env:
...
# Asset path
#
# Used for asset bridging across deployments, default to `nil`.
#
# If there are changes to CSS or JS files, we may get requests
# for the old versions on the new container, and vice versa.
#
# To avoid 404s, we can specify an asset path.
# Kamal will replace that path in the container with a mapped
# volume containing both sets of files.
# This requires that file names change when the contents change
# (e.g., by including a hash of the contents in the name).
#
# To configure this, set the path to the assets:
asset_path: /path/to/assets
# Hooks path
#
# Path to hooks, defaults to `.kamal/hooks`.
# See https://kamal-deploy.org/docs/hooks for more information:
hooks_path: /user_home/kamal/hooks
# Require destinations
#
# Whether deployments require a destination to be specified, defaults to `false`:
require_destination: true
# Primary role
#
# This defaults to `web`, but if you have no web role, you can change this:
primary_role: workers
# Allowing empty roles
#
# Whether roles with no servers are allowed. Defaults to `false`:
allow_empty_roles: false
# Retain containers
#
# How many old containers and images we retain, defaults to 5:
retain_containers: 3
# Minimum version
#
# The minimum version of Kamal required to deploy this configuration, defaults to `nil`:
minimum_version: 1.3.0
# Readiness delay
#
# Seconds to wait for a container to boot after it is running, default 7.
#
# This only applies to containers that do not run a proxy or specify a healthcheck:
readiness_delay: 4
# Deploy timeout
#
# How long to wait for a container to become ready, default 30:
deploy_timeout: 10
# Drain timeout
#
# How long to wait for a container to drain, default 30:
drain_timeout: 10
# Run directory
#
# Directory to store kamal runtime files in on the host, default `.kamal`:
run_directory: /etc/kamal
# SSH options
#
# See kamal docs ssh:
ssh:
...
# Builder options
#
# See kamal docs builder:
builder:
...
# Accessories
#
# Additional services to run in Docker, see kamal docs accessory:
accessories:
...
# Proxy
#
# Configuration for kamal-proxy, see kamal docs proxy:
proxy:
...
# SSHKit
#
# See kamal docs sshkit:
sshkit:
...
# Boot options
#
# See kamal docs boot:
boot:
...
# Logging
#
# Docker logging configuration, see kamal docs logging:
logging:
...
# Aliases
#
# Alias configuration, see kamal docs alias:
aliases:
...

View File

@@ -0,0 +1,85 @@
# Environment variables
#
# Environment variables can be set directly in the Kamal configuration or
# read from `.kamal/secrets`.
# Reading environment variables from the configuration
#
# Environment variables can be set directly in the configuration file.
#
# These are passed to the `docker run` command when deploying.
env:
DATABASE_HOST: mysql-db1
DATABASE_PORT: 3306
# Secrets
#
# Kamal uses dotenv to automatically load environment variables set in the `.kamal/secrets` file.
#
# If you are using destinations, secrets will instead be read from `.kamal/secrets.<DESTINATION>` if
# it exists.
#
# Common secrets across all destinations can be set in `.kamal/secrets-common`.
#
# This file can be used to set variables like `KAMAL_REGISTRY_PASSWORD` or database passwords.
# You can use variable or command substitution in the secrets file.
#
# ```shell
# KAMAL_REGISTRY_PASSWORD=$KAMAL_REGISTRY_PASSWORD
# RAILS_MASTER_KEY=$(cat config/master.key)
# ```
#
# You can also use [secret helpers](../../commands/secrets) for some common password managers.
#
# ```shell
# SECRETS=$(kamal secrets fetch ...)
#
# REGISTRY_PASSWORD=$(kamal secrets extract REGISTRY_PASSWORD $SECRETS)
# DB_PASSWORD=$(kamal secrets extract DB_PASSWORD $SECRETS)
# ```
#
# If you store secrets directly in `.kamal/secrets`, ensure that it is not checked into version control.
#
# To pass the secrets, you should list them under the `secret` key. When you do this, the
# other variables need to be moved under the `clear` key.
#
# Unlike clear values, secrets are not passed directly to the container
# but are stored in an env file on the host:
env:
clear:
DB_USER: app
secret:
- DB_PASSWORD
# Tags
#
# Tags are used to add extra env variables to specific hosts.
# See kamal docs servers for how to tag hosts.
#
# Tags are only allowed in the top-level env configuration (i.e., not under a role-specific env).
#
# The env variables can be specified with secret and clear values as explained above.
env:
tags:
<tag1>:
MYSQL_USER: monitoring
<tag2>:
clear:
MYSQL_USER: readonly
secret:
- MYSQL_PASSWORD
# Example configuration
env:
clear:
MYSQL_USER: app
secret:
- MYSQL_PASSWORD
tags:
monitoring:
MYSQL_USER: monitoring
replica:
clear:
MYSQL_USER: readonly
secret:
- READONLY_PASSWORD

View File

@@ -0,0 +1,21 @@
# Custom logging configuration
#
# Set these to control the Docker logging driver and options.
# Logging settings
#
# These go under the logging key in the configuration file.
#
# This can be specified at the root level or for a specific role.
logging:
# Driver
#
# The logging driver to use, passed to Docker via `--log-driver`:
driver: json-file
# Options
#
# Any logging options to pass to the driver, passed to Docker via `--log-opt`:
options:
max-size: 100m

View File

@@ -0,0 +1,108 @@
# Proxy
#
# Kamal uses [kamal-proxy](https://github.com/basecamp/kamal-proxy) to provide
# gapless deployments. It runs on ports 80 and 443 and forwards requests to the
# application container.
#
# The proxy is configured in the root configuration under `proxy`. These are
# options that are set when deploying the application, not when booting the proxy.
#
# They are application-specific, so they are not shared when multiple applications
# run on the same proxy.
#
# The proxy is enabled by default on the primary role but can be disabled by
# setting `proxy: false`.
#
# It is disabled by default on all other roles but can be enabled by setting
# `proxy: true` or providing a proxy configuration.
proxy:
# Hosts
#
# The hosts that will be used to serve the app. The proxy will only route requests
# to this host to your app.
#
# If no hosts are set, then all requests will be forwarded, except for matching
# requests for other apps deployed on that server that do have a host set.
#
# Specify one of `host` or `hosts`.
host: foo.example.com
hosts:
- foo.example.com
- bar.example.com
# App port
#
# The port the application container is exposed on.
#
# Defaults to 80:
app_port: 3000
# SSL
#
# kamal-proxy can provide automatic HTTPS for your application via Let's Encrypt.
#
# This requires that we are deploying to one server and the host option is set.
# The host value must point to the server we are deploying to, and port 443 must be
# open for the Let's Encrypt challenge to succeed.
#
# If you set `ssl` to `true`, `kamal-proxy` will stop forwarding headers to your app,
# unless you explicitly set `forward_headers: true`
#
# Defaults to `false`:
ssl: true
# Forward headers
#
# Whether to forward the `X-Forwarded-For` and `X-Forwarded-Proto` headers.
#
# If you are behind a trusted proxy, you can set this to `true` to forward the headers.
#
# By default, kamal-proxy will not forward the headers if the `ssl` option is set to `true`, and
# will forward them if it is set to `false`.
forward_headers: true
# Response timeout
#
# How long to wait for requests to complete before timing out, defaults to 30 seconds:
response_timeout: 10
# Healthcheck
#
# When deploying, the proxy will by default hit `/up` once every second until we hit
# the deploy timeout, with a 5-second timeout for each request.
#
# Once the app is up, the proxy will stop hitting the healthcheck endpoint.
healthcheck:
interval: 3
path: /health
timeout: 3
# Buffering
#
# Whether to buffer request and response bodies in the proxy.
#
# By default, buffering is enabled with a max request body size of 1GB and no limit
# for response size.
#
# You can also set the memory limit for buffering, which defaults to 1MB; anything
# larger than that is written to disk.
buffering:
requests: true
responses: true
max_request_body: 40_000_000
max_response_body: 0
memory: 2_000_000
# Logging
#
# Configure request logging for the proxy.
# You can specify request and response headers to log.
# By default, `Cache-Control`, `Last-Modified`, and `User-Agent` request headers are logged:
logging:
request_headers:
- Cache-Control
- X-Forwarded-Proto
response_headers:
- X-Request-ID
- X-Request-Start

View File

@@ -0,0 +1,56 @@
# Registry
#
# The default registry is Docker Hub, but you can change it using `registry/server`.
#
# By default, Docker Hub creates public repositories. To avoid making your images public,
# set up a private repository before deploying, or change the default repository privacy
# settings to private in your [Docker Hub settings](https://hub.docker.com/repository-settings/default-privacy).
#
# A reference to a secret (in this case, `DOCKER_REGISTRY_TOKEN`) will look up the secret
# in the local environment:
registry:
server: registry.digitalocean.com
username:
- DOCKER_REGISTRY_TOKEN
password:
- DOCKER_REGISTRY_TOKEN
# Using AWS ECR as the container registry
#
# You will need to have the AWS CLI installed locally for this to work.
# AWS ECRs access token is only valid for 12 hours. In order to avoid having to manually regenerate the token every time, you can use ERB in the `deploy.yml` file to shell out to the AWS CLI command and obtain the token:
registry:
server: <your aws account id>.dkr.ecr.<your aws region id>.amazonaws.com
username: AWS
password: <%= %x(aws ecr get-login-password) %>
# Using GCP Artifact Registry as the container registry
#
# To sign into Artifact Registry, you need to
# [create a service account](https://cloud.google.com/iam/docs/service-accounts-create#creating)
# and [set up roles and permissions](https://cloud.google.com/artifact-registry/docs/access-control#permissions).
# Normally, assigning the `roles/artifactregistry.writer` role should be sufficient.
#
# Once the service account is ready, you need to generate and download a JSON key and base64 encode it:
#
# ```shell
# base64 -i /path/to/key.json | tr -d "\\n"
# ```
#
# You'll then need to set the `KAMAL_REGISTRY_PASSWORD` secret to that value.
#
# Use the environment variable as the password along with `_json_key_base64` as the username.
# Heres the final configuration:
registry:
server: <your registry region>-docker.pkg.dev
username: _json_key_base64
password:
- KAMAL_REGISTRY_PASSWORD
# Validating the configuration
#
# You can validate the configuration by running:
#
# ```shell
# kamal registry login
# ```

View File

@@ -0,0 +1,53 @@
# Roles
#
# Roles are used to configure different types of servers in the deployment.
# The most common use for this is to run web servers and job servers.
#
# Kamal expects there to be a `web` role, unless you set a different `primary_role`
# in the root configuration.
# Role configuration
#
# Roles are specified under the servers key:
servers:
# Simple role configuration
#
# This can be a list of hosts if you don't need custom configuration for the role.
#
# You can set tags on the hosts for custom env variables (see kamal docs env):
web:
- 172.1.0.1
- 172.1.0.2: experiment1
- 172.1.0.2: [ experiment1, experiment2 ]
# Custom role configuration
#
# When there are other options to set, the list of hosts goes under the `hosts` key.
#
# By default, only the primary role uses a proxy.
#
# For other roles, you can set it to `proxy: true` to enable it and inherit the root proxy
# configuration or provide a map of options to override the root configuration.
#
# For the primary role, you can set `proxy: false` to disable the proxy.
#
# You can also set a custom `cmd` to run in the container and overwrite other settings
# from the root configuration.
workers:
hosts:
- 172.1.0.3
- 172.1.0.4: experiment1
cmd: "bin/jobs"
options:
memory: 2g
cpus: 4
logging:
...
proxy:
...
labels:
my-label: workers
env:
...
asset_path: /public

View File

@@ -0,0 +1,27 @@
# Servers
#
# Servers are split into different roles, with each role having its own configuration.
#
# For simpler deployments, though, where all servers are identical, you can just specify a list of servers.
# They will be implicitly assigned to the `web` role.
servers:
- 172.0.0.1
- 172.0.0.2
- 172.0.0.3
# Tagging servers
#
# Servers can be tagged, with the tags used to add custom env variables (see kamal docs env).
servers:
- 172.0.0.1
- 172.0.0.2: experiments
- 172.0.0.3: [ experiments, three ]
# Roles
#
# For more complex deployments (e.g., if you are running job hosts), you can specify roles and configure each separately (see kamal docs role):
servers:
web:
...
workers:
...

View File

@@ -0,0 +1,70 @@
# SSH configuration
#
# Kamal uses SSH to connect and run commands on your hosts.
# By default, it will attempt to connect to the root user on port 22.
#
# If you are using a non-root user, you may need to bootstrap your servers manually before using them with Kamal. On Ubuntu, youd do:
#
# ```shell
# sudo apt update
# sudo apt upgrade -y
# sudo apt install -y docker.io curl git
# sudo usermod -a -G docker app
# ```
# SSH options
#
# The options are specified under the ssh key in the configuration file.
ssh:
# The SSH user
#
# Defaults to `root`:
user: app
# The SSH port
#
# Defaults to 22:
port: "2222"
# Proxy host
#
# Specified in the form <host> or <user>@<host>:
proxy: root@proxy-host
# Proxy command
#
# A custom proxy command, required for older versions of SSH:
proxy_command: "ssh -W %h:%p user@proxy"
# Log level
#
# Defaults to `fatal`. Set this to `debug` if you are having SSH connection issues.
log_level: debug
# Keys only
#
# Set to `true` to use only private keys from the `keys` and `key_data` parameters,
# even if ssh-agent offers more identities. This option is intended for
# situations where ssh-agent offers many different identities or you
# need to overwrite all identities and force a single one.
keys_only: false
# Keys
#
# An array of file names of private keys to use for public key
# and host-based authentication:
keys: [ "~/.ssh/id.pem" ]
# Key data
#
# An array of strings, with each element of the array being
# a raw private key in PEM format.
key_data: [ "-----BEGIN OPENSSH PRIVATE KEY-----" ]
# Config
#
# Set to true to load the default OpenSSH config files (~/.ssh/config,
# /etc/ssh_config), to false ignore config files, or to a file path
# (or array of paths) to load specific configuration. Defaults to true.
config: true

View File

@@ -0,0 +1,23 @@
# SSHKit
#
# [SSHKit](https://github.com/capistrano/sshkit) is the SSH toolkit used by Kamal.
#
# The default, settings should be sufficient for most use cases, but
# when connecting to a large number of hosts, you may need to adjust.
# SSHKit options
#
# The options are specified under the sshkit key in the configuration file.
sshkit:
# Max concurrent starts
#
# Creating SSH connections concurrently can be an issue when deploying to many servers.
# By default, Kamal will limit concurrent connection starts to 30 at a time.
max_concurrent_starts: 10
# Pool idle timeout
#
# Kamal sets a long idle timeout of 900 seconds on connections to try to avoid
# re-connection storms after an idle period, such as building an image or waiting for CI.
pool_idle_timeout: 300

View File

@@ -0,0 +1,29 @@
class Kamal::Configuration::Env
include Kamal::Configuration::Validation
attr_reader :context, :secrets
attr_reader :clear, :secret_keys
delegate :argumentize, to: Kamal::Utils
def initialize(config:, secrets:, context: "env")
@clear = config.fetch("clear", config.key?("secret") || config.key?("tags") ? {} : config)
@secrets = secrets
@secret_keys = config.fetch("secret", [])
@context = context
validate! config, context: context, with: Kamal::Configuration::Validator::Env
end
def clear_args
argumentize("--env", clear)
end
def secrets_io
Kamal::EnvFile.new(secret_keys.to_h { |key| [ key, secrets[key] ] }).to_io
end
def merge(other)
self.class.new \
config: { "clear" => clear.merge(other.clear), "secret" => secret_keys | other.secret_keys },
secrets: secrets
end
end

13
lib/kamal/configuration/env/tag.rb vendored Normal file
View File

@@ -0,0 +1,13 @@
class Kamal::Configuration::Env::Tag
attr_reader :name, :config, :secrets
def initialize(name, config:, secrets:)
@name = name
@config = config
@secrets = secrets
end
def env
Kamal::Configuration::Env.new(config: config, secrets: secrets)
end
end

View File

@@ -0,0 +1,33 @@
class Kamal::Configuration::Logging
delegate :optionize, :argumentize, to: Kamal::Utils
include Kamal::Configuration::Validation
attr_reader :logging_config
def initialize(logging_config:, context: "logging")
@logging_config = logging_config || {}
validate! @logging_config, context: context
end
def driver
logging_config["driver"]
end
def options
logging_config.fetch("options", {})
end
def merge(other)
self.class.new logging_config: logging_config.deep_merge(other.logging_config)
end
def args
if driver.present? || options.present?
optionize({ "log-driver" => driver }.compact) +
argumentize("--log-opt", options)
else
argumentize("--log-opt", { "max-size" => "10m" })
end
end
end

View File

@@ -0,0 +1,62 @@
class Kamal::Configuration::Proxy
include Kamal::Configuration::Validation
DEFAULT_LOG_REQUEST_HEADERS = [ "Cache-Control", "Last-Modified", "User-Agent" ]
CONTAINER_NAME = "kamal-proxy"
delegate :argumentize, :optionize, to: Kamal::Utils
attr_reader :config, :proxy_config
def initialize(config:, proxy_config:, context: "proxy")
@config = config
@proxy_config = proxy_config
validate! @proxy_config, with: Kamal::Configuration::Validator::Proxy, context: context
end
def app_port
proxy_config.fetch("app_port", 80)
end
def ssl?
proxy_config.fetch("ssl", false)
end
def hosts
proxy_config["hosts"] || proxy_config["host"]&.split(",") || []
end
def deploy_options
{
host: hosts,
tls: proxy_config["ssl"].presence,
"deploy-timeout": seconds_duration(config.deploy_timeout),
"drain-timeout": seconds_duration(config.drain_timeout),
"health-check-interval": seconds_duration(proxy_config.dig("healthcheck", "interval")),
"health-check-timeout": seconds_duration(proxy_config.dig("healthcheck", "timeout")),
"health-check-path": proxy_config.dig("healthcheck", "path"),
"target-timeout": seconds_duration(proxy_config["response_timeout"]),
"buffer-requests": proxy_config.fetch("buffering", { "requests": true }).fetch("requests", true),
"buffer-responses": proxy_config.fetch("buffering", { "responses": true }).fetch("responses", true),
"buffer-memory": proxy_config.dig("buffering", "memory"),
"max-request-body": proxy_config.dig("buffering", "max_request_body"),
"max-response-body": proxy_config.dig("buffering", "max_response_body"),
"forward-headers": proxy_config.dig("forward_headers"),
"log-request-header": proxy_config.dig("logging", "request_headers") || DEFAULT_LOG_REQUEST_HEADERS,
"log-response-header": proxy_config.dig("logging", "response_headers")
}.compact
end
def deploy_command_args(target:)
optionize ({ target: "#{target}:#{app_port}" }).merge(deploy_options), with: "="
end
def merge(other)
self.class.new config: config, proxy_config: proxy_config.deep_merge(other.proxy_config)
end
private
def seconds_duration(value)
value ? "#{value}s" : nil
end
end

View File

@@ -0,0 +1,32 @@
class Kamal::Configuration::Registry
include Kamal::Configuration::Validation
def initialize(config:, secrets:, context: "registry")
@registry_config = config["registry"] || {}
@secrets = secrets
validate! registry_config, context: context, with: Kamal::Configuration::Validator::Registry
end
def server
registry_config["server"]
end
def username
lookup("username")
end
def password
lookup("password")
end
private
attr_reader :registry_config, :secrets
def lookup(key)
if registry_config[key].is_a?(Array)
secrets[registry_config[key].first]
else
registry_config[key]
end
end
end

View File

@@ -0,0 +1,220 @@
class Kamal::Configuration::Role
include Kamal::Configuration::Validation
delegate :argumentize, :optionize, to: Kamal::Utils
attr_reader :name, :config, :specialized_env, :specialized_logging, :specialized_proxy
alias to_s name
def initialize(name, config:)
@name, @config = name.inquiry, config
validate! \
role_config,
example: validation_yml["servers"]["workers"],
context: "servers/#{name}",
with: Kamal::Configuration::Validator::Role
@specialized_env = Kamal::Configuration::Env.new \
config: specializations.fetch("env", {}),
secrets: config.secrets,
context: "servers/#{name}/env"
@specialized_logging = Kamal::Configuration::Logging.new \
logging_config: specializations.fetch("logging", {}),
context: "servers/#{name}/logging"
initialize_specialized_proxy
end
def primary_host
hosts.first
end
def hosts
tagged_hosts.keys
end
def env_tags(host)
tagged_hosts.fetch(host).collect { |tag| config.env_tag(tag) }
end
def cmd
specializations["cmd"]
end
def option_args
if args = specializations["options"]
optionize args
else
[]
end
end
def labels
default_labels.merge(custom_labels)
end
def label_args
argumentize "--label", labels
end
def logging_args
logging.args
end
def logging
@logging ||= config.logging.merge(specialized_logging)
end
def proxy
@proxy ||= config.proxy.merge(specialized_proxy) if running_proxy?
end
def running_proxy?
@running_proxy
end
def ssl?
running_proxy? && proxy.ssl?
end
def stop_args
# When deploying with the proxy, kamal-proxy will drain request before returning so we don't need to wait.
timeout = running_proxy? ? nil : config.drain_timeout
[ *argumentize("-t", timeout) ]
end
def env(host)
@envs ||= {}
@envs[host] ||= [ config.env, specialized_env, *env_tags(host).map(&:env) ].reduce(:merge)
end
def env_args(host)
[ *env(host).clear_args, *argumentize("--env-file", secrets_path) ]
end
def env_directory
File.join(config.env_directory, "roles")
end
def secrets_io(host)
env(host).secrets_io
end
def secrets_path
File.join(config.env_directory, "roles", "#{name}.env")
end
def asset_volume_args
asset_volume&.docker_args
end
def primary?
name == @config.primary_role_name
end
def container_name(version = nil)
[ container_prefix, version || config.version ].compact.join("-")
end
def container_prefix
[ config.service, name, config.destination ].compact.join("-")
end
def asset_path
specializations["asset_path"] || config.asset_path
end
def assets?
asset_path.present? && running_proxy?
end
def asset_volume(version = config.version)
if assets?
Kamal::Configuration::Volume.new \
host_path: asset_volume_directory(version), container_path: asset_path
end
end
def asset_extracted_directory(version = config.version)
File.join config.assets_directory, "extracted", [ name, version ].join("-")
end
def asset_volume_directory(version = config.version)
File.join config.assets_directory, "volumes", [ name, version ].join("-")
end
def ensure_one_host_for_ssl
if running_proxy? && proxy.ssl? && hosts.size > 1
raise Kamal::ConfigurationError, "SSL is only supported on a single server, found #{hosts.size} servers for role #{name}"
end
end
private
def initialize_specialized_proxy
proxy_specializations = specializations["proxy"]
if primary?
# only false means no proxy for non-primary roles
@running_proxy = proxy_specializations != false
else
# false and nil both mean no proxy for non-primary roles
@running_proxy = !!proxy_specializations
end
if running_proxy?
proxy_config = proxy_specializations == true || proxy_specializations.nil? ? {} : proxy_specializations
@specialized_proxy = Kamal::Configuration::Proxy.new \
config: config,
proxy_config: proxy_config,
context: "servers/#{name}/proxy"
end
end
def tagged_hosts
{}.tap do |tagged_hosts|
extract_hosts_from_config.map do |host_config|
if host_config.is_a?(Hash)
host, tags = host_config.first
tagged_hosts[host] = Array(tags)
elsif host_config.is_a?(String)
tagged_hosts[host_config] = []
end
end
end
end
def extract_hosts_from_config
if config.raw_config.servers.is_a?(Array)
config.raw_config.servers
else
servers = config.raw_config.servers[name]
servers.is_a?(Array) ? servers : Array(servers["hosts"])
end
end
def default_labels
{ "service" => config.service, "role" => name, "destination" => config.destination }
end
def specializations
@specializations ||= role_config.is_a?(Array) ? {} : role_config
end
def role_config
@role_config ||= config.raw_config.servers.is_a?(Array) ? {} : config.raw_config.servers[name]
end
def custom_labels
Hash.new.tap do |labels|
labels.merge!(config.labels) if config.labels.present?
labels.merge!(specializations["labels"]) if specializations["labels"].present?
end
end
end

View File

@@ -0,0 +1,18 @@
class Kamal::Configuration::Servers
include Kamal::Configuration::Validation
attr_reader :config, :servers_config, :roles
def initialize(config:)
@config = config
@servers_config = config.raw_config.servers
validate! servers_config, with: Kamal::Configuration::Validator::Servers
@roles = role_names.map { |role_name| Kamal::Configuration::Role.new role_name, config: config }
end
private
def role_names
servers_config.is_a?(Array) ? [ "web" ] : servers_config.keys.sort
end
end

View File

@@ -0,0 +1,57 @@
class Kamal::Configuration::Ssh
LOGGER = ::Logger.new(STDERR)
include Kamal::Configuration::Validation
attr_reader :ssh_config
def initialize(config:)
@ssh_config = config.raw_config.ssh || {}
validate! ssh_config
end
def user
ssh_config.fetch("user", "root")
end
def port
ssh_config.fetch("port", 22)
end
def proxy
if (proxy = ssh_config["proxy"])
Net::SSH::Proxy::Jump.new(proxy.include?("@") ? proxy : "root@#{proxy}")
elsif (proxy_command = ssh_config["proxy_command"])
Net::SSH::Proxy::Command.new(proxy_command)
end
end
def keys_only
ssh_config["keys_only"]
end
def keys
ssh_config["keys"]
end
def key_data
ssh_config["key_data"]
end
def options
{ user: user, port: port, proxy: proxy, logger: logger, keepalive: true, keepalive_interval: 30, keys_only: keys_only, keys: keys, key_data: key_data }.compact
end
def to_h
options.except(:logger).merge(log_level: log_level)
end
private
def logger
LOGGER.tap { |logger| logger.level = log_level }
end
def log_level
ssh_config.fetch("log_level", :fatal)
end
end

View File

@@ -0,0 +1,22 @@
class Kamal::Configuration::Sshkit
include Kamal::Configuration::Validation
attr_reader :sshkit_config
def initialize(config:)
@sshkit_config = config.raw_config.sshkit || {}
validate! sshkit_config
end
def max_concurrent_starts
sshkit_config.fetch("max_concurrent_starts", 30)
end
def pool_idle_timeout
sshkit_config.fetch("pool_idle_timeout", 900)
end
def to_h
sshkit_config
end
end

View File

@@ -0,0 +1,27 @@
require "yaml"
require "active_support/inflector"
module Kamal::Configuration::Validation
extend ActiveSupport::Concern
class_methods do
def validation_doc
@validation_doc ||= File.read(File.join(File.dirname(__FILE__), "docs", "#{validation_config_key}.yml"))
end
def validation_config_key
@validation_config_key ||= name.demodulize.underscore
end
end
def validate!(config, example: nil, context: nil, with: Kamal::Configuration::Validator)
context ||= self.class.validation_config_key
example ||= validation_yml[self.class.validation_config_key]
with.new(config, example: example, context: context).validate!
end
def validation_yml
@validation_yml ||= YAML.load(self.class.validation_doc)
end
end

Some files were not shown because too many files have changed in this diff Show More