Compare commits

..

65 Commits

Author SHA1 Message Date
Donal McBreen
459ba95bbf Revert "Simplify builders config" 2024-08-29 20:16:34 +01:00
Donal McBreen
6adf3c117f Merge pull request #905 from basecamp/simplify-builders-config
Simplify builders config
2024-08-29 09:28:51 +01:00
Donal McBreen
9f0b10425c Fix aliases tests 2024-08-29 09:16:07 +01:00
Donal McBreen
5f2384f123 Use docker info to get arch 2024-08-29 08:46:18 +01:00
Donal McBreen
eab7d3adc5 Keep buildx build, in case of old docker versions which don't default to buildkit 2024-08-29 08:45:51 +01:00
Donal McBreen
d2d0223c37 Require an arch to be set, and default to amd64 in the template 2024-08-29 08:45:51 +01:00
Donal McBreen
56268d724d Simplify the builders configuration
1. Add driver as an option, defaulting to `docker-container`. For a
   "native" build you can set it to `docker`
2. Set arch as a array of architectures to build for, defaulting to
   `[ "amd64", "arm64" ]` unless you are using the docker driver in
   which case we default to not setting a platform
3. Remote is now just a connection string for the remote builder
4. If remote is set, we only use it for non-local arches, if we are
   only building for the local arch, we'll ignore it.

Examples:

On arm64, build for arm64 locally, amd64 remotely or
On amd64, build for amd64 locally, arm64 remotely:

```yaml
builder:
  remote: ssh://docker@docker-builder
```

On arm64, build amd64 on remote,
On amd64 build locally:

```yaml
builder:
  arch:
    - amd64
  remote:
    host: ssh://docker@docker-builder
```

Build amd64 on local:

```yaml
builder:
  arch:
    - amd64
```

Use docker driver, building for local arch:

```yaml
builder:
  driver: docker
```
2024-08-29 08:45:48 +01:00
Donal McBreen
cffb6c3d7e Allow the driver to be set 2024-08-29 08:44:11 +01:00
Donal McBreen
bd1726f305 docker buildx build -> docker build 2024-08-29 08:44:11 +01:00
Donal McBreen
7ddb122a22 Get tests passing 2024-08-29 08:44:11 +01:00
Donal McBreen
98c951bbdb Simplfy choosing a builder 2024-08-29 08:44:11 +01:00
Donal McBreen
374c117b79 Validate multiarch configuration
Remote and local are only allowed when multiarch is enabled.
Remote requires a host and arch, local only requires an arch.
2024-08-29 08:44:11 +01:00
Donal McBreen
d6a5cf3c78 Rip out context_hosts checks
The remote host is now encoded in the builder name so we don't need
to check it. We'll just do an inspect to confirm the builder exists.
2024-08-29 08:44:11 +01:00
Donal McBreen
2aeabda455 Move multiarch remote builder to hybrid builder
Include the host name in the builder name, so we can have one builder
per host/arch across all kamal projects.

Inherit from the remote builder. The difference in the hybrid builder
is that we create a local buildx instance and append the remote context
to it.
2024-08-29 08:44:11 +01:00
Donal McBreen
c048c097ed Create a context for local builds
This ensures we use the docker-container driver and not whatever the
local default is.
2024-08-29 08:44:11 +01:00
Donal McBreen
ed148628fb Local build doesn't need a builder 2024-08-29 08:44:11 +01:00
Donal McBreen
d48080c772 Dump native builder
We already ensure that buildx is installed, so let's always use it.
2024-08-29 08:44:11 +01:00
Donal McBreen
3f64338929 Move native remote to just remote
It's just a remote builder, that will build whichever platform is asked
for, so let's remove the "native" part.

We'll also remove the service name from the builder name, so multiple
services can share the same builder.
2024-08-29 08:44:11 +01:00
Donal McBreen
0ab838bc25 Combine multiarch and native/cache builders
Combine the two builders, as they are almost identical. The only
difference was whether the platforms were set.

The native cached builder wasn't using the context it created, so now
we do.

We'll set the driver to `docker-container` - it seems to be the default
but the Docker docs claim it is `docker`.
2024-08-29 08:44:11 +01:00
Donal McBreen
b7382ceeaf Merge pull request #912 from basecamp/alias
Add aliases to Kamal
2024-08-29 08:43:35 +01:00
Donal McBreen
69367fbc6b Merge pull request #917 from basecamp/v2.0-alpha
Switch the version on main to 2.0.0.alpha
2024-08-29 08:43:19 +01:00
Donal McBreen
2515bd705c Switch the version on main to 2.0.0.alpha
All development is now for the 2.0.0 release.
2024-08-29 08:33:21 +01:00
Donal McBreen
579e169be2 Allow multiple arguments for exec commands
If you can have an alias like:

```
aliases:
  rails: app exec -p rails
```

Then `kamal rails db:migrate:status` will execute
`kamal app exec -p rails db:migrate:status`.

So this works, we'll allow multiple arguments `app exec` and
`server exec` to accept multiple arguments.

The arguments are combined by simply joining them with a space. This
means that these are equivalent:

```
kamal app exec -p rails db:migrate:status
kamal app exec -p "rails db:migrate:status"
```

If you want to pass an argument with spaces, you'll need to quote it:

```
kamal app exec -p "git commit -am \"My comment\""
kamal app exec -p git commit -am "\"My comment\""
```
2024-08-28 10:58:25 +01:00
Donal McBreen
d6f5da92be Bump version for 1.8.2 2024-08-28 09:43:06 +01:00
Donal McBreen
9ccfe20b10 Fix up tests 2024-08-26 11:20:26 +01:00
Donal McBreen
e871d347d5 Merge pull request #889 from xiaohui-zhangxh/git-clone-update-submodules
git clone with --recurse-submodules
2024-08-26 11:20:05 +01:00
Donal McBreen
b8af719bb7 Add aliases to Kamal
Aliases are defined in the configuration file under the `aliases` key.

The configuration is a map of alias name to command. When we run the
command the we just do a literal replacement of the alias with the
string.

So if we have:

```yaml
aliases:
  console: app exec -r console -i --reuse "rails console"
```

Then running `kamal console -r workers` will run the command

```sh
$ kamal app exec -r console -i --reuse "rails console" -r workers
```

Because of the order Thor parses the arguments, this allows us to
override the role from the alias command.

There might be cases where we need to munge the command a bit more but
that would involve getting into Thor command parsing internals,
which are complicated and possibly subject to change.

There's a chance that your aliases could conflict with future built-in
commands, but there's not likely to be many of those and if it happens
you'll get a validation error when you upgrade.

Thanks to @dhnaranjo for the idea!
2024-08-26 10:47:43 +01:00
Donal McBreen
f48987aa03 Merge pull request #903 from basecamp/integration-test-insecure-registry
Integration test insecure registry
2024-08-01 09:57:17 +01:00
Donal McBreen
ef051eca1b Merge pull request #904 from galori/main
Fixed typo in `env.yml`: "valies" --> "values"
2024-08-01 09:57:03 +01:00
Gall Steinitz
173d44ee0a fixed typo in env.yml: valies --> values 2024-07-31 22:12:21 -07:00
Donal McBreen
4e811372f8 Integration test insecure registry
The integrations tests use their own registry so avoid hitting docker
hub rate limits.

This was using a self signed certificate but instead use
`--insecure-registry` to let the docker daemon use HTTP.
2024-07-31 16:54:00 +01:00
Donal McBreen
ec4aa45852 Bump version for 1.8.1 2024-07-29 09:09:57 +01:00
Donal McBreen
5e11a64181 Merge pull request #891 from basecamp/single-pull
Pull once from hosts that warm registry mirrors
2024-07-22 08:18:48 +01:00
Jeremy Daer
57d9ce177a Pull once from hosts that warm registry mirrors 2024-07-18 09:14:22 -07:00
xiaohui
b12de87388 git clone with --recurse-submodules 2024-07-17 10:36:58 +08:00
Donal McBreen
8a98949634 Merge pull request #886 from guoard/patch-2
Remove `--update` flag from `apk add` command
2024-07-16 15:46:37 +01:00
Donal McBreen
0eb9f48082 Merge pull request #887 from basecamp/fix-tests-with-git-config
Fix the tests when you have a git config email set
2024-07-16 13:08:18 +01:00
Donal McBreen
9db6fc0704 Fix the tests when you have a git config email set
The ran ok on CI where we fall back to `whoami`, but failed locally
where there was a git email set.
2024-07-16 12:09:05 +01:00
Donal McBreen
27fede3caa Merge pull request #884 from basecamp/x-config
Add support for configuration extensions
2024-07-16 11:38:28 +01:00
Donal McBreen
29c723f7ec Add support for configuration extensions
Allow blocks prefixed with `x-` in the configuration as a place to
declare reusable blocks with YAML anchors and aliases.

Borrowed from the Docker Compose configuration file format -
https://github.com/compose-spec/compose-spec/blob/main/spec.md#extension

Thanks to @ruyrocha for the suggestion.
2024-07-15 20:47:55 +01:00
Ali Afsharzadeh
2755582c47 Remove --update flag from apk add command 2024-07-15 22:15:25 +03:30
Donal McBreen
fa73d722ea Bump version for 1.8.0 2024-07-15 14:21:23 +01:00
Donal McBreen
c535e4e44f Merge pull request #883 from basecamp/revert-840-main
Revert "Add x25519 gem, support Curve25519"
2024-07-15 13:56:49 +01:00
Donal McBreen
0ea07b1760 Merge pull request #878 from pagbrl/main
feat: Use git email as performer when available
2024-07-15 13:41:17 +01:00
Donal McBreen
03b531f179 Merge pull request #865 from basecamp/clean-envify-env
Ensure envify templates aren't polluted by existing env
2024-07-15 13:41:03 +01:00
Donal McBreen
d8570d1c2c Merge pull request #847 from basecamp/remove-ruby-2.7-from-ci
Remove Ruby 2.7 from CI
2024-07-15 13:40:37 +01:00
Donal McBreen
3fe70b458d Merge pull request #862 from jeromedalbert/bump-sshkit
Bump sshkit to support unbracketed IPv6 addresses
2024-07-15 13:40:18 +01:00
Donal McBreen
ade8b43599 Merge pull request #866 from acidtib/ssh-key-overwrite
Configurable SSH Identity
2024-07-15 13:39:51 +01:00
Donal McBreen
d24fc3ca4e Revert "Add x25519 gem, support Curve25519" 2024-07-15 13:36:50 +01:00
Donal McBreen
7c244bbb98 Merge pull request #879 from basecamp/seed-mirror
Seed docker mirrors by pulling once per mirror first
2024-07-15 13:30:53 +01:00
Donal McBreen
1369c46a83 Seed docker mirrors by pulling once per mirror first
Find the first registry mirror on each host. If we find any, pull the
images on one host per mirror, then do the remainder concurrently.

The initial pulls will seed the mirrors ensuring that we pull the image
from Docker Hub once each.

This works best if there is only one mirror on each host.
2024-07-11 16:20:37 +01:00
Paul Gabriel
deccf1cfaf feat: Use git email as performer when available 2024-07-11 11:19:44 +02:00
Donal McBreen
1573cebadf Merge pull request #868 from nickhammond/env/service
Add ENV['KAMAL_SERVICE'] to hooks
2024-07-10 10:26:59 +01:00
Nick Hammond
85a2926cde Remove the deprecated docker compose version (#869) 2024-06-28 15:00:23 -07:00
Nick Hammond
58a51b079e Add KAMAL_SERVICE to custom hooks and exclude from auditor 2024-06-27 10:52:55 -06:00
Nick Hammond
f1f3fc566f Add ENV['SERVICE'] to hooks 2024-06-27 10:26:11 -06:00
acidtib
44726ff65a overwrite ssh identity 2024-06-26 17:14:13 -06:00
Jerome Dalbert
fd0d4af21f Bump sshkit to support unbracketed IPv6 addresses
Set sshkit minimum version to 1.23.0, which includes an enhancement to
support unbracketed IPv6 addresses.

See https://github.com/capistrano/sshkit/pull/538
2024-06-25 12:17:40 -07:00
Jeremy Daer
13409ada5a Ensure envify templates aren't polluted by existing env
Setting `GITHUB_TOKEN` as in the docs results in reusing the existing
`GITHUB_TOKEN` since `gh` returns that env var if it's set:
```bash
GITHUB_TOKEN=junk gh config get -h github.com oauth_token
junk
```

Using the original env ensures that the templates will be evaluated the
same way regardless of whether envify had been previously invoked.
2024-06-25 11:14:34 -07:00
Donal McBreen
9a1379be6c Bump version for 1.7.3 2024-06-25 15:03:02 +01:00
Donal McBreen
31d6c198da Merge pull request #861 from K4sku/update-docker-setup-sample-hook
Expand on docker-setup.sample hook
2024-06-25 14:44:13 +01:00
Donal McBreen
22afe4de77 Merge pull request #864 from basecamp/allow-arrays-in-args
Allow arrays in args
2024-06-25 14:41:07 +01:00
Donal McBreen
b63982c3a7 Allow arrays in args
Just check that args is a Hash without checking the value types.

Fixes: https://github.com/basecamp/kamal/issues/863
2024-06-25 14:18:23 +01:00
Cezary Kłos
9e12d32cc3 Expand on docker-setup.sample script so it creates docker network "kamal" on each of the defined hosts. 2024-06-24 12:45:56 +02:00
Donal McBreen
e160852e4d Remove Ruby 2.7 from CI
It's EOL since March 2023.
2024-06-20 08:54:55 +01:00
56 changed files with 524 additions and 132 deletions

View File

@@ -24,25 +24,12 @@ jobs:
strategy: strategy:
matrix: matrix:
ruby-version: ruby-version:
- "2.7"
- "3.1" - "3.1"
- "3.2" - "3.2"
- "3.3" - "3.3"
gemfile: gemfile:
- Gemfile - Gemfile
- gemfiles/ruby_2.7.gemfile
- gemfiles/rails_edge.gemfile - gemfiles/rails_edge.gemfile
exclude:
- ruby-version: "2.7"
gemfile: Gemfile
- ruby-version: "2.7"
gemfile: gemfiles/rails_edge.gemfile
- ruby-version: "3.1"
gemfile: gemfiles/ruby_2.7.gemfile
- ruby-version: "3.2"
gemfile: gemfiles/ruby_2.7.gemfile
- ruby-version: "3.3"
gemfile: gemfiles/ruby_2.7.gemfile
name: ${{ format('Tests (Ruby {0})', matrix.ruby-version) }} name: ${{ format('Tests (Ruby {0})', matrix.ruby-version) }}
runs-on: ubuntu-latest runs-on: ubuntu-latest
continue-on-error: true continue-on-error: true

View File

@@ -1,7 +1,7 @@
# Use the official Ruby 3.2.0 Alpine image as the base image # Use the official Ruby 3.2.0 Alpine image as the base image
FROM ruby:3.2.0-alpine FROM ruby:3.2.0-alpine
# Install docker/buildx-bin # Install docker/buildx-bin
COPY --from=docker/buildx-bin /buildx /usr/libexec/docker/cli-plugins/docker-buildx COPY --from=docker/buildx-bin /buildx /usr/libexec/docker/cli-plugins/docker-buildx
# Set the working directory to /kamal # Set the working directory to /kamal
@@ -14,7 +14,7 @@ COPY Gemfile Gemfile.lock kamal.gemspec ./
COPY lib/kamal/version.rb /kamal/lib/kamal/version.rb COPY lib/kamal/version.rb /kamal/lib/kamal/version.rb
# Install system dependencies # Install system dependencies
RUN apk add --no-cache --update build-base git docker openrc openssh-client-default \ RUN apk add --no-cache build-base git docker openrc openssh-client-default \
&& rc-update add docker boot \ && rc-update add docker boot \
&& gem install bundler --version=2.4.3 \ && gem install bundler --version=2.4.3 \
&& bundle install && bundle install

View File

@@ -1,7 +1,7 @@
PATH PATH
remote: . remote: .
specs: specs:
kamal (1.7.2) kamal (2.0.0.alpha)
activesupport (>= 7.0) activesupport (>= 7.0)
base64 (~> 0.2) base64 (~> 0.2)
bcrypt_pbkdf (~> 1.0) bcrypt_pbkdf (~> 1.0)
@@ -9,9 +9,8 @@ PATH
dotenv (~> 2.8) dotenv (~> 2.8)
ed25519 (~> 1.2) ed25519 (~> 1.2)
net-ssh (~> 7.0) net-ssh (~> 7.0)
sshkit (>= 1.22.2, < 2.0) sshkit (>= 1.23.0, < 2.0)
thor (~> 1.2) thor (~> 1.3)
x25519 (~> 1.0, >= 1.0.10)
zeitwerk (~> 2.5) zeitwerk (~> 2.5)
GEM GEM
@@ -154,9 +153,8 @@ GEM
rubocop-rails rubocop-rails
ruby-progressbar (1.13.0) ruby-progressbar (1.13.0)
ruby2_keywords (0.0.5) ruby2_keywords (0.0.5)
sshkit (1.22.2) sshkit (1.23.0)
base64 base64
mutex_m
net-scp (>= 1.1.2) net-scp (>= 1.1.2)
net-sftp (>= 2.1.2) net-sftp (>= 2.1.2)
net-ssh (>= 2.8.0) net-ssh (>= 2.8.0)
@@ -166,7 +164,6 @@ GEM
concurrent-ruby (~> 1.0) concurrent-ruby (~> 1.0)
unicode-display_width (2.5.0) unicode-display_width (2.5.0)
webrick (1.8.1) webrick (1.8.1)
x25519 (1.0.10)
zeitwerk (2.6.12) zeitwerk (2.6.12)
PLATFORMS PLATFORMS

View File

@@ -17,6 +17,7 @@ end
DOCS = { DOCS = {
"accessory" => "Accessories", "accessory" => "Accessories",
"alias" => "Aliases",
"boot" => "Booting", "boot" => "Booting",
"builder" => "Builders", "builder" => "Builders",
"configuration" => "Configuration overview", "configuration" => "Configuration overview",
@@ -67,26 +68,27 @@ class DocWriter
output.puts output.puts
place = :new_section place = :new_section
elsif line =~ /^ *#/ elsif line =~ /^ *#/
generate_line(line, place: place) generate_line(line, heading: place == :new_section)
place = :in_section place = :in_section
else else
output.puts "```yaml" output.puts "```yaml"
output.print line output.puts line
place = :in_yaml place = :in_yaml
end end
when :in_yaml when :in_yaml, :in_empty_line_yaml
if line =~ /^ *#/ if line =~ /^ *#/
output.puts "```" output.puts "```"
generate_line(line, place: :new_section) generate_line(line, heading: place == :in_empty_line_yaml)
place = :in_section place = :in_section
elsif line.empty?
place = :in_empty_line_yaml
else else
output.puts output.puts line
output.print line
end end
end end
end end
output.puts "\n```" if place == :in_yaml output.puts "```" if place == :in_yaml
end end
def generate_header def generate_header
@@ -98,7 +100,7 @@ class DocWriter
output.puts output.puts
end end
def generate_line(line, place: :in_section) def generate_line(line, heading: false)
line = line.gsub(/^ *#\s?/, "") line = line.gsub(/^ *#\s?/, "")
if line =~ /(.*)kamal docs ([a-z]*)(.*)/ if line =~ /(.*)kamal docs ([a-z]*)(.*)/
@@ -109,7 +111,7 @@ class DocWriter
line = "#{$1}[#{titlify($2.split("/").last)}](#{$2})#{$3}" line = "#{$1}[#{titlify($2.split("/").last)}](#{$2})#{$3}"
end end
if place == :new_section if heading
output.puts "## [#{line}](##{linkify(line)})" output.puts "## [#{line}](##{linkify(line)})"
else else
output.puts line output.puts line

View File

@@ -1,6 +0,0 @@
source 'https://rubygems.org'
git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gemspec path: "../"
gem "nokogiri", "~> 1.15.0"

View File

@@ -12,13 +12,12 @@ Gem::Specification.new do |spec|
spec.executables = %w[ kamal ] spec.executables = %w[ kamal ]
spec.add_dependency "activesupport", ">= 7.0" spec.add_dependency "activesupport", ">= 7.0"
spec.add_dependency "sshkit", ">= 1.22.2", "< 2.0" spec.add_dependency "sshkit", ">= 1.23.0", "< 2.0"
spec.add_dependency "net-ssh", "~> 7.0" spec.add_dependency "net-ssh", "~> 7.0"
spec.add_dependency "thor", "~> 1.2" spec.add_dependency "thor", "~> 1.3"
spec.add_dependency "dotenv", "~> 2.8" spec.add_dependency "dotenv", "~> 2.8"
spec.add_dependency "zeitwerk", "~> 2.5" spec.add_dependency "zeitwerk", "~> 2.5"
spec.add_dependency "ed25519", "~> 1.2" spec.add_dependency "ed25519", "~> 1.2"
spec.add_dependency "x25519", "~> 1.0", ">= 1.0.10"
spec.add_dependency "bcrypt_pbkdf", "~> 1.0" spec.add_dependency "bcrypt_pbkdf", "~> 1.0"
spec.add_dependency "concurrent-ruby", "~> 1.2" spec.add_dependency "concurrent-ruby", "~> 1.2"
spec.add_dependency "base64", "~> 0.2" spec.add_dependency "base64", "~> 0.2"

View File

@@ -0,0 +1,9 @@
class Kamal::Cli::Alias::Command < Thor::DynamicCommand
def run(instance, args = [])
if (_alias = KAMAL.config.aliases[name])
Kamal::Cli::Main.start(Shellwords.split(_alias.command) + ARGV[1..-1])
else
super
end
end
end

View File

@@ -71,11 +71,12 @@ class Kamal::Cli::App < Kamal::Cli::Base
end end
end end
desc "exec [CMD]", "Execute a custom command on servers within the app container (use --help to show options)" desc "exec [CMD...]", "Execute a custom command on servers within the app container (use --help to show options)"
option :interactive, aliases: "-i", type: :boolean, default: false, desc: "Execute command over ssh for an interactive shell (use for console/bash)" option :interactive, aliases: "-i", type: :boolean, default: false, desc: "Execute command over ssh for an interactive shell (use for console/bash)"
option :reuse, type: :boolean, default: false, desc: "Reuse currently running container instead of starting a new one" option :reuse, type: :boolean, default: false, desc: "Reuse currently running container instead of starting a new one"
option :env, aliases: "-e", type: :hash, desc: "Set environment variables for the command" option :env, aliases: "-e", type: :hash, desc: "Set environment variables for the command"
def exec(cmd) def exec(*cmd)
cmd = Kamal::Utils.join_commands(cmd)
env = options[:env] env = options[:env]
case case
when options[:interactive] && options[:reuse] when options[:interactive] && options[:reuse]

View File

@@ -6,7 +6,8 @@ module Kamal::Cli
class Base < Thor class Base < Thor
include SSHKit::DSL include SSHKit::DSL
def self.exit_on_failure?() true end def self.exit_on_failure?() false end
def self.dynamic_command_class() Kamal::Cli::Alias::Command end
class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging" class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging"
class_option :quiet, type: :boolean, aliases: "-q", desc: "Minimal logging" class_option :quiet, type: :boolean, aliases: "-q", desc: "Minimal logging"
@@ -22,15 +23,26 @@ module Kamal::Cli
class_option :skip_hooks, aliases: "-H", type: :boolean, default: false, desc: "Don't run hooks" class_option :skip_hooks, aliases: "-H", type: :boolean, default: false, desc: "Don't run hooks"
def initialize(*) def initialize(args = [], local_options = {}, config = {})
super if config[:current_command].is_a?(Kamal::Cli::Alias::Command)
# When Thor generates a dynamic command, it doesn't attempt to parse the arguments.
# For our purposes, it means the arguments are passed in args rather than local_options.
super([], args, config)
else
super
end
@original_env = ENV.to_h.dup @original_env = ENV.to_h.dup
load_envs load_env
initialize_commander(options_with_subcommand_class_options) initialize_commander(options_with_subcommand_class_options)
end end
private private
def load_envs def reload_env
reset_env
load_env
end
def load_env
if destination = options[:destination] if destination = options[:destination]
Dotenv.load(".env.#{destination}", ".env") Dotenv.load(".env.#{destination}", ".env")
else else
@@ -38,10 +50,27 @@ module Kamal::Cli
end end
end end
def reload_envs def reset_env
replace_env @original_env
end
def replace_env(env)
ENV.clear ENV.clear
ENV.update(@original_env) ENV.update(env)
load_envs end
def with_original_env
keeping_current_env do
reset_env
yield
end
end
def keeping_current_env
current_env = ENV.to_h.dup
yield
ensure
replace_env(current_env)
end end
def options_with_subcommand_class_options def options_with_subcommand_class_options

View File

@@ -59,11 +59,14 @@ class Kamal::Cli::Build < Kamal::Cli::Base
desc "pull", "Pull app image from registry onto servers" desc "pull", "Pull app image from registry onto servers"
def pull def pull
on(KAMAL.hosts) do if (first_hosts = mirror_hosts).any?
execute *KAMAL.auditor.record("Pulled image with version #{KAMAL.config.version}"), verbosity: :debug #  Pull on a single host per mirror first to seed them
execute *KAMAL.builder.clean, raise_on_non_zero_exit: false say "Pulling image on #{first_hosts.join(", ")} to seed the #{"mirror".pluralize(first_hosts.count)}...", :magenta
execute *KAMAL.builder.pull pull_on_hosts(first_hosts)
execute *KAMAL.builder.validate_image say "Pulling image on remaining hosts...", :magenta
pull_on_hosts(KAMAL.hosts - first_hosts)
else
pull_on_hosts(KAMAL.hosts)
end end
end end
@@ -131,4 +134,28 @@ class Kamal::Cli::Build < Kamal::Cli::Base
end end
end end
end end
def mirror_hosts
if KAMAL.hosts.many?
mirror_hosts = Concurrent::Hash.new
on(KAMAL.hosts) do |host|
first_mirror = capture_with_info(*KAMAL.builder.first_mirror).strip.presence
mirror_hosts[first_mirror] ||= host.to_s if first_mirror
rescue SSHKit::Command::Failed => e
raise unless e.message =~ /error calling index: reflect: slice index out of range/
end
mirror_hosts.values
else
[]
end
end
def pull_on_hosts(hosts)
on(hosts) do
execute *KAMAL.auditor.record("Pulled image with version #{KAMAL.config.version}"), verbosity: :debug
execute *KAMAL.builder.clean, raise_on_non_zero_exit: false
execute *KAMAL.builder.pull
execute *KAMAL.builder.validate_image
end
end
end end

View File

@@ -191,10 +191,12 @@ class Kamal::Cli::Main < Kamal::Cli::Base
end end
if Pathname.new(File.expand_path(env_template_path)).exist? if Pathname.new(File.expand_path(env_template_path)).exist?
File.write(env_path, ERB.new(File.read(env_template_path), trim_mode: "-").result, perm: 0600) # Ensure existing env doesn't pollute template evaluation
content = with_original_env { ERB.new(File.read(env_template_path), trim_mode: "-").result }
File.write(env_path, content, perm: 0600)
unless options[:skip_push] unless options[:skip_push]
reload_envs reload_env
invoke "kamal:cli:env:push", options invoke "kamal:cli:env:push", options
end end
else else

View File

@@ -1,7 +1,8 @@
class Kamal::Cli::Server < Kamal::Cli::Base class Kamal::Cli::Server < Kamal::Cli::Base
desc "exec", "Run a custom command on the server (use --help to show options)" desc "exec", "Run a custom command on the server (use --help to show options)"
option :interactive, type: :boolean, aliases: "-i", default: false, desc: "Run the command interactively (use for console/bash)" option :interactive, type: :boolean, aliases: "-i", default: false, desc: "Run the command interactively (use for console/bash)"
def exec(cmd) def exec(*cmd)
cmd = Kamal::Utils.join_commands(cmd)
hosts = KAMAL.hosts | KAMAL.accessory_hosts hosts = KAMAL.hosts | KAMAL.accessory_hosts
case case

View File

@@ -1,7 +1,13 @@
#!/bin/sh #!/usr/bin/env ruby
# A sample docker-setup hook # A sample docker-setup hook
# #
# Sets up a Docker network which can then be used by the applications containers # Sets up a Docker network on defined hosts which can then be used by the applications containers
ssh user@example.com docker network create kamal hosts = ENV["KAMAL_HOSTS"].split(",")
hosts.each do |ip|
destination = "root@#{ip}"
puts "Creating a Docker network \"kamal\" on #{destination}"
`ssh #{destination} docker network create kamal`
end

View File

@@ -27,7 +27,11 @@ class Kamal::Commander
def specific_primary! def specific_primary!
@specifics = nil @specifics = nil
self.specific_hosts = [ config.primary_host ] if specific_roles.present?
self.specific_hosts = [ specific_roles.first.primary_host ]
else
self.specific_hosts = [ config.primary_host ]
end
end end
def specific_roles=(role_names) def specific_roles=(role_names)
@@ -113,6 +117,10 @@ class Kamal::Commander
@traefik ||= Kamal::Commands::Traefik.new(config) @traefik ||= Kamal::Commands::Traefik.new(config)
end end
def alias(name)
config.aliases[name]
end
def with_verbosity(level) def with_verbosity(level)
old_level = self.verbosity old_level = self.verbosity

View File

@@ -9,7 +9,7 @@ class Kamal::Commands::Auditor < Kamal::Commands::Base
# Runs remotely # Runs remotely
def record(line, **details) def record(line, **details)
append \ append \
[ :echo, audit_tags(**details).except(:version, :service_version).to_s, line ], [ :echo, audit_tags(**details).except(:version, :service_version, :service).to_s, line ],
audit_log_file audit_log_file
end end

View File

@@ -2,7 +2,7 @@ require "active_support/core_ext/string/filters"
class Kamal::Commands::Builder < Kamal::Commands::Base class Kamal::Commands::Builder < Kamal::Commands::Base
delegate :create, :remove, :push, :clean, :pull, :info, :context_hosts, :config_context_hosts, :validate_image, delegate :create, :remove, :push, :clean, :pull, :info, :context_hosts, :config_context_hosts, :validate_image,
to: :target :first_mirror, to: :target
include Clone include Clone

View File

@@ -40,6 +40,10 @@ class Kamal::Commands::Builder::Base < Kamal::Commands::Base
[] []
end end
def first_mirror
docker(:info, "--format '{{index .RegistryConfig.Mirrors 0}}'")
end
private private
def build_tags def build_tags
[ "-t", config.absolute_image, "-t", config.latest_image ] [ "-t", config.absolute_image, "-t", config.latest_image ]

View File

@@ -6,7 +6,7 @@ module Kamal::Commands::Builder::Clone
end end
def clone def clone
git :clone, Kamal::Git.root, path: clone_directory git :clone, Kamal::Git.root, "--recurse-submodules", path: clone_directory
end end
def clone_reset_steps def clone_reset_steps
@@ -14,7 +14,8 @@ module Kamal::Commands::Builder::Clone
git(:remote, "set-url", :origin, Kamal::Git.root, path: build_directory), git(:remote, "set-url", :origin, Kamal::Git.root, path: build_directory),
git(:fetch, :origin, path: build_directory), git(:fetch, :origin, path: build_directory),
git(:reset, "--hard", Kamal::Git.revision, path: build_directory), git(:reset, "--hard", Kamal::Git.revision, path: build_directory),
git(:clean, "-fdx", path: build_directory) git(:clean, "-fdx", path: build_directory),
git(:submodule, :update, "--init", path: build_directory)
] ]
end end

View File

@@ -11,7 +11,7 @@ class Kamal::Configuration
delegate :argumentize, :optionize, to: Kamal::Utils delegate :argumentize, :optionize, to: Kamal::Utils
attr_reader :destination, :raw_config attr_reader :destination, :raw_config
attr_reader :accessories, :boot, :builder, :env, :healthcheck, :logging, :traefik, :servers, :ssh, :sshkit, :registry attr_reader :accessories, :aliases, :boot, :builder, :env, :healthcheck, :logging, :traefik, :servers, :ssh, :sshkit, :registry
include Validation include Validation
@@ -47,13 +47,14 @@ class Kamal::Configuration
@destination = destination @destination = destination
@declared_version = version @declared_version = version
validate! raw_config, example: validation_yml.symbolize_keys, context: "" validate! raw_config, example: validation_yml.symbolize_keys, context: "", with: Kamal::Configuration::Validator::Configuration
# Eager load config to validate it, these are first as they have dependencies later on # Eager load config to validate it, these are first as they have dependencies later on
@servers = Servers.new(config: self) @servers = Servers.new(config: self)
@registry = Registry.new(config: self) @registry = Registry.new(config: self)
@accessories = @raw_config.accessories&.keys&.collect { |name| Accessory.new(name, config: self) } || [] @accessories = @raw_config.accessories&.keys&.collect { |name| Accessory.new(name, config: self) } || []
@aliases = @raw_config.aliases&.keys&.to_h { |name| [ name, Alias.new(name, config: self) ] } || {}
@boot = Boot.new(config: self) @boot = Boot.new(config: self)
@builder = Builder.new(config: self) @builder = Builder.new(config: self)
@env = Env.new(config: @raw_config.env || {}) @env = Env.new(config: @raw_config.env || {})

View File

@@ -0,0 +1,15 @@
class Kamal::Configuration::Alias
include Kamal::Configuration::Validation
attr_reader :name, :command
def initialize(name, config:)
@name, @command = name.inquiry, config.raw_config["aliases"][name]
validate! \
command,
example: validation_yml["aliases"]["uname"],
context: "aliases/#{name}",
with: Kamal::Configuration::Validator::Alias
end
end

View File

@@ -0,0 +1,26 @@
# Aliases
#
# Aliases are shortcuts for Kamal commands.
#
# For example, for a Rails app, you might open a console with:
#
# ```shell
# kamal app exec -i -r console "rails console"
# ```
#
# By defining an alias, like this:
aliases:
console: app exec -r console -i "rails console"
# You can now open the console with:
# ```shell
# kamal console
# ```
# Configuring aliases
#
# Aliases are defined in the root config under the alias key
#
# Each alias is named and can only contain lowercase letters, numbers, dashes and underscores.
aliases:
uname: app exec -p -q -r web "uname -a"

View File

@@ -2,13 +2,24 @@
# #
# Configuration is read from the `config/deploy.yml` # Configuration is read from the `config/deploy.yml`
# #
# Destinations
#
# When running commands, you can specify a destination with the `-d` flag, # When running commands, you can specify a destination with the `-d` flag,
# e.g. `kamal deploy -d staging` # e.g. `kamal deploy -d staging`
# #
# In this case the configuration will also be read from `config/deploy.staging.yml` # In this case the configuration will also be read from `config/deploy.staging.yml`
# and merged with the base configuration. # and merged with the base configuration.
# Extensions
# #
# The available configuration options are explained below. # Kamal will not accept unrecognized keys in the configuration file.
#
# However, you might want to declare a configuration block using YAML anchors
# and aliases to avoid repetition.
#
# You can use prefix a configuration section with `x-` to indicate that it is an
# extension. Kamal will ignore the extension and not raise an error.
# The service name # The service name
# This is a required value. It is used as the container name prefix. # This is a required value. It is used as the container name prefix.
@@ -155,3 +166,9 @@ healthcheck:
# Docker logging configuration, see kamal docs logging # Docker logging configuration, see kamal docs logging
logging: logging:
... ...
# Aliases
#
# Alias configuration, see kamal docs alias
aliases:
...

View File

@@ -29,7 +29,7 @@ env:
# To pass the secrets you should list them under the `secret` key. When you do this the # To pass the secrets you should list them under the `secret` key. When you do this the
# other variables need to be moved under the `clear` key. # other variables need to be moved under the `clear` key.
# #
# Unlike clear valies, secrets are not passed directly to the container, # Unlike clear values, secrets are not passed directly to the container,
# but are stored in an env file on the host # but are stored in an env file on the host
# The file is not updated when deploying, only when running `kamal envify` or `kamal env push`. # The file is not updated when deploying, only when running `kamal envify` or `kamal env push`.
env: env:

View File

@@ -44,3 +44,23 @@ ssh:
# Defaults to `fatal`. Set this to debug if you are having # Defaults to `fatal`. Set this to debug if you are having
# SSH connection issues. # SSH connection issues.
log_level: debug log_level: debug
# Keys Only
#
# Set to true to use only private keys from keys and key_data parameters,
# even if ssh-agent offers more identities. This option is intended for
# situations where ssh-agent offers many different identites or you have
# a need to overwrite all identites and force a single one.
keys_only: false
# Keys
#
# An array of file names of private keys to use for publickey
# and hostbased authentication
keys: [ "~/.ssh/id.pem" ]
# Key Data
#
# An array of strings, with each element of the array being
# a raw private key in PEM format.
key_data: [ "-----BEGIN OPENSSH PRIVATE KEY-----" ]

View File

@@ -26,8 +26,20 @@ class Kamal::Configuration::Ssh
end end
end end
def keys_only
ssh_config["keys_only"]
end
def keys
ssh_config["keys"]
end
def key_data
ssh_config["key_data"]
end
def options def options
{ user: user, port: port, proxy: proxy, logger: logger, keepalive: true, keepalive_interval: 30 }.compact { user: user, port: port, proxy: proxy, logger: logger, keepalive: true, keepalive_interval: 30, keys_only: keys_only, keys: keys, key_data: key_data }.compact
end end
def to_h def to_h

View File

@@ -13,33 +13,34 @@ class Kamal::Configuration::Validator
private private
def validate_against_example!(validation_config, example) def validate_against_example!(validation_config, example)
validate_type! validation_config, Hash validate_type! validation_config, example.class
if (unknown_keys = validation_config.keys - example.keys).any? if example.class == Hash
unknown_keys_error unknown_keys check_unknown_keys! validation_config, example
end
validation_config.each do |key, value| validation_config.each do |key, value|
with_context(key) do next if extension?(key)
example_value = example[key] with_context(key) do
example_value = example[key]
if example_value == "..." if example_value == "..."
validate_type! value, *(Array if key == :servers), Hash validate_type! value, *(Array if key == :servers), Hash
elsif key == "hosts" elsif key == "hosts"
validate_servers! value validate_servers! value
elsif example_value.is_a?(Array) elsif example_value.is_a?(Array)
validate_array_of! value, example_value.first.class validate_array_of! value, example_value.first.class
elsif example_value.is_a?(Hash) elsif example_value.is_a?(Hash)
case key.to_s case key.to_s
when "options" when "options", "args"
validate_type! value, Hash validate_type! value, Hash
when "args", "labels" when "labels"
validate_hash_of! value, example_value.first[1].class validate_hash_of! value, example_value.first[1].class
else
validate_against_example! value, example_value
end
else else
validate_against_example! value, example_value validate_type! value, example_value.class
end end
else
validate_type! value, example_value.class
end end
end end
end end
@@ -137,4 +138,18 @@ class Kamal::Configuration::Validator
ensure ensure
@context = old_context @context = old_context
end end
def allow_extensions?
false
end
def extension?(key)
key.to_s.start_with?("x-")
end
def check_unknown_keys!(config, example)
unknown_keys = config.keys - example.keys
unknown_keys.reject! { |key| extension?(key) } if allow_extensions?
unknown_keys_error unknown_keys if unknown_keys.present?
end
end end

View File

@@ -0,0 +1,15 @@
class Kamal::Configuration::Validator::Alias < Kamal::Configuration::Validator
def validate!
super
name = context.delete_prefix("aliases/")
if name !~ /\A[a-z0-9_-]+\z/
error "Invalid alias name: '#{name}'. Must only contain lowercase letters, alphanumeric, hyphens and underscores."
end
if Kamal::Cli::Main.commands.include?(name)
error "Alias '#{name}' conflicts with a built-in command."
end
end
end

View File

@@ -0,0 +1,6 @@
class Kamal::Configuration::Validator::Configuration < Kamal::Configuration::Validator
private
def allow_extensions?
true
end
end

View File

@@ -9,6 +9,10 @@ module Kamal::Git
`git config user.name`.strip `git config user.name`.strip
end end
def email
`git config user.email`.strip
end
def revision def revision
`git rev-parse HEAD`.strip `git rev-parse HEAD`.strip
end end

View File

@@ -10,10 +10,11 @@ class Kamal::Tags
def default_tags(config) def default_tags(config)
{ recorded_at: Time.now.utc.iso8601, { recorded_at: Time.now.utc.iso8601,
performer: `whoami`.chomp, performer: Kamal::Git.email.presence || `whoami`.chomp,
destination: config.destination, destination: config.destination,
version: config.version, version: config.version,
service_version: service_version(config) } service_version: service_version(config),
service: config.service }
end end
def service_version(config) def service_version(config)

View File

@@ -77,4 +77,8 @@ module Kamal::Utils
def stable_sort!(elements, &block) def stable_sort!(elements, &block)
elements.sort_by!.with_index { |element, index| [ block.call(element), index ] } elements.sort_by!.with_index { |element, index| [ block.call(element), index ] }
end end
def join_commands(commands)
commands.map(&:strip).join(" ")
end
end end

View File

@@ -1,3 +1,3 @@
module Kamal module Kamal
VERSION = "1.7.2" VERSION = "2.0.0.alpha"
end end

View File

@@ -247,6 +247,12 @@ class CliAppTest < CliTestCase
end end
end end
test "exec separate arguments" do
run_command("exec", "ruby", " -v").tap do |output|
assert_match "docker run --rm --env-file .kamal/env/roles/app-web.env dhh/app:latest ruby -v", output
end
end
test "exec with reuse" do test "exec with reuse" do
run_command("exec", "--reuse", "ruby -v").tap do |output| run_command("exec", "--reuse", "ruby -v").tap do |output|
assert_match "sh -c 'docker ps --latest --format '\\''{{.Names}}'\\'' --filter label=service=app --filter label=role=web --filter status=running --filter status=restarting --filter ancestor=$(docker image ls --filter reference=dhh/app:latest --format '\\''{{.ID}}'\\'') ; docker ps --latest --format '\\''{{.Names}}'\\'' --filter label=service=app --filter label=role=web --filter status=running --filter status=restarting' | head -1 | while read line; do echo ${line#app-web-}; done", output # Get current version assert_match "sh -c 'docker ps --latest --format '\\''{{.Names}}'\\'' --filter label=service=app --filter label=role=web --filter status=running --filter status=restarting --filter ancestor=$(docker image ls --filter reference=dhh/app:latest --format '\\''{{.ID}}'\\'') ; docker ps --latest --format '\\''{{.Names}}'\\'' --filter label=service=app --filter label=role=web --filter status=running --filter status=restarting' | head -1 | while read line; do echo ${line#app-web-}; done", output # Get current version

View File

@@ -42,7 +42,7 @@ class CliBuildTest < CliTestCase
SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:docker, "--version", "&&", :docker, :buildx, "version") SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:docker, "--version", "&&", :docker, :buildx, "version")
SSHKit::Backend::Abstract.any_instance.expects(:execute) SSHKit::Backend::Abstract.any_instance.expects(:execute)
.with(:git, "-C", "#{Dir.tmpdir}/kamal-clones/app-#{pwd_sha}", :clone, Dir.pwd) .with(:git, "-C", "#{Dir.tmpdir}/kamal-clones/app-#{pwd_sha}", :clone, Dir.pwd, "--recurse-submodules")
.raises(SSHKit::Command::Failed.new("fatal: destination path 'kamal' already exists and is not an empty directory")) .raises(SSHKit::Command::Failed.new("fatal: destination path 'kamal' already exists and is not an empty directory"))
.then .then
.returns(true) .returns(true)
@@ -50,6 +50,7 @@ class CliBuildTest < CliTestCase
SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:git, "-C", build_directory, :fetch, :origin) SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:git, "-C", build_directory, :fetch, :origin)
SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:git, "-C", build_directory, :reset, "--hard", Kamal::Git.revision) SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:git, "-C", build_directory, :reset, "--hard", Kamal::Git.revision)
SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:git, "-C", build_directory, :clean, "-fdx") SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:git, "-C", build_directory, :clean, "-fdx")
SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:git, "-C", build_directory, :submodule, :update, "--init")
SSHKit::Backend::Abstract.any_instance.expects(:execute) SSHKit::Backend::Abstract.any_instance.expects(:execute)
.with(:docker, :buildx, :build, "--push", "--platform", "linux/amd64,linux/arm64", "--builder", "kamal-app-multiarch", "-t", "dhh/app:999", "-t", "dhh/app:latest", "--label", "service=\"app\"", "--file", "Dockerfile", ".") .with(:docker, :buildx, :build, "--push", "--platform", "linux/amd64,linux/arm64", "--builder", "kamal-app-multiarch", "-t", "dhh/app:999", "-t", "dhh/app:latest", "--label", "service=\"app\"", "--file", "Dockerfile", ".")
@@ -88,7 +89,7 @@ class CliBuildTest < CliTestCase
SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:docker, "--version", "&&", :docker, :buildx, "version") SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:docker, "--version", "&&", :docker, :buildx, "version")
SSHKit::Backend::Abstract.any_instance.expects(:execute) SSHKit::Backend::Abstract.any_instance.expects(:execute)
.with(:git, "-C", "#{Dir.tmpdir}/kamal-clones/app-#{pwd_sha}", :clone, Dir.pwd) .with(:git, "-C", "#{Dir.tmpdir}/kamal-clones/app-#{pwd_sha}", :clone, Dir.pwd, "--recurse-submodules")
.raises(SSHKit::Command::Failed.new("fatal: destination path 'kamal' already exists and is not an empty directory")) .raises(SSHKit::Command::Failed.new("fatal: destination path 'kamal' already exists and is not an empty directory"))
.then .then
.returns(true) .returns(true)
@@ -169,12 +170,41 @@ class CliBuildTest < CliTestCase
test "pull" do test "pull" do
run_command("pull").tap do |output| run_command("pull").tap do |output|
assert_match /docker info --format '{{index .RegistryConfig.Mirrors 0}}'/, output
assert_match /docker image rm --force dhh\/app:999/, output assert_match /docker image rm --force dhh\/app:999/, output
assert_match /docker pull dhh\/app:999/, output assert_match /docker pull dhh\/app:999/, output
assert_match "docker inspect -f '{{ .Config.Labels.service }}' dhh/app:999 | grep -x app || (echo \"Image dhh/app:999 is missing the 'service' label\" && exit 1)", output assert_match "docker inspect -f '{{ .Config.Labels.service }}' dhh/app:999 | grep -x app || (echo \"Image dhh/app:999 is missing the 'service' label\" && exit 1)", output
end end
end end
test "pull with mirror" do
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info)
.with(:docker, :info, "--format '{{index .RegistryConfig.Mirrors 0}}'")
.returns("registry-mirror.example.com")
.at_least_once
run_command("pull").tap do |output|
assert_match /Pulling image on 1\.1\.1\.\d to seed the mirror\.\.\./, output
assert_match "Pulling image on remaining hosts...", output
assert_equal 4, output.scan(/docker pull dhh\/app:999/).size, output
assert_match "docker inspect -f '{{ .Config.Labels.service }}' dhh/app:999 | grep -x app || (echo \"Image dhh/app:999 is missing the 'service' label\" && exit 1)", output
end
end
test "pull with mirrors" do
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info)
.with(:docker, :info, "--format '{{index .RegistryConfig.Mirrors 0}}'")
.returns("registry-mirror.example.com", "registry-mirror2.example.com")
.at_least_once
run_command("pull").tap do |output|
assert_match /Pulling image on 1\.1\.1\.\d, 1\.1\.1\.\d to seed the mirrors\.\.\./, output
assert_match "Pulling image on remaining hosts...", output
assert_equal 4, output.scan(/docker pull dhh\/app:999/).size, output
assert_match "docker inspect -f '{{ .Config.Labels.service }}' dhh/app:999 | grep -x app || (echo \"Image dhh/app:999 is missing the 'service' label\" && exit 1)", output
end
end
test "create" do test "create" do
run_command("create").tap do |output| run_command("create").tap do |output|
assert_match /docker buildx create --use --name kamal-app-multiarch/, output assert_match /docker buildx create --use --name kamal-app-multiarch/, output

View File

@@ -42,16 +42,19 @@ class CliTestCase < ActiveSupport::TestCase
end end
def assert_hook_ran(hook, output, version:, service_version:, hosts:, command:, subcommand: nil, runtime: false) def assert_hook_ran(hook, output, version:, service_version:, hosts:, command:, subcommand: nil, runtime: false)
performer = `whoami`.strip whoami = `whoami`.chomp
performer = Kamal::Git.email.presence || whoami
service = service_version.split("@").first
assert_match "Running the #{hook} hook...\n", output assert_match "Running the #{hook} hook...\n", output
expected = %r{Running\s/usr/bin/env\s\.kamal/hooks/#{hook}\sas\s#{performer}@localhost\n\s expected = %r{Running\s/usr/bin/env\s\.kamal/hooks/#{hook}\sas\s#{whoami}@localhost\n\s
DEBUG\s\[[0-9a-f]*\]\sCommand:\s\(\sexport\s DEBUG\s\[[0-9a-f]*\]\sCommand:\s\(\sexport\s
KAMAL_RECORDED_AT=\"\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\dZ\"\s KAMAL_RECORDED_AT=\"\d\d\d\d-\d\d-\d\dT\d\d:\d\d:\d\dZ\"\s
KAMAL_PERFORMER=\"#{performer}\"\s KAMAL_PERFORMER=\"#{performer}\"\s
KAMAL_VERSION=\"#{version}\"\s KAMAL_VERSION=\"#{version}\"\s
KAMAL_SERVICE_VERSION=\"#{service_version}\"\s KAMAL_SERVICE_VERSION=\"#{service_version}\"\s
KAMAL_SERVICE=\"#{service}\"\s
KAMAL_HOSTS=\"#{hosts}\"\s KAMAL_HOSTS=\"#{hosts}\"\s
KAMAL_COMMAND=\"#{command}\"\s KAMAL_COMMAND=\"#{command}\"\s
#{"KAMAL_SUBCOMMAND=\\\"#{subcommand}\\\"\\s" if subcommand} #{"KAMAL_SUBCOMMAND=\\\"#{subcommand}\\\"\\s" if subcommand}
@@ -60,4 +63,12 @@ class CliTestCase < ActiveSupport::TestCase
assert_match expected, output assert_match expected, output
end end
def with_argv(*argv)
old_argv = ARGV
ARGV.replace(*argv)
yield
ensure
ARGV.replace(old_argv)
end
end end

View File

@@ -1,6 +1,9 @@
require_relative "cli_test_case" require_relative "cli_test_case"
class CliMainTest < CliTestCase class CliMainTest < CliTestCase
setup { @original_env = ENV.to_h.dup }
teardown { ENV.clear; ENV.update @original_env }
test "setup" do test "setup" do
invoke_options = { "config_file" => "test/fixtures/deploy_simple.yml", "version" => "999", "skip_hooks" => false } invoke_options = { "config_file" => "test/fixtures/deploy_simple.yml", "version" => "999", "skip_hooks" => false }
@@ -122,6 +125,11 @@ class CliMainTest < CliTestCase
.with(:docker, :buildx, :inspect, "kamal-app-multiarch", "> /dev/null") .with(:docker, :buildx, :inspect, "kamal-app-multiarch", "> /dev/null")
.returns("") .returns("")
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info)
.with(:docker, :info, "--format '{{index .RegistryConfig.Mirrors 0}}'")
.returns("")
.at_least_once
assert_raises(Kamal::Cli::LockError) do assert_raises(Kamal::Cli::LockError) do
run_command("deploy") run_command("deploy")
end end
@@ -155,6 +163,11 @@ class CliMainTest < CliTestCase
.with(:docker, :buildx, :inspect, "kamal-app-multiarch", "> /dev/null") .with(:docker, :buildx, :inspect, "kamal-app-multiarch", "> /dev/null")
.returns("") .returns("")
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info)
.with(:docker, :info, "--format '{{index .RegistryConfig.Mirrors 0}}'")
.returns("")
.at_least_once
assert_raises(SSHKit::Runner::ExecuteError) do assert_raises(SSHKit::Runner::ExecuteError) do
run_command("deploy") run_command("deploy")
end end
@@ -434,7 +447,7 @@ class CliMainTest < CliTestCase
end end
test "envify" do test "envify" do
with_test_dot_env_erb(contents: "HELLO=<%= 'world' %>") do with_test_dotenv(".env.erb": "HELLO=<%= 'world' %>") do
run_command("envify") run_command("envify")
assert_equal("HELLO=world", File.read(".env")) assert_equal("HELLO=world", File.read(".env"))
end end
@@ -448,14 +461,14 @@ class CliMainTest < CliTestCase
<% end -%> <% end -%>
EOF EOF
with_test_dot_env_erb(contents: file) do with_test_dotenv(".env.erb": file) do
run_command("envify") run_command("envify")
assert_equal("HELLO=world\nKEY=value\n", File.read(".env")) assert_equal("HELLO=world\nKEY=value\n", File.read(".env"))
end end
end end
test "envify with destination" do test "envify with destination" do
with_test_dot_env_erb(contents: "HELLO=<%= 'world' %>", file: ".env.world.erb") do with_test_dotenv(".env.world.erb": "HELLO=<%= 'world' %>") do
run_command("envify", "-d", "world", config_file: "deploy_for_dest") run_command("envify", "-d", "world", config_file: "deploy_for_dest")
assert_equal "HELLO=world", File.read(".env.world") assert_equal "HELLO=world", File.read(".env.world")
end end
@@ -470,6 +483,13 @@ class CliMainTest < CliTestCase
run_command("envify", "--skip-push") run_command("envify", "--skip-push")
end end
test "envify with clean env" do
with_test_dotenv(".env": "HELLO=already", ".env.erb": "HELLO=<%= ENV.fetch 'HELLO', 'never' %>") do
run_command("envify", "--skip-push")
assert_equal "HELLO=never", File.read(".env")
end
end
test "remove with confirmation" do test "remove with confirmation" do
run_command("remove", "-y", config_file: "deploy_with_accessories").tap do |output| run_command("remove", "-y", config_file: "deploy_with_accessories").tap do |output|
assert_match /docker container stop traefik/, output assert_match /docker container stop traefik/, output
@@ -517,19 +537,59 @@ class CliMainTest < CliTestCase
assert_equal Kamal::VERSION, version assert_equal Kamal::VERSION, version
end end
test "run an alias for details" do
Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:traefik:details")
Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:app:details")
Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:accessory:details", [ "all" ])
run_command("info", config_file: "deploy_with_aliases")
end
test "run an alias for a console" do
run_command("console", config_file: "deploy_with_aliases").tap do |output|
assert_match "docker exec app-console-999 bin/console on 1.1.1.5", output
assert_match "App Host: 1.1.1.5", output
end
end
test "run an alias for a console overriding role" do
run_command("console", "-r", "workers", config_file: "deploy_with_aliases").tap do |output|
assert_match "docker exec app-workers-999 bin/console on 1.1.1.3", output
assert_match "App Host: 1.1.1.3", output
end
end
test "run an alias for a console passing command" do
run_command("exec", "bin/job", config_file: "deploy_with_aliases").tap do |output|
assert_match "docker exec app-console-999 bin/job on 1.1.1.5", output
assert_match "App Host: 1.1.1.5", output
end
end
test "append to command with an alias" do
run_command("rails", "db:migrate:status", config_file: "deploy_with_aliases").tap do |output|
assert_match "docker exec app-console-999 rails db:migrate:status on 1.1.1.5", output
assert_match "App Host: 1.1.1.5", output
end
end
private private
def run_command(*command, config_file: "deploy_simple") def run_command(*command, config_file: "deploy_simple")
stdouted { Kamal::Cli::Main.start([ *command, "-c", "test/fixtures/#{config_file}.yml" ]) } with_argv([ *command, "-c", "test/fixtures/#{config_file}.yml" ]) do
stdouted { Kamal::Cli::Main.start }
end
end end
def with_test_dot_env_erb(contents:, file: ".env.erb") def with_test_dotenv(**files)
Dir.mktmpdir do |dir| Dir.mktmpdir do |dir|
fixtures_dup = File.join(dir, "test") fixtures_dup = File.join(dir, "test")
FileUtils.mkdir_p(fixtures_dup) FileUtils.mkdir_p(fixtures_dup)
FileUtils.cp_r("test/fixtures/", fixtures_dup) FileUtils.cp_r("test/fixtures/", fixtures_dup)
Dir.chdir(dir) do Dir.chdir(dir) do
File.write(file, contents) files.each do |filename, contents|
File.binwrite(filename.to_s, contents)
end
yield yield
end end
end end

View File

@@ -3,8 +3,8 @@ require_relative "cli_test_case"
class CliServerTest < CliTestCase class CliServerTest < CliTestCase
test "running a command with exec" do test "running a command with exec" do
SSHKit::Backend::Abstract.any_instance.stubs(:capture) SSHKit::Backend::Abstract.any_instance.stubs(:capture)
.with("date", verbosity: 1) .with("date", verbosity: 1)
.returns("Today") .returns("Today")
hosts = "1.1.1.1".."1.1.1.4" hosts = "1.1.1.1".."1.1.1.4"
run_command("exec", "date").tap do |output| run_command("exec", "date").tap do |output|
@@ -15,6 +15,20 @@ class CliServerTest < CliTestCase
end end
end end
test "running a command with exec multiple arguments" do
SSHKit::Backend::Abstract.any_instance.stubs(:capture)
.with("date -j", verbosity: 1)
.returns("Today")
hosts = "1.1.1.1".."1.1.1.4"
run_command("exec", "date", "-j").tap do |output|
hosts.map do |host|
assert_match "Running 'date -j' on #{hosts.to_a.join(', ')}...", output
assert_match "App Host: #{host}\nToday", output
end
end
end
test "bootstrap already installed" do test "bootstrap already installed" do
stub_setup stub_setup
SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:docker, "-v", raise_on_non_zero_exit: false).returns(true).at_least_once SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:docker, "-v", raise_on_non_zero_exit: false).returns(true).at_least_once

View File

@@ -12,7 +12,7 @@ class CommandsAuditorTest < ActiveSupport::TestCase
} }
@auditor = new_command @auditor = new_command
@performer = `whoami`.strip @performer = Kamal::Git.email.presence || `whoami`.chomp
@recorded_at = Time.now.utc.iso8601 @recorded_at = Time.now.utc.iso8601
end end

View File

@@ -200,6 +200,11 @@ class CommandsBuilderTest < ActiveSupport::TestCase
assert_equal [ "unix:///var/run/docker.sock", "ssh://host" ], command.config_context_hosts assert_equal [ "unix:///var/run/docker.sock", "ssh://host" ], command.config_context_hosts
end end
test "mirror count" do
command = new_builder_command
assert_equal "docker info --format '{{index .RegistryConfig.Mirrors 0}}'", command.first_mirror.join(" ")
end
private private
def new_builder_command(additional_config = {}) def new_builder_command(additional_config = {})
Kamal::Commands::Builder.new(Kamal::Configuration.new(@config.merge(additional_config), version: "123")) Kamal::Commands::Builder.new(Kamal::Configuration.new(@config.merge(additional_config), version: "123"))

View File

@@ -11,7 +11,7 @@ class CommandsHookTest < ActiveSupport::TestCase
traefik: { "args" => { "accesslog.format" => "json", "metrics.prometheus.buckets" => "0.1,0.3,1.2,5.0" } } traefik: { "args" => { "accesslog.format" => "json", "metrics.prometheus.buckets" => "0.1,0.3,1.2,5.0" } }
} }
@performer = `whoami`.strip @performer = Kamal::Git.email.presence || `whoami`.chomp
@recorded_at = Time.now.utc.iso8601 @recorded_at = Time.now.utc.iso8601
end end
@@ -22,7 +22,8 @@ class CommandsHookTest < ActiveSupport::TestCase
"KAMAL_RECORDED_AT" => @recorded_at, "KAMAL_RECORDED_AT" => @recorded_at,
"KAMAL_PERFORMER" => @performer, "KAMAL_PERFORMER" => @performer,
"KAMAL_VERSION" => "123", "KAMAL_VERSION" => "123",
"KAMAL_SERVICE_VERSION" => "app@123" } } "KAMAL_SERVICE_VERSION" => "app@123",
"KAMAL_SERVICE" => "app" } }
], new_command.run("foo") ], new_command.run("foo")
end end
@@ -33,7 +34,8 @@ class CommandsHookTest < ActiveSupport::TestCase
"KAMAL_RECORDED_AT" => @recorded_at, "KAMAL_RECORDED_AT" => @recorded_at,
"KAMAL_PERFORMER" => @performer, "KAMAL_PERFORMER" => @performer,
"KAMAL_VERSION" => "123", "KAMAL_VERSION" => "123",
"KAMAL_SERVICE_VERSION" => "app@123" } } "KAMAL_SERVICE_VERSION" => "app@123",
"KAMAL_SERVICE" => "app" } }
], new_command(hooks_path: "custom/hooks/path").run("foo") ], new_command(hooks_path: "custom/hooks/path").run("foo")
end end

View File

@@ -111,6 +111,11 @@ class CommandsTraefikTest < ActiveSupport::TestCase
new_command.run.join(" ") new_command.run.join(" ")
end end
test "run with args array" do
@config[:traefik]["args"] = { "entrypoints.web.forwardedheaders.trustedips" => %w[ 127.0.0.1 127.0.0.2 ] }
assert_equal "docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --env-file .kamal/env/traefik/traefik.env --log-opt max-size=\"10m\" --label traefik.http.routers.catchall.entryPoints=\"http\" --label traefik.http.routers.catchall.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.routers.catchall.service=\"unavailable\" --label traefik.http.routers.catchall.priority=\"1\" --label traefik.http.services.unavailable.loadbalancer.server.port=\"0\" traefik:test --providers.docker --log.level=\"DEBUG\" --entrypoints.web.forwardedheaders.trustedips=\"127.0.0.1\" --entrypoints.web.forwardedheaders.trustedips=\"127.0.0.2\"", new_command.run.join(" ")
end
test "traefik start" do test "traefik start" do
assert_equal \ assert_equal \
"docker container start traefik", "docker container start traefik",

View File

@@ -94,7 +94,7 @@ class ConfigurationValidationTest < ActiveSupport::TestCase
assert_error "builder/remote: unknown key: foo", builder: { "remote" => { "foo" => "bar" } } assert_error "builder/remote: unknown key: foo", builder: { "remote" => { "foo" => "bar" } }
assert_error "builder/local: unknown key: foo", builder: { "local" => { "foo" => "bar" } } assert_error "builder/local: unknown key: foo", builder: { "local" => { "foo" => "bar" } }
assert_error "builder/remote/arch: should be a string", builder: { "remote" => { "arch" => [] } } assert_error "builder/remote/arch: should be a string", builder: { "remote" => { "arch" => [] } }
assert_error "builder/args/foo: should be a string", builder: { "args" => { "foo" => [] } } assert_error "builder/args: should be a hash", builder: { "args" => [ "foo" ] }
assert_error "builder/cache/options: should be a string", builder: { "cache" => { "options" => [] } } assert_error "builder/cache/options: should be a string", builder: { "cache" => { "options" => [] } }
end end

View File

@@ -344,4 +344,12 @@ class ConfigurationTest < ActiveSupport::TestCase
assert_raises(Kamal::ConfigurationError) { Kamal::Configuration.new(@deploy_with_roles.merge(retain_containers: 0)) } assert_raises(Kamal::ConfigurationError) { Kamal::Configuration.new(@deploy_with_roles.merge(retain_containers: 0)) }
end end
test "extensions" do
dest_config_file = Pathname.new(File.expand_path("fixtures/deploy_with_extensions.yml", __dir__))
config = Kamal::Configuration.create_from config_file: dest_config_file
assert_equal config.role(:web_tokyo).running_traefik?, true
assert_equal config.role(:web_chicago).running_traefik?, true
end
end end

21
test/fixtures/deploy_with_aliases.yml vendored Normal file
View File

@@ -0,0 +1,21 @@
service: app
image: dhh/app
servers:
web:
- 1.1.1.1
- 1.1.1.2
workers:
hosts:
- 1.1.1.3
- 1.1.1.4
console:
hosts:
- 1.1.1.5
registry:
username: user
password: pw
aliases:
info: details
console: app exec --reuse -p -r console "bin/console"
exec: app exec --reuse -p -r console
rails: app exec --reuse -p -r console rails

View File

@@ -0,0 +1,24 @@
x-web: &web
traefik: true
service: app
image: dhh/app
servers:
web_chicago:
<<: *web
hosts:
- 1.1.1.1
- 1.1.1.2
web_tokyo:
<<: *web
hosts:
- 1.1.1.3
- 1.1.1.4
env:
REDIS_URL: redis://x/y
registry:
server: registry.digitalocean.com
username: user
password: pw
primary_role: web_tokyo

View File

@@ -1,4 +1,3 @@
version: "3.7"
name: "kamal-test" name: "kamal-test"
volumes: volumes:
@@ -30,8 +29,6 @@ services:
context: docker/registry context: docker/registry
environment: environment:
- REGISTRY_HTTP_ADDR=0.0.0.0:4443 - REGISTRY_HTTP_ADDR=0.0.0.0:4443
- REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt
- REGISTRY_HTTP_TLS_KEY=/certs/domain.key
volumes: volumes:
- shared:/shared - shared:/shared
- registry:/var/lib/registry/ - registry:/var/lib/registry/

View File

@@ -22,7 +22,6 @@ COPY app_with_roles/ app_with_roles/
RUN rm -rf /root/.ssh RUN rm -rf /root/.ssh
RUN ln -s /shared/ssh /root/.ssh RUN ln -s /shared/ssh /root/.ssh
RUN mkdir -p /etc/docker/certs.d/registry:4443 && ln -s /shared/certs/domain.crt /etc/docker/certs.d/registry:4443/ca.crt
RUN git config --global user.email "deployer@example.com" RUN git config --global user.email "deployer@example.com"
RUN git config --global user.name "Deployer" RUN git config --global user.name "Deployer"

View File

@@ -1,8 +1,4 @@
#!/bin/sh #!/bin/sh
echo "About to lock..." echo "About to lock..."
if [ "$KAMAL_HOSTS" != "vm1,vm2" ]; then
echo "Expected hosts to be 'vm1,vm2', got $KAMAL_HOSTS"
exit 1
fi
mkdir -p /tmp/${TEST_ID} && touch /tmp/${TEST_ID}/pre-connect mkdir -p /tmp/${TEST_ID} && touch /tmp/${TEST_ID}/pre-connect

View File

@@ -1,8 +1,4 @@
#!/bin/sh #!/bin/sh
echo "About to lock..." echo "About to lock..."
if [ "$KAMAL_HOSTS" != "vm1,vm2,vm3" ]; then
echo "Expected hosts to be 'vm1,vm2,vm3', got $KAMAL_HOSTS"
exit 1
fi
mkdir -p /tmp/${TEST_ID} && touch /tmp/${TEST_ID}/pre-connect mkdir -p /tmp/${TEST_ID} && touch /tmp/${TEST_ID}/pre-connect

View File

@@ -37,3 +37,7 @@ accessories:
- web - web
stop_wait_time: 1 stop_wait_time: 1
readiness_delay: 0 readiness_delay: 0
aliases:
whome: version
worker_hostname: app exec -r workers -q --reuse hostname
uname: server exec -q -p uname

View File

@@ -1,5 +1,5 @@
#!/bin/bash #!/bin/bash
dockerd --max-concurrent-downloads 1 & dockerd --max-concurrent-downloads 1 --insecure-registry registry:4443 &
exec sleep infinity exec sleep infinity

View File

@@ -1,5 +1,3 @@
#!/bin/sh #!/bin/sh
while [ ! -f /certs/domain.crt ]; do sleep 1; done
exec /entrypoint.sh /etc/docker/registry/config.yml exec /entrypoint.sh /etc/docker/registry/config.yml

View File

@@ -10,8 +10,6 @@ RUN mkdir ssh && \
COPY registry-dns.conf . COPY registry-dns.conf .
COPY boot.sh . COPY boot.sh .
RUN mkdir certs && openssl req -newkey rsa:4096 -nodes -sha256 -keyout certs/domain.key -x509 -days 365 -out certs/domain.crt -subj '/CN=registry' -extensions EXT -config registry-dns.conf
HEALTHCHECK --interval=1s CMD pgrep sleep HEALTHCHECK --interval=1s CMD pgrep sleep
CMD ["./boot.sh"] CMD ["./boot.sh"]

View File

@@ -5,7 +5,6 @@ WORKDIR /work
RUN apt-get update --fix-missing && apt-get -y install openssh-client openssh-server docker.io RUN apt-get update --fix-missing && apt-get -y install openssh-client openssh-server docker.io
RUN mkdir /root/.ssh && ln -s /shared/ssh/id_rsa.pub /root/.ssh/authorized_keys RUN mkdir /root/.ssh && ln -s /shared/ssh/id_rsa.pub /root/.ssh/authorized_keys
RUN mkdir -p /etc/docker/certs.d/registry:4443 && ln -s /shared/certs/domain.crt /etc/docker/certs.d/registry:4443/ca.crt
RUN echo "HOST_TOKEN=abcd" >> /etc/environment RUN echo "HOST_TOKEN=abcd" >> /etc/environment

View File

@@ -4,6 +4,6 @@ while [ ! -f /root/.ssh/authorized_keys ]; do echo "Waiting for ssh keys"; sleep
service ssh restart service ssh restart
dockerd --max-concurrent-downloads 1 & dockerd --max-concurrent-downloads 1 --insecure-registry registry:4443 &
exec sleep infinity exec sleep infinity

View File

@@ -82,6 +82,22 @@ class MainTest < IntegrationTest
assert_equal({ "cmd"=>"wget -qO- http://localhost > /dev/null || exit 1", "interval"=>"1s", "max_attempts"=>3, "port"=>3000, "path"=>"/up", "cord"=>"/tmp/kamal-cord", "log_lines"=>50 }, config[:healthcheck]) assert_equal({ "cmd"=>"wget -qO- http://localhost > /dev/null || exit 1", "interval"=>"1s", "max_attempts"=>3, "port"=>3000, "path"=>"/up", "cord"=>"/tmp/kamal-cord", "log_lines"=>50 }, config[:healthcheck])
end end
test "aliases" do
@app = "app_with_roles"
kamal :envify
kamal :deploy
output = kamal :whome, capture: true
assert_equal Kamal::VERSION, output
output = kamal :worker_hostname, capture: true
assert_match /App Host: vm3\nvm3-[0-9a-f]{12}$/, output
output = kamal :uname, "-o", capture: true
assert_match "App Host: vm1\nGNU/Linux", output
end
test "setup and remove" do test "setup and remove" do
# Check remove completes when nothing has been setup yet # Check remove completes when nothing has been setup yet
kamal :remove, "-y" kamal :remove, "-y"