Compare commits

..

32 Commits

Author SHA1 Message Date
Donal McBreen
b21af0f5b3 Merge remote-tracking branch 'origin/revert-905-simplify-builders-config' into proxy-experimental-and-revert-905-simplify-builders-config
* origin/revert-905-simplify-builders-config:
  Revert "Simplify builders config"
2024-08-29 20:18:17 +01:00
Donal McBreen
459ba95bbf Revert "Simplify builders config" 2024-08-29 20:16:34 +01:00
Donal McBreen
01d08738ff Fix merge error 2024-08-29 09:49:03 +01:00
Donal McBreen
19cf94457f Read buffer not buffering 2024-08-29 09:49:03 +01:00
Donal McBreen
269a5ff3e6 Set request and response headers 2024-08-29 09:49:03 +01:00
Donal McBreen
c8adda1550 Split buffer requests/responses 2024-08-29 09:49:03 +01:00
Donal McBreen
d891eb91e4 Add forward headers support 2024-08-29 09:49:03 +01:00
Donal McBreen
268ec1c6e0 Set extra fields 2024-08-29 09:49:03 +01:00
Donal McBreen
42fdbd98cb Add kamal-proxy in experimental mode
The proxy can be enabled via the config:

```
proxy:
  enabled: true
  hosts:
    - 10.0.0.1
    - 10.0.0.2
```

This will enable the proxy and cause it to be run on the hosts listed
under `hosts`, after running `kamal proxy reboot`.

Enabling the proxy disables `kamal traefik` commands and replaces them
with `kamal proxy` ones. However only the marked hosts will run the
kamal-proxy container, the rest will run Traefik as before.
2024-08-29 09:49:03 +01:00
Donal McBreen
6adf3c117f Merge pull request #905 from basecamp/simplify-builders-config
Simplify builders config
2024-08-29 09:28:51 +01:00
Donal McBreen
9f0b10425c Fix aliases tests 2024-08-29 09:16:07 +01:00
Donal McBreen
5f2384f123 Use docker info to get arch 2024-08-29 08:46:18 +01:00
Donal McBreen
eab7d3adc5 Keep buildx build, in case of old docker versions which don't default to buildkit 2024-08-29 08:45:51 +01:00
Donal McBreen
d2d0223c37 Require an arch to be set, and default to amd64 in the template 2024-08-29 08:45:51 +01:00
Donal McBreen
56268d724d Simplify the builders configuration
1. Add driver as an option, defaulting to `docker-container`. For a
   "native" build you can set it to `docker`
2. Set arch as a array of architectures to build for, defaulting to
   `[ "amd64", "arm64" ]` unless you are using the docker driver in
   which case we default to not setting a platform
3. Remote is now just a connection string for the remote builder
4. If remote is set, we only use it for non-local arches, if we are
   only building for the local arch, we'll ignore it.

Examples:

On arm64, build for arm64 locally, amd64 remotely or
On amd64, build for amd64 locally, arm64 remotely:

```yaml
builder:
  remote: ssh://docker@docker-builder
```

On arm64, build amd64 on remote,
On amd64 build locally:

```yaml
builder:
  arch:
    - amd64
  remote:
    host: ssh://docker@docker-builder
```

Build amd64 on local:

```yaml
builder:
  arch:
    - amd64
```

Use docker driver, building for local arch:

```yaml
builder:
  driver: docker
```
2024-08-29 08:45:48 +01:00
Donal McBreen
cffb6c3d7e Allow the driver to be set 2024-08-29 08:44:11 +01:00
Donal McBreen
bd1726f305 docker buildx build -> docker build 2024-08-29 08:44:11 +01:00
Donal McBreen
7ddb122a22 Get tests passing 2024-08-29 08:44:11 +01:00
Donal McBreen
98c951bbdb Simplfy choosing a builder 2024-08-29 08:44:11 +01:00
Donal McBreen
374c117b79 Validate multiarch configuration
Remote and local are only allowed when multiarch is enabled.
Remote requires a host and arch, local only requires an arch.
2024-08-29 08:44:11 +01:00
Donal McBreen
d6a5cf3c78 Rip out context_hosts checks
The remote host is now encoded in the builder name so we don't need
to check it. We'll just do an inspect to confirm the builder exists.
2024-08-29 08:44:11 +01:00
Donal McBreen
2aeabda455 Move multiarch remote builder to hybrid builder
Include the host name in the builder name, so we can have one builder
per host/arch across all kamal projects.

Inherit from the remote builder. The difference in the hybrid builder
is that we create a local buildx instance and append the remote context
to it.
2024-08-29 08:44:11 +01:00
Donal McBreen
c048c097ed Create a context for local builds
This ensures we use the docker-container driver and not whatever the
local default is.
2024-08-29 08:44:11 +01:00
Donal McBreen
ed148628fb Local build doesn't need a builder 2024-08-29 08:44:11 +01:00
Donal McBreen
d48080c772 Dump native builder
We already ensure that buildx is installed, so let's always use it.
2024-08-29 08:44:11 +01:00
Donal McBreen
3f64338929 Move native remote to just remote
It's just a remote builder, that will build whichever platform is asked
for, so let's remove the "native" part.

We'll also remove the service name from the builder name, so multiple
services can share the same builder.
2024-08-29 08:44:11 +01:00
Donal McBreen
0ab838bc25 Combine multiarch and native/cache builders
Combine the two builders, as they are almost identical. The only
difference was whether the platforms were set.

The native cached builder wasn't using the context it created, so now
we do.

We'll set the driver to `docker-container` - it seems to be the default
but the Docker docs claim it is `docker`.
2024-08-29 08:44:11 +01:00
Donal McBreen
b7382ceeaf Merge pull request #912 from basecamp/alias
Add aliases to Kamal
2024-08-29 08:43:35 +01:00
Donal McBreen
69367fbc6b Merge pull request #917 from basecamp/v2.0-alpha
Switch the version on main to 2.0.0.alpha
2024-08-29 08:43:19 +01:00
Donal McBreen
2515bd705c Switch the version on main to 2.0.0.alpha
All development is now for the 2.0.0 release.
2024-08-29 08:33:21 +01:00
Donal McBreen
579e169be2 Allow multiple arguments for exec commands
If you can have an alias like:

```
aliases:
  rails: app exec -p rails
```

Then `kamal rails db:migrate:status` will execute
`kamal app exec -p rails db:migrate:status`.

So this works, we'll allow multiple arguments `app exec` and
`server exec` to accept multiple arguments.

The arguments are combined by simply joining them with a space. This
means that these are equivalent:

```
kamal app exec -p rails db:migrate:status
kamal app exec -p "rails db:migrate:status"
```

If you want to pass an argument with spaces, you'll need to quote it:

```
kamal app exec -p "git commit -am \"My comment\""
kamal app exec -p git commit -am "\"My comment\""
```
2024-08-28 10:58:25 +01:00
Donal McBreen
b8af719bb7 Add aliases to Kamal
Aliases are defined in the configuration file under the `aliases` key.

The configuration is a map of alias name to command. When we run the
command the we just do a literal replacement of the alias with the
string.

So if we have:

```yaml
aliases:
  console: app exec -r console -i --reuse "rails console"
```

Then running `kamal console -r workers` will run the command

```sh
$ kamal app exec -r console -i --reuse "rails console" -r workers
```

Because of the order Thor parses the arguments, this allows us to
override the role from the alias command.

There might be cases where we need to munge the command a bit more but
that would involve getting into Thor command parsing internals,
which are complicated and possibly subject to change.

There's a chance that your aliases could conflict with future built-in
commands, but there's not likely to be many of those and if it happens
you'll get a validation error when you upgrade.

Thanks to @dhnaranjo for the idea!
2024-08-26 10:47:43 +01:00
52 changed files with 1204 additions and 306 deletions

View File

@@ -3,7 +3,6 @@ on:
push: push:
branches: branches:
- main - main
- 1-8-stable
pull_request: pull_request:
jobs: jobs:
rubocop: rubocop:

View File

@@ -1,7 +1,7 @@
PATH PATH
remote: . remote: .
specs: specs:
kamal (1.9.0) kamal (2.0.0.alpha)
activesupport (>= 7.0) activesupport (>= 7.0)
base64 (~> 0.2) base64 (~> 0.2)
bcrypt_pbkdf (~> 1.0) bcrypt_pbkdf (~> 1.0)
@@ -10,7 +10,7 @@ PATH
ed25519 (~> 1.2) ed25519 (~> 1.2)
net-ssh (~> 7.0) net-ssh (~> 7.0)
sshkit (>= 1.23.0, < 2.0) sshkit (>= 1.23.0, < 2.0)
thor (~> 1.2) thor (~> 1.3)
zeitwerk (~> 2.5) zeitwerk (~> 2.5)
GEM GEM

View File

@@ -17,12 +17,14 @@ end
DOCS = { DOCS = {
"accessory" => "Accessories", "accessory" => "Accessories",
"alias" => "Aliases",
"boot" => "Booting", "boot" => "Booting",
"builder" => "Builders", "builder" => "Builders",
"configuration" => "Configuration overview", "configuration" => "Configuration overview",
"env" => "Environment variables", "env" => "Environment variables",
"healthcheck" => "Healthchecks", "healthcheck" => "Healthchecks",
"logging" => "Logging", "logging" => "Logging",
"proxy" => "Proxy (Experimental)",
"registry" => "Docker Registry", "registry" => "Docker Registry",
"role" => "Roles", "role" => "Roles",
"servers" => "Servers", "servers" => "Servers",
@@ -67,26 +69,27 @@ class DocWriter
output.puts output.puts
place = :new_section place = :new_section
elsif line =~ /^ *#/ elsif line =~ /^ *#/
generate_line(line, place: place) generate_line(line, heading: place == :new_section)
place = :in_section place = :in_section
else else
output.puts "```yaml" output.puts "```yaml"
output.print line output.puts line
place = :in_yaml place = :in_yaml
end end
when :in_yaml when :in_yaml, :in_empty_line_yaml
if line =~ /^ *#/ if line =~ /^ *#/
output.puts "```" output.puts "```"
generate_line(line, place: :new_section) generate_line(line, heading: place == :in_empty_line_yaml)
place = :in_section place = :in_section
elsif line.empty?
place = :in_empty_line_yaml
else else
output.puts output.puts line
output.print line
end end
end end
end end
output.puts "\n```" if place == :in_yaml output.puts "```" if place == :in_yaml
end end
def generate_header def generate_header
@@ -98,7 +101,7 @@ class DocWriter
output.puts output.puts
end end
def generate_line(line, place: :in_section) def generate_line(line, heading: false)
line = line.gsub(/^ *#\s?/, "") line = line.gsub(/^ *#\s?/, "")
if line =~ /(.*)kamal docs ([a-z]*)(.*)/ if line =~ /(.*)kamal docs ([a-z]*)(.*)/
@@ -109,7 +112,7 @@ class DocWriter
line = "#{$1}[#{titlify($2.split("/").last)}](#{$2})#{$3}" line = "#{$1}[#{titlify($2.split("/").last)}](#{$2})#{$3}"
end end
if place == :new_section if heading
output.puts "## [#{line}](##{linkify(line)})" output.puts "## [#{line}](##{linkify(line)})"
else else
output.puts line output.puts line

View File

@@ -14,7 +14,7 @@ Gem::Specification.new do |spec|
spec.add_dependency "activesupport", ">= 7.0" spec.add_dependency "activesupport", ">= 7.0"
spec.add_dependency "sshkit", ">= 1.23.0", "< 2.0" spec.add_dependency "sshkit", ">= 1.23.0", "< 2.0"
spec.add_dependency "net-ssh", "~> 7.0" spec.add_dependency "net-ssh", "~> 7.0"
spec.add_dependency "thor", "~> 1.2" spec.add_dependency "thor", "~> 1.3"
spec.add_dependency "dotenv", "~> 2.8" spec.add_dependency "dotenv", "~> 2.8"
spec.add_dependency "zeitwerk", "~> 2.5" spec.add_dependency "zeitwerk", "~> 2.5"
spec.add_dependency "ed25519", "~> 1.2" spec.add_dependency "ed25519", "~> 1.2"

View File

@@ -1,4 +1,5 @@
module Kamal::Cli module Kamal::Cli
class BootError < StandardError; end
class HookError < StandardError; end class HookError < StandardError; end
class LockError < StandardError; end class LockError < StandardError; end
end end

View File

@@ -222,25 +222,6 @@ class Kamal::Cli::Accessory < Kamal::Cli::Base
end end
end end
desc "downgrade", "Downgrade accessories from Kamal 2 to 1.9"
option :rolling, type: :boolean, default: false, desc: "Upgrade one host at a time"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def downgrade(name)
confirming "This will restart all accessories" do
with_lock do
host_groups = options[:rolling] ? KAMAL.accessory_hosts : [ KAMAL.accessory_hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
KAMAL.with_specific_hosts(hosts) do
say "Downgrading #{name} accessories on #{host_list}...", :magenta
reboot name
say "Downgraded #{name} accessories on #{host_list}...", :magenta
end
end
end
end
end
private private
def with_accessory(name) def with_accessory(name)
if KAMAL.config.accessory(name) if KAMAL.config.accessory(name)

View File

@@ -0,0 +1,9 @@
class Kamal::Cli::Alias::Command < Thor::DynamicCommand
def run(instance, args = [])
if (_alias = KAMAL.config.aliases[name])
Kamal::Cli::Main.start(Shellwords.split(_alias.command) + ARGV[1..-1])
else
super
end
end
end

View File

@@ -38,8 +38,17 @@ class Kamal::Cli::App < Kamal::Cli::Base
roles = KAMAL.roles_on(host) roles = KAMAL.roles_on(host)
roles.each do |role| roles.each do |role|
app = KAMAL.app(role: role, host: host)
execute *KAMAL.auditor.record("Started app version #{KAMAL.config.version}"), verbosity: :debug execute *KAMAL.auditor.record("Started app version #{KAMAL.config.version}"), verbosity: :debug
execute *KAMAL.app(role: role, host: host).start, raise_on_non_zero_exit: false execute *app.start, raise_on_non_zero_exit: false
if role.running_traefik? && KAMAL.proxy_host?(host)
version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
endpoint = capture_with_info(*app.container_endpoint(version: version)).strip
raise Kamal::Cli::BootError, "Failed to get endpoint for #{role} on #{host}, did the container boot?" if endpoint.empty?
execute *KAMAL.proxy.deploy(role.container_prefix, target: endpoint)
end
end end
end end
end end
@@ -52,8 +61,18 @@ class Kamal::Cli::App < Kamal::Cli::Base
roles = KAMAL.roles_on(host) roles = KAMAL.roles_on(host)
roles.each do |role| roles.each do |role|
app = KAMAL.app(role: role, host: host)
execute *KAMAL.auditor.record("Stopped app", role: role), verbosity: :debug execute *KAMAL.auditor.record("Stopped app", role: role), verbosity: :debug
execute *KAMAL.app(role: role, host: host).stop, raise_on_non_zero_exit: false
if role.running_traefik? && KAMAL.proxy_host?(host)
version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
endpoint = capture_with_info(*app.container_endpoint(version: version)).strip
if endpoint.present?
execute *KAMAL.proxy.remove(role.container_prefix, target: endpoint), raise_on_non_zero_exit: false
end
end
execute *app.stop, raise_on_non_zero_exit: false
end end
end end
end end
@@ -71,11 +90,12 @@ class Kamal::Cli::App < Kamal::Cli::Base
end end
end end
desc "exec [CMD]", "Execute a custom command on servers within the app container (use --help to show options)" desc "exec [CMD...]", "Execute a custom command on servers within the app container (use --help to show options)"
option :interactive, aliases: "-i", type: :boolean, default: false, desc: "Execute command over ssh for an interactive shell (use for console/bash)" option :interactive, aliases: "-i", type: :boolean, default: false, desc: "Execute command over ssh for an interactive shell (use for console/bash)"
option :reuse, type: :boolean, default: false, desc: "Reuse currently running container instead of starting a new one" option :reuse, type: :boolean, default: false, desc: "Reuse currently running container instead of starting a new one"
option :env, aliases: "-e", type: :hash, desc: "Set environment variables for the command" option :env, aliases: "-e", type: :hash, desc: "Set environment variables for the command"
def exec(cmd) def exec(*cmd)
cmd = Kamal::Utils.join_commands(cmd)
env = options[:env] env = options[:env]
case case
when options[:interactive] && options[:reuse] when options[:interactive] && options[:reuse]

View File

@@ -45,11 +45,22 @@ class Kamal::Cli::App::Boot
def start_new_version def start_new_version
audit "Booted app version #{version}" audit "Booted app version #{version}"
execute *app.tie_cord(role.cord_host_file) if uses_cord?
hostname = "#{host.to_s[0...51].gsub(/\.+$/, '')}-#{SecureRandom.hex(6)}" hostname = "#{host.to_s[0...51].gsub(/\.+$/, '')}-#{SecureRandom.hex(6)}"
execute *app.run(hostname: hostname)
Kamal::Cli::Healthcheck::Poller.wait_for_healthy(pause_after_ready: true) { capture_with_info(*app.status(version: version)) } if proxy_host?
execute *app.run_for_proxy(hostname: hostname)
if running_traefik?
endpoint = capture_with_info(*app.container_endpoint(version: version)).strip
raise Kamal::Cli::BootError, "Failed to get endpoint for #{role} on #{host}, did the container boot?" if endpoint.empty?
execute *KAMAL.proxy.deploy(role.container_prefix, target: endpoint)
else
Kamal::Cli::Healthcheck::Poller.wait_for_healthy(pause_after_ready: true) { capture_with_info(*app.status(version: version)) }
end
else
execute *app.tie_cord(role.cord_host_file) if uses_cord?
execute *app.run(hostname: hostname)
Kamal::Cli::Healthcheck::Poller.wait_for_healthy(pause_after_ready: true) { capture_with_info(*app.status(version: version)) }
end
end end
def stop_new_version def stop_new_version
@@ -57,7 +68,7 @@ class Kamal::Cli::App::Boot
end end
def stop_old_version(version) def stop_old_version(version)
if uses_cord? if uses_cord? && !proxy_host?
cord = capture_with_info(*app.cord(version: version), raise_on_non_zero_exit: false).strip cord = capture_with_info(*app.cord(version: version), raise_on_non_zero_exit: false).strip
if cord.present? if cord.present?
execute *app.cut_cord(cord) execute *app.cut_cord(cord)
@@ -116,4 +127,8 @@ class Kamal::Cli::App::Boot
def queuer? def queuer?
barrier && !barrier_role? barrier && !barrier_role?
end end
def proxy_host?
KAMAL.proxy_host?(host)
end
end end

View File

@@ -6,7 +6,8 @@ module Kamal::Cli
class Base < Thor class Base < Thor
include SSHKit::DSL include SSHKit::DSL
def self.exit_on_failure?() true end def self.exit_on_failure?() false end
def self.dynamic_command_class() Kamal::Cli::Alias::Command end
class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging" class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging"
class_option :quiet, type: :boolean, aliases: "-q", desc: "Minimal logging" class_option :quiet, type: :boolean, aliases: "-q", desc: "Minimal logging"
@@ -22,8 +23,14 @@ module Kamal::Cli
class_option :skip_hooks, aliases: "-H", type: :boolean, default: false, desc: "Don't run hooks" class_option :skip_hooks, aliases: "-H", type: :boolean, default: false, desc: "Don't run hooks"
def initialize(*) def initialize(args = [], local_options = {}, config = {})
super if config[:current_command].is_a?(Kamal::Cli::Alias::Command)
# When Thor generates a dynamic command, it doesn't attempt to parse the arguments.
# For our purposes, it means the arguments are passed in args rather than local_options.
super([], args, config)
else
super
end
@original_env = ENV.to_h.dup @original_env = ENV.to_h.dup
load_env load_env
initialize_commander(options_with_subcommand_class_options) initialize_commander(options_with_subcommand_class_options)
@@ -206,10 +213,6 @@ module Kamal::Cli
instance_variable_get("@_invocations").first instance_variable_get("@_invocations").first
end end
def reset_invocation(cli_class)
instance_variable_get("@_invocations")[cli_class].pop
end
def ensure_run_and_locks_directory def ensure_run_and_locks_directory
on(KAMAL.hosts) do on(KAMAL.hosts) do
execute(*KAMAL.server.ensure_run_directory) execute(*KAMAL.server.ensure_run_directory)

View File

@@ -39,8 +39,13 @@ class Kamal::Cli::Main < Kamal::Cli::Base
with_lock do with_lock do
run_hook "pre-deploy" run_hook "pre-deploy"
say "Ensure Traefik is running...", :magenta if KAMAL.config.proxy.enabled?
invoke "kamal:cli:traefik:boot", [], invoke_options say "Ensure Traefik/kamal-proxy is running...", :magenta
invoke "kamal:cli:proxy:boot", [], invoke_options
else
say "Ensure Traefik is running...", :magenta
invoke "kamal:cli:traefik:boot", [], invoke_options
end
say "Detect stale containers...", :magenta say "Detect stale containers...", :magenta
invoke "kamal:cli:app:stale_containers", [], invoke_options.merge(stop: true) invoke "kamal:cli:app:stale_containers", [], invoke_options.merge(stop: true)
@@ -108,7 +113,11 @@ class Kamal::Cli::Main < Kamal::Cli::Base
desc "details", "Show details about all containers" desc "details", "Show details about all containers"
def details def details
invoke "kamal:cli:traefik:details" if KAMAL.config.proxy.enabled?
invoke "kamal:cli:proxy:details"
else
invoke "kamal:cli:traefik:details"
end
invoke "kamal:cli:app:details" invoke "kamal:cli:app:details"
invoke "kamal:cli:accessory:details", [ "all" ] invoke "kamal:cli:accessory:details", [ "all" ]
end end
@@ -209,7 +218,11 @@ class Kamal::Cli::Main < Kamal::Cli::Base
def remove def remove
confirming "This will remove all containers and images. Are you sure?" do confirming "This will remove all containers and images. Are you sure?" do
with_lock do with_lock do
invoke "kamal:cli:traefik:remove", [], options.without(:confirmed) if KAMAL.config.proxy.enabled?
invoke "kamal:cli:proxy:remove", [], options.without(:confirmed)
else
invoke "kamal:cli:traefik:remove", [], options.without(:confirmed)
end
invoke "kamal:cli:app:remove", [], options.without(:confirmed) invoke "kamal:cli:app:remove", [], options.without(:confirmed)
invoke "kamal:cli:accessory:remove", [ "all" ], options invoke "kamal:cli:accessory:remove", [ "all" ], options
invoke "kamal:cli:registry:logout", [], options.without(:confirmed).merge(skip_local: true) invoke "kamal:cli:registry:logout", [], options.without(:confirmed).merge(skip_local: true)
@@ -217,37 +230,6 @@ class Kamal::Cli::Main < Kamal::Cli::Base
end end
end end
desc "downgrade", "Downgrade from Kamal 2 to 1.9"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
option :rolling, type: :boolean, default: false, desc: "Downgrade one host at a time"
def downgrade
confirming "This will replace Traefik with kamal-proxy and restart all accessories" do
with_lock do
if options[:rolling]
(KAMAL.hosts | KAMAL.accessory_hosts).each do |host|
KAMAL.with_specific_hosts(host) do
say "Downgrading #{host}...", :magenta
if KAMAL.hosts.include?(host)
invoke "kamal:cli:traefik:downgrade", [], options.merge(confirmed: true, rolling: false)
reset_invocation(Kamal::Cli::Traefik)
end
if KAMAL.accessory_hosts.include?(host)
invoke "kamal:cli:accessory:downgrade", [ "all" ], options.merge(confirmed: true, rolling: false)
reset_invocation(Kamal::Cli::Accessory)
end
say "Downgraded #{host}", :magenta
end
end
else
say "Downgrading all hosts...", :magenta
invoke "kamal:cli:traefik:downgrade", [], options.merge(confirmed: true)
invoke "kamal:cli:accessory:downgrade", [ "all" ], options.merge(confirmed: true)
say "Downgraded all hosts", :magenta
end
end
end
end
desc "version", "Show Kamal version" desc "version", "Show Kamal version"
def version def version
puts Kamal::VERSION puts Kamal::VERSION
@@ -268,6 +250,9 @@ class Kamal::Cli::Main < Kamal::Cli::Base
desc "lock", "Manage the deploy lock" desc "lock", "Manage the deploy lock"
subcommand "lock", Kamal::Cli::Lock subcommand "lock", Kamal::Cli::Lock
desc "proxy", "Prune old application images and containers"
subcommand "proxy", Kamal::Cli::Proxy
desc "prune", "Prune old application images and containers" desc "prune", "Prune old application images and containers"
subcommand "prune", Kamal::Cli::Prune subcommand "prune", Kamal::Cli::Prune

160
lib/kamal/cli/proxy.rb Normal file
View File

@@ -0,0 +1,160 @@
class Kamal::Cli::Proxy < Kamal::Cli::Base
desc "boot", "Boot proxy on servers"
def boot
raise_unless_kamal_proxy_enabled!
with_lock do
on(KAMAL.traefik_hosts) do |host|
execute *KAMAL.registry.login
execute *KAMAL.traefik_or_proxy(host).start_or_run
end
end
end
desc "reboot", "Reboot proxy on servers (stop container, remove container, start new container)"
option :rolling, type: :boolean, default: false, desc: "Reboot proxy on hosts in sequence, rather than in parallel"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def reboot
raise_unless_kamal_proxy_enabled!
confirming "This will cause a brief outage on each host. Are you sure?" do
with_lock do
host_groups = options[:rolling] ? KAMAL.traefik_hosts : [ KAMAL.traefik_hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
run_hook "pre-traefik-reboot", hosts: host_list
on(hosts) do |host|
execute *KAMAL.auditor.record("Rebooted proxy"), verbosity: :debug
execute *KAMAL.registry.login
"Stopping and removing Traefik on #{host}, if running..."
execute *KAMAL.traefik.stop, raise_on_non_zero_exit: false
execute *KAMAL.traefik.remove_container
"Stopping and removing kamal-proxy on #{host}, if running..."
execute *KAMAL.proxy.stop, raise_on_non_zero_exit: false
execute *KAMAL.proxy.remove_container
execute *KAMAL.traefik_or_proxy(host).run
if KAMAL.proxy_host?(host)
KAMAL.roles_on(host).select(&:running_traefik?).each do |role|
app = KAMAL.app(role: role, host: host)
version = capture_with_info(*app.current_running_version, raise_on_non_zero_exit: false).strip
endpoint = capture_with_info(*app.container_endpoint(version: version)).strip
if endpoint.present?
info "Deploying #{endpoint} for role `#{role}` on #{host}..."
execute *KAMAL.proxy.deploy(role.container_prefix, target: endpoint)
end
end
end
end
run_hook "post-traefik-reboot", hosts: host_list
end
end
end
end
desc "start", "Start existing proxy container on servers"
def start
raise_unless_kamal_proxy_enabled!
with_lock do
on(KAMAL.traefik_hosts) do |host|
execute *KAMAL.auditor.record("Started proxy"), verbosity: :debug
execute *KAMAL.traefik_or_proxy(host).start
end
end
end
desc "stop", "Stop existing proxy container on servers"
def stop
raise_unless_kamal_proxy_enabled!
with_lock do
on(KAMAL.traefik_hosts) do |host|
execute *KAMAL.auditor.record("Stopped proxy"), verbosity: :debug
execute *KAMAL.traefik_or_proxy(host).stop, raise_on_non_zero_exit: false
end
end
end
desc "restart", "Restart existing proxy container on servers"
def restart
raise_unless_kamal_proxy_enabled!
with_lock do
stop
start
end
end
desc "details", "Show details about proxy container from servers"
def details
raise_unless_kamal_proxy_enabled!
on(KAMAL.traefik_hosts) { |host| puts_by_host host, capture_with_info(*KAMAL.traefik_or_proxy(host).info), type: "Proxy" }
end
desc "logs", "Show log lines from proxy on servers"
option :since, aliases: "-s", desc: "Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)"
option :lines, type: :numeric, aliases: "-n", desc: "Number of log lines to pull from each server"
option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)"
option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)"
def logs
raise_unless_kamal_proxy_enabled!
grep = options[:grep]
if options[:follow]
run_locally do
info "Following logs on #{KAMAL.primary_host}..."
info KAMAL.traefik_or_proxy(KAMAL.primary_host).follow_logs(host: KAMAL.primary_host, grep: grep)
exec KAMAL.traefik_or_proxy(KAMAL.primary_host).follow_logs(host: KAMAL.primary_host, grep: grep)
end
else
since = options[:since]
lines = options[:lines].presence || ((since || grep) ? nil : 100) # Default to 100 lines if since or grep isn't set
on(KAMAL.traefik_hosts) do |host|
puts_by_host host, capture(*KAMAL.traefik_or_proxy(host).logs(since: since, lines: lines, grep: grep)), type: "Proxy"
end
end
end
desc "remove", "Remove proxy container and image from servers"
def remove
raise_unless_kamal_proxy_enabled!
with_lock do
stop
remove_container
remove_image
end
end
desc "remove_container", "Remove proxy container from servers", hide: true
def remove_container
raise_unless_kamal_proxy_enabled!
with_lock do
on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Removed proxy container"), verbosity: :debug
execute *KAMAL.proxy.remove_container
execute *KAMAL.traefik.remove_container
end
end
end
desc "remove_image", "Remove proxy image from servers", hide: true
def remove_image
raise_unless_kamal_proxy_enabled!
with_lock do
on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Removed proxy image"), verbosity: :debug
execute *KAMAL.proxy.remove_image
execute *KAMAL.traefik.remove_image
end
end
end
private
def raise_unless_kamal_proxy_enabled!
unless KAMAL.config.proxy.enabled?
raise "kamal proxy commands are disabled unless experimental proxy support is enabled. Use `kamal traefik` commands instead."
end
end
end

View File

@@ -1,7 +1,8 @@
class Kamal::Cli::Server < Kamal::Cli::Base class Kamal::Cli::Server < Kamal::Cli::Base
desc "exec", "Run a custom command on the server (use --help to show options)" desc "exec", "Run a custom command on the server (use --help to show options)"
option :interactive, type: :boolean, aliases: "-i", default: false, desc: "Run the command interactively (use for console/bash)" option :interactive, type: :boolean, aliases: "-i", default: false, desc: "Run the command interactively (use for console/bash)"
def exec(cmd) def exec(*cmd)
cmd = Kamal::Utils.join_commands(cmd)
hosts = KAMAL.hosts | KAMAL.accessory_hosts hosts = KAMAL.hosts | KAMAL.accessory_hosts
case case

View File

@@ -1,6 +1,7 @@
class Kamal::Cli::Traefik < Kamal::Cli::Base class Kamal::Cli::Traefik < Kamal::Cli::Base
desc "boot", "Boot Traefik on servers" desc "boot", "Boot Traefik on servers"
def boot def boot
raise_if_kamal_proxy_enabled!
with_lock do with_lock do
on(KAMAL.traefik_hosts) do on(KAMAL.traefik_hosts) do
execute *KAMAL.registry.login execute *KAMAL.registry.login
@@ -13,6 +14,7 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
option :rolling, type: :boolean, default: false, desc: "Reboot traefik on hosts in sequence, rather than in parallel" option :rolling, type: :boolean, default: false, desc: "Reboot traefik on hosts in sequence, rather than in parallel"
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question" option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question"
def reboot def reboot
raise_if_kamal_proxy_enabled!
confirming "This will cause a brief outage on each host. Are you sure?" do confirming "This will cause a brief outage on each host. Are you sure?" do
with_lock do with_lock do
host_groups = options[:rolling] ? KAMAL.traefik_hosts : [ KAMAL.traefik_hosts ] host_groups = options[:rolling] ? KAMAL.traefik_hosts : [ KAMAL.traefik_hosts ]
@@ -34,6 +36,7 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
desc "start", "Start existing Traefik container on servers" desc "start", "Start existing Traefik container on servers"
def start def start
raise_if_kamal_proxy_enabled!
with_lock do with_lock do
on(KAMAL.traefik_hosts) do on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Started traefik"), verbosity: :debug execute *KAMAL.auditor.record("Started traefik"), verbosity: :debug
@@ -44,6 +47,7 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
desc "stop", "Stop existing Traefik container on servers" desc "stop", "Stop existing Traefik container on servers"
def stop def stop
raise_if_kamal_proxy_enabled!
with_lock do with_lock do
on(KAMAL.traefik_hosts) do on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Stopped traefik"), verbosity: :debug execute *KAMAL.auditor.record("Stopped traefik"), verbosity: :debug
@@ -54,6 +58,7 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
desc "restart", "Restart existing Traefik container on servers" desc "restart", "Restart existing Traefik container on servers"
def restart def restart
raise_if_kamal_proxy_enabled!
with_lock do with_lock do
stop stop
start start
@@ -62,6 +67,7 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
desc "details", "Show details about Traefik container from servers" desc "details", "Show details about Traefik container from servers"
def details def details
raise_if_kamal_proxy_enabled!
on(KAMAL.traefik_hosts) { |host| puts_by_host host, capture_with_info(*KAMAL.traefik.info), type: "Traefik" } on(KAMAL.traefik_hosts) { |host| puts_by_host host, capture_with_info(*KAMAL.traefik.info), type: "Traefik" }
end end
@@ -72,6 +78,7 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
option :grep_options, aliases: "-o", desc: "Additional options supplied to grep" option :grep_options, aliases: "-o", desc: "Additional options supplied to grep"
option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)" option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)"
def logs def logs
raise_if_kamal_proxy_enabled!
grep = options[:grep] grep = options[:grep]
grep_options = options[:grep_options] grep_options = options[:grep_options]
@@ -93,6 +100,7 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
desc "remove", "Remove Traefik container and image from servers" desc "remove", "Remove Traefik container and image from servers"
def remove def remove
raise_if_kamal_proxy_enabled!
with_lock do with_lock do
stop stop
remove_container remove_container
@@ -102,6 +110,7 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
desc "remove_container", "Remove Traefik container from servers", hide: true desc "remove_container", "Remove Traefik container from servers", hide: true
def remove_container def remove_container
raise_if_kamal_proxy_enabled!
with_lock do with_lock do
on(KAMAL.traefik_hosts) do on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Removed traefik container"), verbosity: :debug execute *KAMAL.auditor.record("Removed traefik container"), verbosity: :debug
@@ -112,6 +121,7 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
desc "remove_image", "Remove Traefik image from servers", hide: true desc "remove_image", "Remove Traefik image from servers", hide: true
def remove_image def remove_image
raise_if_kamal_proxy_enabled!
with_lock do with_lock do
on(KAMAL.traefik_hosts) do on(KAMAL.traefik_hosts) do
execute *KAMAL.auditor.record("Removed traefik image"), verbosity: :debug execute *KAMAL.auditor.record("Removed traefik image"), verbosity: :debug
@@ -120,43 +130,10 @@ class Kamal::Cli::Traefik < Kamal::Cli::Base
end end
end end
desc "downgrade", "Downgrade to Traefik on servers (stop container, remove container, start new container, reboot app)" private
option :rolling, type: :boolean, default: false, desc: "Reboot proxy on hosts in sequence, rather than in parallel" def raise_if_kamal_proxy_enabled!
option :confirmed, aliases: "-y", type: :boolean, default: false, desc: "Proceed without confirmation question" if KAMAL.config.proxy.enabled?
def downgrade raise "kamal traefik commands are disabled when experimental proxy support is enabled. Use `kamal proxy` commands instead."
invoke_options = { "version" => KAMAL.config.latest_tag }.merge(options)
confirming "This will cause a brief outage on each host. Are you sure?" do
host_groups = options[:rolling] ? KAMAL.hosts : [ KAMAL.hosts ]
host_groups.each do |hosts|
host_list = Array(hosts).join(",")
say "Downgrading to Traefik on #{host_list}...", :magenta
run_hook "pre-traefik-reboot", hosts: host_list
on(hosts) do |host|
execute *KAMAL.auditor.record("Rebooted Traefik"), verbosity: :debug
execute *KAMAL.registry.login
"Stopping and removing kamal-proxy on #{host}, if running..."
execute *KAMAL.traefik.cleanup_kamal_proxy
"Stopping and removing Traefik on #{host}, if running..."
execute *KAMAL.traefik.stop, raise_on_non_zero_exit: false
execute *KAMAL.traefik.remove_container
execute *KAMAL.traefik.remove_image
end
KAMAL.with_specific_hosts(hosts) do
invoke "kamal:cli:traefik:boot", [], invoke_options
reset_invocation(Kamal::Cli::Traefik)
invoke "kamal:cli:app:boot", [], invoke_options
reset_invocation(Kamal::Cli::App)
invoke "kamal:cli:prune:all", [], invoke_options
reset_invocation(Kamal::Cli::Prune)
end
run_hook "post-traefik-reboot", hosts: host_list
say "Downgraded to Traefik on #{host_list}", :magenta
end end
end end
end
end end

View File

@@ -3,7 +3,7 @@ require "active_support/core_ext/module/delegation"
class Kamal::Commander class Kamal::Commander
attr_accessor :verbosity, :holding_lock, :connected attr_accessor :verbosity, :holding_lock, :connected
delegate :hosts, :roles, :primary_host, :primary_role, :roles_on, :traefik_hosts, :accessory_hosts, to: :specifics delegate :hosts, :roles, :primary_host, :primary_role, :roles_on, :traefik_hosts, :proxy_hosts, :proxy_host?, :accessory_hosts, to: :specifics
def initialize def initialize
self.verbosity = :info self.verbosity = :info
@@ -27,7 +27,11 @@ class Kamal::Commander
def specific_primary! def specific_primary!
@specifics = nil @specifics = nil
self.specific_hosts = [ config.primary_host ] if specific_roles.present?
self.specific_hosts = [ specific_roles.first.primary_host ]
else
self.specific_hosts = [ config.primary_host ]
end
end end
def specific_roles=(role_names) def specific_roles=(role_names)
@@ -56,13 +60,6 @@ class Kamal::Commander
end end
end end
def with_specific_hosts(hosts)
original_hosts, self.specific_hosts = specific_hosts, hosts
yield
ensure
self.specific_hosts = original_hosts
end
def accessory_names def accessory_names
config.accessories&.collect(&:name) || [] config.accessories&.collect(&:name) || []
end end
@@ -104,6 +101,10 @@ class Kamal::Commander
@lock ||= Kamal::Commands::Lock.new(config) @lock ||= Kamal::Commands::Lock.new(config)
end end
def proxy
@proxy ||= Kamal::Commands::Proxy.new(config)
end
def prune def prune
@prune ||= Kamal::Commands::Prune.new(config) @prune ||= Kamal::Commands::Prune.new(config)
end end
@@ -120,6 +121,14 @@ class Kamal::Commander
@traefik ||= Kamal::Commands::Traefik.new(config) @traefik ||= Kamal::Commands::Traefik.new(config)
end end
def alias(name)
config.aliases[name]
end
def traefik_or_proxy(host)
proxy_host?(host) ? proxy : traefik
end
def with_verbosity(level) def with_verbosity(level)
old_level = self.verbosity old_level = self.verbosity

View File

@@ -22,8 +22,17 @@ class Kamal::Commander::Specifics
config.traefik_hosts & specified_hosts config.traefik_hosts & specified_hosts
end end
def proxy_hosts
traefik_hosts & config.proxy_hosts
end
def proxy_host?(host)
host = host.hostname if host.is_a?(SSHKit::Host)
proxy_hosts.include?(host)
end
def accessory_hosts def accessory_hosts
config.accessories.flat_map(&:hosts) & specified_hosts specific_hosts || config.accessories.flat_map(&:hosts)
end end
private private

View File

@@ -30,6 +30,24 @@ class Kamal::Commands::App < Kamal::Commands::Base
role.cmd role.cmd
end end
def run_for_proxy(hostname: nil)
docker :run,
"--detach",
"--restart unless-stopped",
"--name", container_name,
*([ "--hostname", hostname ] if hostname),
"-e", "KAMAL_CONTAINER_NAME=\"#{container_name}\"",
"-e", "KAMAL_VERSION=\"#{config.version}\"",
*role.env_args(host),
*role.logging_args,
*config.volume_args,
*role.asset_volume_args,
*role.label_args_for_proxy,
*role.option_args,
config.absolute_image,
role.cmd
end
def start def start
docker :start, container_name docker :start, container_name
end end

View File

@@ -28,4 +28,11 @@ module Kamal::Commands::App::Containers
container_id_for(container_name: container_name(version)), container_id_for(container_name: container_name(version)),
xargs(docker(:inspect, "--format", DOCKER_HEALTH_LOG_FORMAT)) xargs(docker(:inspect, "--format", DOCKER_HEALTH_LOG_FORMAT))
end end
def container_endpoint(version:)
pipe \
container_id_for(container_name: container_name(version)),
xargs(docker(:inspect, "--format", "'{{.NetworkSettings.IPAddress}}{{range $k, $v := .NetworkSettings.Ports}}{{printf \":%s\" $k}}{{break}}{{end}}'")),
[ :sed, "-e", "'s/\\/tcp$//'" ]
end
end end

View File

@@ -58,8 +58,4 @@ class Kamal::Commands::Builder::Multiarch::Remote < Kamal::Commands::Builder::Mu
def remove_context(arch) def remove_context(arch)
docker :context, :rm, builder_name_with_arch(arch) docker :context, :rm, builder_name_with_arch(arch)
end end
def platform_names
"linux/#{local_arch},linux/#{remote_arch}"
end
end end

View File

@@ -0,0 +1,68 @@
class Kamal::Commands::Proxy < Kamal::Commands::Base
delegate :argumentize, :optionize, to: Kamal::Utils
delegate :container_name, to: :proxy_config
attr_reader :proxy_config
def initialize(config)
super
@proxy_config = config.proxy
end
def run
docker :run,
"--name", container_name,
"--detach",
"--restart", "unless-stopped",
*proxy_config.publish_args,
"--volume", "/var/run/docker.sock:/var/run/docker.sock",
"--volume", "#{container_name}:/root/.config/kamal-proxy",
*config.logging_args,
proxy_config.image
end
def start
docker :container, :start, container_name
end
def stop(name: container_name)
docker :container, :stop, name
end
def start_or_run
combine start, run, by: "||"
end
def deploy(service, target:)
docker :exec, container_name, "kamal-proxy", :deploy, service, *optionize({ target: target }), *proxy_config.deploy_command_args
end
def remove(service, target:)
docker :exec, container_name, "kamal-proxy", :remove, service, *optionize({ target: target })
end
def info
docker :ps, "--filter", "name=^#{container_name}$"
end
def logs(since: nil, lines: nil, grep: nil, grep_options: nil)
pipe \
docker(:logs, container_name, (" --since #{since}" if since), (" --tail #{lines}" if lines), "--timestamps", "2>&1"),
("grep '#{grep}'#{" #{grep_options}" if grep_options}" if grep)
end
def follow_logs(host:, grep: nil, grep_options: nil)
run_over_ssh pipe(
docker(:logs, container_name, "--timestamps", "--tail", "10", "--follow", "2>&1"),
(%(grep "#{grep}"#{" #{grep_options}" if grep_options}) if grep)
).join(" "), host: host
end
def remove_container
docker :container, :prune, "--force", "--filter", "label=org.opencontainers.image.title=kamal-proxy"
end
def remove_image
docker :image, :prune, "--all", "--force", "--filter", "label=org.opencontainers.image.title=kamal-proxy"
end
end

View File

@@ -62,15 +62,6 @@ class Kamal::Commands::Traefik < Kamal::Commands::Base
[ :rm, "-f", env.secrets_file ] [ :rm, "-f", env.secrets_file ]
end end
def cleanup_kamal_proxy
chain \
docker(:container, :stop, "kamal-proxy"),
combine(
docker(:container, :prune, "--force", "--filter", "label=org.opencontainers.image.title=kamal-proxy"),
docker(:image, :prune, "--all", "--force", "--filter", "label=org.opencontainers.image.title=kamal-proxy")
)
end
private private
def publish_args def publish_args
argumentize "--publish", port if publish? argumentize "--publish", port if publish?

View File

@@ -11,7 +11,7 @@ class Kamal::Configuration
delegate :argumentize, :optionize, to: Kamal::Utils delegate :argumentize, :optionize, to: Kamal::Utils
attr_reader :destination, :raw_config attr_reader :destination, :raw_config
attr_reader :accessories, :boot, :builder, :env, :healthcheck, :logging, :traefik, :servers, :ssh, :sshkit, :registry attr_reader :accessories, :aliases, :boot, :builder, :env, :healthcheck, :logging, :proxy, :traefik, :servers, :ssh, :sshkit, :registry
include Validation include Validation
@@ -54,12 +54,14 @@ class Kamal::Configuration
@registry = Registry.new(config: self) @registry = Registry.new(config: self)
@accessories = @raw_config.accessories&.keys&.collect { |name| Accessory.new(name, config: self) } || [] @accessories = @raw_config.accessories&.keys&.collect { |name| Accessory.new(name, config: self) } || []
@aliases = @raw_config.aliases&.keys&.to_h { |name| [ name, Alias.new(name, config: self) ] } || {}
@boot = Boot.new(config: self) @boot = Boot.new(config: self)
@builder = Builder.new(config: self) @builder = Builder.new(config: self)
@env = Env.new(config: @raw_config.env || {}) @env = Env.new(config: @raw_config.env || {})
@healthcheck = Healthcheck.new(healthcheck_config: @raw_config.healthcheck) @healthcheck = Healthcheck.new(healthcheck_config: @raw_config.healthcheck)
@logging = Logging.new(logging_config: @raw_config.logging) @logging = Logging.new(logging_config: @raw_config.logging)
@proxy = Proxy.new(config: self)
@traefik = Traefik.new(config: self) @traefik = Traefik.new(config: self)
@ssh = Ssh.new(config: self) @ssh = Ssh.new(config: self)
@sshkit = Sshkit.new(config: self) @sshkit = Sshkit.new(config: self)
@@ -141,6 +143,10 @@ class Kamal::Configuration
traefik_roles.flat_map(&:hosts).uniq traefik_roles.flat_map(&:hosts).uniq
end end
def proxy_hosts
proxy.hosts
end
def repository def repository
[ registry.server, image ].compact.join("/") [ registry.server, image ].compact.join("/")
end end

View File

@@ -0,0 +1,15 @@
class Kamal::Configuration::Alias
include Kamal::Configuration::Validation
attr_reader :name, :command
def initialize(name, config:)
@name, @command = name.inquiry, config.raw_config["aliases"][name]
validate! \
command,
example: validation_yml["aliases"]["uname"],
context: "aliases/#{name}",
with: Kamal::Configuration::Validator::Alias
end
end

View File

@@ -0,0 +1,26 @@
# Aliases
#
# Aliases are shortcuts for Kamal commands.
#
# For example, for a Rails app, you might open a console with:
#
# ```shell
# kamal app exec -i -r console "rails console"
# ```
#
# By defining an alias, like this:
aliases:
console: app exec -r console -i "rails console"
# You can now open the console with:
# ```shell
# kamal console
# ```
# Configuring aliases
#
# Aliases are defined in the root config under the alias key
#
# Each alias is named and can only contain lowercase letters, numbers, dashes and underscores.
aliases:
uname: app exec -p -q -r web "uname -a"

View File

@@ -143,6 +143,12 @@ accessories:
traefik: traefik:
... ...
# Proxy
#
# **Experimental** Configuration for kamal-proxy the replacement for Traefik, see kamal docs proxy
proxy:
...
# SSHKit # SSHKit
# #
# See kamal docs sshkit # See kamal docs sshkit
@@ -166,3 +172,9 @@ healthcheck:
# Docker logging configuration, see kamal docs logging # Docker logging configuration, see kamal docs logging
logging: logging:
... ...
# Aliases
#
# Alias configuration, see kamal docs alias
aliases:
...

View File

@@ -0,0 +1,114 @@
# Proxy
#
# **Experimental** [kamal-proxy](http://github.com/basecamp/kamal-proxy) is a
# custom built specifically for Kamal. It will replace Traefik in Kamal v2.0,
# but currently is available as an experimental feature.
#
# When this is enabled, the proxy will be started on the hosts listed under the hosts key.
# In addition, the kamal traefik command will be disabled and replaced by kamal proxy.
#
# The kamal proxy command works identically to kamal traefik on hosts that have not
# been included. It will also handle switching between Traefik and kamal-proxy when you
# run kamal proxy reboot.
# Limitations
#
# Currently the proxy will run on ports 80 and 443 and will bind to those
# ports on the host.
#
# There is no way to set custom options for `docker run` when booting the proxy.
#
# If you have custom Traefik configuration via labels or boot arguments they may
# not have an equivalent in kamal-proxy.
# Proxy settings
#
# The proxy is configured in the root configuration under `traefik`. These are
# options that are set when deploying the application, not when booting the proxy
#
# They are application specific, so are not shared when multiple applications
# with the same proxy.
proxy:
# Enabled
#
# Whether to enable experimental proxy support. Defaults to false
enabled: true
# Hosts
#
# The hosts to run the proxy on, instead of Traefik
# This is a temporary setting and will be removed when we full switch to kamal-proxy
#
# If you run `kamal traefik reboot`, then the proxy will be started on these hosts
# in place of traefik.
hosts:
- 10.0.0.1
- 10.0.0.2
# Host
#
# This is the host that will be used to serve the app. By setting this you can run
# multiple apps on the same server sharing the same instance of the proxy.
#
# If this is set only requests that match this host will be forwarded by the proxy.
# if this is not set, then all requests will be forwarded, except for matching
# requests for other apps that do have a host set.
host: foo.example.com
# Deploy timeout
#
# How long to wait for the app to boot when deploying, defaults to 30 seconds
deploy_timeout: 10s
# Response timeout
#
# How long to wait for requests to complete before timing out, defaults to 30 seconds
response_timeout: 10
# Healthcheck
#
# When deploying, the proxy will by default hit /up once every second until we hit
# the deploy timeout, with a 5 second timeout for each request.
#
# Once the app is up, the proxy will stop hitting the healthcheck endpoint.
healthcheck:
interval: 3
path: /health
timeout: 3
# Buffering
#
# Whether to buffer request and response bodies in the proxy
#
# By default buffering is enabled with a max request body size of 1GB and no limit
# for response size.
#
# You can also set the memory limit for buffering, which defaults to 1MB, anything
# larger than that is written to disk.
buffering:
requests: true
responses: true
max_request_body: 40_000_000
max_response_body: 0
memory: 2_000_000
# Logging
#
# Configure request logging for the proxy
# You can specify request and response headers to log.
# By default, Cache-Control and Last-Modified request headers are logged
logging:
request_headers:
- Cache-Control
- X-Forwarded-Proto
response_headers:
- X-Request-ID
- X-Request-Start
# Forward headers
#
# Whether to forward the X-Forwarded-For and X-Forwarded-Proto headers (defaults to false)
#
# If you are behind a trusted proxy, you can set this to true to forward the headers.
forward_headers: true

View File

@@ -0,0 +1,66 @@
class Kamal::Configuration::Proxy
include Kamal::Configuration::Validation
DEFAULT_HTTP_PORT = 80
DEFAULT_HTTPS_PORT = 443
DEFAULT_IMAGE = "basecamp/kamal-proxy:latest"
DEFAULT_LOG_REQUEST_HEADERS = [ "Cache-Control", "Last-Modified" ]
delegate :argumentize, :optionize, to: Kamal::Utils
def initialize(config:)
@proxy_config = config.raw_config.proxy || {}
validate! proxy_config
end
def enabled?
!!proxy_config.fetch("enabled", false)
end
def hosts
if enabled?
proxy_config.fetch("hosts", [])
else
[]
end
end
def image
proxy_config.fetch("image", DEFAULT_IMAGE)
end
def container_name
"kamal-proxy"
end
def publish_args
argumentize "--publish", [ "#{DEFAULT_HTTP_PORT}:#{DEFAULT_HTTP_PORT}", "#{DEFAULT_HTTPS_PORT}:#{DEFAULT_HTTPS_PORT}" ]
end
def deploy_options
{
host: proxy_config["host"],
"deploy-timeout": proxy_config["deploy_timeout"],
"drain-timeout": proxy_config["drain_timeout"],
"health-check-interval": proxy_config.dig("health_check", "interval"),
"health-check-timeout": proxy_config.dig("health_check", "timeout"),
"health-check-path": proxy_config.dig("health_check", "path"),
"target-timeout": proxy_config["response_timeout"],
"buffer-requests": proxy_config.fetch("buffering", { "requests": true }).fetch("requests", true),
"buffer-responses": proxy_config.fetch("buffering", { "responses": true }).fetch("responses", true),
"buffer-memory": proxy_config.dig("buffering", "memory"),
"max-request-body": proxy_config.dig("buffering", "max_request_body"),
"max-response-body": proxy_config.dig("buffering", "max_response_body"),
"forward-headers": proxy_config.dig("forward_headers"),
"log-request-header": proxy_config.dig("logging", "request_headers") || DEFAULT_LOG_REQUEST_HEADERS,
"log-response-header": proxy_config.dig("logging", "response_headers")
}.compact
end
def deploy_command_args
optionize deploy_options
end
private
attr_accessor :proxy_config
end

View File

@@ -58,10 +58,18 @@ class Kamal::Configuration::Role
default_labels.merge(traefik_labels).merge(custom_labels) default_labels.merge(traefik_labels).merge(custom_labels)
end end
def labels_for_proxy
default_labels.merge(custom_labels)
end
def label_args def label_args
argumentize "--label", labels argumentize "--label", labels
end end
def label_args_for_proxy
argumentize "--label", labels_for_proxy
end
def logging_args def logging_args
logging.args logging.args
end end

View File

@@ -13,32 +13,34 @@ class Kamal::Configuration::Validator
private private
def validate_against_example!(validation_config, example) def validate_against_example!(validation_config, example)
validate_type! validation_config, Hash validate_type! validation_config, example.class
check_unknown_keys! validation_config, example if example.class == Hash
check_unknown_keys! validation_config, example
validation_config.each do |key, value| validation_config.each do |key, value|
next if extension?(key) next if extension?(key)
with_context(key) do with_context(key) do
example_value = example[key] example_value = example[key]
if example_value == "..." if example_value == "..."
validate_type! value, *(Array if key == :servers), Hash validate_type! value, *(Array if key == :servers), Hash
elsif key == "hosts" elsif key == "hosts"
validate_servers! value validate_servers! value
elsif example_value.is_a?(Array) elsif example_value.is_a?(Array)
validate_array_of! value, example_value.first.class validate_array_of! value, example_value.first.class
elsif example_value.is_a?(Hash) elsif example_value.is_a?(Hash)
case key.to_s case key.to_s
when "options", "args" when "options", "args"
validate_type! value, Hash validate_type! value, Hash
when "labels" when "labels"
validate_hash_of! value, example_value.first[1].class validate_hash_of! value, example_value.first[1].class
else
validate_against_example! value, example_value
end
else else
validate_against_example! value, example_value validate_type! value, example_value.class
end end
else
validate_type! value, example_value.class
end end
end end
end end

View File

@@ -0,0 +1,15 @@
class Kamal::Configuration::Validator::Alias < Kamal::Configuration::Validator
def validate!
super
name = context.delete_prefix("aliases/")
if name !~ /\A[a-z0-9_-]+\z/
error "Invalid alias name: '#{name}'. Must only contain lowercase letters, alphanumeric, hyphens and underscores."
end
if Kamal::Cli::Main.commands.include?(name)
error "Alias '#{name}' conflicts with a built-in command."
end
end
end

View File

@@ -77,4 +77,8 @@ module Kamal::Utils
def stable_sort!(elements, &block) def stable_sort!(elements, &block)
elements.sort_by!.with_index { |element, index| [ block.call(element), index ] } elements.sort_by!.with_index { |element, index| [ block.call(element), index ] }
end end
def join_commands(commands)
commands.map(&:strip).join(" ")
end
end end

View File

@@ -1,3 +1,3 @@
module Kamal module Kamal
VERSION = "1.9.0" VERSION = "2.0.0.alpha"
end end

View File

@@ -209,24 +209,6 @@ class CliAccessoryTest < CliTestCase
end end
end end
test "downgrade" do
run_command("downgrade", "-y", "all").tap do |output|
assert_match "Downgrading all accessories on 1.1.1.3,1.1.1.1,1.1.1.2...", output
assert_match "docker container stop app-mysql on 1.1.1.3", output
assert_match "docker run --name app-mysql --detach --restart unless-stopped --log-opt max-size=\"10m\" --publish 3306:3306 --env-file .kamal/env/accessories/app-mysql.env --env MYSQL_ROOT_HOST=\"%\" --volume $PWD/app-mysql/etc/mysql/my.cnf:/etc/mysql/my.cnf --volume $PWD/app-mysql/data:/var/lib/mysql --label service=\"app-mysql\" mysql:5.7 on 1.1.1.3", output
assert_match "Downgraded all accessories on 1.1.1.3,1.1.1.1,1.1.1.2", output
end
end
test "downgrade rolling" do
run_command("downgrade", "--rolling", "-y", "all").tap do |output|
assert_match "Downgrading all accessories on 1.1.1.3...", output
assert_match "docker container stop app-mysql on 1.1.1.3", output
assert_match "docker run --name app-mysql --detach --restart unless-stopped --log-opt max-size=\"10m\" --publish 3306:3306 --env-file .kamal/env/accessories/app-mysql.env --env MYSQL_ROOT_HOST=\"%\" --volume $PWD/app-mysql/etc/mysql/my.cnf:/etc/mysql/my.cnf --volume $PWD/app-mysql/data:/var/lib/mysql --label service=\"app-mysql\" mysql:5.7 on 1.1.1.3", output
assert_match "Downgraded all accessories on 1.1.1.3", output
end
end
private private
def run_command(*command) def run_command(*command)
stdouted { Kamal::Cli::Accessory.start([ *command, "-c", "test/fixtures/deploy_with_accessories.yml" ]) } stdouted { Kamal::Cli::Accessory.start([ *command, "-c", "test/fixtures/deploy_with_accessories.yml" ]) }

View File

@@ -247,6 +247,12 @@ class CliAppTest < CliTestCase
end end
end end
test "exec separate arguments" do
run_command("exec", "ruby", " -v").tap do |output|
assert_match "docker run --rm --env-file .kamal/env/roles/app-web.env dhh/app:latest ruby -v", output
end
end
test "exec with reuse" do test "exec with reuse" do
run_command("exec", "--reuse", "ruby -v").tap do |output| run_command("exec", "--reuse", "ruby -v").tap do |output|
assert_match "sh -c 'docker ps --latest --format '\\''{{.Names}}'\\'' --filter label=service=app --filter label=role=web --filter status=running --filter status=restarting --filter ancestor=$(docker image ls --filter reference=dhh/app:latest --format '\\''{{.ID}}'\\'') ; docker ps --latest --format '\\''{{.Names}}'\\'' --filter label=service=app --filter label=role=web --filter status=running --filter status=restarting' | head -1 | while read line; do echo ${line#app-web-}; done", output # Get current version assert_match "sh -c 'docker ps --latest --format '\\''{{.Names}}'\\'' --filter label=service=app --filter label=role=web --filter status=running --filter status=restarting --filter ancestor=$(docker image ls --filter reference=dhh/app:latest --format '\\''{{.ID}}'\\'') ; docker ps --latest --format '\\''{{.Names}}'\\'' --filter label=service=app --filter label=role=web --filter status=running --filter status=restarting' | head -1 | while read line; do echo ${line#app-web-}; done", output # Get current version
@@ -350,6 +356,18 @@ class CliAppTest < CliTestCase
end end
end end
test "boot proxy" do
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info).returns("123") # old version
run_command("boot", config: :with_proxy).tap do |output|
assert_match /Renaming container .* to .* as already deployed on 1.1.1.1/, output # Rename
assert_match /docker rename app-web-latest app-web-latest_replaced_[0-9a-f]{16}/, output
assert_match /docker run --detach --restart unless-stopped --name app-web-latest --hostname 1.1.1.1-[0-9a-f]{12} -e KAMAL_CONTAINER_NAME="app-web-latest" -e KAMAL_VERSION="latest" --env-file .kamal\/env\/roles\/app-web.env --log-opt max-size="10m" --label service="app" --label role="web" --label destination dhh\/app:latest/, output
assert_match /docker exec kamal-proxy kamal-proxy deploy app-web --target "123"/, output
assert_match "docker container ls --all --filter name=^app-web-123$ --quiet | xargs docker stop", output
end
end
private private
def run_command(*command, config: :with_accessories, host: "1.1.1.1", allow_execute_error: false) def run_command(*command, config: :with_accessories, host: "1.1.1.1", allow_execute_error: false)
stdouted do stdouted do

View File

@@ -63,4 +63,12 @@ class CliTestCase < ActiveSupport::TestCase
assert_match expected, output assert_match expected, output
end end
def with_argv(*argv)
old_argv = ARGV
ARGV.replace(*argv)
yield
ensure
ARGV.replace(old_argv)
end
end end

View File

@@ -537,37 +537,47 @@ class CliMainTest < CliTestCase
assert_equal Kamal::VERSION, version assert_equal Kamal::VERSION, version
end end
test "downgrade" do test "run an alias for details" do
invoke_options = { "config_file" => "test/fixtures/deploy_with_accessories.yml", "skip_hooks" => false, "confirmed" => true, "rolling" => false } Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:traefik:details")
Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:traefik:downgrade", [], invoke_options) Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:app:details")
Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:accessory:downgrade", [ "all" ], invoke_options) Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:accessory:details", [ "all" ])
run_command("downgrade", "-y", config_file: "deploy_with_accessories").tap do |output| run_command("info", config_file: "deploy_with_aliases")
assert_match "Downgrading all hosts...", output end
assert_match "Downgraded all hosts", output
test "run an alias for a console" do
run_command("console", config_file: "deploy_with_aliases").tap do |output|
assert_match "docker exec app-console-999 bin/console on 1.1.1.5", output
assert_match "App Host: 1.1.1.5", output
end end
end end
test "downgrade rolling" do test "run an alias for a console overriding role" do
invoke_options = { "config_file" => "test/fixtures/deploy_with_accessories.yml", "skip_hooks" => false, "confirmed" => true, "rolling" => false } run_command("console", "-r", "workers", config_file: "deploy_with_aliases").tap do |output|
Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:traefik:downgrade", [], invoke_options).times(4) assert_match "docker exec app-workers-999 bin/console on 1.1.1.3", output
Kamal::Cli::Main.any_instance.expects(:invoke).with("kamal:cli:accessory:downgrade", [ "all" ], invoke_options).times(3) assert_match "App Host: 1.1.1.3", output
end
end
run_command("downgrade", "--rolling", "-y", config_file: "deploy_with_accessories").tap do |output| test "run an alias for a console passing command" do
assert_match "Downgrading 1.1.1.1...", output run_command("exec", "bin/job", config_file: "deploy_with_aliases").tap do |output|
assert_match "Downgraded 1.1.1.1", output assert_match "docker exec app-console-999 bin/job on 1.1.1.5", output
assert_match "Downgrading 1.1.1.2...", output assert_match "App Host: 1.1.1.5", output
assert_match "Downgraded 1.1.1.2", output end
assert_match "Downgrading 1.1.1.3...", output end
assert_match "Downgraded 1.1.1.3", output
assert_match "Downgrading 1.1.1.4...", output test "append to command with an alias" do
assert_match "Downgraded 1.1.1.4", output run_command("rails", "db:migrate:status", config_file: "deploy_with_aliases").tap do |output|
assert_match "docker exec app-console-999 rails db:migrate:status on 1.1.1.5", output
assert_match "App Host: 1.1.1.5", output
end end
end end
private private
def run_command(*command, config_file: "deploy_simple") def run_command(*command, config_file: "deploy_simple")
stdouted { Kamal::Cli::Main.start([ *command, "-c", "test/fixtures/#{config_file}.yml" ]) } with_argv([ *command, "-c", "test/fixtures/#{config_file}.yml" ]) do
stdouted { Kamal::Cli::Main.start }
end
end end
def with_test_dotenv(**files) def with_test_dotenv(**files)

141
test/cli/proxy_test.rb Normal file
View File

@@ -0,0 +1,141 @@
require_relative "cli_test_case"
class CliProxyTest < CliTestCase
test "boot" do
run_command("boot").tap do |output|
assert_match "docker login", output
assert_match "docker run --name kamal-proxy --detach --restart unless-stopped --publish 80:80 --publish 443:443 --volume /var/run/docker.sock:/var/run/docker.sock --volume kamal-proxy:/root/.config/kamal-proxy --log-opt max-size=\"10m\" #{Kamal::Configuration::Proxy::DEFAULT_IMAGE}", output
end
end
test "reboot" do
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-web-123$", "--quiet", "|", :xargs, :docker, :inspect, "--format", "'{{.NetworkSettings.IPAddress}}{{range $k, $v := .NetworkSettings.Ports}}{{printf \":%s\" $k}}{{break}}{{end}}'", "|", :sed, "-e", "'s/\\/tcp$//'")
.returns("172.1.0.2:80")
.at_least_once
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info)
.with { |*args| args[0..1] == [ :sh, "-c" ] }
.returns("123")
.at_least_once
run_command("reboot", "-y").tap do |output|
assert_match "docker container stop kamal-proxy on 1.1.1.1", output
assert_match "docker container stop traefik on 1.1.1.1", output
assert_match "docker container prune --force --filter label=org.opencontainers.image.title=kamal-proxy on 1.1.1.1", output
assert_match "docker container prune --force --filter label=org.opencontainers.image.title=Traefik on 1.1.1.1", output
assert_match "docker run --name kamal-proxy --detach --restart unless-stopped --publish 80:80 --publish 443:443 --volume /var/run/docker.sock:/var/run/docker.sock --volume kamal-proxy:/root/.config/kamal-proxy --log-opt max-size=\"10m\" #{Kamal::Configuration::Proxy::DEFAULT_IMAGE} on 1.1.1.1", output
assert_match "docker exec kamal-proxy kamal-proxy deploy app-web --target \"172.1.0.2:80\" --deploy-timeout \"6s\" on 1.1.1.1", output
assert_match "docker container stop kamal-proxy on 1.1.1.2", output
assert_match "docker container stop traefik on 1.1.1.2", output
assert_match "docker container prune --force --filter label=org.opencontainers.image.title=kamal-proxy on 1.1.1.2", output
assert_match "docker container prune --force --filter label=org.opencontainers.image.title=Traefik on 1.1.1.2", output
assert_match "docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --env-file .kamal/env/traefik/traefik.env --log-opt max-size=\"10m\" --label traefik.http.routers.catchall.entryPoints=\"http\" --label traefik.http.routers.catchall.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.routers.catchall.service=\"unavailable\" --label traefik.http.routers.catchall.priority=\"1\" --label traefik.http.services.unavailable.loadbalancer.server.port=\"0\" traefik:v2.10 --providers.docker --log.level=\"DEBUG\" on 1.1.1.2", output
end
end
test "reboot --rolling" do
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-web-123$", "--quiet", "|", :xargs, :docker, :inspect, "--format", "'{{.NetworkSettings.IPAddress}}{{range $k, $v := .NetworkSettings.Ports}}{{printf \":%s\" $k}}{{break}}{{end}}'", "|", :sed, "-e", "'s/\\/tcp$//'")
.returns("172.1.0.2:80")
.at_least_once
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info)
.with { |*args| args[0..1] == [ :sh, "-c" ] }
.returns("123")
.at_least_once
run_command("reboot", "--rolling", "-y").tap do |output|
assert_match "Running docker container prune --force --filter label=org.opencontainers.image.title=kamal-proxy on 1.1.1.1", output
end
end
test "start" do
run_command("start").tap do |output|
assert_match "docker container start kamal-proxy", output
end
end
test "stop" do
run_command("stop").tap do |output|
assert_match "docker container stop kamal-proxy", output
end
end
test "restart" do
Kamal::Cli::Proxy.any_instance.expects(:stop)
Kamal::Cli::Proxy.any_instance.expects(:start)
run_command("restart")
end
test "details" do
run_command("details").tap do |output|
assert_match "docker ps --filter name=^kamal-proxy$", output
end
end
test "logs" do
SSHKit::Backend::Abstract.any_instance.stubs(:capture)
.with(:docker, :logs, "kamal-proxy", " --tail 100", "--timestamps", "2>&1")
.returns("Log entry")
SSHKit::Backend::Abstract.any_instance.stubs(:capture)
.with(:docker, :logs, "traefik", " --tail 100", "--timestamps", "2>&1")
.returns("Log entry")
run_command("logs").tap do |output|
assert_match "Proxy Host: 1.1.1.1", output
assert_match "Log entry", output
end
end
test "logs with follow" do
SSHKit::Backend::Abstract.any_instance.stubs(:exec)
.with("ssh -t root@1.1.1.1 -p 22 'docker logs kamal-proxy --timestamps --tail 10 --follow 2>&1'")
assert_match "docker logs kamal-proxy --timestamps --tail 10 --follow", run_command("logs", "--follow")
end
test "remove" do
Kamal::Cli::Proxy.any_instance.expects(:stop)
Kamal::Cli::Proxy.any_instance.expects(:remove_container)
Kamal::Cli::Proxy.any_instance.expects(:remove_image)
run_command("remove")
end
test "remove_container" do
run_command("remove_container").tap do |output|
assert_match "docker container prune --force --filter label=org.opencontainers.image.title=kamal-proxy", output
end
end
test "remove_image" do
run_command("remove_image").tap do |output|
assert_match "docker image prune --all --force --filter label=org.opencontainers.image.title=kamal-proxy", output
end
end
test "commands disallowed when proxy is disabled" do
assert_raises_when_disabled "boot"
assert_raises_when_disabled "reboot"
assert_raises_when_disabled "start"
assert_raises_when_disabled "stop"
assert_raises_when_disabled "details"
assert_raises_when_disabled "logs"
assert_raises_when_disabled "remove"
end
private
def run_command(*command, fixture: :with_proxy)
stdouted { Kamal::Cli::Proxy.start([ *command, "-c", "test/fixtures/deploy_#{fixture}.yml" ]) }
end
def assert_raises_when_disabled(command)
assert_raises "kamal proxy commands are disabled unless experimental proxy support is enabled. Use `kamal traefik` commands instead." do
run_command(command, fixture: :with_accessories)
end
end
end

View File

@@ -3,8 +3,8 @@ require_relative "cli_test_case"
class CliServerTest < CliTestCase class CliServerTest < CliTestCase
test "running a command with exec" do test "running a command with exec" do
SSHKit::Backend::Abstract.any_instance.stubs(:capture) SSHKit::Backend::Abstract.any_instance.stubs(:capture)
.with("date", verbosity: 1) .with("date", verbosity: 1)
.returns("Today") .returns("Today")
hosts = "1.1.1.1".."1.1.1.4" hosts = "1.1.1.1".."1.1.1.4"
run_command("exec", "date").tap do |output| run_command("exec", "date").tap do |output|
@@ -15,6 +15,20 @@ class CliServerTest < CliTestCase
end end
end end
test "running a command with exec multiple arguments" do
SSHKit::Backend::Abstract.any_instance.stubs(:capture)
.with("date -j", verbosity: 1)
.returns("Today")
hosts = "1.1.1.1".."1.1.1.4"
run_command("exec", "date", "-j").tap do |output|
hosts.map do |host|
assert_match "Running 'date -j' on #{hosts.to_a.join(', ')}...", output
assert_match "App Host: #{host}\nToday", output
end
end
end
test "bootstrap already installed" do test "bootstrap already installed" do
stub_setup stub_setup
SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:docker, "-v", raise_on_non_zero_exit: false).returns(true).at_least_once SSHKit::Backend::Abstract.any_instance.expects(:execute).with(:docker, "-v", raise_on_non_zero_exit: false).returns(true).at_least_once

View File

@@ -103,90 +103,6 @@ class CliTraefikTest < CliTestCase
end end
end end
test "downgrade" do
Object.any_instance.stubs(:sleep)
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-web-latest$", "--quiet", raise_on_non_zero_exit: false)
.returns("12345678")
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-workers-latest$", "--quiet", raise_on_non_zero_exit: false)
.returns("12345678")
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with { |*args| args[0..1] == [ :sh, "-c" ] }
.returns("123") # old version
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-web-latest$", "--quiet", "|", :xargs, :docker, :inspect, "--format", "'{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}'")
.returns("running") # health check
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-workers-latest$", "--quiet", "|", :xargs, :docker, :inspect, "--format", "'{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}'")
.returns("running").at_least_once # workers health check
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :inspect, "-f '{{ range .Mounts }}{{printf \"%s %s\\n\" .Source .Destination}}{{ end }}'", "app-web-123", "|", :awk, "'$2 == \"/tmp/kamal-cord\" {print $1}'", raise_on_non_zero_exit: false)
.returns("") # old version
run_command("downgrade", "-y").tap do |output|
assert_match "Downgrading to Traefik on 1.1.1.1,1.1.1.2,1.1.1.3,1.1.1.4...", output
assert_match "docker login -u [REDACTED] -p [REDACTED]", output
assert_match "docker container stop kamal-proxy ; docker container prune --force --filter label=org.opencontainers.image.title=kamal-proxy && docker image prune --all --force --filter label=org.opencontainers.image.title=kamal-proxy", output
assert_match "docker container stop traefik", output
assert_match "docker container prune --force --filter label=org.opencontainers.image.title=Traefik", output
assert_match "docker image prune --all --force --filter label=org.opencontainers.image.title=Traefik", output
assert_match "/usr/bin/env mkdir -p .kamal", output
assert_match "docker login -u [REDACTED] -p [REDACTED]", output
assert_match "docker container start traefik || docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --env-file .kamal/env/traefik/traefik.env --log-opt max-size=\"10m\" --label traefik.http.routers.catchall.entryPoints=\"http\" --label traefik.http.routers.catchall.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.routers.catchall.service=\"unavailable\" --label traefik.http.routers.catchall.priority=\"1\" --label traefik.http.services.unavailable.loadbalancer.server.port=\"0\" traefik:v2.10 --providers.docker --log.level=\"DEBUG\"", output
assert_match "/usr/bin/env mkdir -p .kamal", output
assert_match %r{docker rename app-web-latest app-web-latest_replaced_.*}, output
assert_match %r{docker run --detach --restart unless-stopped --name app-web-latest --hostname 1.1.1.1-.* -e KAMAL_CONTAINER_NAME="app-web-latest" -e KAMAL_VERSION="latest" --env-file .kamal/env/roles/app-web.env --health-cmd}, output
assert_match "docker tag dhh/app:latest dhh/app:latest", output
assert_match "/usr/bin/env mkdir -p .kamal", output
assert_match "docker ps -q -a --filter label=service=app --filter status=created --filter status=exited --filter status=dead | tail -n +6 | while read container_id; do docker rm $container_id; done", output
assert_match "docker image prune --force --filter label=service=app", output
assert_match "Downgraded to Traefik on 1.1.1.1,1.1.1.2,1.1.1.3,1.1.1.4", output
end
end
test "downgrade rolling" do
Object.any_instance.stubs(:sleep)
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-web-latest$", "--quiet", raise_on_non_zero_exit: false)
.returns("12345678")
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-workers-latest$", "--quiet", raise_on_non_zero_exit: false)
.returns("12345678")
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with { |*args| args[0..1] == [ :sh, "-c" ] }
.returns("123") # old version
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-web-latest$", "--quiet", "|", :xargs, :docker, :inspect, "--format", "'{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}'")
.returns("running") # health check
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :container, :ls, "--all", "--filter", "name=^app-workers-latest$", "--quiet", "|", :xargs, :docker, :inspect, "--format", "'{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}'")
.returns("running").at_least_once # workers health check
SSHKit::Backend::Abstract.any_instance.stubs(:capture_with_info)
.with(:docker, :inspect, "-f '{{ range .Mounts }}{{printf \"%s %s\\n\" .Source .Destination}}{{ end }}'", "app-web-123", "|", :awk, "'$2 == \"/tmp/kamal-cord\" {print $1}'", raise_on_non_zero_exit: false)
.returns("") # old version
run_command("downgrade", "--rolling", "-y",).tap do |output|
%w[1.1.1.1 1.1.1.2 1.1.1.3 1.1.1.4].each do |host|
assert_match "Downgrading to Traefik on #{host}...", output
assert_match "docker container stop kamal-proxy ; docker container prune --force --filter label=org.opencontainers.image.title=kamal-proxy && docker image prune --all --force --filter label=org.opencontainers.image.title=kamal-proxy", output
assert_match "Downgraded to Traefik on #{host}", output
end
end
end
private private
def run_command(*command) def run_command(*command)
stdouted { Kamal::Cli::Traefik.start([ *command, "-c", "test/fixtures/deploy_with_accessories.yml" ]) } stdouted { Kamal::Cli::Traefik.start([ *command, "-c", "test/fixtures/deploy_with_accessories.yml" ]) }

View File

@@ -30,10 +30,10 @@ class CommandsBuilderTest < ActiveSupport::TestCase
end end
test "target multiarch remote when local and remote is set" do test "target multiarch remote when local and remote is set" do
builder = new_builder_command(builder: { "local" => { "arch" => "arm64" }, "remote" => { "arch" => "amd64" }, "cache" => { "type" => "gha" } }) builder = new_builder_command(builder: { "local" => {}, "remote" => {}, "cache" => { "type" => "gha" } })
assert_equal "multiarch/remote", builder.name assert_equal "multiarch/remote", builder.name
assert_equal \ assert_equal \
"docker buildx build --push --platform linux/arm64,linux/amd64 --builder kamal-app-multiarch-remote -t dhh/app:123 -t dhh/app:latest --cache-to type=gha --cache-from type=gha --label service=\"app\" --file Dockerfile .", "docker buildx build --push --platform linux/amd64,linux/arm64 --builder kamal-app-multiarch-remote -t dhh/app:123 -t dhh/app:latest --cache-to type=gha --cache-from type=gha --label service=\"app\" --file Dockerfile .",
builder.push.join(" ") builder.push.join(" ")
end end

126
test/commands/proxy_test.rb Normal file
View File

@@ -0,0 +1,126 @@
require "test_helper"
class CommandsProxyTest < ActiveSupport::TestCase
setup do
@config = {
service: "app", image: "dhh/app", registry: { "username" => "dhh", "password" => "secret" }, servers: [ "1.1.1.1" ]
}
ENV["EXAMPLE_API_KEY"] = "456"
end
teardown do
ENV.delete("EXAMPLE_API_KEY")
end
test "run" do
assert_equal \
"docker run --name kamal-proxy --detach --restart unless-stopped --publish 80:80 --publish 443:443 --volume /var/run/docker.sock:/var/run/docker.sock --volume kamal-proxy:/root/.config/kamal-proxy --log-opt max-size=\"10m\" #{Kamal::Configuration::Proxy::DEFAULT_IMAGE}",
new_command.run.join(" ")
end
test "run with ports configured" do
assert_equal \
"docker run --name kamal-proxy --detach --restart unless-stopped --publish 80:80 --publish 443:443 --volume /var/run/docker.sock:/var/run/docker.sock --volume kamal-proxy:/root/.config/kamal-proxy --log-opt max-size=\"10m\" #{Kamal::Configuration::Proxy::DEFAULT_IMAGE}",
new_command.run.join(" ")
end
test "run without configuration" do
@config.delete(:proxy)
assert_equal \
"docker run --name kamal-proxy --detach --restart unless-stopped --publish 80:80 --publish 443:443 --volume /var/run/docker.sock:/var/run/docker.sock --volume kamal-proxy:/root/.config/kamal-proxy --log-opt max-size=\"10m\" #{Kamal::Configuration::Proxy::DEFAULT_IMAGE}",
new_command.run.join(" ")
end
test "run with logging config" do
@config[:logging] = { "driver" => "local", "options" => { "max-size" => "100m", "max-file" => "3" } }
assert_equal \
"docker run --name kamal-proxy --detach --restart unless-stopped --publish 80:80 --publish 443:443 --volume /var/run/docker.sock:/var/run/docker.sock --volume kamal-proxy:/root/.config/kamal-proxy --log-driver \"local\" --log-opt max-size=\"100m\" --log-opt max-file=\"3\" #{Kamal::Configuration::Proxy::DEFAULT_IMAGE}",
new_command.run.join(" ")
end
test "proxy start" do
assert_equal \
"docker container start kamal-proxy",
new_command.start.join(" ")
end
test "proxy stop" do
assert_equal \
"docker container stop kamal-proxy",
new_command.stop.join(" ")
end
test "proxy info" do
assert_equal \
"docker ps --filter name=^kamal-proxy$",
new_command.info.join(" ")
end
test "proxy logs" do
assert_equal \
"docker logs kamal-proxy --timestamps 2>&1",
new_command.logs.join(" ")
end
test "proxy logs since 2h" do
assert_equal \
"docker logs kamal-proxy --since 2h --timestamps 2>&1",
new_command.logs(since: "2h").join(" ")
end
test "proxy logs last 10 lines" do
assert_equal \
"docker logs kamal-proxy --tail 10 --timestamps 2>&1",
new_command.logs(lines: 10).join(" ")
end
test "proxy logs with grep hello!" do
assert_equal \
"docker logs kamal-proxy --timestamps 2>&1 | grep 'hello!'",
new_command.logs(grep: "hello!").join(" ")
end
test "proxy remove container" do
assert_equal \
"docker container prune --force --filter label=org.opencontainers.image.title=kamal-proxy",
new_command.remove_container.join(" ")
end
test "proxy remove image" do
assert_equal \
"docker image prune --all --force --filter label=org.opencontainers.image.title=kamal-proxy",
new_command.remove_image.join(" ")
end
test "proxy follow logs" do
assert_equal \
"ssh -t root@1.1.1.1 -p 22 'docker logs kamal-proxy --timestamps --tail 10 --follow 2>&1'",
new_command.follow_logs(host: @config[:servers].first)
end
test "proxy follow logs with grep hello!" do
assert_equal \
"ssh -t root@1.1.1.1 -p 22 'docker logs kamal-proxy --timestamps --tail 10 --follow 2>&1 | grep \"hello!\"'",
new_command.follow_logs(host: @config[:servers].first, grep: "hello!")
end
test "deploy" do
assert_equal \
"docker exec kamal-proxy kamal-proxy deploy service --target \"172.1.0.2:80\"",
new_command.deploy("service", target: "172.1.0.2:80").join(" ")
end
test "remove" do
assert_equal \
"docker exec kamal-proxy kamal-proxy remove service --target \"172.1.0.2:80\"",
new_command.remove("service", target: "172.1.0.2:80").join(" ")
end
private
def new_command
Kamal::Commands::Proxy.new(Kamal::Configuration.new(@config, version: "123"))
end
end

21
test/fixtures/deploy_with_aliases.yml vendored Normal file
View File

@@ -0,0 +1,21 @@
service: app
image: dhh/app
servers:
web:
- 1.1.1.1
- 1.1.1.2
workers:
hosts:
- 1.1.1.3
- 1.1.1.4
console:
hosts:
- 1.1.1.5
registry:
username: user
password: pw
aliases:
info: details
console: app exec --reuse -p -r console "bin/console"
exec: app exec --reuse -p -r console
rails: app exec --reuse -p -r console rails

42
test/fixtures/deploy_with_proxy.yml vendored Normal file
View File

@@ -0,0 +1,42 @@
service: app
image: dhh/app
servers:
web:
- "1.1.1.1"
- "1.1.1.2"
workers:
- "1.1.1.3"
- "1.1.1.4"
registry:
username: user
password: pw
proxy:
enabled: true
hosts:
- "1.1.1.1"
deploy_timeout: 6s
accessories:
mysql:
image: mysql:5.7
host: 1.1.1.3
port: 3306
env:
clear:
MYSQL_ROOT_HOST: '%'
secret:
- MYSQL_ROOT_PASSWORD
files:
- test/fixtures/files/my.cnf:/etc/mysql/my.cnf
directories:
- data:/var/lib/mysql
redis:
image: redis:latest
roles:
- web
port: 6379
directories:
- data:/data
readiness_delay: 0

View File

@@ -29,6 +29,5 @@ class BrokenDeployTest < IntegrationTest
assert_match /First web container is unhealthy on vm[12], not booting any other roles/, output assert_match /First web container is unhealthy on vm[12], not booting any other roles/, output
assert_match "First web container is unhealthy, not booting workers on vm3", output assert_match "First web container is unhealthy, not booting workers on vm3", output
assert_match "nginx: [emerg] unexpected end of file, expecting \";\" or \"}\" in /etc/nginx/conf.d/default.conf:2", output assert_match "nginx: [emerg] unexpected end of file, expecting \";\" or \"}\" in /etc/nginx/conf.d/default.conf:2", output
assert_match 'ERROR {"Status":"unhealthy","FailingStreak":0,"Log":[]}', output
end end
end end

View File

@@ -1,8 +1,4 @@
#!/bin/sh #!/bin/sh
echo "About to lock..." echo "About to lock..."
if [ "$KAMAL_HOSTS" != "vm1,vm2" ]; then
echo "Expected hosts to be 'vm1,vm2', got $KAMAL_HOSTS"
exit 1
fi
mkdir -p /tmp/${TEST_ID} && touch /tmp/${TEST_ID}/pre-connect mkdir -p /tmp/${TEST_ID} && touch /tmp/${TEST_ID}/pre-connect

View File

@@ -6,4 +6,4 @@ ARG COMMIT_SHA
RUN echo $COMMIT_SHA > /usr/share/nginx/html/version RUN echo $COMMIT_SHA > /usr/share/nginx/html/version
RUN mkdir -p /usr/share/nginx/html/versions && echo "version" > /usr/share/nginx/html/versions/$COMMIT_SHA RUN mkdir -p /usr/share/nginx/html/versions && echo "version" > /usr/share/nginx/html/versions/$COMMIT_SHA
RUN mkdir -p /usr/share/nginx/html/versions && echo "hidden" > /usr/share/nginx/html/versions/.hidden RUN mkdir -p /usr/share/nginx/html/versions && echo "hidden" > /usr/share/nginx/html/versions/.hidden
RUN echo "Up!" > /usr/share/nginx/html/up

View File

@@ -1,8 +1,4 @@
#!/bin/sh #!/bin/sh
echo "About to lock..." echo "About to lock..."
if [ "$KAMAL_HOSTS" != "vm1,vm2,vm3" ]; then
echo "Expected hosts to be 'vm1,vm2,vm3', got $KAMAL_HOSTS"
exit 1
fi
mkdir -p /tmp/${TEST_ID} && touch /tmp/${TEST_ID}/pre-connect mkdir -p /tmp/${TEST_ID} && touch /tmp/${TEST_ID}/pre-connect

View File

@@ -6,4 +6,4 @@ ARG COMMIT_SHA
RUN echo $COMMIT_SHA > /usr/share/nginx/html/version RUN echo $COMMIT_SHA > /usr/share/nginx/html/version
RUN mkdir -p /usr/share/nginx/html/versions && echo "version" > /usr/share/nginx/html/versions/$COMMIT_SHA RUN mkdir -p /usr/share/nginx/html/versions && echo "version" > /usr/share/nginx/html/versions/$COMMIT_SHA
RUN mkdir -p /usr/share/nginx/html/versions && echo "hidden" > /usr/share/nginx/html/versions/.hidden RUN mkdir -p /usr/share/nginx/html/versions && echo "hidden" > /usr/share/nginx/html/versions/.hidden
RUN echo "Up!" > /usr/share/nginx/html/up

View File

@@ -9,6 +9,11 @@ servers:
hosts: hosts:
- vm3 - vm3
cmd: sleep infinity cmd: sleep infinity
proxy:
enabled: true
hosts:
- vm2
deploy_timeout: 2s
asset_path: /usr/share/nginx/html/versions asset_path: /usr/share/nginx/html/versions
@@ -37,3 +42,7 @@ accessories:
- web - web
stop_wait_time: 1 stop_wait_time: 1
readiness_delay: 0 readiness_delay: 0
aliases:
whome: version
worker_hostname: app exec -r workers -q --reuse hostname
uname: server exec -q -p uname

View File

@@ -31,7 +31,7 @@ class IntegrationTest < ActiveSupport::TestCase
succeeded = system("cd test/integration && #{command}") succeeded = system("cd test/integration && #{command}")
end end
raise "Command `#{command}` failed with error code `#{$?}`" if !succeeded && raise_on_error raise "Command `#{command}` failed with error code `#{$?}`, and output:\n#{result}" if !succeeded && raise_on_error
result result
end end

View File

@@ -82,6 +82,22 @@ class MainTest < IntegrationTest
assert_equal({ "cmd"=>"wget -qO- http://localhost > /dev/null || exit 1", "interval"=>"1s", "max_attempts"=>3, "port"=>3000, "path"=>"/up", "cord"=>"/tmp/kamal-cord", "log_lines"=>50 }, config[:healthcheck]) assert_equal({ "cmd"=>"wget -qO- http://localhost > /dev/null || exit 1", "interval"=>"1s", "max_attempts"=>3, "port"=>3000, "path"=>"/up", "cord"=>"/tmp/kamal-cord", "log_lines"=>50 }, config[:healthcheck])
end end
test "aliases" do
@app = "app_with_roles"
kamal :envify
kamal :deploy
output = kamal :whome, capture: true
assert_equal Kamal::VERSION, output
output = kamal :worker_hostname, capture: true
assert_match /App Host: vm3\nvm3-[0-9a-f]{12}$/, output
output = kamal :uname, "-o", capture: true
assert_match "App Host: vm1\nGNU/Linux", output
end
test "setup and remove" do test "setup and remove" do
# Check remove completes when nothing has been setup yet # Check remove completes when nothing has been setup yet
kamal :remove, "-y" kamal :remove, "-y"

View File

@@ -0,0 +1,84 @@
require_relative "integration_test"
class ProxyTest < IntegrationTest
setup do
@app = "app_with_roles"
end
test "boot, reboot, stop, start, restart, logs, remove" do
kamal :envify
kamal :proxy, :boot
assert_proxy_running
output = kamal :proxy, :reboot, "-y", "--verbose", capture: true
assert_proxy_running
assert_hooks_ran "pre-traefik-reboot", "post-traefik-reboot"
assert_match /Rebooting Traefik on vm1,vm2.../, output
assert_match /Rebooted Traefik on vm1,vm2/, output
output = kamal :proxy, :reboot, "--rolling", "-y", "--verbose", capture: true
assert_proxy_running
assert_hooks_ran "pre-traefik-reboot", "post-traefik-reboot"
assert_match /Rebooting Traefik on vm1.../, output
assert_match /Rebooted Traefik on vm1/, output
assert_match /Rebooting Traefik on vm2.../, output
assert_match /Rebooted Traefik on vm2/, output
kamal :proxy, :boot
assert_proxy_running
assert_traefik_running
# Check booting when booted doesn't raise an error
kamal :proxy, :stop
assert_proxy_not_running
assert_traefik_not_running
# Check booting when stopped works
kamal :proxy, :boot
assert_proxy_running
assert_traefik_running
kamal :proxy, :stop
assert_proxy_not_running
assert_traefik_not_running
kamal :proxy, :start
assert_proxy_running
assert_traefik_running
kamal :proxy, :restart
assert_proxy_running
assert_traefik_running
logs = kamal :proxy, :logs, capture: true
assert_match /Traefik version [\d.]+ built on/, logs
kamal :proxy, :remove
assert_proxy_not_running
assert_traefik_not_running
kamal :env, :delete
end
private
def assert_proxy_running
assert_match /basecamp\/kamal-proxy:latest \"kamal-proxy run\"/, proxy_details
end
def assert_proxy_not_running
assert_no_match /basecamp\/kamal-proxy:latest \"kamal-proxy run\"/, proxy_details
end
def assert_traefik_running
assert_match /traefik:v2.10 "\/entrypoint.sh/, proxy_details
end
def assert_traefik_not_running
assert_no_match /traefik:v2.10 "\/entrypoint.sh/, proxy_details
end
def proxy_details
kamal :proxy, :details, capture: true
end
end