Compare commits

..

44 Commits

Author SHA1 Message Date
David Heinemeier Hansson
60a19f0b30 Bump version for 0.11.0 2023-04-12 11:45:33 +02:00
David Heinemeier Hansson
2d0a7e1b67 Merge pull request #208 from tannakartikey/add_labels_to_traefik
Labels can be added to Traefik
2023-04-12 11:35:28 +02:00
David Heinemeier Hansson
49df19fb0d Merge pull request #209 from ncreuschling/fix-roles-documentation
fix typo role to roles
2023-04-12 11:34:02 +02:00
David Heinemeier Hansson
cef8fddfb4 Merge pull request #210 from basecamp/role-aware-rollbacks
Make rollbacks role-aware
2023-04-12 11:33:45 +02:00
Kartikey Tanna
c59eb00dd0 Labels can be added to Traefik 2023-04-12 14:53:48 +05:30
Donal McBreen
43f7409de0 Make rollbacks role-aware
Rollbacks stopped working after https://github.com/mrsked/mrsk/pull/99.

We'll confirm that a container is available for the first role on the
primary host before attempting to rollback.
2023-04-12 09:59:39 +01:00
Nicolai Reuschling
448ea7719f fix typo role to roles 2023-04-12 10:53:10 +02:00
David Heinemeier Hansson
ab54dbdb8b Merge pull request #206 from tannakartikey/traefik_rule_docs
Explained the latest modifications of Traefik container labels
2023-04-11 14:18:31 +02:00
David Heinemeier Hansson
ac3771447a Merge pull request #203 from matharvard/main
Require net-ssh ~> 7.0 for SHA-2 support
2023-04-11 14:17:52 +02:00
David Heinemeier Hansson
daa0c9b5be Merge pull request #196 from handy-la/main
Configurable max_attempts for healthcheck
2023-04-11 14:17:17 +02:00
Kartikey Tanna
d3936363d0 Explained the latest modifications of Traefik container labels 2023-04-11 10:20:16 +05:30
Arturo Ojeda
cfc8fa0590 Remove .idea folder 2023-04-10 22:33:20 -06:00
Arturo Ojeda
161ebe4bc1 Updated README.md with new healthcheck.max_attempts option 2023-04-10 22:26:10 -06:00
Arturo Ojeda
514b2aa243 Fix test case: console output message was not updated to display the current/total attempts 2023-04-10 09:29:19 -06:00
David Heinemeier Hansson
18031bc552 Merge pull request #202 from basecamp/deploy-lock-acquisition
Improved deploy lock acquisition
2023-04-10 16:42:03 +02:00
Mat Harvard
d8c61004e4 Require net-ssh ~> 7.0 for SHA-2 support
Versions of net-ssh before 7.0 do not support the SHA-2 algorithm and result in mrsk not being able to connect to hosts using keys generated with it. net-ssh is also a dependency of sshkit, however, sshkit has a version requirement of >= 2.8.0 for net-ssh, so is not effective at ensuring mrsk has the version it needs to be the most compatible.
2023-04-10 07:29:07 -07:00
Donal McBreen
c4df440c79 Improved deploy lock acquisition
1. Don't raise lock error for non-lock issues during lock acquire
  (see https://github.com/mrsked/mrsk/pull/181)
2. If there is an error while the lock is held, don't release the lock
  and send a warning to stderr
2023-04-10 15:23:00 +01:00
David Heinemeier Hansson
fb1718ca6d Merge pull request #197 from tannakartikey/traefik_rules_with_destination
Traefik service name to be derived from role and destination
2023-04-10 15:11:07 +02:00
David Heinemeier Hansson
7d17a6c3b5 Excess CR 2023-04-10 15:10:08 +02:00
David Heinemeier Hansson
f4133de896 Merge pull request #176 from dilpreet92/enable_ssh_over_proxy_command
Enable ssh over proxy command
2023-04-10 14:41:45 +02:00
David Heinemeier Hansson
a9488e935d Style 2023-04-10 14:39:18 +02:00
David Heinemeier Hansson
ac61528dfc Merge pull request #189 from basecamp/traefik-image
Traefik image config for version pinning, upgrades, and custom images
2023-04-10 14:35:30 +02:00
David Heinemeier Hansson
0eb7a8d087 Merge branch 'main' into pr/176
* main:
  Simpler
  Make it explicit, focus on Ubuntu
  More explicit
  Not that --bundle is a Rails 7+ option
  Update README.md
  Update README.md
  Add github discussions link to readme
  Bump debug to fix missing deps in CI
  Only redact the non-sensitive bits of build args and env vars.
  improve code sample (traefik configuration)
2023-04-10 14:31:43 +02:00
David Heinemeier Hansson
7559f439e9 Merge pull request #195 from nickhammond/patch-1
Add github discussions link to readme
2023-04-10 14:28:59 +02:00
David Heinemeier Hansson
54a5b90d8f Simpler 2023-04-10 14:28:52 +02:00
David Heinemeier Hansson
a245adfad2 Merge pull request #200 from huksley/main
Add sample commands to bootstrap non-root ssh server
2023-04-10 14:27:13 +02:00
David Heinemeier Hansson
f386c3bdab Make it explicit, focus on Ubuntu 2023-04-10 14:26:49 +02:00
David Heinemeier Hansson
2a3e576182 More explicit 2023-04-10 14:24:51 +02:00
David Heinemeier Hansson
f3e3196ce5 Not that --bundle is a Rails 7+ option 2023-04-10 14:22:58 +02:00
Ruslan Gainutdinov
fca5b11682 Update README.md
Use docker.io on Ubuntu
2023-04-10 12:26:57 +03:00
Ruslan Gainutdinov
d09cddde8d Update README.md
Add sample commands to bootstrap non-root ssh server.
2023-04-10 12:23:06 +03:00
Arturo Ojeda
3969f56fa6 Improved: configurable max_attempts for healthcheck 2023-04-09 12:07:27 -06:00
Kartikey Tanna
c60cc92dfe Traefik service name to be derived from role and destination 2023-04-09 13:44:57 +05:30
Arturo Ojeda
cb3c5a53f4 Configurable max_attempts for healthcheck 2023-04-08 19:52:53 -06:00
Nick Hammond
ef04410d77 Add github discussions link to readme
I realize that there's a discussions link on github but I didn't realize mrsk actually utilized it until I saw it mentioned on Discord. I was thinking adding it to the readme would help push people there.
2023-04-08 13:33:31 -07:00
Jeremy Daer
bd8f13dd5e Traefik image config for version pinning, upgrades, and custom images
Accounts for the 2.9.10 security release and allows testing Traefik 3 betas.

* Use `image` to configure a specific Traefik Docker image.
* Default to `traefik:v2.9` to track future 2.9.x minor releases rather
  than tightly pinning to `v2.9.9`.
* Support images from the configured registry.

References #165
2023-04-07 14:15:25 -07:00
David Heinemeier Hansson
2146f6d0ec Merge pull request #182 from basecamp/sensitive-args
Only redact the non-sensitive bits of build args and env vars.
2023-04-06 16:19:41 +02:00
David Heinemeier Hansson
52d8c112d3 Merge branch 'main' into pr/182
* main:
  Bump debug to fix missing deps in CI
2023-04-06 16:18:22 +02:00
David Heinemeier Hansson
c9afd66222 Merge pull request #184 from basecamp/fix-ci 2023-04-06 13:21:26 +02:00
Jeremy Daer
36c458407f Bump debug to fix missing deps in CI 2023-04-05 12:00:15 -07:00
Jeremy Daer
c137b38c87 Only redact the non-sensitive bits of build args and env vars.
* `-e [REDACTED]` → `-e SOME_SECRET=[REDACTED]`
* Replaces `Utils.redact` with `Utils.sensitive` to clarify that we're
  indicating redactability, not actually performing redaction.
* Redacts from YAML output, including `mrsk config` (fixes #96)
2023-04-05 09:45:28 -07:00
David Heinemeier Hansson
f851d6528d Merge pull request #169 from ncreuschling/patch-1
improve code sample (traefik configuration)
2023-04-05 16:31:10 +02:00
Dilpreet Singh
12632aa7f9 Enable ssh over proxy command 2023-04-03 17:14:06 +05:30
Nicolai Reuschling
2f97bc488f improve code sample (traefik configuration)
fixed yaml format (code sample traefik configuration)
2023-03-31 11:50:43 +02:00
28 changed files with 352 additions and 109 deletions

View File

@@ -1,11 +1,12 @@
PATH
remote: .
specs:
mrsk (0.10.1)
mrsk (0.11.0)
activesupport (>= 7.0)
bcrypt_pbkdf (~> 1.0)
dotenv (~> 2.8)
ed25519 (~> 1.2)
net-ssh (~> 7.0)
sshkit (~> 1.21)
thor (~> 1.2)
zeitwerk (~> 2.5)
@@ -35,13 +36,18 @@ GEM
builder (3.2.4)
concurrent-ruby (1.2.2)
crass (1.0.6)
debug (1.7.1)
debug (1.7.2)
irb (>= 1.5.0)
reline (>= 0.3.1)
dotenv (2.8.1)
ed25519 (1.3.0)
erubi (1.12.0)
i18n (1.12.0)
concurrent-ruby (~> 1.0)
loofah (2.19.1)
io-console (0.6.0)
irb (1.6.3)
reline (>= 0.3.0)
loofah (2.20.0)
crass (~> 1.0.2)
nokogiri (>= 1.5.9)
method_source (1.0.0)
@@ -74,6 +80,8 @@ GEM
thor (~> 1.0)
zeitwerk (~> 2.5)
rake (13.0.6)
reline (0.3.3)
io-console (~> 0.5)
ruby2_keywords (0.0.5)
sshkit (1.21.4)
net-scp (>= 1.1.2)

View File

@@ -6,6 +6,8 @@ Watch the screencast: https://www.youtube.com/watch?v=LL1cV2FXZ5I
Join us on Discord: https://discord.gg/YgHVT7GCXS
Ask questions: https://github.com/mrsked/mrsk/discussions
## Installation
If you have a Ruby environment available, you can install MRSK globally with:
@@ -14,13 +16,13 @@ If you have a Ruby environment available, you can install MRSK globally with:
gem install mrsk
```
...otherwise, you can run a dockerized version via an alias (add this to your ${SHELL}rc to simplify re-use):
...otherwise, you can run a dockerized version via an alias (add this to your .bashrc or similar to simplify re-use):
```sh
alias mrsk='docker run --rm -it -v $HOME/.ssh:/root/.ssh -v /var/run/docker.sock:/var/run/docker.sock -v ${PWD}/:/workdir ghcr.io/mrsked/mrsk'
```
Then, inside your app directory, run `mrsk init` (or `mrsk init --bundle` within Rails apps where you want a bin/mrsk binstub). Now edit the new file `config/deploy.yml`. It could look as simple as this:
Then, inside your app directory, run `mrsk init` (or `mrsk init --bundle` within Rails 7+ apps where you want a bin/mrsk binstub). Now edit the new file `config/deploy.yml`. It could look as simple as this:
```yaml
service: hey
@@ -191,6 +193,15 @@ ssh:
user: app
```
If you are using non-root user, you need to bootstrap your servers manually, before using them with MRSK. On Ubuntu, you'd do:
```bash
sudo apt update
sudo apt upgrade -y
sudo apt install -y docker.io curl git
sudo usermod -a -G docker ubuntu
```
### Using a proxy SSH host
If you need to connect to server through a proxy host, you can use `ssh/proxy`:
@@ -207,6 +218,13 @@ ssh:
proxy: "app@192.168.0.1"
```
Also if you need specific proxy command to connect to the server:
```yaml
ssh:
proxy_command: aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p' --region=us-east-1 ## ssh via aws ssm
```
### Using env variables
You can inject env variables into the app containers using `env`:
@@ -288,8 +306,9 @@ You can specialize the default Traefik rules by setting labels on the containers
```yaml
labels:
traefik.http.routers.hey.rule: Host(`app.hey.com`)
traefik.http.routers.hey-web.rule: Host(`app.hey.com`)
```
Traefik rules are in the "service-role-destination" format. The default role will be `web` if no rule is specified. If the destination is not specified, it is not included. To give an example, the above rule would become "traefik.http.routers.hey-web.rule" if it was for the "staging" destination.
Note: The backticks are needed to ensure the rule is passed in correctly and not treated as command substitution by Bash!
@@ -439,9 +458,9 @@ RUN --mount=type=secret,id=GITHUB_TOKEN \
rm -rf /usr/local/bundle/cache
```
### Using command arguments for Traefik
### Traefik command arguments
You can customize the traefik command line:
Customize the Traefik command line using `args`:
```yaml
traefik:
@@ -450,37 +469,70 @@ traefik:
accesslog.format: json
```
This will start the traefik container with `--accesslog=true accesslog.format=json`.
This starts the Traefik container with `--accesslog=true --accesslog.format=json` arguments.
### Traefik's host port binding
### Traefik host port binding
By default Traefik binds to port 80 of the host machine, it can be configured to use an alternative port:
Traefik binds to port 80 by default. Specify an alternative port using `host_port`:
```yaml
traefik:
host_port: 8080
```
### Configure docker options for traefik
### Traefik version, upgrades, and custom images
We allow users to pass additional docker options to the trafik container like
MRSK runs the traefik:v2.9 image to track Traefik 2.9.x releases.
To pin Traefik to a specific version or an image published to your registry,
specify `image`:
```yaml
traefik:
image: traefik:v2.10.0-rc1
```
This is useful for downgrading Traefik if there's an unexpected breaking
change in a minor version release, upgrading Traefik to test forthcoming
releases, or running your own Traefik-derived image.
MRSK has not been tested for compatibility with Traefik 3 betas. Please do!
### Traefik container configuration
Pass additional Docker configuration for the Traefik container using `options`:
```yaml
traefik:
options:
publish:
- 8080:8080
- 8080:8080
volumes:
- /tmp/example.json:/tmp/example.json
- /tmp/example.json:/tmp/example.json
memory: 512m
```
This will start the traefik container with a command like: `docker run ... --volume /tmp/example.json:/tmp/example.json --publish 8080:8080 `
This starts the Traefik container with `--volume /tmp/example.json:/tmp/example.json --publish 8080:8080 --memory 512m` arguments to `docker run`.
### Traefik container lables
### Configure alternate entrypoints for traefik
Add labels to Traefik Docker container.
You can configure multiple entrypoints for traefik like so:
```yaml
traefik:
lables:
- traefik.enable: true
- traefik.http.routers.dashboard.rule: Host(`traefik.example.com`) && (PathPrefix(`/api`) || PathPrefix(`/dashboard`))
- traefik.http.routers.dashboard.service: api@internal
- traefik.http.routers.dashboard.middlewares: auth
- traefik.http.middlewares.auth.basicauth.users: test:$2y$05$H2o72tMaO.TwY1wNQUV1K.fhjRgLHRDWohFvUZOJHBEtUXNKrqUKi # test:password
```
This labels Traefik container with `--label traefik.http.routers.dashboard.middlewares=\"auth\"` and so on.
### Traefik alternate entrypoints
You can configure multiple entrypoints for Traefik like so:
```yaml
service: myservice
@@ -540,7 +592,7 @@ accessories:
memory: "2GB"
redis:
image: redis:latest
role:
roles:
- web
port: "36379:6379"
volumes:
@@ -610,18 +662,21 @@ That'll post a line like follows to a preconfigured chatbot in Basecamp:
[My App] [dhh] Rolled back to version d264c4e92470ad1bd18590f04466787262f605de
```
### Using custom healthcheck path or port
### Custom healthcheck
MRSK defaults to checking the health of your application again `/up` on port 3000. You can tailor both with the `healthcheck` setting:
MRSK defaults to checking the health of your application again `/up` on port 3000 up to 7 times. You can tailor the behaviour with the `healthcheck` setting:
```yaml
healthcheck:
path: /healthz
port: 4000
max_attempts: 7
```
This will ensure your application is configured with a traefik label for the healthcheck against `/healthz` and that the pre-deploy healthcheck that MRSK performs is done against the same path on port 4000.
The healthcheck also allows for an optional `max_attempts` setting, which will attempt the healthcheck up to the specified number of times before failing the deploy. This is useful for applications that take a while to start up. The default is 7.
## Commands
### Running commands on servers

View File

@@ -1,4 +1,5 @@
module Mrsk::Cli
class LockError < StandardError; end
end
# SSHKit uses instance eval, so we need a global const for ergonomics

View File

@@ -6,8 +6,6 @@ module Mrsk::Cli
class Base < Thor
include SSHKit::DSL
class LockError < StandardError; end
def self.exit_on_failure?() true end
class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging"
@@ -82,8 +80,11 @@ module Mrsk::Cli
acquire_lock
yield
ensure
release_lock
rescue
error " \e[31mDeploy lock was not released\e[0m" if MRSK.lock_count > 0
raise
end
def acquire_lock
@@ -95,9 +96,10 @@ module Mrsk::Cli
rescue SSHKit::Runner::ExecuteError => e
if e.message =~ /cannot create directory/
invoke "mrsk:cli:lock:status", []
raise LockError, "Deploy lock found"
else
raise e
end
raise LockError, "Deploy lock found"
end
def release_lock

View File

@@ -1,5 +1,4 @@
class Mrsk::Cli::Healthcheck < Mrsk::Cli::Base
MAX_ATTEMPTS = 7
class HealthcheckError < StandardError; end
@@ -13,6 +12,7 @@ class Mrsk::Cli::Healthcheck < Mrsk::Cli::Base
target = "Health check against #{MRSK.config.healthcheck["path"]}"
attempt = 1
max_attempts = MRSK.config.healthcheck["max_attempts"]
begin
status = capture_with_info(*MRSK.healthcheck.curl)
@@ -23,8 +23,8 @@ class Mrsk::Cli::Healthcheck < Mrsk::Cli::Base
raise HealthcheckError, "#{target} failed with status #{status}"
end
rescue SSHKit::Command::Failed
if attempt <= MAX_ATTEMPTS
info "#{target} failed to respond, retrying in #{attempt}s..."
if attempt <= max_attempts
info "#{target} failed to respond, retrying in #{attempt}s (attempt #{attempt}/#{max_attempts})..."
sleep attempt
attempt += 1

View File

@@ -12,7 +12,7 @@ class Mrsk::Cli::Lock < Mrsk::Cli::Base
message = options[:message]
handle_missing_lock do
on(MRSK.primary_host) { execute *MRSK.lock.acquire(message, MRSK.config.version) }
say "Set the deploy lock"
say "Acquired the deploy lock"
end
end
@@ -20,7 +20,7 @@ class Mrsk::Cli::Lock < Mrsk::Cli::Base
def release
handle_missing_lock do
on(MRSK.primary_host) { execute *MRSK.lock.release }
say "Removed the deploy lock"
say "Released the deploy lock"
end
end

View File

@@ -77,21 +77,26 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
with_lock do
MRSK.config.version = version
if container_name_available?(MRSK.config.service_with_version)
if container_available?(version)
say "Start version #{version}, then wait #{MRSK.config.readiness_delay}s for app to boot before stopping the old version...", :magenta
cli = self
old_version = nil
on(MRSK.hosts) do |host|
old_version = capture_with_info(*MRSK.app.current_running_version).strip.presence
roles = MRSK.roles_on(host)
execute *MRSK.app.start
roles.each do |role|
app = MRSK.app(role: role)
old_version = capture_with_info(*app.current_running_version).strip.presence
if old_version
sleep MRSK.config.readiness_delay
execute *app.start
execute *MRSK.app.stop(version: old_version), raise_on_non_zero_exit: false
if old_version
sleep MRSK.config.readiness_delay
execute *app.stop(version: old_version), raise_on_non_zero_exit: false
end
end
end
@@ -119,7 +124,7 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
desc "config", "Show combined config (including secrets!)"
def config
run_locally do
puts MRSK.config.to_h.to_yaml
puts Mrsk::Utils.redacted(MRSK.config.to_h).to_yaml
end
end
@@ -214,10 +219,15 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
subcommand "lock", Mrsk::Cli::Lock
private
def container_name_available?(container_name, host: MRSK.primary_host)
container_names = nil
on(host) { container_names = capture_with_info(*MRSK.app.list_container_names).split("\n") }
Array(container_names).include?(container_name)
def container_available?(version, host: MRSK.primary_host)
available = nil
on(host) do
first_role = MRSK.roles_on(host).first
available = capture_with_info(*MRSK.app(role: first_role).container_id_for_version(version)).present?
end
available
end
def deploy_options

View File

@@ -2,7 +2,10 @@ class Mrsk::Cli::Traefik < Mrsk::Cli::Base
desc "boot", "Boot Traefik on servers"
def boot
with_lock do
on(MRSK.traefik_hosts) { execute *MRSK.traefik.run, raise_on_non_zero_exit: false }
on(MRSK.traefik_hosts) do
execute *MRSK.registry.login
execute *MRSK.traefik.run, raise_on_non_zero_exit: false
end
end
end

View File

@@ -1,6 +1,6 @@
module Mrsk::Commands
class Base
delegate :redact, :argumentize, to: Mrsk::Utils
delegate :sensitive, :argumentize, to: Mrsk::Utils
attr_accessor :config

View File

@@ -28,7 +28,7 @@ class Mrsk::Commands::Builder::Base < Mrsk::Commands::Base
end
def build_args
argumentize "--build-arg", args, redacted: true
argumentize "--build-arg", args, sensitive: true
end
def build_secrets

View File

@@ -2,7 +2,7 @@ class Mrsk::Commands::Registry < Mrsk::Commands::Base
delegate :registry, to: :config
def login
docker :login, registry["server"], "-u", redact(lookup("username")), "-p", redact(lookup("password"))
docker :login, registry["server"], "-u", sensitive(lookup("username")), "-p", sensitive(lookup("password"))
end
def logout

View File

@@ -1,7 +1,7 @@
class Mrsk::Commands::Traefik < Mrsk::Commands::Base
delegate :optionize, to: Mrsk::Utils
delegate :argumentize, :optionize, to: Mrsk::Utils
IMAGE = "traefik:v2.9.9"
DEFAULT_IMAGE = "traefik:v2.9"
CONTAINER_PORT = 80
def run
@@ -11,8 +11,9 @@ class Mrsk::Commands::Traefik < Mrsk::Commands::Base
"--publish", port,
"--volume", "/var/run/docker.sock:/var/run/docker.sock",
*config.logging_args,
*label_args,
*docker_options_args,
IMAGE,
image,
"--providers.docker",
"--log.level=DEBUG",
*cmd_option_args
@@ -56,6 +57,18 @@ class Mrsk::Commands::Traefik < Mrsk::Commands::Base
end
private
def label_args
argumentize "--label", labels
end
def labels
config.traefik["labels"] || []
end
def image
config.traefik.fetch("image") { DEFAULT_IMAGE }
end
def docker_options_args
optionize(config.traefik["options"] || {})
end

View File

@@ -143,6 +143,8 @@ class Mrsk::Configuration
if raw_config.ssh.present? && raw_config.ssh["proxy"]
Net::SSH::Proxy::Jump.new \
raw_config.ssh["proxy"].include?("@") ? raw_config.ssh["proxy"] : "root@#{raw_config.ssh["proxy"]}"
elsif raw_config.ssh.present? && raw_config.ssh["proxy_command"]
Net::SSH::Proxy::Command.new(raw_config.ssh["proxy_command"])
end
end
@@ -156,7 +158,7 @@ class Mrsk::Configuration
end
def healthcheck
{ "path" => "/up", "port" => 3000 }.merge(raw_config.healthcheck || {})
{ "path" => "/up", "port" => 3000, "max_attempts" => 7 }.merge(raw_config.healthcheck || {})
end
def readiness_delay

View File

@@ -74,18 +74,22 @@ class Mrsk::Configuration::Role
def traefik_labels
if running_traefik?
{
"traefik.http.routers.#{config.service}.rule" => "PathPrefix(`/`)",
"traefik.http.services.#{config.service}.loadbalancer.healthcheck.path" => config.healthcheck["path"],
"traefik.http.services.#{config.service}.loadbalancer.healthcheck.interval" => "1s",
"traefik.http.middlewares.#{config.service}-retry.retry.attempts" => "5",
"traefik.http.middlewares.#{config.service}-retry.retry.initialinterval" => "500ms",
"traefik.http.routers.#{config.service}.middlewares" => "#{config.service}-retry@docker"
"traefik.http.routers.#{traefik_service}.rule" => "PathPrefix(`/`)",
"traefik.http.services.#{traefik_service}.loadbalancer.healthcheck.path" => config.healthcheck["path"],
"traefik.http.services.#{traefik_service}.loadbalancer.healthcheck.interval" => "1s",
"traefik.http.middlewares.#{traefik_service}-retry.retry.attempts" => "5",
"traefik.http.middlewares.#{traefik_service}-retry.retry.initialinterval" => "500ms",
"traefik.http.routers.#{traefik_service}.middlewares" => "#{traefik_service}-retry@docker"
}
else
{}
end
end
def traefik_service
[ config.service, name, config.destination ].compact.join("-")
end
def custom_labels
Hash.new.tap do |labels|
labels.merge!(config.labels) if config.labels.present?

View File

@@ -2,11 +2,12 @@ module Mrsk::Utils
extend self
# Return a list of escaped shell arguments using the same named argument against the passed attributes (hash or array).
def argumentize(argument, attributes, redacted: false)
def argumentize(argument, attributes, sensitive: false)
Array(attributes).flat_map do |key, value|
if value.present?
escaped_pair = [ key, escape_shell_value(value) ].join("=")
[ argument, redacted ? redact(escaped_pair) : escaped_pair ]
attr = "#{key}=#{escape_shell_value(value)}"
attr = self.sensitive(attr, redaction: "#{key}=[REDACTED]") if sensitive
[ argument, attr]
else
[ argument, key ]
end
@@ -17,7 +18,7 @@ module Mrsk::Utils
# but redacts and expands secrets.
def argumentize_env_with_secrets(env)
if (secrets = env["secret"]).present?
argumentize("-e", secrets.to_h { |key| [ key, ENV.fetch(key) ] }, redacted: true) + argumentize("-e", env["clear"])
argumentize("-e", secrets.to_h { |key| [ key, ENV.fetch(key) ] }, sensitive: true) + argumentize("-e", env["clear"])
else
argumentize "-e", env.fetch("clear", env)
end
@@ -39,9 +40,37 @@ module Mrsk::Utils
args.flat_map { |key, value| value.try(:map) { |entry| [key, entry] } || [ [ key, value ] ] }
end
# Copied from SSHKit::Backend::Abstract#redact to be available inside Commands classes
def redact(arg) # Used in execute_command to hide redact() args a user passes in
arg.to_s.extend(SSHKit::Redaction) # to_s due to our inability to extend Integer, etc
# Marks sensitive values for redaction in logs and human-visible output.
# Pass `redaction:` to change the default `"[REDACTED]"` redaction, e.g.
# `sensitive "#{arg}=#{secret}", redaction: "#{arg}=xxxx"
def sensitive(...)
Mrsk::Utils::Sensitive.new(...)
end
def redacted(value)
case
when value.respond_to?(:redaction)
value.redaction
when value.respond_to?(:transform_values)
value.transform_values { |value| redacted value }
when value.respond_to?(:map)
value.map { |element| redacted element }
else
value
end
end
def unredacted(value)
case
when value.respond_to?(:unredacted)
value.unredacted
when value.respond_to?(:transform_values)
value.transform_values { |value| unredacted value }
when value.respond_to?(:map)
value.map { |element| unredacted element }
else
value
end
end
# Escape a value to make it safe for shell use.

View File

@@ -0,0 +1,19 @@
require "active_support/core_ext/module/delegation"
class Mrsk::Utils::Sensitive
# So SSHKit knows to redact these values.
include SSHKit::Redaction
attr_reader :unredacted, :redaction
delegate :to_s, to: :unredacted
delegate :inspect, to: :redaction
def initialize(value, redaction: "[REDACTED]")
@unredacted, @redaction = value, redaction
end
# Sensitive values won't leak into YAML output.
def encode_with(coder)
coder.represent_scalar nil, redaction
end
end

View File

@@ -1,3 +1,3 @@
module Mrsk
VERSION = "0.10.1"
VERSION = "0.11.0"
end

View File

@@ -14,6 +14,7 @@ Gem::Specification.new do |spec|
spec.add_dependency "activesupport", ">= 7.0"
spec.add_dependency "sshkit", "~> 1.21"
spec.add_dependency "net-ssh", "~> 7.0"
spec.add_dependency "thor", "~> 1.2"
spec.add_dependency "dotenv", "~> 2.8"
spec.add_dependency "zeitwerk", "~> 2.5"

View File

@@ -22,4 +22,8 @@ class CliTestCase < ActiveSupport::TestCase
def stdouted
capture(:stdout) { yield }.strip
end
end
def stderred
capture(:stderr) { yield }.strip
end
end

View File

@@ -23,8 +23,8 @@ class CliHealthcheckTest < CliTestCase
.returns("200")
run_command("perform").tap do |output|
assert_match "Health check against /up failed to respond, retrying in 1s...", output
assert_match "Health check against /up failed to respond, retrying in 2s...", output
assert_match "Health check against /up failed to respond, retrying in 1s (attempt 1/7)...", output
assert_match "Health check against /up failed to respond, retrying in 2s (attempt 2/7)...", output
assert_match "Health check against /up succeeded with 200 OK!", output
end
end

View File

@@ -53,6 +53,46 @@ class CliMainTest < CliTestCase
end
end
test "deploy when locked" do
Thread.report_on_exception = false
SSHKit::Backend::Abstract.any_instance.stubs(:execute)
.with { |*arg| arg[0..1] == [:mkdir, :mrsk_lock] }
.raises(RuntimeError, "mkdir: cannot create directory mrsk_lock: File exists")
Mrsk::Cli::Base.any_instance.expects(:invoke).with("mrsk:cli:lock:status", [])
assert_raises(Mrsk::Cli::LockError) do
run_command("deploy")
end
end
test "deploy error when locking" do
Thread.report_on_exception = false
SSHKit::Backend::Abstract.any_instance.stubs(:execute)
.with { |*arg| arg[0..1] == [:mkdir, :mrsk_lock] }
.raises(SocketError, "getaddrinfo: nodename nor servname provided, or not known")
assert_raises(SSHKit::Runner::ExecuteError) do
run_command("deploy")
end
end
test "deploy errors leave lock in place" do
invoke_options = { "config_file" => "test/fixtures/deploy_simple.yml", "skip_broadcast" => false, "version" => "999" }
Mrsk::Cli::Main.any_instance.expects(:invoke)
.with("mrsk:cli:server:bootstrap", [], invoke_options)
.raises(RuntimeError)
assert_equal 0, MRSK.lock_count
assert_raises(RuntimeError) do
stderred { run_command("deploy") }
end
assert_equal 1, MRSK.lock_count
end
test "redeploy" do
invoke_options = { "config_file" => "test/fixtures/deploy_simple.yml", "skip_broadcast" => false, "version" => "999" }
@@ -80,32 +120,34 @@ class CliMainTest < CliTestCase
end
test "rollback bad version" do
# Mrsk::Cli::Main.any_instance.stubs(:container_available?).returns(false)
run_command("details") # Preheat MRSK const
run_command("rollback", "nonsense").tap do |output|
assert_match /docker container ls --all --filter label=service=app --format '{{ .Names }}'/, output
assert_match /docker container ls --all --filter name=\^app-web-nonsense\$ --quiet/, output
assert_match /The app version 'nonsense' is not available as a container/, output
end
end
test "rollback good version" do
Mrsk::Cli::Main.any_instance.stubs(:container_name_available?).returns(true)
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info).with(:docker, :ps, "--filter", "label=service=app", "--format", "\"{{.Names}}\"", "|", "sed 's/-/\\n/g'", "|", "tail -n 1").returns("version-to-rollback\n").at_least_once
Mrsk::Cli::Main.any_instance.stubs(:container_available?).returns(true)
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info).with(:docker, :ps, "--filter", "label=service=app", "--filter", "label=role=web", "--format", "\"{{.Names}}\"", "|", "sed 's/-/\\n/g'", "|", "tail -n 1").returns("version-to-rollback\n").at_least_once
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info).with(:docker, :ps, "--filter", "label=service=app", "--filter", "label=role=workers", "--format", "\"{{.Names}}\"", "|", "sed 's/-/\\n/g'", "|", "tail -n 1").returns("version-to-rollback\n").at_least_once
run_command("rollback", "123", config_file: "deploy_with_accessories").tap do |output|
assert_match "Start version 123", output
assert_match "docker start app-123", output
assert_match "docker container ls --all --filter name=^app-version-to-rollback$ --quiet | xargs docker stop", output, "Should stop the container that was previously running"
assert_match "docker start app-web-123", output
assert_match "docker container ls --all --filter name=^app-web-version-to-rollback$ --quiet | xargs docker stop", output, "Should stop the container that was previously running"
end
end
test "rollback without old version" do
Mrsk::Cli::Main.any_instance.stubs(:container_name_available?).returns(true)
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info).with(:docker, :ps, "--filter", "label=service=app", "--format", "\"{{.Names}}\"", "|", "sed 's/-/\\n/g'", "|", "tail -n 1").returns("").at_least_once
Mrsk::Cli::Main.any_instance.stubs(:container_available?).returns(true)
SSHKit::Backend::Abstract.any_instance.expects(:capture_with_info).with(:docker, :ps, "--filter", "label=service=app", "--filter", "label=role=web", "--format", "\"{{.Names}}\"", "|", "sed 's/-/\\n/g'", "|", "tail -n 1").returns("").at_least_once
run_command("rollback", "123").tap do |output|
assert_match "Start version 123", output
assert_match "docker start app-123", output
assert_match "docker start app-web-123", output
assert_no_match "docker stop", output
end
end

View File

@@ -3,7 +3,8 @@ require_relative "cli_test_case"
class CliTraefikTest < CliTestCase
test "boot" do
run_command("boot").tap do |output|
assert_match "docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" traefik:v2.9.9 --providers.docker --log.level=DEBUG", output
assert_match "docker login", output
assert_match "docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" #{Mrsk::Commands::Traefik::DEFAULT_IMAGE} --providers.docker --log.level=DEBUG", output
end
end

View File

@@ -13,7 +13,7 @@ class CommandsAppTest < ActiveSupport::TestCase
test "run" do
assert_equal \
"docker run --detach --restart unless-stopped --name app-web-999 -e MRSK_CONTAINER_NAME=\"app-web-999\" -e RAILS_MASTER_KEY=\"456\" --log-opt max-size=\"10m\" --label service=\"app\" --label role=\"web\" --label traefik.http.routers.app.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.services.app.loadbalancer.healthcheck.path=\"/up\" --label traefik.http.services.app.loadbalancer.healthcheck.interval=\"1s\" --label traefik.http.middlewares.app-retry.retry.attempts=\"5\" --label traefik.http.middlewares.app-retry.retry.initialinterval=\"500ms\" --label traefik.http.routers.app.middlewares=\"app-retry@docker\" dhh/app:999",
"docker run --detach --restart unless-stopped --name app-web-999 -e MRSK_CONTAINER_NAME=\"app-web-999\" -e RAILS_MASTER_KEY=\"456\" --log-opt max-size=\"10m\" --label service=\"app\" --label role=\"web\" --label traefik.http.routers.app-web.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.services.app-web.loadbalancer.healthcheck.path=\"/up\" --label traefik.http.services.app-web.loadbalancer.healthcheck.interval=\"1s\" --label traefik.http.middlewares.app-web-retry.retry.attempts=\"5\" --label traefik.http.middlewares.app-web-retry.retry.initialinterval=\"500ms\" --label traefik.http.routers.app-web.middlewares=\"app-web-retry@docker\" dhh/app:999",
new_command.run.join(" ")
end
@@ -21,7 +21,7 @@ class CommandsAppTest < ActiveSupport::TestCase
@config[:volumes] = ["/local/path:/container/path" ]
assert_equal \
"docker run --detach --restart unless-stopped --name app-web-999 -e MRSK_CONTAINER_NAME=\"app-web-999\" -e RAILS_MASTER_KEY=\"456\" --log-opt max-size=\"10m\" --volume /local/path:/container/path --label service=\"app\" --label role=\"web\" --label traefik.http.routers.app.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.services.app.loadbalancer.healthcheck.path=\"/up\" --label traefik.http.services.app.loadbalancer.healthcheck.interval=\"1s\" --label traefik.http.middlewares.app-retry.retry.attempts=\"5\" --label traefik.http.middlewares.app-retry.retry.initialinterval=\"500ms\" --label traefik.http.routers.app.middlewares=\"app-retry@docker\" dhh/app:999",
"docker run --detach --restart unless-stopped --name app-web-999 -e MRSK_CONTAINER_NAME=\"app-web-999\" -e RAILS_MASTER_KEY=\"456\" --log-opt max-size=\"10m\" --volume /local/path:/container/path --label service=\"app\" --label role=\"web\" --label traefik.http.routers.app-web.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.services.app-web.loadbalancer.healthcheck.path=\"/up\" --label traefik.http.services.app-web.loadbalancer.healthcheck.interval=\"1s\" --label traefik.http.middlewares.app-web-retry.retry.attempts=\"5\" --label traefik.http.middlewares.app-web-retry.retry.initialinterval=\"500ms\" --label traefik.http.routers.app-web.middlewares=\"app-web-retry@docker\" dhh/app:999",
new_command.run.join(" ")
end
@@ -29,7 +29,7 @@ class CommandsAppTest < ActiveSupport::TestCase
@config[:healthcheck] = { "path" => "/healthz" }
assert_equal \
"docker run --detach --restart unless-stopped --name app-web-999 -e MRSK_CONTAINER_NAME=\"app-web-999\" -e RAILS_MASTER_KEY=\"456\" --log-opt max-size=\"10m\" --label service=\"app\" --label role=\"web\" --label traefik.http.routers.app.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.services.app.loadbalancer.healthcheck.path=\"/healthz\" --label traefik.http.services.app.loadbalancer.healthcheck.interval=\"1s\" --label traefik.http.middlewares.app-retry.retry.attempts=\"5\" --label traefik.http.middlewares.app-retry.retry.initialinterval=\"500ms\" --label traefik.http.routers.app.middlewares=\"app-retry@docker\" dhh/app:999",
"docker run --detach --restart unless-stopped --name app-web-999 -e MRSK_CONTAINER_NAME=\"app-web-999\" -e RAILS_MASTER_KEY=\"456\" --log-opt max-size=\"10m\" --label service=\"app\" --label role=\"web\" --label traefik.http.routers.app-web.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.services.app-web.loadbalancer.healthcheck.path=\"/healthz\" --label traefik.http.services.app-web.loadbalancer.healthcheck.interval=\"1s\" --label traefik.http.middlewares.app-web-retry.retry.attempts=\"5\" --label traefik.http.middlewares.app-web-retry.retry.initialinterval=\"500ms\" --label traefik.http.routers.app-web.middlewares=\"app-web-retry@docker\" dhh/app:999",
new_command.run.join(" ")
end
@@ -44,7 +44,7 @@ class CommandsAppTest < ActiveSupport::TestCase
@config[:logging] = { "driver" => "local", "options" => { "max-size" => "100m", "max-file" => "3" } }
assert_equal \
"docker run --detach --restart unless-stopped --name app-web-999 -e MRSK_CONTAINER_NAME=\"app-web-999\" -e RAILS_MASTER_KEY=\"456\" --log-driver \"local\" --log-opt max-size=\"100m\" --log-opt max-file=\"3\" --label service=\"app\" --label role=\"web\" --label traefik.http.routers.app.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.services.app.loadbalancer.healthcheck.path=\"/up\" --label traefik.http.services.app.loadbalancer.healthcheck.interval=\"1s\" --label traefik.http.middlewares.app-retry.retry.attempts=\"5\" --label traefik.http.middlewares.app-retry.retry.initialinterval=\"500ms\" --label traefik.http.routers.app.middlewares=\"app-retry@docker\" dhh/app:999",
"docker run --detach --restart unless-stopped --name app-web-999 -e MRSK_CONTAINER_NAME=\"app-web-999\" -e RAILS_MASTER_KEY=\"456\" --log-driver \"local\" --log-opt max-size=\"100m\" --log-opt max-file=\"3\" --label service=\"app\" --label role=\"web\" --label traefik.http.routers.app-web.rule=\"PathPrefix(\\`/\\`)\" --label traefik.http.services.app-web.loadbalancer.healthcheck.path=\"/up\" --label traefik.http.services.app-web.loadbalancer.healthcheck.interval=\"1s\" --label traefik.http.middlewares.app-web-retry.retry.attempts=\"5\" --label traefik.http.middlewares.app-web-retry.retry.initialinterval=\"500ms\" --label traefik.http.routers.app-web.middlewares=\"app-web-retry@docker\" dhh/app:999",
new_command.run.join(" ")
end

View File

@@ -2,53 +2,66 @@ require "test_helper"
class CommandsTraefikTest < ActiveSupport::TestCase
setup do
@image = "traefik:test"
@config = {
service: "app", image: "dhh/app", registry: { "username" => "dhh", "password" => "secret" }, servers: [ "1.1.1.1" ],
traefik: { "args" => { "accesslog.format" => "json", "api.insecure" => true, "metrics.prometheus.buckets" => "0.1,0.3,1.2,5.0" } }
traefik: { "image" => @image, "args" => { "accesslog.format" => "json", "api.insecure" => true, "metrics.prometheus.buckets" => "0.1,0.3,1.2,5.0" } }
}
end
test "run" do
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" traefik:v2.9.9 --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
@config[:traefik]["host_port"] = "8080"
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 8080:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" traefik:v2.9.9 --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
"docker run --name traefik --detach --restart unless-stopped --publish 8080:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
end
test "run with ports configured" do
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" traefik:v2.9.9 --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
@config[:traefik]["options"] = {"publish" => %w[9000:9000 9001:9001]}
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" --publish \"9000:9000\" --publish \"9001:9001\" traefik:v2.9.9 --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" --publish \"9000:9000\" --publish \"9001:9001\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
end
test "run with volumes configured" do
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" traefik:v2.9.9 --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
@config[:traefik]["options"] = {"volume" => %w[./letsencrypt/acme.json:/letsencrypt/acme.json] }
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" --volume \"./letsencrypt/acme.json:/letsencrypt/acme.json\" traefik:v2.9.9 --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" --volume \"./letsencrypt/acme.json:/letsencrypt/acme.json\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
end
test "run with several options configured" do
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" traefik:v2.9.9 --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
@config[:traefik]["options"] = {"volume" => %w[./letsencrypt/acme.json:/letsencrypt/acme.json], "publish" => %w[8080:8080], "memory" => "512m"}
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" --volume \"./letsencrypt/acme.json:/letsencrypt/acme.json\" --publish \"8080:8080\" --memory \"512m\" traefik:v2.9.9 --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" --volume \"./letsencrypt/acme.json:/letsencrypt/acme.json\" --publish \"8080:8080\" --memory \"512m\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
end
test "run with labels configured" do
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
@config[:traefik]["labels"] = { "traefik.http.routers.dashboard.service" => "api@internal", "traefik.http.routers.dashboard.middlewares" => "auth" }
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" --label traefik.http.routers.dashboard.service=\"api@internal\" --label traefik.http.routers.dashboard.middlewares=\"auth\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
end
@@ -56,7 +69,7 @@ class CommandsTraefikTest < ActiveSupport::TestCase
@config.delete(:traefik)
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" traefik:v2.9.9 --providers.docker --log.level=DEBUG",
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-opt max-size=\"10m\" #{Mrsk::Commands::Traefik::DEFAULT_IMAGE} --providers.docker --log.level=DEBUG",
new_command.run.join(" ")
end
@@ -64,7 +77,7 @@ class CommandsTraefikTest < ActiveSupport::TestCase
@config[:logging] = { "driver" => "local", "options" => { "max-size" => "100m", "max-file" => "3" } }
assert_equal \
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-driver \"local\" --log-opt max-size=\"100m\" --log-opt max-file=\"3\" traefik:v2.9.9 --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
"docker run --name traefik --detach --restart unless-stopped --publish 80:80 --volume /var/run/docker.sock:/var/run/docker.sock --log-driver \"local\" --log-opt max-size=\"100m\" --log-opt max-file=\"3\" #{@image} --providers.docker --log.level=DEBUG --accesslog.format=\"json\" --api.insecure --metrics.prometheus.buckets=\"0.1,0.3,1.2,5.0\"",
new_command.run.join(" ")
end

View File

@@ -112,8 +112,11 @@ class ConfigurationAccessoryTest < ActiveSupport::TestCase
test "env args with secret" do
ENV["MYSQL_ROOT_PASSWORD"] = "secret123"
assert_equal ["-e", "MYSQL_ROOT_PASSWORD=\"secret123\"", "-e", "MYSQL_ROOT_HOST=\"%\""], @config.accessory(:mysql).env_args
assert @config.accessory(:mysql).env_args[1].is_a?(SSHKit::Redaction)
@config.accessory(:mysql).env_args.tap do |env_args|
assert_equal ["-e", "MYSQL_ROOT_PASSWORD=\"secret123\"", "-e", "MYSQL_ROOT_HOST=\"%\""], Mrsk::Utils.unredacted(env_args)
assert_equal ["-e", "MYSQL_ROOT_PASSWORD=[REDACTED]", "-e", "MYSQL_ROOT_HOST=\"%\""], Mrsk::Utils.redacted(env_args)
end
ensure
ENV["MYSQL_ROOT_PASSWORD"] = nil
end

View File

@@ -42,7 +42,7 @@ class ConfigurationRoleTest < ActiveSupport::TestCase
end
test "special label args for web" do
assert_equal [ "--label", "service=\"app\"", "--label", "role=\"web\"", "--label", "traefik.http.routers.app.rule=\"PathPrefix(\\`/\\`)\"", "--label", "traefik.http.services.app.loadbalancer.healthcheck.path=\"/up\"", "--label", "traefik.http.services.app.loadbalancer.healthcheck.interval=\"1s\"", "--label", "traefik.http.middlewares.app-retry.retry.attempts=\"5\"", "--label", "traefik.http.middlewares.app-retry.retry.initialinterval=\"500ms\"", "--label", "traefik.http.routers.app.middlewares=\"app-retry@docker\"" ], @config.role(:web).label_args
assert_equal [ "--label", "service=\"app\"", "--label", "role=\"web\"", "--label", "traefik.http.routers.app-web.rule=\"PathPrefix(\\`/\\`)\"", "--label", "traefik.http.services.app-web.loadbalancer.healthcheck.path=\"/up\"", "--label", "traefik.http.services.app-web.loadbalancer.healthcheck.interval=\"1s\"", "--label", "traefik.http.middlewares.app-web-retry.retry.attempts=\"5\"", "--label", "traefik.http.middlewares.app-web-retry.retry.initialinterval=\"500ms\"", "--label", "traefik.http.routers.app-web.middlewares=\"app-web-retry@docker\"" ], @config.role(:web).label_args
end
test "custom labels" do
@@ -66,7 +66,15 @@ class ConfigurationRoleTest < ActiveSupport::TestCase
c[:servers]["beta"] = { "traefik" => "true", "hosts" => [ "1.1.1.5" ] }
})
assert_equal [ "--label", "service=\"app\"", "--label", "role=\"beta\"", "--label", "traefik.http.routers.app.rule=\"PathPrefix(\\`/\\`)\"", "--label", "traefik.http.services.app.loadbalancer.healthcheck.path=\"/up\"", "--label", "traefik.http.services.app.loadbalancer.healthcheck.interval=\"1s\"", "--label", "traefik.http.middlewares.app-retry.retry.attempts=\"5\"", "--label", "traefik.http.middlewares.app-retry.retry.initialinterval=\"500ms\"", "--label", "traefik.http.routers.app.middlewares=\"app-retry@docker\"" ], config.role(:beta).label_args
assert_equal [ "--label", "service=\"app\"", "--label", "role=\"beta\"", "--label", "traefik.http.routers.app-beta.rule=\"PathPrefix(\\`/\\`)\"", "--label", "traefik.http.services.app-beta.loadbalancer.healthcheck.path=\"/up\"", "--label", "traefik.http.services.app-beta.loadbalancer.healthcheck.interval=\"1s\"", "--label", "traefik.http.middlewares.app-beta-retry.retry.attempts=\"5\"", "--label", "traefik.http.middlewares.app-beta-retry.retry.initialinterval=\"500ms\"", "--label", "traefik.http.routers.app-beta.middlewares=\"app-beta-retry@docker\"" ], config.role(:beta).label_args
end
test "default traefik label for non-web role with destination" do
config = Mrsk::Configuration.new(@deploy_with_roles.tap { |c|
c[:servers]["beta"] = { "traefik" => "true", "hosts" => [ "1.1.1.5" ] }
}, destination: "staging")
assert_equal [ "--label", "service=\"app\"", "--label", "role=\"beta\"", "--label", "destination=\"staging\"", "--label", "traefik.http.routers.app-beta-staging.rule=\"PathPrefix(\\`/\\`)\"", "--label", "traefik.http.services.app-beta-staging.loadbalancer.healthcheck.path=\"/up\"", "--label", "traefik.http.services.app-beta-staging.loadbalancer.healthcheck.interval=\"1s\"", "--label", "traefik.http.middlewares.app-beta-staging-retry.retry.attempts=\"5\"", "--label", "traefik.http.middlewares.app-beta-staging-retry.retry.initialinterval=\"500ms\"", "--label", "traefik.http.routers.app-beta-staging.middlewares=\"app-beta-staging-retry@docker\"" ], config.role(:beta).label_args
end
test "env overwritten by role" do
@@ -97,7 +105,10 @@ class ConfigurationRoleTest < ActiveSupport::TestCase
ENV["REDIS_PASSWORD"] = "secret456"
ENV["DB_PASSWORD"] = "secret&\"123"
assert_equal ["-e", "REDIS_PASSWORD=\"secret456\"", "-e", "DB_PASSWORD=\"secret&\\\"123\"", "-e", "REDIS_URL=\"redis://a/b\"", "-e", "WEB_CONCURRENCY=\"4\""], @config_with_roles.role(:workers).env_args
@config_with_roles.role(:workers).env_args.tap do |env_args|
assert_equal ["-e", "REDIS_PASSWORD=\"secret456\"", "-e", "DB_PASSWORD=\"secret&\\\"123\"", "-e", "REDIS_URL=\"redis://a/b\"", "-e", "WEB_CONCURRENCY=\"4\""], Mrsk::Utils.unredacted(env_args)
assert_equal ["-e", "REDIS_PASSWORD=[REDACTED]", "-e", "DB_PASSWORD=[REDACTED]", "-e", "REDIS_URL=\"redis://a/b\"", "-e", "WEB_CONCURRENCY=\"4\""], Mrsk::Utils.redacted(env_args)
end
ensure
ENV["REDIS_PASSWORD"] = nil
ENV["DB_PASSWORD"] = nil
@@ -116,7 +127,10 @@ class ConfigurationRoleTest < ActiveSupport::TestCase
ENV["DB_PASSWORD"] = "secret123"
assert_equal ["-e", "DB_PASSWORD=\"secret123\"", "-e", "REDIS_URL=\"redis://a/b\"", "-e", "WEB_CONCURRENCY=\"4\""], @config_with_roles.role(:workers).env_args
@config_with_roles.role(:workers).env_args.tap do |env_args|
assert_equal ["-e", "DB_PASSWORD=\"secret123\"", "-e", "REDIS_URL=\"redis://a/b\"", "-e", "WEB_CONCURRENCY=\"4\""], Mrsk::Utils.unredacted(env_args)
assert_equal ["-e", "DB_PASSWORD=[REDACTED]", "-e", "REDIS_URL=\"redis://a/b\"", "-e", "WEB_CONCURRENCY=\"4\""], Mrsk::Utils.redacted(env_args)
end
ensure
ENV["DB_PASSWORD"] = nil
end
@@ -133,7 +147,10 @@ class ConfigurationRoleTest < ActiveSupport::TestCase
ENV["REDIS_PASSWORD"] = "secret456"
assert_equal ["-e", "REDIS_PASSWORD=\"secret456\"", "-e", "REDIS_URL=\"redis://a/b\"", "-e", "WEB_CONCURRENCY=\"4\""], @config_with_roles.role(:workers).env_args
@config_with_roles.role(:workers).env_args.tap do |env_args|
assert_equal ["-e", "REDIS_PASSWORD=\"secret456\"", "-e", "REDIS_URL=\"redis://a/b\"", "-e", "WEB_CONCURRENCY=\"4\""], Mrsk::Utils.unredacted(env_args)
assert_equal ["-e", "REDIS_PASSWORD=[REDACTED]", "-e", "REDIS_URL=\"redis://a/b\"", "-e", "WEB_CONCURRENCY=\"4\""], Mrsk::Utils.redacted(env_args)
end
ensure
ENV["REDIS_PASSWORD"] = nil
end

View File

@@ -113,12 +113,13 @@ class ConfigurationTest < ActiveSupport::TestCase
test "env args with clear and secrets" do
ENV["PASSWORD"] = "secret123"
config = Mrsk::Configuration.new(@deploy.tap { |c| c.merge!({
env: { "clear" => { "PORT" => "3000" }, "secret" => [ "PASSWORD" ] }
}) })
assert_equal [ "-e", "PASSWORD=\"secret123\"", "-e", "PORT=\"3000\"" ], config.env_args
assert config.env_args[1].is_a?(SSHKit::Redaction)
assert_equal [ "-e", "PASSWORD=\"secret123\"", "-e", "PORT=\"3000\"" ], Mrsk::Utils.unredacted(config.env_args)
assert_equal [ "-e", "PASSWORD=[REDACTED]", "-e", "PORT=\"3000\"" ], Mrsk::Utils.redacted(config.env_args)
ensure
ENV["PASSWORD"] = nil
end
@@ -133,12 +134,13 @@ class ConfigurationTest < ActiveSupport::TestCase
test "env args with only secrets" do
ENV["PASSWORD"] = "secret123"
config = Mrsk::Configuration.new(@deploy.tap { |c| c.merge!({
env: { "secret" => [ "PASSWORD" ] }
}) })
assert_equal [ "-e", "PASSWORD=\"secret123\"" ], config.env_args
assert config.env_args[1].is_a?(SSHKit::Redaction)
assert_equal [ "-e", "PASSWORD=\"secret123\"" ], Mrsk::Utils.unredacted(config.env_args)
assert_equal [ "-e", "PASSWORD=[REDACTED]" ], Mrsk::Utils.redacted(config.env_args)
ensure
ENV["PASSWORD"] = nil
end
@@ -247,6 +249,6 @@ class ConfigurationTest < ActiveSupport::TestCase
end
test "to_h" do
assert_equal({ :roles=>["web"], :hosts=>["1.1.1.1", "1.1.1.2"], :primary_host=>"1.1.1.1", :version=>"missing", :repository=>"dhh/app", :absolute_image=>"dhh/app:missing", :service_with_version=>"app-missing", :env_args=>["-e", "REDIS_URL=\"redis://x/y\""], :ssh_options=>{:user=>"root", :auth_methods=>["publickey"]}, :volume_args=>["--volume", "/local/path:/container/path"], :logging=>["--log-opt", "max-size=\"10m\""], :healthcheck=>{"path"=>"/up", "port"=>3000 }}, @config.to_h)
assert_equal({ :roles=>["web"], :hosts=>["1.1.1.1", "1.1.1.2"], :primary_host=>"1.1.1.1", :version=>"missing", :repository=>"dhh/app", :absolute_image=>"dhh/app:missing", :service_with_version=>"app-missing", :env_args=>["-e", "REDIS_URL=\"redis://x/y\""], :ssh_options=>{:user=>"root", :auth_methods=>["publickey"]}, :volume_args=>["--volume", "/local/path:/container/path"], :logging=>["--log-opt", "max-size=\"10m\""], :healthcheck=>{"path"=>"/up", "port"=>3000, "max_attempts" => 7 }}, @config.to_h)
end
end

View File

@@ -8,13 +8,16 @@ class UtilsTest < ActiveSupport::TestCase
test "argumentize with redacted" do
assert_kind_of SSHKit::Redaction, \
Mrsk::Utils.argumentize("--label", { foo: "bar" }, redacted: true).last
Mrsk::Utils.argumentize("--label", { foo: "bar" }, sensitive: true).last
end
test "argumentize_env_with_secrets" do
ENV.expects(:fetch).with("FOO").returns("secret")
assert_equal [ "-e", "FOO=\"secret\"", "-e", "BAZ=\"qux\"" ], \
Mrsk::Utils.argumentize_env_with_secrets({ "secret" => [ "FOO" ], "clear" => { BAZ: "qux" } })
args = Mrsk::Utils.argumentize_env_with_secrets({ "secret" => [ "FOO" ], "clear" => { BAZ: "qux" } })
assert_equal [ "-e", "FOO=[REDACTED]", "-e", "BAZ=\"qux\"" ], Mrsk::Utils.redacted(args)
assert_equal [ "-e", "FOO=\"secret\"", "-e", "BAZ=\"qux\"" ], Mrsk::Utils.unredacted(args)
end
test "optionize" do
@@ -27,9 +30,20 @@ class UtilsTest < ActiveSupport::TestCase
Mrsk::Utils.optionize({ foo: "bar", baz: "qux", quux: true }, with: "=")
end
test "redact" do
assert_kind_of SSHKit::Redaction, Mrsk::Utils.redact("secret")
assert_equal "secret", Mrsk::Utils.redact("secret")
test "no redaction from #to_s" do
assert_equal "secret", Mrsk::Utils.sensitive("secret").to_s
end
test "redact from #inspect" do
assert_equal "[REDACTED]".inspect, Mrsk::Utils.sensitive("secret").inspect
end
test "redact from SSHKit output" do
assert_kind_of SSHKit::Redaction, Mrsk::Utils.sensitive("secret")
end
test "redact from YAML output" do
assert_equal "--- ! '[REDACTED]'\n", YAML.dump(Mrsk::Utils.sensitive("secret"))
end
test "escape_shell_value" do