Commit Graph

257 Commits

Author SHA1 Message Date
Kevin McConnell
f530009a6e Allow performing boot & start operations in groups
Adds top-level configuration options for `group_limit` and `group_wait`.
When a `group_limit` is present, we'll perform app boot & start
operations on no more than `group_limit` hosts at a time, optionally
sleeping for `group_wait` seconds after each batch.

We currently only do this batching on boot & start operations (including
when they are part of a deployment). Other commands, like `app stop` or
`app details` still work on all hosts in parallel.
2023-04-13 15:58:27 +01:00
David Heinemeier Hansson
bc8875e020 Merge pull request #183 from basecamp/cleanup-excessive-containers-running
Clear stale containers
2023-04-12 15:58:59 +02:00
David Heinemeier Hansson
b6f7d94ac3 Merge pull request #144 from monorkin/shell-escape-dollar-signs
Shell escape dollar signs
2023-04-12 15:57:37 +02:00
Stanko K.R
3ab16c8994 Shell escape dollar signs
But allow for shell expansion using curly braces e.g. ${PWD}
2023-04-12 15:55:54 +02:00
Jacopo
9ddb181f50 Merge branch 'main' into cleanup-excessive-containers-running
* main:
  Pull the primary host from the role
  Minimise holding the deploy lock
2023-04-12 15:19:19 +02:00
David Heinemeier Hansson
2f1393cd92 Merge pull request #212 from basecamp/role-primary-hosts
Pull the primary host from the role
2023-04-12 14:09:38 +02:00
Donal McBreen
fb62f2e6e1 Pull the primary host from the role
So commands like this run on a host with the specified role:
```
mrsk app exec -r=console -i "/bin/bash`
mrsk app logs -f -r=workers
```
2023-04-12 13:03:02 +01:00
Donal McBreen
051556674f Minimise holding the deploy lock
If we get an error we'll only hold the deploy lock if it occurs while
trying to switch the running containers.

We'll also move tagging the latest image from when the image is pulled
to just before the container switch. This ensures that earlier errors
don't leave the hosts with an updated latest tag while still running the
older version.
2023-04-12 12:09:56 +01:00
Jacopo
5ed431b807 Merge branch 'main' into cleanup-excessive-containers-running
* main: (24 commits)
  Bump version for 0.11.0
  Labels can be added to Traefik
  Make rollbacks role-aware
  fix typo role to roles
  Explained the latest modifications of Traefik container labels
  Remove .idea folder
  Updated README.md with new healthcheck.max_attempts option
  Fix test case: console output message was not updated to display the current/total attempts
  Require net-ssh ~> 7.0 for SHA-2 support
  Improved deploy lock acquisition
  Excess CR
  Style
  Simpler
  Make it explicit, focus on Ubuntu
  More explicit
  Not that --bundle is a Rails 7+ option
  Update README.md
  Update README.md
  Improved: configurable max_attempts for healthcheck
  Traefik service name to be derived from role and destination
  ...
2023-04-12 11:52:47 +02:00
David Heinemeier Hansson
2d0a7e1b67 Merge pull request #208 from tannakartikey/add_labels_to_traefik
Labels can be added to Traefik
2023-04-12 11:35:28 +02:00
Kartikey Tanna
c59eb00dd0 Labels can be added to Traefik 2023-04-12 14:53:48 +05:30
Donal McBreen
43f7409de0 Make rollbacks role-aware
Rollbacks stopped working after https://github.com/mrsked/mrsk/pull/99.

We'll confirm that a container is available for the first role on the
primary host before attempting to rollback.
2023-04-12 09:59:39 +01:00
David Heinemeier Hansson
daa0c9b5be Merge pull request #196 from handy-la/main
Configurable max_attempts for healthcheck
2023-04-11 14:17:17 +02:00
Jacopo
03d933d10b Add Role to the message 2023-04-11 10:59:25 +02:00
Jacopo
579b4cd9aa Simplify
By using and ad-hoc command to detect and stop stale containers.
By default stale containers are only detected.
2023-04-11 10:22:03 +02:00
Jacopo
8ae5331d97 Boot stop all the old containers 2023-04-11 08:53:33 +02:00
Jacopo
4d47fbdf41 Merge stop and stop_stale_containers 2023-04-11 08:53:33 +02:00
Jacopo
e980f1164e Avoid using GNU-only Perl Regepx Grep 2023-04-11 08:53:33 +02:00
Jacopo
e2f6db5cae Clear stale containers
By stopping all the older containers with matching /#{service}-#{role}-#{dest}-.*/ running on the same host.
2023-04-11 08:53:33 +02:00
Arturo Ojeda
514b2aa243 Fix test case: console output message was not updated to display the current/total attempts 2023-04-10 09:29:19 -06:00
Donal McBreen
c4df440c79 Improved deploy lock acquisition
1. Don't raise lock error for non-lock issues during lock acquire
  (see https://github.com/mrsked/mrsk/pull/181)
2. If there is an error while the lock is held, don't release the lock
  and send a warning to stderr
2023-04-10 15:23:00 +01:00
David Heinemeier Hansson
fb1718ca6d Merge pull request #197 from tannakartikey/traefik_rules_with_destination
Traefik service name to be derived from role and destination
2023-04-10 15:11:07 +02:00
David Heinemeier Hansson
7d17a6c3b5 Excess CR 2023-04-10 15:10:08 +02:00
Arturo Ojeda
3969f56fa6 Improved: configurable max_attempts for healthcheck 2023-04-09 12:07:27 -06:00
Kartikey Tanna
c60cc92dfe Traefik service name to be derived from role and destination 2023-04-09 13:44:57 +05:30
Jeremy Daer
bd8f13dd5e Traefik image config for version pinning, upgrades, and custom images
Accounts for the 2.9.10 security release and allows testing Traefik 3 betas.

* Use `image` to configure a specific Traefik Docker image.
* Default to `traefik:v2.9` to track future 2.9.x minor releases rather
  than tightly pinning to `v2.9.9`.
* Support images from the configured registry.

References #165
2023-04-07 14:15:25 -07:00
Jeremy Daer
c137b38c87 Only redact the non-sensitive bits of build args and env vars.
* `-e [REDACTED]` → `-e SOME_SECRET=[REDACTED]`
* Replaces `Utils.redact` with `Utils.sensitive` to clarify that we're
  indicating redactability, not actually performing redaction.
* Redacts from YAML output, including `mrsk config` (fixes #96)
2023-04-05 09:45:28 -07:00
David Heinemeier Hansson
1f83b5f6be Fix failure to pass on class options to subcommands 2023-03-28 18:04:16 +02:00
Kevin McConnell
2957388bf6 Pin Traefik to v2.9.9 2023-03-28 14:59:03 +01:00
David Heinemeier Hansson
7f178101f7 Merge pull request #164 from basecamp/accessory-hosts-or-roles
Run accessories on multiple hosts or roles
2023-03-28 14:31:24 +02:00
Donal McBreen
c06585fef4 Daemon/host/role accessories
Allow the hosts for accessories to be specified by host or role, or on
all app hosts by setting `daemon: true`.

```
  # Single host
  mysql:
    host: 1.1.1.1
  # Multiple hosts
  redis:
    hosts:
      - 1.1.1.1
      - 1.1.1.2
  # By role
  monitoring:
    roles:
      - web
      - jobs
```
2023-03-28 13:26:27 +01:00
David Heinemeier Hansson
fd5313ec3e Merge pull request #163 from milk1000cc/rolify-app-logs
Rolify app logs cli/command
2023-03-28 14:13:02 +02:00
milk1000cc
03614bfb79 Rolify app logs cli/command 2023-03-27 23:08:46 +09:00
Tobias Bühlmann
078d68b170 Push <image>:latest in addition to <image>:<git-ref> 2023-03-27 12:52:11 +02:00
Donal McBreen
05488e4c1e Zero downtime redeploys
When deploying check if there is already a container with the existing
name. If there is rename it to "<version>_<random_hex_string>" to remove
the name clash with the new container we want to boot.

We can then do the normal zero downtime run/wait/stop.

While implementing this I discovered the --filter name=foo does a
substring match for foo, so I've updated those filters to do an exact
match instead.
2023-03-24 17:09:20 +00:00
David Heinemeier Hansson
84540cee7b Merge branch 'main' into pr/154
* main: (32 commits)
  Inline default as with other options
  Symbols!
  Fix tests
  test stop with custom stop wait time
  No need to replicate Docker default
  Describe purpose rather than elements
  Style and ordering
  Customizable stop wait time
  Fix tests
  Ensure it also works when configuring just log options without setting a driver
  Add accessory test
  Undo change
  Improve test
  Update README
  Ensure default log option `max-size=10m`
  #142 Allow to customize container options in accessories
  Fix flaky test
  Fix tests
  More resilient tests
  Fix other tests
  ...
2023-03-24 15:43:17 +01:00
David Heinemeier Hansson
b5ccc1fa5d Merge branch 'main' into global-logging-config 2023-03-24 15:32:41 +01:00
David Heinemeier Hansson
4fa71834ad Symbols! 2023-03-24 15:27:11 +01:00
David Heinemeier Hansson
65663ae2ea Merge branch 'main' into pr/155
* main:
  Describe purpose rather than elements
  Style and ordering
  #142 Allow to customize container options in accessories
2023-03-24 15:25:45 +01:00
Samuel Sieg
4044abdde1 Fix tests 2023-03-24 15:25:29 +01:00
Samuel Sieg
bc64a07a95 Merge branch 'main' into global-logging-config 2023-03-24 15:24:06 +01:00
David Heinemeier Hansson
fdb2502216 test stop with custom stop wait time 2023-03-24 15:22:34 +01:00
David Heinemeier Hansson
348f5844d5 Merge pull request #153 from javierav/feature/accessory-options
#142 Allow to customize container options in accessories
2023-03-24 15:09:12 +01:00
Jacopo
9b43a6b23b Customizable stop wait time
Configurable via a global `stop_wait_time` option.
The default is `10` which matches Docker defaults.
2023-03-24 15:04:45 +01:00
Samuel Sieg
86e99fb079 Merge branch 'main' into global-logging-config 2023-03-24 14:40:27 +01:00
David Heinemeier Hansson
494e29d672 Fix tests 2023-03-24 14:35:17 +01:00
David Heinemeier Hansson
93423f2f20 Merge branch 'main' into pr/99
* main:
  Wording
  Remove accessory images using tags rather than labels
  Update readme to point to ghcr.io/mrsked/mrsk
  Validate that all roles have hosts
  Commander needn't accumulate configuration
  Pull latest image tag, so we can identity it
  Default to deploying the config version
  Remove unneeded Dockerfile.dind, update Readme
  add D-in-D dockerfile, update Readme
2023-03-24 14:26:31 +01:00
Donal McBreen
8d8f9f6ada Deploy locks
Add a deploy lock for commands that are unsafe to run concurrently.

The lock is taken by creating a `mrsk_lock` directory on the primary
host. Details of who took the lock are added to a details file in that
directory.

Additional CLI commands have been added to manual release and acquire
the lock and to check its status.

```
Commands:
  mrsk lock acquire -m, --message=MESSAGE  # Acquire the deploy lock
  mrsk lock help [COMMAND]                 # Describe subcommands or one specific subcommand
  mrsk lock release                        # Release the deploy lock
  mrsk lock status                         # Report lock status

Options:
  -v, [--verbose], [--no-verbose]                # Detailed logging
  -q, [--quiet], [--no-quiet]                    # Minimal logging
      [--version=VERSION]                        # Run commands against a specific app version
  -p, [--primary], [--no-primary]                # Run commands only on primary host instead of all
  -h, [--hosts=HOSTS]                            # Run commands on these hosts instead of all (separate by comma)
  -r, [--roles=ROLES]                            # Run commands on these roles instead of all (separate by comma)
  -c, [--config-file=CONFIG_FILE]                # Path to config file
                                                 # Default: config/deploy.yml
  -d, [--destination=DESTINATION]                # Specify destination to be used for config file (staging -> deploy.staging.yml)
  -B, [--skip-broadcast], [--no-skip-broadcast]  # Skip audit broadcasts
```

If we add support for running multiple deployments on a single server
we'll need to extend the locking to lock per deployment.
2023-03-24 12:28:08 +00:00
David Heinemeier Hansson
17e74910e4 Merge pull request #150 from basecamp/remove-accessory-image
Remove accessory images using tags rather than labels
2023-03-24 13:21:15 +01:00
David Heinemeier Hansson
c89b77127b Merge pull request #143 from djmb/default-to-deploying-config-version
Default to deploying the config version
2023-03-24 12:36:20 +01:00