Commit Graph

545 Commits

Author SHA1 Message Date
David Heinemeier Hansson
71bc9bcf54 Merge pull request #222 from basecamp/deploy-groups
Allow booting containers in groups for rolling restarts
2023-05-02 13:14:32 +02:00
David Heinemeier Hansson
c83b74dcb7 Simplify domain language to just "boot" and unscoped config keys 2023-05-02 13:11:31 +02:00
Donal McBreen
d0f66db33c Extend traefik delay by 1 second 2023-05-01 18:58:46 +01:00
David Heinemeier Hansson
4fa6a6c06d Merge pull request #219 from basecamp/docker-health-checks 2023-04-28 11:43:33 +02:00
David Heinemeier Hansson
2ad0dc0703 Merge pull request #241 from Jberczel/fix-traefik-subcommand-typo 2023-04-28 11:38:46 +02:00
Donal McBreen
cd668066ff Get lock status by executing directly
Getting the lock status with invoke passes through any options from the
original command which will raise an exception if they are not also
valid for the lock status command.

Fixes https://github.com/mrsked/mrsk/issues/239
2023-04-25 16:57:02 +01:00
Jberczel
126e0bbd06 Fix traefik remove_image desc typo 2023-04-24 17:40:28 -04:00
Kevin McConnell
f055766918 Allow percentage-based rolling deployments 2023-04-14 12:46:14 +01:00
Kevin McConnell
a8726be20e Move group_limit & group_wait under boot
Also make formatting the group strategy the responsibility of the
commander.
2023-04-14 11:31:51 +01:00
Kevin McConnell
100b72e4b4 Limit rolling deployment to boot operation 2023-04-14 10:41:07 +01:00
Kevin McConnell
df202d6ef4 Move health checks into Docker
Replaces our current host-based HTTP healthchecks with Docker
healthchecks, and adds a new `healthcheck.cmd` config option that can be
used to define a custom health check command. Also removes Traefik's
healthchecks, since they are no longer necessary.

When deploying a container that has a healthcheck defined, we wait for
it to report a healthy status before stopping the old container that it
replaces. Containers that don't have a healthcheck defined continue to
wait for `MRSK.config.readiness_delay`.

There are some pros and cons to using Docker healthchecks rather than
checking from the host. The main advantages are:

- Supports non-HTTP checks, and app-specific check scripts provided by a
  container.
- When booting a container, allows MRSK to wait for a container to be
  healthy before shutting down the old container it replaces. This
  should be safer than relying on a timeout.
- Containers with healthchecks won't be active in Traefik until they
  reach a healthy state, which prevents any traffic from being routed to
  them before they are ready.

The main _disadvantage_ is that containers are now required to provide
some way to check their health. Our default check assumes that `curl` is
available in the container which, while common, won't always be the
case.
2023-04-13 16:08:43 +01:00
Kevin McConnell
f530009a6e Allow performing boot & start operations in groups
Adds top-level configuration options for `group_limit` and `group_wait`.
When a `group_limit` is present, we'll perform app boot & start
operations on no more than `group_limit` hosts at a time, optionally
sleeping for `group_wait` seconds after each batch.

We currently only do this batching on boot & start operations (including
when they are part of a deployment). Other commands, like `app stop` or
`app details` still work on all hosts in parallel.
2023-04-13 15:58:27 +01:00
David Heinemeier Hansson
bc8875e020 Merge pull request #183 from basecamp/cleanup-excessive-containers-running
Clear stale containers
2023-04-12 15:58:59 +02:00
David Heinemeier Hansson
b6f7d94ac3 Merge pull request #144 from monorkin/shell-escape-dollar-signs
Shell escape dollar signs
2023-04-12 15:57:37 +02:00
Stanko K.R
3ab16c8994 Shell escape dollar signs
But allow for shell expansion using curly braces e.g. ${PWD}
2023-04-12 15:55:54 +02:00
Jacopo
9ddb181f50 Merge branch 'main' into cleanup-excessive-containers-running
* main:
  Pull the primary host from the role
  Minimise holding the deploy lock
2023-04-12 15:19:19 +02:00
David Heinemeier Hansson
2f1393cd92 Merge pull request #212 from basecamp/role-primary-hosts
Pull the primary host from the role
2023-04-12 14:09:38 +02:00
Donal McBreen
fb62f2e6e1 Pull the primary host from the role
So commands like this run on a host with the specified role:
```
mrsk app exec -r=console -i "/bin/bash`
mrsk app logs -f -r=workers
```
2023-04-12 13:03:02 +01:00
Donal McBreen
051556674f Minimise holding the deploy lock
If we get an error we'll only hold the deploy lock if it occurs while
trying to switch the running containers.

We'll also move tagging the latest image from when the image is pulled
to just before the container switch. This ensures that earlier errors
don't leave the hosts with an updated latest tag while still running the
older version.
2023-04-12 12:09:56 +01:00
Jacopo
3cbf4aea46 Make method private method and use :send 2023-04-12 11:53:49 +02:00
Jacopo
5ed431b807 Merge branch 'main' into cleanup-excessive-containers-running
* main: (24 commits)
  Bump version for 0.11.0
  Labels can be added to Traefik
  Make rollbacks role-aware
  fix typo role to roles
  Explained the latest modifications of Traefik container labels
  Remove .idea folder
  Updated README.md with new healthcheck.max_attempts option
  Fix test case: console output message was not updated to display the current/total attempts
  Require net-ssh ~> 7.0 for SHA-2 support
  Improved deploy lock acquisition
  Excess CR
  Style
  Simpler
  Make it explicit, focus on Ubuntu
  More explicit
  Not that --bundle is a Rails 7+ option
  Update README.md
  Update README.md
  Improved: configurable max_attempts for healthcheck
  Traefik service name to be derived from role and destination
  ...
2023-04-12 11:52:47 +02:00
David Heinemeier Hansson
60a19f0b30 Bump version for 0.11.0 2023-04-12 11:45:33 +02:00
David Heinemeier Hansson
2d0a7e1b67 Merge pull request #208 from tannakartikey/add_labels_to_traefik
Labels can be added to Traefik
2023-04-12 11:35:28 +02:00
Kartikey Tanna
c59eb00dd0 Labels can be added to Traefik 2023-04-12 14:53:48 +05:30
Donal McBreen
43f7409de0 Make rollbacks role-aware
Rollbacks stopped working after https://github.com/mrsked/mrsk/pull/99.

We'll confirm that a container is available for the first role on the
primary host before attempting to rollback.
2023-04-12 09:59:39 +01:00
Jacopo
72b70e3e9e More compact 2023-04-11 16:22:47 +02:00
Jacopo
e8697327fa Use no_commands block 2023-04-11 16:20:16 +02:00
Jacopo
0bfd4ca780 Use cli = self approach 2023-04-11 16:04:46 +02:00
Jacopo
12e3a562c4 Extract helper 2023-04-11 15:26:55 +02:00
David Heinemeier Hansson
daa0c9b5be Merge pull request #196 from handy-la/main
Configurable max_attempts for healthcheck
2023-04-11 14:17:17 +02:00
Jacopo
c3393c8213 Remove dot 2023-04-11 11:03:11 +02:00
Jacopo
03d933d10b Add Role to the message 2023-04-11 10:59:25 +02:00
Jacopo
579b4cd9aa Simplify
By using and ad-hoc command to detect and stop stale containers.
By default stale containers are only detected.
2023-04-11 10:22:03 +02:00
Jacopo
f9436d5673 Style 2023-04-11 08:53:33 +02:00
Jacopo
8ae5331d97 Boot stop all the old containers 2023-04-11 08:53:33 +02:00
Jacopo
4d47fbdf41 Merge stop and stop_stale_containers 2023-04-11 08:53:33 +02:00
Jacopo
e980f1164e Avoid using GNU-only Perl Regepx Grep 2023-04-11 08:53:33 +02:00
Jacopo
e2f6db5cae Clear stale containers
By stopping all the older containers with matching /#{service}-#{role}-#{dest}-.*/ running on the same host.
2023-04-11 08:53:33 +02:00
Donal McBreen
c4df440c79 Improved deploy lock acquisition
1. Don't raise lock error for non-lock issues during lock acquire
  (see https://github.com/mrsked/mrsk/pull/181)
2. If there is an error while the lock is held, don't release the lock
  and send a warning to stderr
2023-04-10 15:23:00 +01:00
David Heinemeier Hansson
fb1718ca6d Merge pull request #197 from tannakartikey/traefik_rules_with_destination
Traefik service name to be derived from role and destination
2023-04-10 15:11:07 +02:00
David Heinemeier Hansson
f4133de896 Merge pull request #176 from dilpreet92/enable_ssh_over_proxy_command
Enable ssh over proxy command
2023-04-10 14:41:45 +02:00
David Heinemeier Hansson
a9488e935d Style 2023-04-10 14:39:18 +02:00
David Heinemeier Hansson
0eb7a8d087 Merge branch 'main' into pr/176
* main:
  Simpler
  Make it explicit, focus on Ubuntu
  More explicit
  Not that --bundle is a Rails 7+ option
  Update README.md
  Update README.md
  Add github discussions link to readme
  Bump debug to fix missing deps in CI
  Only redact the non-sensitive bits of build args and env vars.
  improve code sample (traefik configuration)
2023-04-10 14:31:43 +02:00
Arturo Ojeda
3969f56fa6 Improved: configurable max_attempts for healthcheck 2023-04-09 12:07:27 -06:00
Kartikey Tanna
c60cc92dfe Traefik service name to be derived from role and destination 2023-04-09 13:44:57 +05:30
Arturo Ojeda
cb3c5a53f4 Configurable max_attempts for healthcheck 2023-04-08 19:52:53 -06:00
Jeremy Daer
bd8f13dd5e Traefik image config for version pinning, upgrades, and custom images
Accounts for the 2.9.10 security release and allows testing Traefik 3 betas.

* Use `image` to configure a specific Traefik Docker image.
* Default to `traefik:v2.9` to track future 2.9.x minor releases rather
  than tightly pinning to `v2.9.9`.
* Support images from the configured registry.

References #165
2023-04-07 14:15:25 -07:00
Jeremy Daer
c137b38c87 Only redact the non-sensitive bits of build args and env vars.
* `-e [REDACTED]` → `-e SOME_SECRET=[REDACTED]`
* Replaces `Utils.redact` with `Utils.sensitive` to clarify that we're
  indicating redactability, not actually performing redaction.
* Redacts from YAML output, including `mrsk config` (fixes #96)
2023-04-05 09:45:28 -07:00
Dilpreet Singh
12632aa7f9 Enable ssh over proxy command 2023-04-03 17:14:06 +05:30
David Heinemeier Hansson
032266a76a Bump version for 0.10.1 2023-03-29 16:23:58 +02:00