Commit Graph

92 Commits

Author SHA1 Message Date
Donal McBreen
58c1096a90 MRSK hooks
Adds hooks to MRSK. Currently just two hooks, pre-build and post-push.

We could break the build and push into two separate commands if we
found the need for post-build and/or pre-push hooks.

Hooks are stored in `.mrsk/hooks`. Running `mrsk init` will now create
that folder and add sample hook scripts.

Hooks returning non-zero exit codes will abort the current command.

Further potential work here:
- We could replace the audit broadcast command with a
post-deploy/post-rollback hook or similar
- Maybe provide pre-command/post-command hooks that run after every
mrsk invocation
- Also look for hooks in `~/.mrsk/hooks`
2023-05-23 13:55:04 +01:00
Donal McBreen
ee25f200d7 Call app:boot to rollback
The code in Mrsk::Cli::Main#rollback was very similar to
Mrsk::Cli::App#boot.

Modify Mrsk::Cli::App#boot so it can handle rollbacks by:
1. Only renaming running containers
2. Trying first to start then run the new container
2023-05-16 08:59:07 +01:00
Donal McBreen
a5ef1f254f Highlight uncommitted changes in version
If there are uncommitted changes in the app repository when building,
then append `_uncommitted_<random>` to it to distinguish the image
from one built from a clean checkout.

Also change the version used when renaming a container on redeploy to
distinguish and explain the version suffixes.
2023-05-12 11:08:48 +01:00
David Heinemeier Hansson
71bc9bcf54 Merge pull request #222 from basecamp/deploy-groups
Allow booting containers in groups for rolling restarts
2023-05-02 13:14:32 +02:00
David Heinemeier Hansson
c83b74dcb7 Simplify domain language to just "boot" and unscoped config keys 2023-05-02 13:11:31 +02:00
Kevin McConnell
f055766918 Allow percentage-based rolling deployments 2023-04-14 12:46:14 +01:00
Kevin McConnell
df202d6ef4 Move health checks into Docker
Replaces our current host-based HTTP healthchecks with Docker
healthchecks, and adds a new `healthcheck.cmd` config option that can be
used to define a custom health check command. Also removes Traefik's
healthchecks, since they are no longer necessary.

When deploying a container that has a healthcheck defined, we wait for
it to report a healthy status before stopping the old container that it
replaces. Containers that don't have a healthcheck defined continue to
wait for `MRSK.config.readiness_delay`.

There are some pros and cons to using Docker healthchecks rather than
checking from the host. The main advantages are:

- Supports non-HTTP checks, and app-specific check scripts provided by a
  container.
- When booting a container, allows MRSK to wait for a container to be
  healthy before shutting down the old container it replaces. This
  should be safer than relying on a timeout.
- Containers with healthchecks won't be active in Traefik until they
  reach a healthy state, which prevents any traffic from being routed to
  them before they are ready.

The main _disadvantage_ is that containers are now required to provide
some way to check their health. Our default check assumes that `curl` is
available in the container which, while common, won't always be the
case.
2023-04-13 16:08:43 +01:00
Kevin McConnell
f530009a6e Allow performing boot & start operations in groups
Adds top-level configuration options for `group_limit` and `group_wait`.
When a `group_limit` is present, we'll perform app boot & start
operations on no more than `group_limit` hosts at a time, optionally
sleeping for `group_wait` seconds after each batch.

We currently only do this batching on boot & start operations (including
when they are part of a deployment). Other commands, like `app stop` or
`app details` still work on all hosts in parallel.
2023-04-13 15:58:27 +01:00
Jacopo
9ddb181f50 Merge branch 'main' into cleanup-excessive-containers-running
* main:
  Pull the primary host from the role
  Minimise holding the deploy lock
2023-04-12 15:19:19 +02:00
Donal McBreen
051556674f Minimise holding the deploy lock
If we get an error we'll only hold the deploy lock if it occurs while
trying to switch the running containers.

We'll also move tagging the latest image from when the image is pulled
to just before the container switch. This ensures that earlier errors
don't leave the hosts with an updated latest tag while still running the
older version.
2023-04-12 12:09:56 +01:00
Jacopo
03d933d10b Add Role to the message 2023-04-11 10:59:25 +02:00
Jacopo
579b4cd9aa Simplify
By using and ad-hoc command to detect and stop stale containers.
By default stale containers are only detected.
2023-04-11 10:22:03 +02:00
Jacopo
8ae5331d97 Boot stop all the old containers 2023-04-11 08:53:33 +02:00
Jacopo
4d47fbdf41 Merge stop and stop_stale_containers 2023-04-11 08:53:33 +02:00
Jacopo
e980f1164e Avoid using GNU-only Perl Regepx Grep 2023-04-11 08:53:33 +02:00
Jacopo
e2f6db5cae Clear stale containers
By stopping all the older containers with matching /#{service}-#{role}-#{dest}-.*/ running on the same host.
2023-04-11 08:53:33 +02:00
David Heinemeier Hansson
1f83b5f6be Fix failure to pass on class options to subcommands 2023-03-28 18:04:16 +02:00
milk1000cc
03614bfb79 Rolify app logs cli/command 2023-03-27 23:08:46 +09:00
Donal McBreen
05488e4c1e Zero downtime redeploys
When deploying check if there is already a container with the existing
name. If there is rename it to "<version>_<random_hex_string>" to remove
the name clash with the new container we want to boot.

We can then do the normal zero downtime run/wait/stop.

While implementing this I discovered the --filter name=foo does a
substring match for foo, so I've updated those filters to do an exact
match instead.
2023-03-24 17:09:20 +00:00
David Heinemeier Hansson
494e29d672 Fix tests 2023-03-24 14:35:17 +01:00
David Heinemeier Hansson
93423f2f20 Merge branch 'main' into pr/99
* main:
  Wording
  Remove accessory images using tags rather than labels
  Update readme to point to ghcr.io/mrsked/mrsk
  Validate that all roles have hosts
  Commander needn't accumulate configuration
  Pull latest image tag, so we can identity it
  Default to deploying the config version
  Remove unneeded Dockerfile.dind, update Readme
  add D-in-D dockerfile, update Readme
2023-03-24 14:26:31 +01:00
Donal McBreen
1ed4a37da2 Pull latest image tag, so we can identity it
`docker image ls` doesn't tell us what the latest deployed image is (e.g
if we've rolled back). Pull the latest image tag through to the server
so we can use it instead.
2023-03-23 14:39:32 +00:00
David Heinemeier Hansson
43d1ecc94b Fix other tests 2023-03-20 17:33:13 +01:00
David Heinemeier Hansson
00b970323b Merge branch 'main' into pr/99
* main:
  Add another assertion for `escape_shell_value`
  Add tests for `Mrsk::Utils`
  Fix indentation
  Don't report exception here too
  Don't report exception
  Add CLI tests for remaining commands that are not tested yet
  Minor: Properly require active_support
2023-03-20 17:31:50 +01:00
Samuel Sieg
aae290cefc Add CLI tests for remaining commands that are not tested yet 2023-03-15 16:48:12 +01:00
Tobias Bühlmann
418bc13ae7 Apply filters correctly 2023-03-10 10:33:55 +01:00
Tobias Bühlmann
fdb0c8ee91 Rolify app cli/command 2023-03-10 08:50:26 +01:00
David Heinemeier Hansson
371f98d67f Start before stopping and longer timeouts 2023-02-22 19:04:23 +01:00
David Heinemeier Hansson
5898fdd8f4 Expand arguments to be more self-explanatory in logs 2023-02-19 18:11:06 +01:00
David Heinemeier Hansson
745b09051e Test app remove 2023-02-19 17:15:57 +01:00
David Heinemeier Hansson
58417f610f Dupe comment 2023-02-03 17:20:14 +01:00
David Heinemeier Hansson
c453b947e0 Add exec tests 2023-02-03 17:18:42 +01:00
David Heinemeier Hansson
a3fe8856c9 Fix test 2023-02-03 16:27:16 +01:00
David Heinemeier Hansson
37e216f2b7 Add some more tests 2023-02-03 15:08:44 +01:00
David Heinemeier Hansson
17e75ec2c9 No more reboot 2023-02-03 15:06:43 +01:00
David Heinemeier Hansson
15a213eec6 Escape pipe and test for xargs 2023-02-03 14:07:52 +01:00
David Heinemeier Hansson
2023c377ab Reboot if running 2023-02-03 13:52:31 +01:00
David Heinemeier Hansson
8024949fe7 Remove only specific container needed for rebooting 2023-02-03 13:20:03 +01:00
David Heinemeier Hansson
35b42cc885 Fix tests 2023-02-02 18:05:56 +01:00
David Heinemeier Hansson
f88685a525 Extract CliTestCase 2023-02-02 15:37:41 +01:00
David Heinemeier Hansson
08908c3925 Fix test 2023-02-02 15:31:33 +01:00
David Heinemeier Hansson
7cc64299c8 Add app reboot 2023-02-02 15:28:36 +01:00