Commit Graph

32 Commits

Author SHA1 Message Date
Donal McBreen
58c1096a90 MRSK hooks
Adds hooks to MRSK. Currently just two hooks, pre-build and post-push.

We could break the build and push into two separate commands if we
found the need for post-build and/or pre-push hooks.

Hooks are stored in `.mrsk/hooks`. Running `mrsk init` will now create
that folder and add sample hook scripts.

Hooks returning non-zero exit codes will abort the current command.

Further potential work here:
- We could replace the audit broadcast command with a
post-deploy/post-rollback hook or similar
- Maybe provide pre-command/post-command hooks that run after every
mrsk invocation
- Also look for hooks in `~/.mrsk/hooks`
2023-05-23 13:55:04 +01:00
Donal McBreen
ee25f200d7 Call app:boot to rollback
The code in Mrsk::Cli::Main#rollback was very similar to
Mrsk::Cli::App#boot.

Modify Mrsk::Cli::App#boot so it can handle rollbacks by:
1. Only renaming running containers
2. Trying first to start then run the new container
2023-05-16 08:59:07 +01:00
Jberczel
0e19ead37c Capture container health log when unhealthy 2023-05-03 15:03:05 -04:00
Kevin McConnell
df202d6ef4 Move health checks into Docker
Replaces our current host-based HTTP healthchecks with Docker
healthchecks, and adds a new `healthcheck.cmd` config option that can be
used to define a custom health check command. Also removes Traefik's
healthchecks, since they are no longer necessary.

When deploying a container that has a healthcheck defined, we wait for
it to report a healthy status before stopping the old container that it
replaces. Containers that don't have a healthcheck defined continue to
wait for `MRSK.config.readiness_delay`.

There are some pros and cons to using Docker healthchecks rather than
checking from the host. The main advantages are:

- Supports non-HTTP checks, and app-specific check scripts provided by a
  container.
- When booting a container, allows MRSK to wait for a container to be
  healthy before shutting down the old container it replaces. This
  should be safer than relying on a timeout.
- Containers with healthchecks won't be active in Traefik until they
  reach a healthy state, which prevents any traffic from being routed to
  them before they are ready.

The main _disadvantage_ is that containers are now required to provide
some way to check their health. Our default check assumes that `curl` is
available in the container which, while common, won't always be the
case.
2023-04-13 16:08:43 +01:00
Jeremy Daer
c137b38c87 Only redact the non-sensitive bits of build args and env vars.
* `-e [REDACTED]` → `-e SOME_SECRET=[REDACTED]`
* Replaces `Utils.redact` with `Utils.sensitive` to clarify that we're
  indicating redactability, not actually performing redaction.
* Redacts from YAML output, including `mrsk config` (fixes #96)
2023-04-05 09:45:28 -07:00
Donal McBreen
05488e4c1e Zero downtime redeploys
When deploying check if there is already a container with the existing
name. If there is rename it to "<version>_<random_hex_string>" to remove
the name clash with the new container we want to boot.

We can then do the normal zero downtime run/wait/stop.

While implementing this I discovered the --filter name=foo does a
substring match for foo, so I've updated those filters to do an exact
match instead.
2023-03-24 17:09:20 +00:00
David Heinemeier Hansson
84540cee7b Merge branch 'main' into pr/154
* main: (32 commits)
  Inline default as with other options
  Symbols!
  Fix tests
  test stop with custom stop wait time
  No need to replicate Docker default
  Describe purpose rather than elements
  Style and ordering
  Customizable stop wait time
  Fix tests
  Ensure it also works when configuring just log options without setting a driver
  Add accessory test
  Undo change
  Improve test
  Update README
  Ensure default log option `max-size=10m`
  #142 Allow to customize container options in accessories
  Fix flaky test
  Fix tests
  More resilient tests
  Fix other tests
  ...
2023-03-24 15:43:17 +01:00
Donal McBreen
8d8f9f6ada Deploy locks
Add a deploy lock for commands that are unsafe to run concurrently.

The lock is taken by creating a `mrsk_lock` directory on the primary
host. Details of who took the lock are added to a details file in that
directory.

Additional CLI commands have been added to manual release and acquire
the lock and to check its status.

```
Commands:
  mrsk lock acquire -m, --message=MESSAGE  # Acquire the deploy lock
  mrsk lock help [COMMAND]                 # Describe subcommands or one specific subcommand
  mrsk lock release                        # Release the deploy lock
  mrsk lock status                         # Report lock status

Options:
  -v, [--verbose], [--no-verbose]                # Detailed logging
  -q, [--quiet], [--no-quiet]                    # Minimal logging
      [--version=VERSION]                        # Run commands against a specific app version
  -p, [--primary], [--no-primary]                # Run commands only on primary host instead of all
  -h, [--hosts=HOSTS]                            # Run commands on these hosts instead of all (separate by comma)
  -r, [--roles=ROLES]                            # Run commands on these roles instead of all (separate by comma)
  -c, [--config-file=CONFIG_FILE]                # Path to config file
                                                 # Default: config/deploy.yml
  -d, [--destination=DESTINATION]                # Specify destination to be used for config file (staging -> deploy.staging.yml)
  -B, [--skip-broadcast], [--no-skip-broadcast]  # Skip audit broadcasts
```

If we add support for running multiple deployments on a single server
we'll need to extend the locking to lock per deployment.
2023-03-24 12:28:08 +00:00
Samuel Sieg
1bdfc217c4 Merge branch 'main' into global-logging-config 2023-03-21 13:20:12 +01:00
David Heinemeier Hansson
49d60a045a Style 2023-03-21 12:41:28 +01:00
Samuel Sieg
b5372988f7 Add global logging configuration 2023-03-19 09:21:08 +01:00
Samuel Sieg
491777221f Fix destination label filter 2023-03-16 16:15:31 +01:00
David Heinemeier Hansson
487fcd4cea Only used for traefik 2023-03-09 11:00:52 +01:00
David Heinemeier Hansson
c8badea6dd Extract argumentization for cmd and add proper escaping 2023-03-09 10:54:53 +01:00
David Heinemeier Hansson
5898fdd8f4 Expand arguments to be more self-explanatory in logs 2023-02-19 18:11:06 +01:00
David Heinemeier Hansson
933ece35ab Add healthcheck before deploy 2023-02-18 16:22:08 +01:00
David Heinemeier Hansson
f371cda8d8 Stick with json logger for filebeat compatibility but cap at 10mb 2023-02-09 19:56:17 +01:00
David Heinemeier Hansson
9ec6f9d74f Merge branch 'main' into allow-bastion-server 2023-02-04 15:33:25 +01:00
David Heinemeier Hansson
e7ac73be5a Join in run_over_ssh instead of all over 2023-02-04 10:14:31 +01:00
David Heinemeier Hansson
dfca9d8c48 Merge branch 'main' into allow-bastion-server 2023-02-04 10:06:15 +01:00
David Heinemeier Hansson
3daecf696a Extract proper auditor and audit everything 2023-02-03 20:45:32 +01:00
David Heinemeier Hansson
d263b0ffa5 Extract xargs helper 2023-02-03 16:27:10 +01:00
Chris de Bruin
7d95472543 Added -J for ssh proxy 2023-02-03 14:31:09 +01:00
David Heinemeier Hansson
71f8f164ca Expose ssh_run 2023-02-01 14:04:51 +01:00
David Heinemeier Hansson
03fdb9a9ac Chain builder setup for better resiliency
Context may already exist while buildx does not
2023-01-27 15:41:28 +01:00
David Heinemeier Hansson
598e7ab97f Add power to follow logs on app and traefik 2023-01-22 11:27:31 +01:00
David Heinemeier Hansson
5d629d0600 Extract puts_by_host 2023-01-20 16:27:05 +01:00
David Heinemeier Hansson
435b558260 Extract pipe pattern 2023-01-20 14:38:27 +01:00
David Heinemeier Hansson
89161b66a1 Use delegation for shorter access 2023-01-15 13:50:38 +01:00
David Heinemeier Hansson
760a87fe06 Redact build args (since they are often tokens) 2023-01-15 13:15:14 +01:00
David Heinemeier Hansson
ea9a50ec95 Extract command #combine 2023-01-13 10:00:11 +01:00
David Heinemeier Hansson
d3ab10be22 Better require setup 2023-01-12 14:57:34 +01:00