Commit Graph

66 Commits

Author SHA1 Message Date
Donal McBreen
5d33fb6c33 Better lock messages
- Debug verbosity commands
- Show lock status when we fail to acquire it
- Include lock acquire/release in runtime
2023-05-09 14:17:58 +01:00
Olivier
5ea552be40 Fix typo in message 2023-05-05 10:43:21 -04:00
Jeremy Daer
048aecf352 Audit details (#1)
Audit details

* Audit logs and broadcasts accept `details` whose values are included as log tags and MRSK_* env vars passed to the broadcast command
* Commands may return execution options to the CLI in their args list
* Introduce `mrsk broadcast` helper for sending audit broadcasts
* Report UTC time, not local time, in audit logs. Standardize on ISO 8601 format
2023-05-02 11:42:05 -07:00
David Heinemeier Hansson
35b5b317af Merge branch 'main' into pr/205
* main:
  Simplify domain language to just "boot" and unscoped config keys
  Retain a fixed number of containers when pruning
  Don't assume rolling back in message
  Check all hosts before rolling back
  Ensure Traefik service name is consistent
  Extend traefik delay by 1 second
  Include traefik access logs
  Check if we are still getting a 404
  Also dump load balancer logs
  Dump traefik logs when app not booted
  Fix missing for apt-get
  Report on container health after failure
  Fix the integration test healthcheck
  Allow percentage-based rolling deployments
  Move `group_limit` & `group_wait` under `boot`
  Limit rolling deployment to boot operation
  Allow performing boot & start operations in groups
2023-05-02 14:29:06 +02:00
Donal McBreen
86d6f8d674 Don't assume rolling back in message 2023-05-02 10:14:50 +01:00
Donal McBreen
7fe24d5048 Check all hosts before rolling back
Hosts could end up out of sync with each other if prune commands are run
manually or when new hosts are added.

Before rolling back confirm that the required container is available on
all hosts and roles.
2023-05-02 10:14:50 +01:00
Jeremy Daer
e85bd5ff63 Bootstrap: use multi-platform installer
* Limit auto-install to root users; otherwise, give manual install guidance
* Support non-Debian/Ubuntu with the multi-OS get.docker.com installer
2023-05-01 13:26:00 -07:00
Jacopo
9ddb181f50 Merge branch 'main' into cleanup-excessive-containers-running
* main:
  Pull the primary host from the role
  Minimise holding the deploy lock
2023-04-12 15:19:19 +02:00
Donal McBreen
051556674f Minimise holding the deploy lock
If we get an error we'll only hold the deploy lock if it occurs while
trying to switch the running containers.

We'll also move tagging the latest image from when the image is pulled
to just before the container switch. This ensures that earlier errors
don't leave the hosts with an updated latest tag while still running the
older version.
2023-04-12 12:09:56 +01:00
Jacopo
5ed431b807 Merge branch 'main' into cleanup-excessive-containers-running
* main: (24 commits)
  Bump version for 0.11.0
  Labels can be added to Traefik
  Make rollbacks role-aware
  fix typo role to roles
  Explained the latest modifications of Traefik container labels
  Remove .idea folder
  Updated README.md with new healthcheck.max_attempts option
  Fix test case: console output message was not updated to display the current/total attempts
  Require net-ssh ~> 7.0 for SHA-2 support
  Improved deploy lock acquisition
  Excess CR
  Style
  Simpler
  Make it explicit, focus on Ubuntu
  More explicit
  Not that --bundle is a Rails 7+ option
  Update README.md
  Update README.md
  Improved: configurable max_attempts for healthcheck
  Traefik service name to be derived from role and destination
  ...
2023-04-12 11:52:47 +02:00
Donal McBreen
43f7409de0 Make rollbacks role-aware
Rollbacks stopped working after https://github.com/mrsked/mrsk/pull/99.

We'll confirm that a container is available for the first role on the
primary host before attempting to rollback.
2023-04-12 09:59:39 +01:00
Jacopo
579b4cd9aa Simplify
By using and ad-hoc command to detect and stop stale containers.
By default stale containers are only detected.
2023-04-11 10:22:03 +02:00
Jacopo
8ae5331d97 Boot stop all the old containers 2023-04-11 08:53:33 +02:00
Jacopo
4d47fbdf41 Merge stop and stop_stale_containers 2023-04-11 08:53:33 +02:00
Jacopo
e2f6db5cae Clear stale containers
By stopping all the older containers with matching /#{service}-#{role}-#{dest}-.*/ running on the same host.
2023-04-11 08:53:33 +02:00
Jeremy Daer
c137b38c87 Only redact the non-sensitive bits of build args and env vars.
* `-e [REDACTED]` → `-e SOME_SECRET=[REDACTED]`
* Replaces `Utils.redact` with `Utils.sensitive` to clarify that we're
  indicating redactability, not actually performing redaction.
* Redacts from YAML output, including `mrsk config` (fixes #96)
2023-04-05 09:45:28 -07:00
Donal McBreen
8d8f9f6ada Deploy locks
Add a deploy lock for commands that are unsafe to run concurrently.

The lock is taken by creating a `mrsk_lock` directory on the primary
host. Details of who took the lock are added to a details file in that
directory.

Additional CLI commands have been added to manual release and acquire
the lock and to check its status.

```
Commands:
  mrsk lock acquire -m, --message=MESSAGE  # Acquire the deploy lock
  mrsk lock help [COMMAND]                 # Describe subcommands or one specific subcommand
  mrsk lock release                        # Release the deploy lock
  mrsk lock status                         # Report lock status

Options:
  -v, [--verbose], [--no-verbose]                # Detailed logging
  -q, [--quiet], [--no-quiet]                    # Minimal logging
      [--version=VERSION]                        # Run commands against a specific app version
  -p, [--primary], [--no-primary]                # Run commands only on primary host instead of all
  -h, [--hosts=HOSTS]                            # Run commands on these hosts instead of all (separate by comma)
  -r, [--roles=ROLES]                            # Run commands on these roles instead of all (separate by comma)
  -c, [--config-file=CONFIG_FILE]                # Path to config file
                                                 # Default: config/deploy.yml
  -d, [--destination=DESTINATION]                # Specify destination to be used for config file (staging -> deploy.staging.yml)
  -B, [--skip-broadcast], [--no-skip-broadcast]  # Skip audit broadcasts
```

If we add support for running multiple deployments on a single server
we'll need to extend the locking to lock per deployment.
2023-03-24 12:28:08 +00:00
David Heinemeier Hansson
c89b77127b Merge pull request #143 from djmb/default-to-deploying-config-version
Default to deploying the config version
2023-03-24 12:36:20 +01:00
Jeremy Daer
1887a6518e Commander needn't accumulate configuration
Commander had version/destination solely to incrementally accumulate CLI
options. Simpler to configure in one shot.

Clarifies responsibility and lets us introduce things like
`abbreviated_version` in one spot - Configuration.
2023-03-23 08:57:32 -07:00
David Heinemeier Hansson
65b90dd5c8 Merge branch 'main' into default-to-deploying-config-version 2023-03-23 14:42:31 +01:00
Jeremy Daer
53d7f9d528 Deploys mention the released service@version
Less work for broadcast commands to take on.

Also fixes a bug where rollback on hosts without a running container
would stop the container they had just started.
2023-03-23 01:09:25 -07:00
Donal McBreen
fb3353084f Default to deploying the config version
If we don't supply a version when deploying we'll use the result of
docker image ls to decide which image to boot. But that doesn't
necessarily correspond to the one we have just built.

E.g. if you do something like:

```
mrsk deploy        # deploys git sha AAAAAAAAAAAAAA
git commit --amend # update the commit message
mrsk deploy        # deploys git sha BBBBBBBBBBBBBB
```

In this case running `docker image ls` will give you the same image
twice (because the contents are identical) with tags for both SHAs but
the image we have just built will not be returned first. Maybe the order
is random, but it always seems to come second as far as I have seen.

i.e you'll get something like:

```
REPOSITORY    TAG              IMAGE ID       CREATED          SIZE
foo/bar       AAAAAAAAAAAAAA   6272349a9619   31 minutes ago   791MB
foo/bar       BBBBBBBBBBBBBB   6272349a9619   31 minutes ago   791MB
```

Since we already know what version we want to deploy from the config,
let's just pass that through.
2023-03-22 16:14:50 +00:00
Samuel Sieg
46dad1ee6c Add tests for main CLI commands 2023-03-14 15:58:12 +01:00
Samuel Sieg
c29d1ddeba Fix 2023-03-13 16:05:21 +01:00
Samuel Sieg
cb15800d25 Move option to deploy/redeploy, rename to skip-push 2023-03-13 16:02:24 +01:00
Samuel Sieg
47af6d9483 Is a global option better? 2023-03-12 10:53:29 +01:00
Samuel Sieg
ff0170076e Simplify 2023-03-12 10:44:33 +01:00
Samuel Sieg
9b39f2f3ab Keep it simple for the proposal 2023-03-12 10:41:04 +01:00
Samuel Sieg
4bf77ccd1b Allow deploy/deliver without building and pushing the image 2023-03-10 11:26:35 +01:00
Simon Rand
9ae3886b2b Ensure curl is installed during bootstrapping 2023-03-05 16:51:07 +00:00
David Heinemeier Hansson
701f6ff237 Move sleep note out of host loop, so we only see it once 2023-02-26 11:27:19 +01:00
David Heinemeier Hansson
27279c6c82 Accessories can individually ask for confirmation 2023-02-23 15:41:49 +01:00
David Heinemeier Hansson
9a4f502cc4 Pass confirmed flag to accessories 2023-02-23 15:31:56 +01:00
David Heinemeier Hansson
f4f2b5cb17 Communicate the readiness delay 2023-02-23 12:04:57 +01:00
David Heinemeier Hansson
e12436a1db Extract readiness_delay to config 2023-02-23 12:02:49 +01:00
David Heinemeier Hansson
b244e919bf Merge branch 'main' into more-resilient-zero-downtime-deploy
* main:
  Add option to skip audit broadcasts (useful when testing)
2023-02-23 11:52:45 +01:00
David Heinemeier Hansson
7ad416f029 Add option to skip audit broadcasts (useful when testing) 2023-02-23 10:04:35 +01:00
David Heinemeier Hansson
371f98d67f Start before stopping and longer timeouts 2023-02-22 19:04:23 +01:00
David Heinemeier Hansson
5356f31e2e Remove also removes accessories but requires confirmation 2023-02-19 17:16:14 +01:00
David Heinemeier Hansson
42bc691758 CLI doc updates
Match word

Language

Suggest what accessories are

There are also accessories

Default already shown

Better example

Warn about secrets being shown

Now also accessories

Wording

Clarifications

Clarify how to see options

General option for all

Options important here too

Hide subcommands

Implied

Simpler as just version

Be concise

Missing word

Wordsmith

Simpler and uniform words are better

Clarify what exactly we're manipulating

Wordsmithing

Implicit

Simpler language

Hide subcommands

Clarify its container management

Just one per server

Simpler
2023-02-19 17:15:44 +01:00
David Heinemeier Hansson
933ece35ab Add healthcheck before deploy 2023-02-18 16:22:08 +01:00
David Heinemeier Hansson
2e06bf59a4 Protect against rolling back to a bad version 2023-02-18 14:33:47 +01:00
David Heinemeier Hansson
854795c2b6 Wording 2023-02-18 12:10:42 +01:00
David Heinemeier Hansson
fb9357b5ba Add audit broadcasts 2023-02-18 11:36:30 +01:00
David Heinemeier Hansson
4f2cb5e184 Shorter 2023-02-11 13:00:22 +01:00
David Heinemeier Hansson
63a065237a Ensure .env file is only accessible to user 2023-02-11 12:56:57 +01:00
David Heinemeier Hansson
02a5726072 Allow destination specific envifying 2023-02-05 16:35:37 +01:00
David Heinemeier Hansson
e865e823d5 Add envify for managing .env file 2023-02-05 16:30:56 +01:00
David Heinemeier Hansson
10cad5c459 Create binstub without bundler, document it all agnostically
You can use MRSK with something other than Rails.
2023-02-05 16:23:34 +01:00
Xavier Noria
539752e9bd Load with Zeitwerk 2023-02-03 22:45:12 +01:00