Commit Graph
8 Commits
Author SHA1 Message Date
Donal McBreen 051556674f Minimise holding the deploy lock
If we get an error we'll only hold the deploy lock if it occurs while
trying to switch the running containers.

We'll also move tagging the latest image from when the image is pulled
to just before the container switch. This ensures that earlier errors
don't leave the hosts with an updated latest tag while still running the
older version.
2023-04-12 12:09:56 +01:00
Donal McBreen 43f7409de0 Make rollbacks role-aware
Rollbacks stopped working after https://github.com/mrsked/mrsk/pull/99.

We'll confirm that a container is available for the first role on the
primary host before attempting to rollback.
2023-04-12 09:59:39 +01:00
Donal McBreen c4df440c79 Improved deploy lock acquisition
1. Don't raise lock error for non-lock issues during lock acquire
  (see https://github.com/mrsked/mrsk/pull/181)
2. If there is an error while the lock is held, don't release the lock
  and send a warning to stderr
2023-04-10 15:23:00 +01:00
Donal McBreen c06585fef4 Daemon/host/role accessories
Allow the hosts for accessories to be specified by host or role, or on
all app hosts by setting `daemon: true`.

```
  # Single host
  mysql:
    host: 1.1.1.1
  # Multiple hosts
  redis:
    hosts:
      - 1.1.1.1
      - 1.1.1.2
  # By role
  monitoring:
    roles:
      - web
      - jobs
```
2023-03-28 13:26:27 +01:00
Donal McBreen 05488e4c1e Zero downtime redeploys
When deploying check if there is already a container with the existing
name. If there is rename it to "<version>_<random_hex_string>" to remove
the name clash with the new container we want to boot.

We can then do the normal zero downtime run/wait/stop.

While implementing this I discovered the --filter name=foo does a
substring match for foo, so I've updated those filters to do an exact
match instead.
2023-03-24 17:09:20 +00:00
Donal McBreen 8d8f9f6ada Deploy locks
Add a deploy lock for commands that are unsafe to run concurrently.

The lock is taken by creating a `mrsk_lock` directory on the primary
host. Details of who took the lock are added to a details file in that
directory.

Additional CLI commands have been added to manual release and acquire
the lock and to check its status.

```
Commands:
  mrsk lock acquire -m, --message=MESSAGE  # Acquire the deploy lock
  mrsk lock help [COMMAND]                 # Describe subcommands or one specific subcommand
  mrsk lock release                        # Release the deploy lock
  mrsk lock status                         # Report lock status

Options:
  -v, [--verbose], [--no-verbose]                # Detailed logging
  -q, [--quiet], [--no-quiet]                    # Minimal logging
      [--version=VERSION]                        # Run commands against a specific app version
  -p, [--primary], [--no-primary]                # Run commands only on primary host instead of all
  -h, [--hosts=HOSTS]                            # Run commands on these hosts instead of all (separate by comma)
  -r, [--roles=ROLES]                            # Run commands on these roles instead of all (separate by comma)
  -c, [--config-file=CONFIG_FILE]                # Path to config file
                                                 # Default: config/deploy.yml
  -d, [--destination=DESTINATION]                # Specify destination to be used for config file (staging -> deploy.staging.yml)
  -B, [--skip-broadcast], [--no-skip-broadcast]  # Skip audit broadcasts
```

If we add support for running multiple deployments on a single server
we'll need to extend the locking to lock per deployment.
2023-03-24 12:28:08 +00:00
Donal McBreen 1ed4a37da2 Pull latest image tag, so we can identity it
`docker image ls` doesn't tell us what the latest deployed image is (e.g
if we've rolled back). Pull the latest image tag through to the server
so we can use it instead.
2023-03-23 14:39:32 +00:00
Donal McBreen fb3353084f Default to deploying the config version
If we don't supply a version when deploying we'll use the result of
docker image ls to decide which image to boot. But that doesn't
necessarily correspond to the one we have just built.

E.g. if you do something like:

```
mrsk deploy        # deploys git sha AAAAAAAAAAAAAA
git commit --amend # update the commit message
mrsk deploy        # deploys git sha BBBBBBBBBBBBBB
```

In this case running `docker image ls` will give you the same image
twice (because the contents are identical) with tags for both SHAs but
the image we have just built will not be returned first. Maybe the order
is random, but it always seems to come second as far as I have seen.

i.e you'll get something like:

```
REPOSITORY    TAG              IMAGE ID       CREATED          SIZE
foo/bar       AAAAAAAAAAAAAA   6272349a9619   31 minutes ago   791MB
foo/bar       BBBBBBBBBBBBBB   6272349a9619   31 minutes ago   791MB
```

Since we already know what version we want to deploy from the config,
let's just pass that through.
2023-03-22 16:14:50 +00:00