Commit Graph

851 Commits

Author SHA1 Message Date
Dan Goodman
1bf4b6b76f Fix staging label bug
I think this is the correct fix based on the `service-role-destination` format, but seeing as it wasn't changed I assumed it was incorrect.
2023-05-04 17:47:17 -04:00
David Heinemeier Hansson
38c85e8021 Bump version for 0.12.0 v0.12.0 2023-05-02 17:23:10 +02:00
David Heinemeier Hansson
787ef96639 Don't run actions twice on PRs 2023-05-02 14:41:18 +02:00
David Heinemeier Hansson
1e8edc25e2 Merge pull request #205 from basecamp/docker-readiness
Bootstrap: multi-OS Docker install
2023-05-02 14:35:26 +02:00
David Heinemeier Hansson
b7877c59b4 Merge branch 'main' into docker-readiness 2023-05-02 14:30:35 +02:00
David Heinemeier Hansson
35b5b317af Merge branch 'main' into pr/205
* main:
  Simplify domain language to just "boot" and unscoped config keys
  Retain a fixed number of containers when pruning
  Don't assume rolling back in message
  Check all hosts before rolling back
  Ensure Traefik service name is consistent
  Extend traefik delay by 1 second
  Include traefik access logs
  Check if we are still getting a 404
  Also dump load balancer logs
  Dump traefik logs when app not booted
  Fix missing for apt-get
  Report on container health after failure
  Fix the integration test healthcheck
  Allow percentage-based rolling deployments
  Move `group_limit` & `group_wait` under `boot`
  Limit rolling deployment to boot operation
  Allow performing boot & start operations in groups
2023-05-02 14:29:06 +02:00
David Heinemeier Hansson
4c448f7eb1 Merge pull request #256 from Jberczel/check-local-dependencies
Add local dependencies check
2023-05-02 14:13:23 +02:00
David Heinemeier Hansson
263a24afe3 Further distinguish dependency verification 2023-05-02 14:09:10 +02:00
David Heinemeier Hansson
a2d99e48bf Naming 2023-05-02 14:08:29 +02:00
David Heinemeier Hansson
a22e27dbf8 Reveal configured dockerfile path 2023-05-02 14:07:47 +02:00
David Heinemeier Hansson
bb74a74dc4 Style 2023-05-02 14:07:30 +02:00
David Heinemeier Hansson
c611a1616a Distinguish from server dependencies 2023-05-02 14:06:06 +02:00
David Heinemeier Hansson
98e7b995d5 Distinguish from local dependency verification 2023-05-02 14:04:37 +02:00
David Heinemeier Hansson
ae2effb80c Improve clarity and intent 2023-05-02 14:04:23 +02:00
David Heinemeier Hansson
f719540e0c Style 2023-05-02 13:35:05 +02:00
David Heinemeier Hansson
cbda851436 Style 2023-05-02 13:34:56 +02:00
David Heinemeier Hansson
8854bb63a1 Merge pull request #254 from basecamp/retain-last-5-containers
Retain a fixed number of containers when pruning
2023-05-02 13:16:49 +02:00
David Heinemeier Hansson
35ea9f3c81 Merge pull request #255 from basecamp/check-all-hosts-for-rollback-container
Check all hosts before rolling back
2023-05-02 13:16:03 +02:00
David Heinemeier Hansson
18312f5191 Merge pull request #253 from basecamp/ensure-consistent-service-name
Ensure Traefik service name is consistent
2023-05-02 13:15:36 +02:00
David Heinemeier Hansson
71bc9bcf54 Merge pull request #222 from basecamp/deploy-groups
Allow booting containers in groups for rolling restarts
2023-05-02 13:14:32 +02:00
David Heinemeier Hansson
c83b74dcb7 Simplify domain language to just "boot" and unscoped config keys 2023-05-02 13:11:31 +02:00
Donal McBreen
971a91da15 Retain a fixed number of containers when pruning
Time based container and image retention can have variable space
requirements depending on how often we deploy.

- Only prune stopped containers, retaining the 5 newest
- Then prune dangling images so we only keep images for the retained
containers.
2023-05-02 10:15:08 +01:00
Donal McBreen
86d6f8d674 Don't assume rolling back in message 2023-05-02 10:14:50 +01:00
Donal McBreen
7fe24d5048 Check all hosts before rolling back
Hosts could end up out of sync with each other if prune commands are run
manually or when new hosts are added.

Before rolling back confirm that the required container is available on
all hosts and roles.
2023-05-02 10:14:50 +01:00
Kevin McConnell
a72f95f44d Ensure Traefik service name is consistent
If we don't specify any service properties when labelling containers,
the generated service will be named according to the container. However,
we change the container name on every deployment (as it is versioned),
which means that the auto-generated service name will be different in
each container.

That is a problem for two reasons:

- Multiple containers share a common router while a deployment is
  happening. At this point, the router configuration will be different
  between the containers; Traefik flags this as an error, and stops
  routing to the containers until it's resolved.
- We allow custom labels to be set in an app's config. In order to
  define custom configuration on the service, we'll need to know what
  it will be called.

Changed to force the service name by setting one of its properties.
2023-05-02 09:43:04 +01:00
David Heinemeier Hansson
dc3be30b16 Style 2023-05-02 10:29:49 +02:00
David Heinemeier Hansson
54881a0298 Merge pull request #250 from basecamp/integration-test-healthcheck-wget
Integration test healthcheck wget
2023-05-02 10:27:49 +02:00
Jberczel
bfb70b2118 Add local dependencies check
Add checks for:

* Docker installed locally
* Docker buildx plugin installed locally
* Dockerfile exists

If checks fail, it will halt deployment and provide more specific error messages.

Also adds a cli subcommand:
`mrsk build dependencies`

Fixes: #109 and #237
2023-05-01 16:32:41 -04:00
Jeremy Daer
e85bd5ff63 Bootstrap: use multi-platform installer
* Limit auto-install to root users; otherwise, give manual install guidance
* Support non-Debian/Ubuntu with the multi-OS get.docker.com installer
2023-05-01 13:26:00 -07:00
Donal McBreen
d0f66db33c Extend traefik delay by 1 second 2023-05-01 18:58:46 +01:00
Donal McBreen
650f9b1fbf Include traefik access logs 2023-05-01 18:55:10 +01:00
Donal McBreen
1170e2311e Check if we are still getting a 404 2023-05-01 18:32:07 +01:00
Donal McBreen
94f87edded Also dump load balancer logs 2023-05-01 18:27:08 +01:00
Donal McBreen
548a1019c1 Dump traefik logs when app not booted 2023-05-01 18:21:22 +01:00
Donal McBreen
ca2e2bac2e Fix missing for apt-get 2023-05-01 12:50:45 +01:00
Donal McBreen
494a1ae089 Report on container health after failure 2023-05-01 12:13:12 +01:00
Donal McBreen
a77428143f Fix the integration test healthcheck
The alpine nginx container doesn't contain curl, so let's override the
healthcheck command to use wget.
2023-05-01 12:11:24 +01:00
David Heinemeier Hansson
4fa6a6c06d Merge pull request #219 from basecamp/docker-health-checks 2023-04-28 11:43:33 +02:00
David Heinemeier Hansson
2ad0dc0703 Merge pull request #241 from Jberczel/fix-traefik-subcommand-typo 2023-04-28 11:38:46 +02:00
David Heinemeier Hansson
df067e4893 Merge pull request #244 from basecamp/get-lock-status-without-invoke 2023-04-25 18:57:05 +02:00
Donal McBreen
cd668066ff Get lock status by executing directly
Getting the lock status with invoke passes through any options from the
original command which will raise an exception if they are not also
valid for the lock status command.

Fixes https://github.com/mrsked/mrsk/issues/239
2023-04-25 16:57:02 +01:00
David Heinemeier Hansson
1a7d123746 Merge pull request #245 from basecamp/integration-test-wait-for-healthy
Wait for healthy containers in integration test
2023-04-25 16:54:22 +02:00
Donal McBreen
52ca5b846a Wait for healthy containers in integration test
Rather than waiting 5 seconds and hoping for the best after we boot
docker compose, add docker healthchecks and wait for all the containers
to be healthy.
2023-04-25 15:41:25 +01:00
Jberczel
126e0bbd06 Fix traefik remove_image desc typo 2023-04-24 17:40:28 -04:00
David Heinemeier Hansson
9ec3895dab Merge pull request #216 from dmrty/add-ssh-client-to-dockerised-mrsk 2023-04-15 08:29:49 +02:00
David Heinemeier Hansson
a6245a6bc9 Merge pull request #221 from iamFIREcracker/patch-1 2023-04-15 08:29:03 +02:00
David Heinemeier Hansson
0d80709e2d Merge pull request #224 from basecamp/integration-tests 2023-04-15 08:25:47 +02:00
Donal McBreen
bcf8a927f5 Run a mrsk deploy integration test
Adds a simple integration test to ensure that `mrsk deploy` works.

Everything required is spun up with docker compose:
- shared: a container that contains an ssh key and a self signed cert to
be shared between the images
- deployer: the image we will deploy from
- registry: a docker registry
- two vm images to deploy into
- load_balancer: an nginx load balancer to use between our images

The other images are in privileged mode so that we can run
docker-in-docker. We need to run docker inside the images - mapping in
the docker socket doesn't work because both VMs would share the host
daemon.

The docker registry requires a self signed cert as you cannot use basic
auth over HTTP except on localhost. It runs on port 4443 rather than 443
because docker refused to accept that "registry" is a docker host and
tries to push images to docker.io/registry. "registry:4443" works fine.

The shared container contains the ssh keys for the deployer and vms, and
the self signed cert for the registry. When the shared container boots,
it copies them into a shared volume.

The other deployer and vm images are built with soft links from the
shared volume to the require locations. Their boot scripts wait for the
files to be copied in before continuing.

The root mrsk folder is mapped into the deployer container. On boot it
builds the gem and installs it.

Right now there's just a single test. We confirm that the load balancer
is returning a 502, run `mrsk deploy` and then confirm it returns 200.
2023-04-14 15:49:43 +01:00
Kevin McConnell
f055766918 Allow percentage-based rolling deployments 2023-04-14 12:46:14 +01:00
Kevin McConnell
a8726be20e Move group_limit & group_wait under boot
Also make formatting the group strategy the responsibility of the
commander.
2023-04-14 11:31:51 +01:00