Commit Graph

271 Commits

Author SHA1 Message Date
Lewis Buckley
ecfd258093 Document the rolling reboot option 2023-07-19 14:58:46 +01:00
Lewis Buckley
313f89a108 Merge branch 'main' into rolling-traefik-restarts
* main:
  Removed not needed MRSK.traefik.run command in Traefil reboot
  Updated README with locking directory name
  Include service name to lock details
  Configurable SSH log levels
  Add registry container output to debug
  Minor tweaks to hooks section in readme
  Update README.md
  Updated README.md to make setup examples consistent
  Login to the registry proactively before stoping Accessory and Traefik
2023-07-19 14:46:16 +01:00
Lewis Buckley
9ab448e186 Support a --rolling option for traefik reboots 2023-07-19 14:39:27 +01:00
Igor Alexandrov
a29e188c90 Removed not needed MRSK.traefik.run command in Traefil reboot 2023-07-19 15:08:42 +04:00
Igor Alexandrov
2746a48e88 Login to the registry proactively before stoping Accessory and Traefik 2023-06-22 15:13:47 +04:00
Donal McBreen
4950f61a87 Only require secrets when mutating
Rename `with_lock` to more generic `mutating` and move the env_args
check to that point. This allows read-only actions to be run without
requiring secrets.
2023-06-20 15:39:51 +01:00
Donal McBreen
12ca865e71 Fix up the sample pre-deploy hook
- Fix the shebang
- Extract a GithubStatusChecks class
2023-06-12 12:47:51 +01:00
David Heinemeier Hansson
504a09ef1d Merge pull request #318 from basecamp/pre-deploy-hook
Add a pre-deploy hook
2023-05-31 17:59:46 +02:00
David Heinemeier Hansson
5a25f073f7 Merge pull request #320 from jsoref/spelling
Spelling
2023-05-31 17:59:18 +02:00
David Heinemeier Hansson
c8f521c0e8 Merge pull request #323 from basecamp/prefix-docker-host-with-real-host
Prefix container hostname with the underlying one
2023-05-31 17:58:55 +02:00
Donal McBreen
28d6a131a9 Prefix container hostname with the underlying one
To make it easier to identity where a docker container is running,
prefix its hostname with the underlying one from the host.

Docker chooses a 12 character random hex string by default, so we'll
keep that as the suffix.
2023-05-31 16:22:25 +01:00
Donal McBreen
079d9538bb Improve image pruning robustness
If you different images with the same git SHA, on the second deploy the
tag is moved and the first image becomes untagged. It may however still
be attached to an existing container.

To handle this:
1. Initially prune dangling images - this will remove any untagged
images that are not attached to an existing image
2. Then filter out the untagged images when deleting tagged images - any
that remain will be attached to a container.

The second issue is that `docker container ls -a --format '{{.Image}}`
will sometimes return the image id rather than a tag. This means that
the image doesn't get filtered out when we grep to remove the active
images.

To fix that we'll grep against both the image id and repo:tag.
2023-05-31 10:17:52 +01:00
Josh Soref
85005be07f spelling: message
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2023-05-29 20:46:34 -04:00
Donal McBreen
db0bf6bb16 Add a pre-deploy hook
Useful for checking the status of CI before deploying. Doing this at
this point in the deployment maximises the parallelisation of building
and running CI.
2023-05-29 16:06:41 +01:00
David Heinemeier Hansson
57e4f08c4c Merge pull request #308 from tannakartikey/hooks_small_fix
Hooks sample files typo fix
2023-05-25 09:02:25 +02:00
Donal McBreen
66f9ce0e90 Add a pre-connect hook
This can be used for hooks that should run before connecting to remote
hosts. An example use case is pre-warming DNS.
2023-05-24 14:39:30 +01:00
Kartikey Tanna
956ab3560b Hooks typo fix 2023-05-23 22:12:49 +05:30
Donal McBreen
19f0f40adf Add skip_hooks option 2023-05-23 15:56:47 +01:00
Donal McBreen
f9cb87e55a Fixup rebase issues 2023-05-23 14:10:38 +01:00
Donal McBreen
cc2b321d93 Combine post-deploy and post-rollback 2023-05-23 13:57:24 +01:00
Donal McBreen
004f1b04e6 Remove the skip_broadcast option 2023-05-23 13:57:00 +01:00
Donal McBreen
3b695ae127 Add service_version and add running hook message 2023-05-23 13:56:19 +01:00
Donal McBreen
258887a451 Set sample hook permissions and preserve when copying 2023-05-23 13:56:19 +01:00
Donal McBreen
9fd184dc32 Add post-deploy and post-rollback hooks
These replace the custom audit_broadcast_cmd code. An additional env
variable MRSK_RUNTIME is passed to them.

The audit broadcast after booting an accessory has been removed.
2023-05-23 13:56:16 +01:00
Donal McBreen
38023fe538 Remove post push hook 2023-05-23 13:55:05 +01:00
David Heinemeier Hansson
5ab630cb03 Style 2023-05-23 13:55:04 +01:00
Donal McBreen
58c1096a90 MRSK hooks
Adds hooks to MRSK. Currently just two hooks, pre-build and post-push.

We could break the build and push into two separate commands if we
found the need for post-build and/or pre-push hooks.

Hooks are stored in `.mrsk/hooks`. Running `mrsk init` will now create
that folder and add sample hook scripts.

Hooks returning non-zero exit codes will abort the current command.

Further potential work here:
- We could replace the audit broadcast command with a
post-deploy/post-rollback hook or similar
- Maybe provide pre-command/post-command hooks that run after every
mrsk invocation
- Also look for hooks in `~/.mrsk/hooks`
2023-05-23 13:55:04 +01:00
Donal McBreen
340ed94fa9 Make verify_local_dependencies private
We don't need to what it returns, it raises if there is a problem.

Move it out of the run_locally block to make it easier to add hooks.
2023-05-23 13:55:04 +01:00
David Heinemeier Hansson
4e9c39f26d Merge pull request #271 from basecamp/app-boot-for-rollback
Call app:boot to rollback
2023-05-23 13:17:30 +02:00
Donal McBreen
7cd25fd163 Add more integration tests
Add tests for main, app, accessory, traefik and lock commands.
Other commands are generally covered by the main tests.

Also adds some changes to speed up the integration specs:
- Use a persistent volume for the registry so we can push images to to
reuse between runs (also gets around docker hub rate limits)
- Use persistent volume for mrsk gem install, to avoid re-installing
between tests
- Shorter stop wait time
- Shorter connection timeouts on the load balancer

Takes just over 2 minutes to run all tests locally on an M1 Mac
after docker caches are primed.
2023-05-16 10:35:35 +01:00
Donal McBreen
ee25f200d7 Call app:boot to rollback
The code in Mrsk::Cli::Main#rollback was very similar to
Mrsk::Cli::App#boot.

Modify Mrsk::Cli::App#boot so it can handle rollbacks by:
1. Only renaming running containers
2. Trying first to start then run the new container
2023-05-16 08:59:07 +01:00
Donal McBreen
a5ef1f254f Highlight uncommitted changes in version
If there are uncommitted changes in the app repository when building,
then append `_uncommitted_<random>` to it to distinguish the image
from one built from a clean checkout.

Also change the version used when renaming a container on redeploy to
distinguish and explain the version suffixes.
2023-05-12 11:08:48 +01:00
Donal McBreen
5d33fb6c33 Better lock messages
- Debug verbosity commands
- Show lock status when we fail to acquire it
- Include lock acquire/release in runtime
2023-05-09 14:17:58 +01:00
Olivier
5ea552be40 Fix typo in message 2023-05-05 10:43:21 -04:00
Kevin McConnell
82be521e66 Merge branch 'main' into customizable-audit-broadcast
* main:
  Fix staging label bug
  Fix typo
  Capture container health log when unhealthy
  Bump version for 0.12.0
2023-05-05 11:40:29 +01:00
Jberczel
0e19ead37c Capture container health log when unhealthy 2023-05-03 15:03:05 -04:00
Jeremy Daer
048aecf352 Audit details (#1)
Audit details

* Audit logs and broadcasts accept `details` whose values are included as log tags and MRSK_* env vars passed to the broadcast command
* Commands may return execution options to the CLI in their args list
* Introduce `mrsk broadcast` helper for sending audit broadcasts
* Report UTC time, not local time, in audit logs. Standardize on ISO 8601 format
2023-05-02 11:42:05 -07:00
David Heinemeier Hansson
88a7413b3e Merge branch 'main' into pr/223
* main:
  Don't run actions twice on PRs
  Further distinguish dependency verification
  Naming
  Reveal configured dockerfile path
  Style
  Distinguish from server dependencies
  Distinguish from local dependency verification
  Improve clarity and intent
  Style
  Style
  Style
  Add local dependencies check
  Bootstrap: use multi-platform installer
2023-05-02 14:44:16 +02:00
David Heinemeier Hansson
9cc73fed9a Merge branch 'main' into pr/223
* main:
  Simplify domain language to just "boot" and unscoped config keys
  Retain a fixed number of containers when pruning
  Don't assume rolling back in message
  Check all hosts before rolling back
  Ensure Traefik service name is consistent
  Extend traefik delay by 1 second
  Include traefik access logs
  Check if we are still getting a 404
  Also dump load balancer logs
  Dump traefik logs when app not booted
  Fix missing for apt-get
  Report on container health after failure
  Fix the integration test healthcheck
  Allow percentage-based rolling deployments
  Move `group_limit` & `group_wait` under `boot`
  Limit rolling deployment to boot operation
  Allow performing boot & start operations in groups
2023-05-02 14:43:17 +02:00
David Heinemeier Hansson
b7877c59b4 Merge branch 'main' into docker-readiness 2023-05-02 14:30:35 +02:00
David Heinemeier Hansson
35b5b317af Merge branch 'main' into pr/205
* main:
  Simplify domain language to just "boot" and unscoped config keys
  Retain a fixed number of containers when pruning
  Don't assume rolling back in message
  Check all hosts before rolling back
  Ensure Traefik service name is consistent
  Extend traefik delay by 1 second
  Include traefik access logs
  Check if we are still getting a 404
  Also dump load balancer logs
  Dump traefik logs when app not booted
  Fix missing for apt-get
  Report on container health after failure
  Fix the integration test healthcheck
  Allow percentage-based rolling deployments
  Move `group_limit` & `group_wait` under `boot`
  Limit rolling deployment to boot operation
  Allow performing boot & start operations in groups
2023-05-02 14:29:06 +02:00
David Heinemeier Hansson
4c448f7eb1 Merge pull request #256 from Jberczel/check-local-dependencies
Add local dependencies check
2023-05-02 14:13:23 +02:00
David Heinemeier Hansson
263a24afe3 Further distinguish dependency verification 2023-05-02 14:09:10 +02:00
David Heinemeier Hansson
c611a1616a Distinguish from server dependencies 2023-05-02 14:06:06 +02:00
David Heinemeier Hansson
98e7b995d5 Distinguish from local dependency verification 2023-05-02 14:04:37 +02:00
David Heinemeier Hansson
ae2effb80c Improve clarity and intent 2023-05-02 14:04:23 +02:00
David Heinemeier Hansson
f719540e0c Style 2023-05-02 13:35:05 +02:00
David Heinemeier Hansson
8854bb63a1 Merge pull request #254 from basecamp/retain-last-5-containers
Retain a fixed number of containers when pruning
2023-05-02 13:16:49 +02:00
David Heinemeier Hansson
35ea9f3c81 Merge pull request #255 from basecamp/check-all-hosts-for-rollback-container
Check all hosts before rolling back
2023-05-02 13:16:03 +02:00
David Heinemeier Hansson
71bc9bcf54 Merge pull request #222 from basecamp/deploy-groups
Allow booting containers in groups for rolling restarts
2023-05-02 13:14:32 +02:00