Compare commits

..

76 Commits

Author SHA1 Message Date
David Heinemeier Hansson
f32ae43138 Bump version for 0.1.0 2023-01-14 12:35:17 +01:00
David Heinemeier Hansson
c3d2888c51 Update summary 2023-01-14 12:34:56 +01:00
David Heinemeier Hansson
6d1a166fdc Simplify 2023-01-14 12:33:05 +01:00
David Heinemeier Hansson
59be40cf12 Merge pull request #12 from rails/convert-to-thor
Switch to proper standalone executable with Thor
2023-01-14 12:28:24 +01:00
David Heinemeier Hansson
78494bdb0f Just rely on ENV for now 2023-01-14 12:27:38 +01:00
David Heinemeier Hansson
cce3d9ccfb Fix rollback 2023-01-14 12:23:34 +01:00
David Heinemeier Hansson
f0a3466d9d Rollback is clearer 2023-01-14 12:23:30 +01:00
David Heinemeier Hansson
e19e7f9bde Explicitly trying to start a specific version should fail if it can't 2023-01-14 12:23:22 +01:00
David Heinemeier Hansson
0b7af9ac14 Simplify 2023-01-14 12:17:04 +01:00
David Heinemeier Hansson
4551a2b9d7 Always try to log the command we're running remotely 2023-01-14 12:13:31 +01:00
David Heinemeier Hansson
e78da2a925 Update README to match new exec approach 2023-01-14 12:09:09 +01:00
David Heinemeier Hansson
94b3cfd0f4 Ship is cuter, but deploy is clearer
Kill your darlings
2023-01-14 12:07:52 +01:00
David Heinemeier Hansson
e3c1992ae9 Move HOST option to real option 2023-01-14 12:04:41 +01:00
David Heinemeier Hansson
ec31e931bf Add version task 2023-01-14 11:51:46 +01:00
David Heinemeier Hansson
e1e768d7cf Log traefik details commands 2023-01-14 11:51:38 +01:00
David Heinemeier Hansson
c44e224587 Add option to skip binstubs for older apps 2023-01-14 11:44:16 +01:00
David Heinemeier Hansson
fed64ef244 Switch to proper standalone executable with Thor 2023-01-14 11:31:37 +01:00
David Heinemeier Hansson
bf98a0308c Namespace buildx and contexts
To prevent clashes on remote builders
2023-01-13 17:29:53 +01:00
David Heinemeier Hansson
5179d0db37 Go with ship and make it the default 2023-01-13 17:12:46 +01:00
David Heinemeier Hansson
100d68d67e Only install docker if missing 2023-01-13 17:11:01 +01:00
David Heinemeier Hansson
eed8165ec1 Not worth the log noise 2023-01-13 15:44:56 +01:00
David Heinemeier Hansson
be89077917 Bump version for 0.0.3 2023-01-13 10:42:19 +01:00
David Heinemeier Hansson
6bfcc582c8 Singular 2023-01-13 10:30:02 +01:00
David Heinemeier Hansson
fd5172266e More expansive info on builder 2023-01-13 10:28:46 +01:00
David Heinemeier Hansson
e85c8161df Style 2023-01-13 10:28:35 +01:00
David Heinemeier Hansson
a1fc01639e Add build:info to check builder 2023-01-13 10:24:23 +01:00
David Heinemeier Hansson
7e764cbcd9 Explain how to use native builder 2023-01-13 10:18:42 +01:00
David Heinemeier Hansson
f177ee4cfe Make remote builder quack as any other builder 2023-01-13 10:16:28 +01:00
David Heinemeier Hansson
ea9a50ec95 Extract command #combine 2023-01-13 10:00:11 +01:00
David Heinemeier Hansson
6ea06fd04e Log the builder used 2023-01-13 09:49:06 +01:00
David Heinemeier Hansson
6ccb3d2319 Allow for fully native builds too
Skipping multiarch if there's a platform match between dev and prod.
2023-01-13 09:31:47 +01:00
David Heinemeier Hansson
05f1ef5ee8 Registry login actually not necessary 2023-01-12 22:22:22 +01:00
David Heinemeier Hansson
f1a98457b0 Pin platforms 2023-01-12 22:14:05 +01:00
David Heinemeier Hansson
7ae596ef60 Document remote native builds 2023-01-12 21:45:45 +01:00
David Heinemeier Hansson
2257c99189 Add local/remote builder combo for multiarch 2023-01-12 21:35:31 +01:00
David Heinemeier Hansson
5afadb10ca Nicer name for CLI 2023-01-12 18:50:18 +01:00
David Heinemeier Hansson
b3992973d6 Extract builder from app
Building is different from running
2023-01-12 18:16:52 +01:00
David Heinemeier Hansson
08c30a14b9 Use a single builder for MRSK 2023-01-12 18:08:33 +01:00
David Heinemeier Hansson
76d34d2a1c Note quoting issue 2023-01-12 17:42:49 +01:00
David Heinemeier Hansson
184ab18667 Style 2023-01-12 17:38:26 +01:00
David Heinemeier Hansson
87abf06076 Note on exception seen 2023-01-12 17:37:57 +01:00
David Heinemeier Hansson
453570b895 Breakout remove so we can do just containers 2023-01-12 17:37:50 +01:00
David Heinemeier Hansson
f61beb6827 Basic binstub 2023-01-12 17:29:26 +01:00
David Heinemeier Hansson
c481938cdb Reference Traefik docs for more routing rules 2023-01-12 17:16:30 +01:00
David Heinemeier Hansson
7e9b73f86a Add custom labels 2023-01-12 17:15:29 +01:00
David Heinemeier Hansson
1f06b1ff94 Switch to just last 100 log lines for now 2023-01-12 16:00:21 +01:00
David Heinemeier Hansson
d554ae8500 Add back prune 2023-01-12 15:51:01 +01:00
David Heinemeier Hansson
730de486b7 More doc changes 2023-01-12 15:29:56 +01:00
David Heinemeier Hansson
b333c4a05b Simplify presentation of configuration 2023-01-12 15:22:48 +01:00
David Heinemeier Hansson
eec6670dbf Tokens are good too 2023-01-12 15:16:29 +01:00
David Heinemeier Hansson
4aa96d6578 Switch to a Commander base to allow lazy loading config 2023-01-12 14:58:17 +01:00
David Heinemeier Hansson
d3ab10be22 Better require setup 2023-01-12 14:57:34 +01:00
David Heinemeier Hansson
d92318e234 Excess line 2023-01-11 17:58:50 +01:00
David Heinemeier Hansson
e62610069b Correct commadn 2023-01-11 17:46:35 +01:00
David Heinemeier Hansson
a0582c1bdf Explain registry 2023-01-11 17:46:00 +01:00
David Heinemeier Hansson
880ce46c39 Match service name 2023-01-11 17:44:26 +01:00
David Heinemeier Hansson
d049d73547 Realistic looking IP 2023-01-11 17:43:36 +01:00
David Heinemeier Hansson
453fea6c45 Don't rely on ERB interpolation that might fail
Error message isn't good
2023-01-11 17:43:28 +01:00
David Heinemeier Hansson
2694cf5d5f Make init more resilient and communicative 2023-01-11 17:43:07 +01:00
David Heinemeier Hansson
5324fbe3d0 Give feedback on what happened 2023-01-11 17:35:53 +01:00
David Heinemeier Hansson
5e214cde3c Explain where to set this 2023-01-11 17:35:46 +01:00
David Heinemeier Hansson
f61f41ad73 Document app console 2023-01-11 17:28:18 +01:00
David Heinemeier Hansson
d9cdbb87f9 Heads up that this could take a while 2023-01-11 17:26:49 +01:00
David Heinemeier Hansson
543af475d5 Create missing buildx builder if missing automatically 2023-01-11 17:24:32 +01:00
David Heinemeier Hansson
1bb9fe9095 Reuse existing exec command 2023-01-11 17:11:57 +01:00
David Heinemeier Hansson
c6fd4399f1 Hint at which version to start 2023-01-11 17:07:34 +01:00
David Heinemeier Hansson
a4a9f619ad Protect against missing envs 2023-01-11 17:07:22 +01:00
David Heinemeier Hansson
4392bf0ee9 Allow you to turn full verbosity on easily 2023-01-11 17:05:20 +01:00
David Heinemeier Hansson
3b3ab48120 Set a different verbosity level for the duration of the yield 2023-01-11 17:01:19 +01:00
David Heinemeier Hansson
606550d46b Reveal what was pruned 2023-01-11 17:01:12 +01:00
David Heinemeier Hansson
e1b327915f Use error logger instead 2023-01-11 17:01:03 +01:00
David Heinemeier Hansson
9d3871d667 Split out proper Prune command 2023-01-11 16:48:10 +01:00
David Heinemeier Hansson
7d83be2d18 Readability 2023-01-11 16:26:26 +01:00
David Heinemeier Hansson
3e2c48782c Explaining consts 2023-01-11 13:31:25 +01:00
David Heinemeier Hansson
bcdeeff94f Start remote Rails console on primary host 2023-01-10 20:45:15 +01:00
David Heinemeier Hansson
c5249b4a9e Host yield not needed 2023-01-10 20:44:54 +01:00
43 changed files with 895 additions and 373 deletions

View File

@@ -4,3 +4,4 @@ git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gemspec gemspec
gem "debug" gem "debug"
gem "railties"

View File

@@ -1,9 +1,10 @@
PATH PATH
remote: . remote: .
specs: specs:
mrsk (0.0.2) mrsk (0.1.0)
railties (>= 7.0.0) activesupport (>= 7.0)
sshkit (~> 1.21) sshkit (~> 1.21)
thor (~> 1.2)
GEM GEM
remote: https://rubygems.org/ remote: https://rubygems.org/
@@ -46,11 +47,11 @@ GEM
net-scp (4.0.0) net-scp (4.0.0)
net-ssh (>= 2.6.5, < 8.0.0) net-ssh (>= 2.6.5, < 8.0.0)
net-ssh (7.0.1) net-ssh (7.0.1)
nokogiri (1.14.0.rc1-arm64-darwin) nokogiri (1.14.0-arm64-darwin)
racc (~> 1.4) racc (~> 1.4)
nokogiri (1.14.0.rc1-x86_64-darwin) nokogiri (1.14.0-x86_64-darwin)
racc (~> 1.4) racc (~> 1.4)
nokogiri (1.14.0.rc1-x86_64-linux) nokogiri (1.14.0-x86_64-linux)
racc (~> 1.4) racc (~> 1.4)
racc (1.6.2) racc (1.6.2)
rack (2.2.5) rack (2.2.5)
@@ -90,6 +91,7 @@ PLATFORMS
DEPENDENCIES DEPENDENCIES
debug debug
mrsk! mrsk!
railties
BUNDLED WITH BUNDLED WITH
2.4.3 2.4.3

190
README.md
View File

@@ -1,80 +1,159 @@
# MRSK # MRSK
MRSK ships zero-downtime deploys of Rails apps packed as containers to any host. It uses the dynamic reverse-proxy Traefik to hold requests while the new application container is started and the old one is wound down. It works across multiple hosts at the same time, using SSHKit to execute commands. MRSK deploys Rails apps in containers to servers running Docker with zero downtime. It uses the dynamic reverse-proxy Traefik to hold requests while the new application container is started and the old one is stopped. It works seamlessly across multiple hosts, using SSHKit to execute commands.
## Installation ## Installation
Add the gem with `bundle add mrsk`, then run `rake mrsk:init`, and then edit the new file in `config/deploy.yml` to use the proper service name, image reference, servers to deploy on, and so on. It could look something like this: Install MRSK globally with `gem install mrsk`. Then, inside your app directory, run `mrsk install`. Now edit the new file `config/deploy.yml`. It could look as simple as this:
```yaml ```yaml
service: hey service: hey
image: 37s/hey image: 37s/hey
servers: servers:
- xxx.xxx.xxx.xxx - 192.168.0.1
- xxx.xxx.xxx.xxx - 192.168.0.2
env:
DATABASE_URL: mysql2://db1/hey_production/
REDIS_URL: redis://redis1:6379/1
registry: registry:
server: registry.digitalocean.com username: registry-user-name
username: <%= Rails.application.credentials.registry["username"] %> password: <%= ENV["MRSK_REGISTRY_PASSWORD"] %>
password: <%= Rails.application.credentials.registry["password"] %>
``` ```
Then ensure your encrypted credentials have the registry username + password by editing them with `rails credentials:edit`: Now you're ready to deploy a multi-arch image to the servers:
``` ```
registry: mrsk deploy
username: real-user-name
password: real-password
```
Now you're ready to deploy a multi-arch image (FIXME: currently you need to manually run `docker buildx create --use` once first):
```
rake mrsk:deploy
``` ```
This will: This will:
1. Log into the registry both locally and remotely 1. Install Docker on any server that might be missing it (using apt-get)
2. Build the image using the standard Dockerfile in the root of the application. 2. Log into the registry both locally and remotely
3. Push the image to the registry. 3. Build the image using the standard Dockerfile in the root of the application.
4. Pull the image from the registry on the servers. 4. Push the image to the registry.
5. Ensure Traefik is running and accepting traffic on port 80. 5. Pull the image from the registry on the servers.
6. Stop any containers running a previous versions of the app. 6. Ensure Traefik is running and accepting traffic on port 80.
7. Start a new container with the version of the app that matches the current git version hash. 7. Stop any containers running a previous versions of the app.
8. Prune unused images and stopped containers to ensure servers don't fill up. 8. Start a new container with the version of the app that matches the current git version hash.
9. Prune unused images and stopped containers to ensure servers don't fill up.
Voila! All the servers are now serving the app on port 80. If you're just running a single server, you're ready to go. If you're running multiple servers, you need to put a load balancer in front of them. Voila! All the servers are now serving the app on port 80. If you're just running a single server, you're ready to go. If you're running multiple servers, you need to put a load balancer in front of them.
## Operations ## Configuration
### Running job hosts separately ### Using another registry than Docker Hub
If your application uses separate job running hosts, or other roles beyond the default web running, you can specify these hosts and their custom command like so: The default registry for Docker is Docker Hub. If you'd like to use a different one, just configure the server, like so:
```yaml
registry:
server: registry.digitalocean.com
username: registry-user-name
password: <%= ENV["MRSK_REGISTRY_PASSWORD"] %>
```
### Using a different SSH user than root
The default SSH user is root, but you can change it using `ssh_user`:
```yaml
ssh_user: app
```
### Adding custom env variables
You can inject custom env variables into the app containers using `env`:
```yaml
env:
DATABASE_URL: mysql2://db1/hey_production/
REDIS_URL: redis://redis1:6379/1
```
### Splitting servers into different roles
If your application uses separate hosts for running jobs or other roles beyond the default web running, you can specify these hosts and their custom entrypoint command like so:
```yaml ```yaml
servers: servers:
web: web:
- xxx.xxx.xxx.xxx - 192.168.0.1
- xxx.xxx.xxx.xxx - 192.168.0.2
job: job:
hosts: hosts:
- xxx.xxx.xxx.xxx - 192.168.0.3
- xxx.xxx.xxx.xxx - 192.168.0.4
cmd: bin/jobs cmd: bin/jobs
``` ```
The application will be deployed to all hosts, but only those in the `web` role will be labeled to run under traefik. If you want to run custom commands on all hosts in a role, you can use `rake mrsk:app:exec:rails CMD=about ROLES=job`. Traefik will only be installed and run on the servers in the `web` role (and on all servers if no roles are defined).
### Executing commands ### Adding custom container labels
If you need to execute commands inside the Rails containers, you can use `rake mrsk:app:exec`, `rake mrsk:app:exec:once`, `rake mrsk:app:exec:rails`, and `rake mrsk:app:exec:once:rails`. Examples: You can specialize the default Traefik rules by setting custom labels on the containers that are being started:
```
labels:
traefik.http.routers.hey.rule: '''Host(`app.hey.com`)'''
```
(Note: The extra quotes are needed to ensure the rule is passed in correctly!)
This allows you to run multiple applications on the same server sharing the same Traefik instance and port.
See https://doc.traefik.io/traefik/routing/routers/#rule for a full list of available routing rules.
The labels can even be applied on a per-role basis:
```yaml
servers:
web:
- 192.168.0.1
- 192.168.0.2
job:
hosts:
- 192.168.0.3
- 192.168.0.4
cmd: bin/jobs
labels:
my-custom-label: "50"
```
### Configuring remote builder for native multi-arch
If you're developing on ARM64 (like Apple Silicon), but you want to deploy on AMD64 (x86 64-bit), you have to use multi-archecture images. By default, MRSK will setup a local buildx configuration that allows for this through QEMU emulation. This can be slow, especially on the first build.
If you want to speed up this process by using a remote AMD64 host to natively build the AMD64 part of the image, while natively building the ARM64 part locally, you can do so using builder options like follows:
```yaml
builder:
local:
arch: arm64
host: unix:///Users/dhh/.docker/run/docker.sock
remote:
arch: amd64
host: ssh://root@192.168.0.1
```
Note: You must have Docker running on the remote host being used as a builder.
With that configuration in place, you can setup the local/remote configuration using `mrsk build create`. If you wish to remove the contexts and buildx instances again, you can run `mrsk build remove`. If you had already built using the standard emulation setup, run `mrsk build remove` before doing `mrsk build remote`.
### Configuring native builder when multi-arch isn't needed
If you're developing on the same architecture as the one you're deploying on, you can speed up the build a lot by forgoing a multi-arch image. This can be done by configuring the builder like so:
```yaml
builder:
multiarch: false
```
## Commands
### Remote execution
If you need to execute commands inside the Rails containers, you can use `mrsk app exec`, `mrsk app exec --once`, `mrsk app runner`, and `mrsk app runner --once`. Examples:
```bash ```bash
# Runs command on all servers # Runs command on all servers
rake mrsk:app:exec CMD='ruby -v' mrsk app exec 'ruby -v'
App Host: xxx.xxx.xxx.xxx App Host: xxx.xxx.xxx.xxx
ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
@@ -82,11 +161,11 @@ App Host: xxx.xxx.xxx.xxx
ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
# Runs command on first server # Runs command on first server
rake mrsk:app:exec:once CMD='cat .ruby-version' mrsk app exec --once 'cat .ruby-version'
3.1.3 3.1.3
# Runs Rails command on all servers # Runs Rails command on all servers
rake mrsk:app:exec:rails CMD=about mrsk app exec 'bin/rails about'
App Host: xxx.xxx.xxx.xxx App Host: xxx.xxx.xxx.xxx
About your application's environment About your application's environment
Rails version 7.1.0.alpha Rails version 7.1.0.alpha
@@ -111,15 +190,18 @@ Environment production
Database adapter sqlite3 Database adapter sqlite3
Database schema version 20221231233303 Database schema version 20221231233303
# Runs Rails command on first server # Runs Rails runner on first server
rake mrsk:app:exec:once:rails CMD='db:version' mrsk app runner 'puts Rails.application.config.time_zone'
database: storage/production.sqlite3 UTC
Current version: 20221231233303
``` ```
### Running a Rails console on the primary host
If you need to interact with the production console for the app, you can use `mrsk app console`, which will start a Rails console session on the primary host. You can start the console on a different host using `mrsk app console --host 192.168.0.2`. Be mindful that this is a live wire! Any changes made to the production database will take effect immeditately.
### Inspecting ### Inspecting
You can see the state of your servers by running `rake mrsk:info`. It'll show something like this: You can see the state of your servers by running `mrsk details`. It'll show something like this:
``` ```
Traefik Host: xxx.xxx.xxx.xxx Traefik Host: xxx.xxx.xxx.xxx
@@ -139,11 +221,11 @@ CONTAINER ID IMAGE
1d3c91ed1f55 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 13 minutes ago Up 13 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123 1d3c91ed1f55 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 13 minutes ago Up 13 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123
``` ```
You can also see just info for app containers with `rake mrsk:app:info` or just for Traefik with `rake mrsk:traefik:info`. You can also see just info for app containers with `mrsk app details` or just for Traefik with `mrsk traefik details`.
### Rollback ### Rollback
If you've discovered a bad deploy, you can quickly rollback by reactivating the old, paused container image. You can see what old containers are available for rollback by running `rake mrsk:app:containers`. It'll give you a presentation similar to `rake mrsk:app:info`, but include all the old containers as well. Showing something like this: If you've discovered a bad deploy, you can quickly rollback by reactivating the old, paused container image. You can see what old containers are available for rollback by running `mrsk app containers`. It'll give you a presentation similar to `mrsk app details`, but include all the old containers as well. Showing something like this:
``` ```
App Host: 164.92.105.119 App Host: 164.92.105.119
@@ -157,21 +239,17 @@ badb1aa51db4 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483
6f170d1172ae registry.digitalocean.com/user/app:e5d9d7c2b898289dfbc5f7f1334140d984eedae4 "/rails/bin/docker-e…" 31 minutes ago Exited (1) 27 minutes ago chat-e5d9d7c2b898289dfbc5f7f1334140d984eedae4 6f170d1172ae registry.digitalocean.com/user/app:e5d9d7c2b898289dfbc5f7f1334140d984eedae4 "/rails/bin/docker-e…" 31 minutes ago Exited (1) 27 minutes ago chat-e5d9d7c2b898289dfbc5f7f1334140d984eedae4
``` ```
From the example above, we can see that `e5d9d7c2b898289dfbc5f7f1334140d984eedae4` was the last version, so it's available as a rollback target. We can perform this rollback by running `rake mrsk:rollback VERSION=e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. That'll stop `6ef8a6a84c525b123c5245345a8483f86d05a123` and then start `e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. Because the old container is still available, this is very quick. Nothing to download from the registry. From the example above, we can see that `e5d9d7c2b898289dfbc5f7f1334140d984eedae4` was the last version, so it's available as a rollback target. We can perform this rollback by running `mrsk rollback e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. That'll stop `6ef8a6a84c525b123c5245345a8483f86d05a123` and then start `e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. Because the old container is still available, this is very quick. Nothing to download from the registry.
Note that by default old containers are pruned after 3 days when you run `rake mrsk:deploy`. Note that by default old containers are pruned after 3 days when you run `mrsk deploy`.
### Removing ### Removing
If you wish to remove the entire application, including Traefik, containers, images, and registry session, you can run `rake mrsk:remove`. This will leave the servers clean. If you wish to remove the entire application, including Traefik, containers, images, and registry session, you can run `mrsk remove`. This will leave the servers clean.
## Stage of development ## Stage of development
This is alpha software. Lots of stuff is missing. Here are some of the areas we seek to improve: This is alpha software. Lots of stuff is missing. Lots of stuff will keep moving around for a while.
- Adapterize commands to work with Podman and other container runners
- Possibly switching to a bin/mrsk command rather than raw rake
- Integrate with cloud CI pipelines
## License ## License

5
bin/mrsk Executable file
View File

@@ -0,0 +1,5 @@
#!/usr/bin/env ruby
require "mrsk/cli"
Mrsk::Cli::Main.start(ARGV)

View File

@@ -2,7 +2,4 @@ module Mrsk
end end
require "mrsk/version" require "mrsk/version"
require "mrsk/engine" require "mrsk/commander"
require "mrsk/configuration"
require "mrsk/commands"

9
lib/mrsk/cli.rb Normal file
View File

@@ -0,0 +1,9 @@
require "mrsk"
MRSK = Mrsk::Commander.new \
config_file: Pathname.new(File.expand_path("config/deploy.yml"))
module Mrsk::Cli
end
require "mrsk/cli/main"

97
lib/mrsk/cli/app.rb Normal file
View File

@@ -0,0 +1,97 @@
require "mrsk/cli/base"
class Mrsk::Cli::App < Mrsk::Cli::Base
desc "boot", "Boot app on servers (or start them if they've already been booted)"
def boot
MRSK.config.roles.each do |role|
on(role.hosts) do |host|
begin
execute *MRSK.app.run(role: role.name)
rescue SSHKit::Command::Failed => e
if e.message =~ /already in use/
error "Container with same version already deployed on #{host}, starting that instead"
execute *MRSK.app.start, host: host
else
raise
end
end
end
end
end
desc "start", "Start existing app on servers (use --version=<git-hash> to designate specific version)"
option :version, desc: "Defaults to the most recent git-hash in local repository"
def start
if (version = options[:version]).present?
on(MRSK.config.hosts) { execute *MRSK.app.start(version: version) }
else
on(MRSK.config.hosts) { execute *MRSK.app.start, raise_on_non_zero_exit: false }
end
end
desc "stop", "Stop app on servers"
def stop
on(MRSK.config.hosts) { execute *MRSK.app.stop, raise_on_non_zero_exit: false }
end
desc "details", "Display details about app containers"
def details
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.info, verbosity: Logger::INFO) + "\n\n" }
end
desc "exec [CMD]", "Execute a custom task on servers passed in as CMD='bin/rake some:task'"
option :once, type: :boolean, default: false
def exec(cmd)
if options[:once]
on(MRSK.config.primary_host) { puts capture(*MRSK.app.exec(cmd), verbosity: Logger::INFO) }
else
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.exec(cmd), verbosity: Logger::INFO) + "\n\n" }
end
end
desc "console", "Start Rails Console on primary host"
option :host, desc: "Start console on a different host"
def console
host = options[:host] || MRSK.config.primary_host
run_locally do
puts "Launching Rails console on #{host}..."
exec MRSK.app.console(host: host)
end
end
desc "runner [EXPRESSION]", "Execute Rails runner with given expression"
option :once, type: :boolean, default: false, desc:
def runner(expression)
if options[:once]
on(MRSK.config.primary_host) { puts capture(*MRSK.app.exec("bin/rails", "runner", "'#{expression}'"), verbosity: Logger::INFO) }
else
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.exec("bin/rails", "runner", "'#{expression}'"), verbosity: Logger::INFO) + "\n\n" }
end
end
desc "containers", "List all the app containers currently on servers"
def containers
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.list_containers, verbosity: Logger::INFO) + "\n\n" }
end
desc "logs", "Show last 100 log lines from app on servers"
def logs
# FIXME: Catch when app containers aren't running
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.logs) + "\n\n" }
end
desc "remove", "Remove app containers and images from servers"
option :only, default: "", desc: "Use 'containers' or 'images'"
def remove
case options[:only]
when "containers"
on(MRSK.config.hosts) { execute *MRSK.app.remove_containers }
when "images"
on(MRSK.config.hosts) { execute *MRSK.app.remove_images }
else
on(MRSK.config.hosts) { execute *MRSK.app.remove_containers }
on(MRSK.config.hosts) { execute *MRSK.app.remove_images }
end
end
end

27
lib/mrsk/cli/base.rb Normal file
View File

@@ -0,0 +1,27 @@
require "thor"
require "sshkit"
require "sshkit/dsl"
module Mrsk::Cli
class Base < Thor
include SSHKit::DSL
def self.exit_on_failure?() true end
class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging"
def initialize(*)
super
MRSK.verbose = options[:verbose]
end
private
def print_runtime
started_at = Time.now
yield
ensure
runtime = Time.now - started_at
puts " Finished all in #{sprintf("%.1f seconds", runtime)}"
end
end
end

53
lib/mrsk/cli/build.rb Normal file
View File

@@ -0,0 +1,53 @@
require "mrsk/cli/base"
class Mrsk::Cli::Build < Mrsk::Cli::Base
desc "deliver", "Deliver a newly built app image to servers"
def deliver
invoke :push
invoke :pull
end
desc "push", "Build locally and push app image to registry"
def push
run_locally do
begin
debug "Using builder: #{MRSK.builder.name}"
info "Building image may take a while (run with --verbose for progress logging)"
execute *MRSK.builder.push
rescue SSHKit::Command::Failed => e
error "Missing compatible builder, so creating a new one first"
execute *MRSK.builder.create
execute *MRSK.builder.push
end
end
end
desc "pull", "Pull app image from the registry onto servers"
def pull
on(MRSK.config.hosts) { execute *MRSK.builder.pull }
end
desc "create", "Create a local build setup"
def create
run_locally do
debug "Using builder: #{MRSK.builder.name}"
execute *MRSK.builder.create
end
end
desc "remove", "Remove local build setup"
def remove
run_locally do
debug "Using builder: #{MRSK.builder.name}"
execute *MRSK.builder.remove
end
end
desc "details", "Show the name of the configured builder"
def details
run_locally do
puts "Builder: #{MRSK.builder.name} (#{MRSK.builder.target.class.name})"
puts capture(*MRSK.builder.info)
end
end
end

99
lib/mrsk/cli/main.rb Normal file
View File

@@ -0,0 +1,99 @@
require "mrsk/cli/base"
require "mrsk/cli/app"
require "mrsk/cli/build"
require "mrsk/cli/prune"
require "mrsk/cli/registry"
require "mrsk/cli/server"
require "mrsk/cli/traefik"
class Mrsk::Cli::Main < Mrsk::Cli::Base
desc "deploy", "Deploy the app to servers"
def deploy
print_runtime do
invoke "mrsk:cli:server:bootstrap"
invoke "mrsk:cli:registry:login"
invoke "mrsk:cli:build:deliver"
invoke "mrsk:cli:traefik:boot"
invoke "mrsk:cli:app:stop"
invoke "mrsk:cli:app:boot"
invoke "mrsk:cli:prune:all"
end
end
desc "redeploy", "Deploy new version of the app to servers (without bootstrapping servers, starting Traefik, pruning, and registry login)"
def redeploy
print_runtime do
invoke "mrsk:cli:build:deliver"
invoke "mrsk:cli:app:stop"
invoke "mrsk:cli:app:boot"
end
end
desc "rollback [VERSION]", "Rollback the app to VERSION (that must already be on servers)"
def rollback(version)
on(MRSK.config.hosts) do
execute *MRSK.app.stop, raise_on_non_zero_exit: false
execute *MRSK.app.start(version: version)
end
end
desc "details", "Display details about Traefik and app containers"
def details
invoke "mrsk:cli:traefik:details"
invoke "mrsk:cli:app:details"
end
desc "install", "Create config stub in config/deploy.yml and binstub in bin/mrsk"
option :skip_binstub, type: :boolean, default: false, desc: "Skip adding MRSK to the Gemfile and creating bin/mrsk binstub"
def install
require "fileutils"
if (deploy_file = Pathname.new(File.expand_path("config/deploy.yml"))).exist?
puts "Config file already exists in config/deploy.yml (remove first to create a new one)"
else
FileUtils.cp_r Pathname.new(File.expand_path("templates/deploy.yml", __dir__)), deploy_file
puts "Created configuration file in config/deploy.yml"
end
unless options[:skip_binstub]
if (binstub = Pathname.new(File.expand_path("bin/mrsk"))).exist?
puts "Binstub already exists in bin/mrsk (remove first to create a new one)"
else
`bundle add mrsk`
`bundle binstubs mrsk`
puts "Created binstub file in bin/mrsk"
end
end
end
desc "remove", "Remove Traefik, app, and registry session from servers"
def remove
invoke "mrsk:cli:traefik:remove"
invoke "mrsk:cli:app:remove"
invoke "mrsk:cli:registry:logout"
end
desc "version", "Display the MRSK version"
def version
puts Mrsk::VERSION
end
desc "app", "Manage the application"
subcommand "app", Mrsk::Cli::App
desc "build", "Build the application image"
subcommand "build", Mrsk::Cli::Build
desc "prune", "Prune old application images and containers"
subcommand "prune", Mrsk::Cli::Prune
desc "registry", "Login and out of the image registry"
subcommand "registry", Mrsk::Cli::Registry
desc "server", "Bootstrap servers with Docker"
subcommand "server", Mrsk::Cli::Server
desc "traefik", "Manage the Traefik load balancer"
subcommand "traefik", Mrsk::Cli::Traefik
end

19
lib/mrsk/cli/prune.rb Normal file
View File

@@ -0,0 +1,19 @@
require "mrsk/cli/base"
class Mrsk::Cli::Prune < Mrsk::Cli::Base
desc "all", "Prune unused images and stopped containers"
def all
invoke :containers
invoke :images
end
desc "images", "Prune unused images older than 30 days"
def images
on(MRSK.config.hosts) { execute *MRSK.prune.images }
end
desc "containers", "Prune stopped containers for the service older than 3 days"
def containers
on(MRSK.config.hosts) { execute *MRSK.prune.containers }
end
end

14
lib/mrsk/cli/registry.rb Normal file
View File

@@ -0,0 +1,14 @@
require "mrsk/cli/base"
class Mrsk::Cli::Registry < Mrsk::Cli::Base
desc "login", "Login to the registry locally and remotely"
def login
run_locally { execute *MRSK.registry.login }
on(MRSK.config.hosts) { execute *MRSK.registry.login }
end
desc "logout", "Logout of the registry remotely"
def logout
on(MRSK.config.hosts) { execute *MRSK.registry.logout }
end
end

8
lib/mrsk/cli/server.rb Normal file
View File

@@ -0,0 +1,8 @@
require "mrsk/cli/base"
class Mrsk::Cli::Server < Mrsk::Cli::Base
desc "bootstrap", "Ensure Docker is installed on the servers"
def bootstrap
on(MRSK.config.hosts) { execute "which docker || apt-get install docker.io -y" }
end
end

View File

@@ -0,0 +1,17 @@
# Name of your application. Used to uniquely configuring Traefik and app containers.
# Your Dockerfile should set LABEL service=the-same-value to ensure image pruning works.
service: my-app
# Name of the container image.
image: user/my-app
# Deploy to these servers.
servers:
- 192.168.0.1
# Credentials for your image host.
registry:
# Specify the registry server, if you're not using Docker Hub
# server: registry.digitalocean.com / ghcr.io / ...
username: my-user
password: my-password-should-go-somewhere-safe

44
lib/mrsk/cli/traefik.rb Normal file
View File

@@ -0,0 +1,44 @@
require "mrsk/cli/base"
class Mrsk::Cli::Traefik < Mrsk::Cli::Base
desc "boot", "Boot Traefik on servers"
def boot
on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.run, raise_on_non_zero_exit: false }
end
desc "start", "Start existing Traefik on servers"
def start
on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.start, raise_on_non_zero_exit: false }
end
desc "stop", "Stop Traefik on servers"
def stop
on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.stop, raise_on_non_zero_exit: false }
end
desc "restart", "Restart Traefik on servers"
def restart
invoke :stop
invoke :start
end
desc "details", "Display details about Traefik containers from servers"
def details
on(MRSK.config.role(:web).hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*MRSK.traefik.info, verbosity: Logger::INFO) + "\n\n" }
end
desc "logs", "Show last 100 log lines from Traefik on servers"
def logs
on(MRSK.config.hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*MRSK.traefik.logs) + "\n\n" }
end
desc "remove", "Remove Traefik container and image from servers"
def remove
invoke :stop
on(MRSK.config.role(:web).hosts) do
execute *MRSK.traefik.remove_container
execute *MRSK.traefik.remove_image
end
end
end

57
lib/mrsk/commander.rb Normal file
View File

@@ -0,0 +1,57 @@
require "mrsk/configuration"
require "mrsk/commands/app"
require "mrsk/commands/builder"
require "mrsk/commands/prune"
require "mrsk/commands/traefik"
require "mrsk/commands/registry"
class Mrsk::Commander
attr_reader :config
attr_accessor :verbose
def initialize(config_file:)
@config_file = config_file
end
def config
@config ||= Mrsk::Configuration.load_file(@config_file).tap { |config| setup_with(config) }
end
def app
@app ||= Mrsk::Commands::App.new(config)
end
def builder
@builder ||= Mrsk::Commands::Builder.new(config)
end
def traefik
@traefik ||= Mrsk::Commands::Traefik.new(config)
end
def registry
@registry ||= Mrsk::Commands::Registry.new(config)
end
def prune
@prune ||= Mrsk::Commands::Prune.new(config)
end
def verbosity(level)
old_level = SSHKit.config.output_verbosity
SSHKit.config.output_verbosity = level
yield
ensure
SSHKit.config.output_verbosity = old_level
end
private
# Lazy setup of SSHKit
def setup_with(config)
SSHKit::Backend::Netssh.configure { |ssh| ssh.ssh_options = config.ssh_options }
SSHKit.config.command_map[:docker] = "docker" # No need to use /usr/bin/env, just clogs up the logs
SSHKit.config.output_verbosity = :debug if verbose
end
end

View File

@@ -1,25 +1,2 @@
require "sshkit"
module Mrsk::Commands module Mrsk::Commands
class Base
attr_accessor :config
def initialize(config)
@config = config
end
private
def docker(*args)
args.compact.unshift :docker
end
# Copied from SSHKit::Backend::Abstract#redact to be available inside Commands classes
def redact(arg) # Used in execute_command to hide redact() args a user passes in
arg.to_s.extend(SSHKit::Redaction) # to_s due to our inability to extend Integer, etc
end
end
end end
require "mrsk/commands/app"
require "mrsk/commands/traefik"
require "mrsk/commands/registry"

View File

@@ -1,14 +1,6 @@
require "mrsk/commands/base"
class Mrsk::Commands::App < Mrsk::Commands::Base class Mrsk::Commands::App < Mrsk::Commands::Base
def push
# TODO: Run 'docker buildx create --use' when needed
# TODO: Make multiarch an option so Linux users can enjoy speedier builds
docker :buildx, :build, "--push", "--platform linux/amd64,linux/arm64", "-t", config.absolute_image, "."
end
def pull
docker :pull, config.absolute_image
end
def run(role: :web) def run(role: :web)
role = config.role(role) role = config.role(role)
@@ -23,8 +15,8 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
role.cmd role.cmd
end end
def start def start(version: config.version)
docker :start, config.service_with_version docker :start, "#{config.service}-#{version}"
end end
def stop def stop
@@ -36,17 +28,22 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
end end
def logs def logs
[ "docker ps -q #{service_filter.join(" ")} | xargs docker logs -f" ] [ "docker ps -q #{service_filter.join(" ")} | xargs docker logs -n 100 -t" ]
end end
def exec(*command) def exec(*command, interactive: false)
docker :exec, docker :exec,
("-it" if interactive),
"-e", redact("RAILS_MASTER_KEY=#{config.master_key}"), "-e", redact("RAILS_MASTER_KEY=#{config.master_key}"),
*config.env_args, *config.env_args,
config.service_with_version, config.service_with_version,
*command *command
end end
def console(host: config.primary_host)
"ssh -t #{config.ssh_user}@#{host} '#{exec("bin/rails", "c", interactive: true).join(" ")}'"
end
def list_containers def list_containers
docker :container, :ls, "-a", *service_filter docker :container, :ls, "-a", *service_filter
end end

27
lib/mrsk/commands/base.rb Normal file
View File

@@ -0,0 +1,27 @@
require "sshkit"
module Mrsk::Commands
class Base
attr_accessor :config
def initialize(config)
@config = config
end
private
def combine(*commands)
commands
.collect { |command| command + [ "&&" ] }.flatten # Join commands with &&
.tap { |commands| commands.pop } # Remove trailing &&
end
def docker(*args)
args.compact.unshift :docker
end
# Copied from SSHKit::Backend::Abstract#redact to be available inside Commands classes
def redact(arg) # Used in execute_command to hide redact() args a user passes in
arg.to_s.extend(SSHKit::Redaction) # to_s due to our inability to extend Integer, etc
end
end
end

View File

@@ -0,0 +1,39 @@
require "mrsk/commands/base"
class Mrsk::Commands::Builder < Mrsk::Commands::Base
delegate :create, :remove, :push, :pull, :info, to: :target
delegate :native?, :multiarch?, :remote?, to: :name
def name
target.class.to_s.demodulize.downcase.inquiry
end
def target
case
when config.builder.nil?
multiarch
when config.builder["multiarch"] == false
native
when config.builder["local"] && config.builder["local"]
multiarch_remote
else
raise ArgumentError, "Builder configuration incorrect: #{config.builder.inspect}"
end
end
def native
@native ||= Mrsk::Commands::Builder::Native.new(config)
end
def multiarch
@multiarch ||= Mrsk::Commands::Builder::Multiarch.new(config)
end
def multiarch_remote
@multiarch_remote ||= Mrsk::Commands::Builder::Multiarch::Remote.new(config)
end
end
require "mrsk/commands/builder/native"
require "mrsk/commands/builder/multiarch"
require "mrsk/commands/builder/multiarch/remote"

View File

@@ -0,0 +1,30 @@
require "mrsk/commands/base"
class Mrsk::Commands::Builder::Multiarch < Mrsk::Commands::Base
def create
docker :buildx, :create, "--use", "--name", builder_name
end
def remove
docker :buildx, :rm, builder_name
end
def push
docker :buildx, :build, "--push", "--platform linux/amd64,linux/arm64", "-t", config.absolute_image, "."
end
def pull
docker :pull, config.absolute_image
end
def info
combine \
docker(:context, :ls),
docker(:buildx, :ls)
end
private
def builder_name
"mrsk-#{config.service}"
end
end

View File

@@ -0,0 +1,58 @@
require "mrsk/commands/builder/multiarch"
class Mrsk::Commands::Builder::Multiarch::Remote < Mrsk::Commands::Builder::Multiarch
def create
combine \
create_contexts,
create_local_buildx,
append_remote_buildx
end
def remove
combine \
remove_contexts,
super
end
private
def create_local_buildx
docker :buildx, :create, "--use", "--name", builder_name, builder_name_with_arch(local["arch"]), "--platform", "linux/#{local["arch"]}"
end
def append_remote_buildx
docker :buildx, :create, "--append", "--name", builder_name, builder_name_with_arch(remote["arch"]), "--platform", "linux/#{remote["arch"]}"
end
def create_contexts
combine \
create_context(local["arch"], local["host"]),
create_context(remote["arch"], remote["host"])
end
def create_context(arch, host)
docker :context, :create, builder_name_with_arch(arch), "--description", "'#{builder_name} #{arch} native host'", "--docker", "'host=#{host}'"
end
def remove_contexts
combine \
remove_context(local["arch"]),
remove_context(remote["arch"])
end
def remove_context(arch)
docker :context, :rm, builder_name_with_arch(arch)
end
def local
config.builder["local"]
end
def remote
config.builder["remote"]
end
private
def builder_name_with_arch(arch)
"#{builder_name}-#{arch}"
end
end

View File

@@ -0,0 +1,25 @@
require "mrsk/commands/base"
class Mrsk::Commands::Builder::Native < Mrsk::Commands::Base
def create
# No-op on native
end
def remove
# No-op on native
end
def push
combine \
docker(:build, "-t", config.absolute_image, "."),
docker(:push, config.absolute_image)
end
def pull
docker :pull, config.absolute_image
end
def info
# No-op on native
end
end

View File

@@ -0,0 +1,17 @@
require "mrsk/commands/base"
require "active_support/duration"
require "active_support/core_ext/numeric/time"
class Mrsk::Commands::Prune < Mrsk::Commands::Base
PRUNE_IMAGES_AFTER = 30.days.in_hours.to_i
PRUNE_CONTAINERS_AFTER = 3.days.in_hours.to_i
def images
docker :image, :prune, "-f", "--filter", "until=#{PRUNE_IMAGES_AFTER}h"
end
def containers
docker :image, :prune, "-f", "--filter", "until=#{PRUNE_IMAGES_AFTER}h"
docker :container, :prune, "-f", "--filter", "label=service=#{config.service}", "--filter", "'until=#{PRUNE_CONTAINERS_AFTER}h'"
end
end

View File

@@ -1,3 +1,5 @@
require "mrsk/commands/base"
class Mrsk::Commands::Registry < Mrsk::Commands::Base class Mrsk::Commands::Registry < Mrsk::Commands::Base
delegate :registry, to: :config delegate :registry, to: :config

View File

@@ -1,3 +1,5 @@
require "mrsk/commands/base"
class Mrsk::Commands::Traefik < Mrsk::Commands::Base class Mrsk::Commands::Traefik < Mrsk::Commands::Base
def run def run
docker :run, "--name traefik", docker :run, "--name traefik",
@@ -22,7 +24,7 @@ class Mrsk::Commands::Traefik < Mrsk::Commands::Base
end end
def logs def logs
docker :logs, "traefik" docker :logs, "traefik", "-n", "100", "-t"
end end
def remove_container def remove_container

View File

@@ -1,9 +1,11 @@
require "active_support/ordered_options" require "active_support/ordered_options"
require "active_support/core_ext/string/inquiry" require "active_support/core_ext/string/inquiry"
require "active_support/core_ext/module/delegation"
require "pathname"
require "erb" require "erb"
class Mrsk::Configuration class Mrsk::Configuration
delegate :service, :image, :servers, :env, :registry, :ssh_user, to: :config, allow_nil: true delegate :service, :image, :servers, :env, :labels, :registry, :builder, to: :config, allow_nil: true
class << self class << self
def load_file(file) def load_file(file)
@@ -75,15 +77,23 @@ class Mrsk::Configuration
def env_args def env_args
self.class.argumentize "-e", config.env if config.env.present? if config.env.present?
self.class.argumentize "-e", config.env
else
[]
end
end
def ssh_user
config.ssh_user || "root"
end end
def ssh_options def ssh_options
{ user: config.ssh_user || "root", auth_methods: [ "publickey" ] } { user: ssh_user, auth_methods: [ "publickey" ] }
end end
def master_key def master_key
ENV["RAILS_MASTER_KEY"] || File.read(Rails.root.join("config/master.key")) ENV["RAILS_MASTER_KEY"] || File.read(Pathname.new(File.expand_path("config/master.key")))
end end

View File

@@ -11,6 +11,14 @@ class Mrsk::Configuration::Role
@hosts ||= extract_hosts_from_config @hosts ||= extract_hosts_from_config
end end
def labels
if name.web?
default_labels.merge(traefik_labels).merge(custom_labels)
else
default_labels.merge(custom_labels)
end
end
def label_args def label_args
argumentize "--label", labels argumentize "--label", labels
end end
@@ -31,14 +39,6 @@ class Mrsk::Configuration::Role
end end
end end
def labels
if name.web?
default_labels.merge(traefik_labels)
else
default_labels
end
end
def default_labels def default_labels
{ "service" => config.service, "role" => name } { "service" => config.service, "role" => name }
end end
@@ -53,6 +53,13 @@ class Mrsk::Configuration::Role
} }
end end
def custom_labels
Hash.new.tap do |labels|
labels.merge!(config.labels) if config.labels.present?
labels.merge!(specializations["labels"]) if specializations["labels"].present?
end
end
def specializations def specializations
if config.servers.is_a?(Array) || config.servers[name].is_a?(Array) if config.servers.is_a?(Array) || config.servers[name].is_a?(Array)
{ } { }

View File

@@ -1,4 +0,0 @@
module Mrsk
class Engine < ::Rails::Engine
end
end

View File

@@ -1,3 +1,3 @@
module Mrsk module Mrsk
VERSION = "0.0.2" VERSION = "0.1.0"
end end

View File

@@ -1,98 +0,0 @@
require_relative "setup"
app = Mrsk::Commands::App.new(MRSK_CONFIG)
namespace :mrsk do
namespace :app do
desc "Deliver a newly built app image to servers"
task deliver: %i[ push pull ]
desc "Build locally and push app image to registry"
task :push do
run_locally { execute *app.push } unless ENV["VERSION"]
end
desc "Pull app image from the registry onto servers"
task :pull do
on(MRSK_CONFIG.hosts) { execute *app.pull }
end
desc "Run app on servers (or start them if they've already been run)"
task :run do
MRSK_CONFIG.roles.each do |role|
on(role.hosts) do |host|
begin
execute *app.run(role: role.name)
rescue SSHKit::Command::Failed => e
if e.message =~ /already in use/
puts "Container with same version already deployed on #{host}, starting that instead"
execute *app.start, host: host
else
raise
end
end
end
end
end
desc "Start existing app on servers"
task :start do
on(MRSK_CONFIG.hosts) { execute *app.start, raise_on_non_zero_exit: false }
end
desc "Stop app on servers"
task :stop do
on(MRSK_CONFIG.hosts) { execute *app.stop, raise_on_non_zero_exit: false }
end
desc "Start app on servers (use VERSION=<git-hash> to designate which version)"
task restart: %i[ stop start ]
desc "Display information about app containers"
task :info do
on(MRSK_CONFIG.hosts) { |host| puts "App Host: #{host}\n" + capture(*app.info) + "\n\n" }
end
desc "Execute a custom task on servers passed in as CMD='bin/rake some:task'"
task :exec do
on(MRSK_CONFIG.hosts) { |host| puts "App Host: #{host}\n" + capture(*app.exec(ENV["CMD"])) + "\n\n" }
end
namespace :exec do
desc "Execute Rails command on servers, like CMD='runner \"puts %(Hello World)\""
task :rails do
on(MRSK_CONFIG.hosts) { |host| puts "App Host: #{host}\n" + capture(*app.exec("bin/rails", ENV["CMD"])) + "\n\n" }
end
desc "Execute a custom task on the first defined server"
task :once do
on(MRSK_CONFIG.primary_host) { |host| puts capture(*app.exec(ENV["CMD"])) }
end
namespace :once do
desc "Execute Rails command on the first defined server, like CMD='runner \"puts %(Hello World)\""
task :rails do
on(MRSK_CONFIG.primary_host) { puts capture(*app.exec("bin/rails", ENV["CMD"])) }
end
end
end
desc "List all the app containers currently on servers"
task :containers do
on(MRSK_CONFIG.hosts) { |host| puts "App Host: #{host}\n" + capture(*app.list_containers) + "\n\n" }
end
desc "Tail logs from app containers"
task :logs do
on(MRSK_CONFIG.hosts) { execute *app.logs }
end
desc "Remove app containers and images from servers"
task remove: %i[ stop ] do
on(MRSK_CONFIG.hosts) do
execute *app.remove_containers
execute *app.remove_images
end
end
end
end

View File

@@ -1,26 +0,0 @@
require_relative "setup"
namespace :mrsk do
desc "Deploy app for the first time to a fresh server"
task fresh: %w[ server:bootstrap registry:login app:deliver traefik:run app:stop app:run ]
desc "Push the latest version of the app, ensure Traefik is running, then restart app"
task deploy: %w[ registry:login app:deliver traefik:run app:stop app:run prune ]
desc "Rollback to VERSION=x that was already run as a container on servers"
task rollback: %w[ app:restart ]
desc "Display information about Traefik and app containers"
task info: %w[ traefik:info app:info ]
desc "Create config stub in config/deploy.yml"
task :init do
require "fileutils"
FileUtils.cp_r \
Pathname.new(File.expand_path("templates/deploy.yml", __dir__)),
Rails.root.join("config/deploy.yml")
end
desc "Remove Traefik, app, and registry session from servers"
task remove: %w[ traefik:remove app:remove registry:logout ]
end

View File

@@ -1,18 +0,0 @@
require_relative "setup"
namespace :mrsk do
desc "Prune unused images and stopped containers"
task prune: %w[ prune:containers prune:images ]
namespace :prune do
desc "Prune unused images older than 30 days"
task :images do
on(MRSK_CONFIG.hosts) { execute "docker image prune -f --filter 'until=720h'" }
end
desc "Prune stopped containers for the service older than 3 days"
task :containers do
on(MRSK_CONFIG.hosts) { execute "docker container prune -f --filter label=service=#{MRSK_CONFIG.service} --filter 'until=72h'" }
end
end
end

View File

@@ -1,18 +0,0 @@
require_relative "setup"
registry = Mrsk::Commands::Registry.new(MRSK_CONFIG)
namespace :mrsk do
namespace :registry do
desc "Login to the registry locally and remotely"
task :login do
run_locally { execute *registry.login }
on(MRSK_CONFIG.hosts) { execute *registry.login }
end
desc "Logout of the registry remotely"
task :logout do
on(MRSK_CONFIG.hosts) { execute *registry.logout }
end
end
end

View File

@@ -1,11 +0,0 @@
require_relative "setup"
namespace :mrsk do
namespace :server do
desc "Setup Docker on the remote servers"
task :bootstrap do
# FIXME: Detect when apt-get is not available and use the appropriate alternative
on(MRSK_CONFIG.hosts) { execute "apt-get install docker.io -y" }
end
end
end

View File

@@ -1,16 +0,0 @@
require "sshkit"
require "sshkit/dsl"
include SSHKit::DSL
if (config_file = Rails.root.join("config/deploy.yml")).exist?
MRSK_CONFIG = Mrsk::Configuration.load_file(config_file)
SSHKit::Backend::Netssh.configure { |ssh| ssh.ssh_options = MRSK_CONFIG.ssh_options }
# No need to use /usr/bin/env, just clogs up the logs
SSHKit.config.command_map[:docker] = "docker"
else
# MRSK is missing config/deploy.yml run 'rake mrsk:init'
MRSK_CONFIG = Mrsk::Configuration.new({}, validate: false)
end

View File

@@ -1,21 +0,0 @@
# Name of your application will be used for uniquely configuring Traefik and app containers.
# Your Dockerfile should set LABEL service=the-same-value to ensure image pruning works.
service: my-app
# Name of the container image
image: user/chat
# All the servers targeted for deploy. You can reference a single server for a command by using SERVERS=xxx.xxx.xxx.xxx
servers:
- xxx.xxx.xxx.xxx
# The following envs are made available to the container when started
env:
# Remember never to put passwords or tokens directly into this file, use encrypted credentials
# REDIS_URL: redis://x/y
registry:
# Specify the registry server, if you're not using Docker Hub
# server: registry.digitalocean.com / ghcr.io / ...
username: <%= Rails.application.credentials.registry["username"] %>
password: <%= Rails.application.credentials.registry["password"] %>

View File

@@ -1,38 +0,0 @@
require_relative "setup"
traefik = Mrsk::Commands::Traefik.new(MRSK_CONFIG)
namespace :mrsk do
namespace :traefik do
desc "Run Traefik on servers"
task :run do
on(MRSK_CONFIG.role(:web).hosts) { execute *traefik.run, raise_on_non_zero_exit: false }
end
desc "Start existing Traefik on servers"
task :start do
on(MRSK_CONFIG.role(:web).hosts) { execute *traefik.start, raise_on_non_zero_exit: false }
end
desc "Stop Traefik on servers"
task :stop do
on(MRSK_CONFIG.role(:web).hosts) { execute *traefik.stop, raise_on_non_zero_exit: false }
end
desc "Restart Traefik on servers"
task restart: %i[ stop start ]
desc "Display information about Traefik containers from servers"
task :info do
on(MRSK_CONFIG.role(:web).hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*traefik.info) + "\n\n" }
end
desc "Remove Traefik container and image from servers"
task remove: %i[ stop ] do
on(MRSK_CONFIG.role(:web).hosts) do
execute *traefik.remove_container
execute *traefik.remove_image
end
end
end
end

View File

@@ -6,11 +6,13 @@ Gem::Specification.new do |spec|
spec.authors = [ "David Heinemeier Hansson" ] spec.authors = [ "David Heinemeier Hansson" ]
spec.email = "dhh@hey.com" spec.email = "dhh@hey.com"
spec.homepage = "https://github.com/rails/mrsk" spec.homepage = "https://github.com/rails/mrsk"
spec.summary = "Deploy Docker containers with zero downtime to any host." spec.summary = "Deploy Rails apps in containers to servers running Docker with zero downtime."
spec.license = "MIT" spec.license = "MIT"
spec.files = Dir["lib/**/*", "MIT-LICENSE", "README.md"] spec.files = Dir["lib/**/*", "MIT-LICENSE", "README.md"]
spec.executables = %w[ mrsk ]
spec.add_dependency "railties", ">= 7.0.0" spec.add_dependency "activesupport", ">= 7.0"
spec.add_dependency "sshkit", "~> 1.21" spec.add_dependency "sshkit", "~> 1.21"
spec.add_dependency "thor", "~> 1.2"
end end

View File

@@ -1,6 +1,6 @@
require "test_helper" require "test_helper"
require "mrsk/configuration" require "mrsk/configuration"
require "mrsk/commands" require "mrsk/commands/app"
ENV["VERSION"] = "123" ENV["VERSION"] = "123"
ENV["RAILS_MASTER_KEY"] = "456" ENV["RAILS_MASTER_KEY"] = "456"

View File

@@ -0,0 +1,24 @@
require "test_helper"
require "mrsk/configuration"
require "mrsk/commands/builder"
class BuilderCommandTest < ActiveSupport::TestCase
setup do
@config = { service: "app", image: "dhh/app", registry: { "username" => "dhh", "password" => "secret" }, servers: [ "1.1.1.1" ] }
end
test "target multiarch by default" do
builder = Mrsk::Commands::Builder.new(Mrsk::Configuration.new(@config))
assert builder.multiarch?
end
test "target native when multiarch is off" do
builder = Mrsk::Commands::Builder.new(Mrsk::Configuration.new(@config.merge({ builder: { "multiarch" => false } })))
assert builder.native?
end
test "target multiarch remote when local and remote is set" do
builder = Mrsk::Commands::Builder.new(Mrsk::Configuration.new(@config.merge({ builder: { "local" => { }, "remote" => { } } })))
assert builder.remote?
end
end

12
test/commander_test.rb Normal file
View File

@@ -0,0 +1,12 @@
require "test_helper"
require "mrsk/commander"
class CommanderTest < ActiveSupport::TestCase
setup do
@mrsk = Mrsk::Commander.new config_file: Pathname.new(File.expand_path("fixtures/deploy.erb.yml", __dir__))
end
test "lazy configuration" do
assert_equal Mrsk::Configuration, @mrsk.config.class
end
end

View File

@@ -30,6 +30,11 @@ class ConfigurationRoleTest < ActiveSupport::TestCase
assert_equal [ "1.1.1.3", "1.1.1.4" ], @config_with_roles.role(:workers).hosts assert_equal [ "1.1.1.3", "1.1.1.4" ], @config_with_roles.role(:workers).hosts
end end
test "cmd" do
assert_nil @config.role(:web).cmd
assert_equal "bin/jobs", @config_with_roles.role(:workers).cmd
end
test "label args" do test "label args" do
assert_equal [ "--label", "service=app", "--label", "role=workers" ], @config_with_roles.role(:workers).label_args assert_equal [ "--label", "service=app", "--label", "role=workers" ], @config_with_roles.role(:workers).label_args
end end
@@ -38,8 +43,19 @@ class ConfigurationRoleTest < ActiveSupport::TestCase
assert_equal [ "--label", "service=app", "--label", "role=web", "--label", "traefik.http.routers.app.rule='PathPrefix(`/`)'", "--label", "traefik.http.services.app.loadbalancer.healthcheck.path=/up", "--label", "traefik.http.services.app.loadbalancer.healthcheck.interval=1s", "--label", "traefik.http.middlewares.app.retry.attempts=3", "--label", "traefik.http.middlewares.app.retry.initialinterval=500ms"], @config.role(:web).label_args assert_equal [ "--label", "service=app", "--label", "role=web", "--label", "traefik.http.routers.app.rule='PathPrefix(`/`)'", "--label", "traefik.http.services.app.loadbalancer.healthcheck.path=/up", "--label", "traefik.http.services.app.loadbalancer.healthcheck.interval=1s", "--label", "traefik.http.middlewares.app.retry.attempts=3", "--label", "traefik.http.middlewares.app.retry.initialinterval=500ms"], @config.role(:web).label_args
end end
test "cmd" do test "custom labels" do
assert_nil @config.role(:web).cmd @deploy[:labels] = { "my.custom.label" => "50" }
assert_equal "bin/jobs", @config_with_roles.role(:workers).cmd assert_equal "50", @config.role(:web).labels["my.custom.label"]
end
test "custom labels via role specialization" do
@deploy_with_roles[:labels] = { "my.custom.label" => "50" }
@deploy_with_roles[:servers]["workers"]["labels"] = { "my.custom.label" => "70" }
assert_equal "70", @config_with_roles.role(:workers).labels["my.custom.label"]
end
test "overwriting default traefik label" do
@deploy[:labels] = { "traefik.http.routers.app.rule" => "'Host(`example.com`) || (Host(`example.org`) && Path(`/traefik`))'" }
assert_equal "'Host(`example.com`) || (Host(`example.org`) && Path(`/traefik`))'", @config.role(:web).labels["traefik.http.routers.app.rule"]
end end
end end