Compare commits

..

21 Commits

Author SHA1 Message Date
David Heinemeier Hansson
f32ae43138 Bump version for 0.1.0 2023-01-14 12:35:17 +01:00
David Heinemeier Hansson
c3d2888c51 Update summary 2023-01-14 12:34:56 +01:00
David Heinemeier Hansson
6d1a166fdc Simplify 2023-01-14 12:33:05 +01:00
David Heinemeier Hansson
59be40cf12 Merge pull request #12 from rails/convert-to-thor
Switch to proper standalone executable with Thor
2023-01-14 12:28:24 +01:00
David Heinemeier Hansson
78494bdb0f Just rely on ENV for now 2023-01-14 12:27:38 +01:00
David Heinemeier Hansson
cce3d9ccfb Fix rollback 2023-01-14 12:23:34 +01:00
David Heinemeier Hansson
f0a3466d9d Rollback is clearer 2023-01-14 12:23:30 +01:00
David Heinemeier Hansson
e19e7f9bde Explicitly trying to start a specific version should fail if it can't 2023-01-14 12:23:22 +01:00
David Heinemeier Hansson
0b7af9ac14 Simplify 2023-01-14 12:17:04 +01:00
David Heinemeier Hansson
4551a2b9d7 Always try to log the command we're running remotely 2023-01-14 12:13:31 +01:00
David Heinemeier Hansson
e78da2a925 Update README to match new exec approach 2023-01-14 12:09:09 +01:00
David Heinemeier Hansson
94b3cfd0f4 Ship is cuter, but deploy is clearer
Kill your darlings
2023-01-14 12:07:52 +01:00
David Heinemeier Hansson
e3c1992ae9 Move HOST option to real option 2023-01-14 12:04:41 +01:00
David Heinemeier Hansson
ec31e931bf Add version task 2023-01-14 11:51:46 +01:00
David Heinemeier Hansson
e1e768d7cf Log traefik details commands 2023-01-14 11:51:38 +01:00
David Heinemeier Hansson
c44e224587 Add option to skip binstubs for older apps 2023-01-14 11:44:16 +01:00
David Heinemeier Hansson
fed64ef244 Switch to proper standalone executable with Thor 2023-01-14 11:31:37 +01:00
David Heinemeier Hansson
bf98a0308c Namespace buildx and contexts
To prevent clashes on remote builders
2023-01-13 17:29:53 +01:00
David Heinemeier Hansson
5179d0db37 Go with ship and make it the default 2023-01-13 17:12:46 +01:00
David Heinemeier Hansson
100d68d67e Only install docker if missing 2023-01-13 17:11:01 +01:00
David Heinemeier Hansson
eed8165ec1 Not worth the log noise 2023-01-13 15:44:56 +01:00
33 changed files with 465 additions and 381 deletions

View File

@@ -4,3 +4,4 @@ git_source(:github) { |repo| "https://github.com/#{repo}.git" }
gemspec
gem "debug"
gem "railties"

View File

@@ -1,9 +1,10 @@
PATH
remote: .
specs:
mrsk (0.0.3)
railties (>= 7.0.0)
mrsk (0.1.0)
activesupport (>= 7.0)
sshkit (~> 1.21)
thor (~> 1.2)
GEM
remote: https://rubygems.org/
@@ -46,11 +47,11 @@ GEM
net-scp (4.0.0)
net-ssh (>= 2.6.5, < 8.0.0)
net-ssh (7.0.1)
nokogiri (1.14.0.rc1-arm64-darwin)
nokogiri (1.14.0-arm64-darwin)
racc (~> 1.4)
nokogiri (1.14.0.rc1-x86_64-darwin)
nokogiri (1.14.0-x86_64-darwin)
racc (~> 1.4)
nokogiri (1.14.0.rc1-x86_64-linux)
nokogiri (1.14.0-x86_64-linux)
racc (~> 1.4)
racc (1.6.2)
rack (2.2.5)
@@ -90,6 +91,7 @@ PLATFORMS
DEPENDENCIES
debug
mrsk!
railties
BUNDLED WITH
2.4.3

View File

@@ -1,10 +1,10 @@
# MRSK
MRSK ships zero-downtime deploys of Rails apps packed as containers to any host. It uses the dynamic reverse-proxy Traefik to hold requests while the new application container is started and the old one is wound down. It works seamlessly across multiple hosts, using SSHKit to execute commands.
MRSK deploys Rails apps in containers to servers running Docker with zero downtime. It uses the dynamic reverse-proxy Traefik to hold requests while the new application container is started and the old one is stopped. It works seamlessly across multiple hosts, using SSHKit to execute commands.
## Installation
Add the gem with `bundle add mrsk`, then run `rake mrsk:init`, and then edit the new file in `config/deploy.yml`. It could look as simple as this:
Install MRSK globally with `gem install mrsk`. Then, inside your app directory, run `mrsk install`. Now edit the new file `config/deploy.yml`. It could look as simple as this:
```yaml
service: hey
@@ -13,34 +13,27 @@ servers:
- 192.168.0.1
- 192.168.0.2
registry:
username: <%= Rails.application.credentials.registry["username"] %>
password: <%= Rails.application.credentials.registry["password"] %>
```
Then ensure your encrypted credentials have the registry username + password by editing them with `rails credentials:edit`:
```
registry:
username: real-user-name
password: real-registry-password-or-token
username: registry-user-name
password: <%= ENV["MRSK_REGISTRY_PASSWORD"] %>
```
Now you're ready to deploy a multi-arch image to the servers:
```
./bin/mrsk deploy
mrsk deploy
```
This will:
1. Log into the registry both locally and remotely
2. Build the image using the standard Dockerfile in the root of the application.
3. Push the image to the registry.
4. Pull the image from the registry on the servers.
5. Ensure Traefik is running and accepting traffic on port 80.
6. Stop any containers running a previous versions of the app.
7. Start a new container with the version of the app that matches the current git version hash.
8. Prune unused images and stopped containers to ensure servers don't fill up.
1. Install Docker on any server that might be missing it (using apt-get)
2. Log into the registry both locally and remotely
3. Build the image using the standard Dockerfile in the root of the application.
4. Push the image to the registry.
5. Pull the image from the registry on the servers.
6. Ensure Traefik is running and accepting traffic on port 80.
7. Stop any containers running a previous versions of the app.
8. Start a new container with the version of the app that matches the current git version hash.
9. Prune unused images and stopped containers to ensure servers don't fill up.
Voila! All the servers are now serving the app on port 80. If you're just running a single server, you're ready to go. If you're running multiple servers, you need to put a load balancer in front of them.
@@ -53,8 +46,8 @@ The default registry for Docker is Docker Hub. If you'd like to use a different
```yaml
registry:
server: registry.digitalocean.com
username: <%= Rails.application.credentials.registry["username"] %>
password: <%= Rails.application.credentials.registry["password"] %>
username: registry-user-name
password: <%= ENV["MRSK_REGISTRY_PASSWORD"] %>
```
### Using a different SSH user than root
@@ -141,7 +134,7 @@ builder:
Note: You must have Docker running on the remote host being used as a builder.
With that configuration in place, you can setup the local/remote configuration using `./bin/mrsk build:remote:create`. If you wish to remove the contexts and buildx instances again, you can run `./bin/mrsk build:remote:remove`. If you had already built using the standard emulation setup, run `./bin/mrsk build:remove` before doing `./bin/mrsk build:remote:create`.
With that configuration in place, you can setup the local/remote configuration using `mrsk build create`. If you wish to remove the contexts and buildx instances again, you can run `mrsk build remove`. If you had already built using the standard emulation setup, run `mrsk build remove` before doing `mrsk build remote`.
### Configuring native builder when multi-arch isn't needed
@@ -156,11 +149,11 @@ builder:
### Remote execution
If you need to execute commands inside the Rails containers, you can use `./bin/mrsk app:exec`, `./bin/mrsk app:exec:once`, `./bin/mrsk app:exec:rails`, and `./bin/mrsk app:exec:once:rails`. Examples:
If you need to execute commands inside the Rails containers, you can use `mrsk app exec`, `mrsk app exec --once`, `mrsk app runner`, and `mrsk app runner --once`. Examples:
```bash
# Runs command on all servers
./bin/mrsk app:exec CMD='ruby -v'
mrsk app exec 'ruby -v'
App Host: xxx.xxx.xxx.xxx
ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
@@ -168,11 +161,11 @@ App Host: xxx.xxx.xxx.xxx
ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux]
# Runs command on first server
./bin/mrsk app:exec:once CMD='cat .ruby-version'
mrsk app exec --once 'cat .ruby-version'
3.1.3
# Runs Rails command on all servers
./bin/mrsk app:exec:rails CMD=about
mrsk app exec 'bin/rails about'
App Host: xxx.xxx.xxx.xxx
About your application's environment
Rails version 7.1.0.alpha
@@ -197,19 +190,18 @@ Environment production
Database adapter sqlite3
Database schema version 20221231233303
# Runs Rails command on first server
./bin/mrsk app:exec:once:rails CMD='db:version'
database: storage/production.sqlite3
Current version: 20221231233303
# Runs Rails runner on first server
mrsk app runner 'puts Rails.application.config.time_zone'
UTC
```
### Running a Rails console on the primary host
If you need to interact with the production console for the app, you can use `./bin/mrsk app:console`, which will start a Rails console session on the primary host. Be mindful that this is a live wire! Any changes made to the production database will take effect immeditately.
If you need to interact with the production console for the app, you can use `mrsk app console`, which will start a Rails console session on the primary host. You can start the console on a different host using `mrsk app console --host 192.168.0.2`. Be mindful that this is a live wire! Any changes made to the production database will take effect immeditately.
### Inspecting
You can see the state of your servers by running `./bin/mrsk info`. It'll show something like this:
You can see the state of your servers by running `mrsk details`. It'll show something like this:
```
Traefik Host: xxx.xxx.xxx.xxx
@@ -229,11 +221,11 @@ CONTAINER ID IMAGE
1d3c91ed1f55 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 13 minutes ago Up 13 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123
```
You can also see just info for app containers with `./bin/mrsk app:info` or just for Traefik with `./bin/mrsk traefik:info`.
You can also see just info for app containers with `mrsk app details` or just for Traefik with `mrsk traefik details`.
### Rollback
If you've discovered a bad deploy, you can quickly rollback by reactivating the old, paused container image. You can see what old containers are available for rollback by running `./bin/mrsk app:containers`. It'll give you a presentation similar to `./bin/mrsk app:info`, but include all the old containers as well. Showing something like this:
If you've discovered a bad deploy, you can quickly rollback by reactivating the old, paused container image. You can see what old containers are available for rollback by running `mrsk app containers`. It'll give you a presentation similar to `mrsk app details`, but include all the old containers as well. Showing something like this:
```
App Host: 164.92.105.119
@@ -247,20 +239,17 @@ badb1aa51db4 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483
6f170d1172ae registry.digitalocean.com/user/app:e5d9d7c2b898289dfbc5f7f1334140d984eedae4 "/rails/bin/docker-e…" 31 minutes ago Exited (1) 27 minutes ago chat-e5d9d7c2b898289dfbc5f7f1334140d984eedae4
```
From the example above, we can see that `e5d9d7c2b898289dfbc5f7f1334140d984eedae4` was the last version, so it's available as a rollback target. We can perform this rollback by running `./bin/mrsk rollback VERSION=e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. That'll stop `6ef8a6a84c525b123c5245345a8483f86d05a123` and then start `e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. Because the old container is still available, this is very quick. Nothing to download from the registry.
From the example above, we can see that `e5d9d7c2b898289dfbc5f7f1334140d984eedae4` was the last version, so it's available as a rollback target. We can perform this rollback by running `mrsk rollback e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. That'll stop `6ef8a6a84c525b123c5245345a8483f86d05a123` and then start `e5d9d7c2b898289dfbc5f7f1334140d984eedae4`. Because the old container is still available, this is very quick. Nothing to download from the registry.
Note that by default old containers are pruned after 3 days when you run `./bin/mrsk deploy`.
Note that by default old containers are pruned after 3 days when you run `mrsk deploy`.
### Removing
If you wish to remove the entire application, including Traefik, containers, images, and registry session, you can run `./bin/mrsk remove`. This will leave the servers clean.
If you wish to remove the entire application, including Traefik, containers, images, and registry session, you can run `mrsk remove`. This will leave the servers clean.
## Stage of development
This is alpha software. Lots of stuff is missing. Here are some of the areas we seek to improve:
- Adapterize commands to work with Podman and other container runners
- Integrate with cloud CI pipelines
This is alpha software. Lots of stuff is missing. Lots of stuff will keep moving around for a while.
## License

5
bin/mrsk Executable file
View File

@@ -0,0 +1,5 @@
#!/usr/bin/env ruby
require "mrsk/cli"
Mrsk::Cli::Main.start(ARGV)

View File

@@ -2,5 +2,4 @@ module Mrsk
end
require "mrsk/version"
require "mrsk/engine"
require "mrsk/commander"

9
lib/mrsk/cli.rb Normal file
View File

@@ -0,0 +1,9 @@
require "mrsk"
MRSK = Mrsk::Commander.new \
config_file: Pathname.new(File.expand_path("config/deploy.yml"))
module Mrsk::Cli
end
require "mrsk/cli/main"

97
lib/mrsk/cli/app.rb Normal file
View File

@@ -0,0 +1,97 @@
require "mrsk/cli/base"
class Mrsk::Cli::App < Mrsk::Cli::Base
desc "boot", "Boot app on servers (or start them if they've already been booted)"
def boot
MRSK.config.roles.each do |role|
on(role.hosts) do |host|
begin
execute *MRSK.app.run(role: role.name)
rescue SSHKit::Command::Failed => e
if e.message =~ /already in use/
error "Container with same version already deployed on #{host}, starting that instead"
execute *MRSK.app.start, host: host
else
raise
end
end
end
end
end
desc "start", "Start existing app on servers (use --version=<git-hash> to designate specific version)"
option :version, desc: "Defaults to the most recent git-hash in local repository"
def start
if (version = options[:version]).present?
on(MRSK.config.hosts) { execute *MRSK.app.start(version: version) }
else
on(MRSK.config.hosts) { execute *MRSK.app.start, raise_on_non_zero_exit: false }
end
end
desc "stop", "Stop app on servers"
def stop
on(MRSK.config.hosts) { execute *MRSK.app.stop, raise_on_non_zero_exit: false }
end
desc "details", "Display details about app containers"
def details
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.info, verbosity: Logger::INFO) + "\n\n" }
end
desc "exec [CMD]", "Execute a custom task on servers passed in as CMD='bin/rake some:task'"
option :once, type: :boolean, default: false
def exec(cmd)
if options[:once]
on(MRSK.config.primary_host) { puts capture(*MRSK.app.exec(cmd), verbosity: Logger::INFO) }
else
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.exec(cmd), verbosity: Logger::INFO) + "\n\n" }
end
end
desc "console", "Start Rails Console on primary host"
option :host, desc: "Start console on a different host"
def console
host = options[:host] || MRSK.config.primary_host
run_locally do
puts "Launching Rails console on #{host}..."
exec MRSK.app.console(host: host)
end
end
desc "runner [EXPRESSION]", "Execute Rails runner with given expression"
option :once, type: :boolean, default: false, desc:
def runner(expression)
if options[:once]
on(MRSK.config.primary_host) { puts capture(*MRSK.app.exec("bin/rails", "runner", "'#{expression}'"), verbosity: Logger::INFO) }
else
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.exec("bin/rails", "runner", "'#{expression}'"), verbosity: Logger::INFO) + "\n\n" }
end
end
desc "containers", "List all the app containers currently on servers"
def containers
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.list_containers, verbosity: Logger::INFO) + "\n\n" }
end
desc "logs", "Show last 100 log lines from app on servers"
def logs
# FIXME: Catch when app containers aren't running
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.logs) + "\n\n" }
end
desc "remove", "Remove app containers and images from servers"
option :only, default: "", desc: "Use 'containers' or 'images'"
def remove
case options[:only]
when "containers"
on(MRSK.config.hosts) { execute *MRSK.app.remove_containers }
when "images"
on(MRSK.config.hosts) { execute *MRSK.app.remove_images }
else
on(MRSK.config.hosts) { execute *MRSK.app.remove_containers }
on(MRSK.config.hosts) { execute *MRSK.app.remove_images }
end
end
end

27
lib/mrsk/cli/base.rb Normal file
View File

@@ -0,0 +1,27 @@
require "thor"
require "sshkit"
require "sshkit/dsl"
module Mrsk::Cli
class Base < Thor
include SSHKit::DSL
def self.exit_on_failure?() true end
class_option :verbose, type: :boolean, aliases: "-v", desc: "Detailed logging"
def initialize(*)
super
MRSK.verbose = options[:verbose]
end
private
def print_runtime
started_at = Time.now
yield
ensure
runtime = Time.now - started_at
puts " Finished all in #{sprintf("%.1f seconds", runtime)}"
end
end
end

53
lib/mrsk/cli/build.rb Normal file
View File

@@ -0,0 +1,53 @@
require "mrsk/cli/base"
class Mrsk::Cli::Build < Mrsk::Cli::Base
desc "deliver", "Deliver a newly built app image to servers"
def deliver
invoke :push
invoke :pull
end
desc "push", "Build locally and push app image to registry"
def push
run_locally do
begin
debug "Using builder: #{MRSK.builder.name}"
info "Building image may take a while (run with --verbose for progress logging)"
execute *MRSK.builder.push
rescue SSHKit::Command::Failed => e
error "Missing compatible builder, so creating a new one first"
execute *MRSK.builder.create
execute *MRSK.builder.push
end
end
end
desc "pull", "Pull app image from the registry onto servers"
def pull
on(MRSK.config.hosts) { execute *MRSK.builder.pull }
end
desc "create", "Create a local build setup"
def create
run_locally do
debug "Using builder: #{MRSK.builder.name}"
execute *MRSK.builder.create
end
end
desc "remove", "Remove local build setup"
def remove
run_locally do
debug "Using builder: #{MRSK.builder.name}"
execute *MRSK.builder.remove
end
end
desc "details", "Show the name of the configured builder"
def details
run_locally do
puts "Builder: #{MRSK.builder.name} (#{MRSK.builder.target.class.name})"
puts capture(*MRSK.builder.info)
end
end
end

99
lib/mrsk/cli/main.rb Normal file
View File

@@ -0,0 +1,99 @@
require "mrsk/cli/base"
require "mrsk/cli/app"
require "mrsk/cli/build"
require "mrsk/cli/prune"
require "mrsk/cli/registry"
require "mrsk/cli/server"
require "mrsk/cli/traefik"
class Mrsk::Cli::Main < Mrsk::Cli::Base
desc "deploy", "Deploy the app to servers"
def deploy
print_runtime do
invoke "mrsk:cli:server:bootstrap"
invoke "mrsk:cli:registry:login"
invoke "mrsk:cli:build:deliver"
invoke "mrsk:cli:traefik:boot"
invoke "mrsk:cli:app:stop"
invoke "mrsk:cli:app:boot"
invoke "mrsk:cli:prune:all"
end
end
desc "redeploy", "Deploy new version of the app to servers (without bootstrapping servers, starting Traefik, pruning, and registry login)"
def redeploy
print_runtime do
invoke "mrsk:cli:build:deliver"
invoke "mrsk:cli:app:stop"
invoke "mrsk:cli:app:boot"
end
end
desc "rollback [VERSION]", "Rollback the app to VERSION (that must already be on servers)"
def rollback(version)
on(MRSK.config.hosts) do
execute *MRSK.app.stop, raise_on_non_zero_exit: false
execute *MRSK.app.start(version: version)
end
end
desc "details", "Display details about Traefik and app containers"
def details
invoke "mrsk:cli:traefik:details"
invoke "mrsk:cli:app:details"
end
desc "install", "Create config stub in config/deploy.yml and binstub in bin/mrsk"
option :skip_binstub, type: :boolean, default: false, desc: "Skip adding MRSK to the Gemfile and creating bin/mrsk binstub"
def install
require "fileutils"
if (deploy_file = Pathname.new(File.expand_path("config/deploy.yml"))).exist?
puts "Config file already exists in config/deploy.yml (remove first to create a new one)"
else
FileUtils.cp_r Pathname.new(File.expand_path("templates/deploy.yml", __dir__)), deploy_file
puts "Created configuration file in config/deploy.yml"
end
unless options[:skip_binstub]
if (binstub = Pathname.new(File.expand_path("bin/mrsk"))).exist?
puts "Binstub already exists in bin/mrsk (remove first to create a new one)"
else
`bundle add mrsk`
`bundle binstubs mrsk`
puts "Created binstub file in bin/mrsk"
end
end
end
desc "remove", "Remove Traefik, app, and registry session from servers"
def remove
invoke "mrsk:cli:traefik:remove"
invoke "mrsk:cli:app:remove"
invoke "mrsk:cli:registry:logout"
end
desc "version", "Display the MRSK version"
def version
puts Mrsk::VERSION
end
desc "app", "Manage the application"
subcommand "app", Mrsk::Cli::App
desc "build", "Build the application image"
subcommand "build", Mrsk::Cli::Build
desc "prune", "Prune old application images and containers"
subcommand "prune", Mrsk::Cli::Prune
desc "registry", "Login and out of the image registry"
subcommand "registry", Mrsk::Cli::Registry
desc "server", "Bootstrap servers with Docker"
subcommand "server", Mrsk::Cli::Server
desc "traefik", "Manage the Traefik load balancer"
subcommand "traefik", Mrsk::Cli::Traefik
end

19
lib/mrsk/cli/prune.rb Normal file
View File

@@ -0,0 +1,19 @@
require "mrsk/cli/base"
class Mrsk::Cli::Prune < Mrsk::Cli::Base
desc "all", "Prune unused images and stopped containers"
def all
invoke :containers
invoke :images
end
desc "images", "Prune unused images older than 30 days"
def images
on(MRSK.config.hosts) { execute *MRSK.prune.images }
end
desc "containers", "Prune stopped containers for the service older than 3 days"
def containers
on(MRSK.config.hosts) { execute *MRSK.prune.containers }
end
end

14
lib/mrsk/cli/registry.rb Normal file
View File

@@ -0,0 +1,14 @@
require "mrsk/cli/base"
class Mrsk::Cli::Registry < Mrsk::Cli::Base
desc "login", "Login to the registry locally and remotely"
def login
run_locally { execute *MRSK.registry.login }
on(MRSK.config.hosts) { execute *MRSK.registry.login }
end
desc "logout", "Logout of the registry remotely"
def logout
on(MRSK.config.hosts) { execute *MRSK.registry.logout }
end
end

8
lib/mrsk/cli/server.rb Normal file
View File

@@ -0,0 +1,8 @@
require "mrsk/cli/base"
class Mrsk::Cli::Server < Mrsk::Cli::Base
desc "bootstrap", "Ensure Docker is installed on the servers"
def bootstrap
on(MRSK.config.hosts) { execute "which docker || apt-get install docker.io -y" }
end
end

View File

@@ -0,0 +1,17 @@
# Name of your application. Used to uniquely configuring Traefik and app containers.
# Your Dockerfile should set LABEL service=the-same-value to ensure image pruning works.
service: my-app
# Name of the container image.
image: user/my-app
# Deploy to these servers.
servers:
- 192.168.0.1
# Credentials for your image host.
registry:
# Specify the registry server, if you're not using Docker Hub
# server: registry.digitalocean.com / ghcr.io / ...
username: my-user
password: my-password-should-go-somewhere-safe

44
lib/mrsk/cli/traefik.rb Normal file
View File

@@ -0,0 +1,44 @@
require "mrsk/cli/base"
class Mrsk::Cli::Traefik < Mrsk::Cli::Base
desc "boot", "Boot Traefik on servers"
def boot
on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.run, raise_on_non_zero_exit: false }
end
desc "start", "Start existing Traefik on servers"
def start
on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.start, raise_on_non_zero_exit: false }
end
desc "stop", "Stop Traefik on servers"
def stop
on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.stop, raise_on_non_zero_exit: false }
end
desc "restart", "Restart Traefik on servers"
def restart
invoke :stop
invoke :start
end
desc "details", "Display details about Traefik containers from servers"
def details
on(MRSK.config.role(:web).hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*MRSK.traefik.info, verbosity: Logger::INFO) + "\n\n" }
end
desc "logs", "Show last 100 log lines from Traefik on servers"
def logs
on(MRSK.config.hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*MRSK.traefik.logs) + "\n\n" }
end
desc "remove", "Remove Traefik container and image from servers"
def remove
invoke :stop
on(MRSK.config.role(:web).hosts) do
execute *MRSK.traefik.remove_container
execute *MRSK.traefik.remove_image
end
end
end

View File

@@ -6,14 +6,15 @@ require "mrsk/commands/traefik"
require "mrsk/commands/registry"
class Mrsk::Commander
attr_reader :config_file, :config, :verbose
attr_reader :config
attr_accessor :verbose
def initialize(config_file:, verbose: false)
@config_file, @verbose = config_file, verbose
def initialize(config_file:)
@config_file = config_file
end
def config
@config ||= Mrsk::Configuration.load_file(config_file).tap { |config| setup_with(config) }
@config ||= Mrsk::Configuration.load_file(@config_file).tap { |config| setup_with(config) }
end

View File

@@ -15,8 +15,8 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
role.cmd
end
def start
docker :start, config.service_with_version
def start(version: config.version)
docker :start, "#{config.service}-#{version}"
end
def stop
@@ -40,8 +40,8 @@ class Mrsk::Commands::App < Mrsk::Commands::Base
*command
end
def console
"ssh -t #{config.ssh_user}@#{config.primary_host} '#{exec("bin/rails", "c", interactive: true).join(" ")}'"
def console(host: config.primary_host)
"ssh -t #{config.ssh_user}@#{host} '#{exec("bin/rails", "c", interactive: true).join(" ")}'"
end
def list_containers

View File

@@ -2,11 +2,11 @@ require "mrsk/commands/base"
class Mrsk::Commands::Builder::Multiarch < Mrsk::Commands::Base
def create
docker :buildx, :create, "--use", "--name", "mrsk"
docker :buildx, :create, "--use", "--name", builder_name
end
def remove
docker :buildx, :rm, "mrsk"
docker :buildx, :rm, builder_name
end
def push
@@ -22,4 +22,9 @@ class Mrsk::Commands::Builder::Multiarch < Mrsk::Commands::Base
docker(:context, :ls),
docker(:buildx, :ls)
end
private
def builder_name
"mrsk-#{config.service}"
end
end

View File

@@ -16,11 +16,11 @@ class Mrsk::Commands::Builder::Multiarch::Remote < Mrsk::Commands::Builder::Mult
private
def create_local_buildx
docker :buildx, :create, "--use", "--name", "mrsk", "mrsk-#{local["arch"]}", "--platform", "linux/#{local["arch"]}"
docker :buildx, :create, "--use", "--name", builder_name, builder_name_with_arch(local["arch"]), "--platform", "linux/#{local["arch"]}"
end
def append_remote_buildx
docker :buildx, :create, "--append", "--name", "mrsk", "mrsk-#{remote["arch"]}", "--platform", "linux/#{remote["arch"]}"
docker :buildx, :create, "--append", "--name", builder_name, builder_name_with_arch(remote["arch"]), "--platform", "linux/#{remote["arch"]}"
end
def create_contexts
@@ -30,7 +30,7 @@ class Mrsk::Commands::Builder::Multiarch::Remote < Mrsk::Commands::Builder::Mult
end
def create_context(arch, host)
docker :context, :create, "mrsk-#{arch}", "--description", "'MRSK #{arch} Native Host'", "--docker", "'host=#{host}'"
docker :context, :create, builder_name_with_arch(arch), "--description", "'#{builder_name} #{arch} native host'", "--docker", "'host=#{host}'"
end
def remove_contexts
@@ -40,7 +40,7 @@ class Mrsk::Commands::Builder::Multiarch::Remote < Mrsk::Commands::Builder::Mult
end
def remove_context(arch)
docker :context, :rm, "mrsk-#{arch}"
docker :context, :rm, builder_name_with_arch(arch)
end
def local
@@ -50,4 +50,9 @@ class Mrsk::Commands::Builder::Multiarch::Remote < Mrsk::Commands::Builder::Mult
def remote
config.builder["remote"]
end
private
def builder_name_with_arch(arch)
"#{builder_name}-#{arch}"
end
end

View File

@@ -1,5 +1,7 @@
require "active_support/ordered_options"
require "active_support/core_ext/string/inquiry"
require "active_support/core_ext/module/delegation"
require "pathname"
require "erb"
class Mrsk::Configuration
@@ -91,7 +93,7 @@ class Mrsk::Configuration
end
def master_key
ENV["RAILS_MASTER_KEY"] || File.read(Rails.root.join("config/master.key"))
ENV["RAILS_MASTER_KEY"] || File.read(Pathname.new(File.expand_path("config/master.key")))
end

View File

@@ -1,4 +0,0 @@
module Mrsk
class Engine < ::Rails::Engine
end
end

View File

@@ -1,3 +1,3 @@
module Mrsk
VERSION = "0.0.3"
VERSION = "0.1.0"
end

View File

@@ -1,97 +0,0 @@
require_relative "setup"
namespace :mrsk do
namespace :app do
desc "Run app on servers (or start them if they've already been run)"
task :run do
MRSK.config.roles.each do |role|
on(role.hosts) do |host|
begin
execute *MRSK.app.run(role: role.name)
rescue SSHKit::Command::Failed => e
if e.message =~ /already in use/
error "Container with same version already deployed on #{host}, starting that instead"
execute *MRSK.app.start, host: host
else
raise
end
end
end
end
end
desc "Start existing app on servers (use VERSION=<git-hash> to designate which version)"
task :start do
on(MRSK.config.hosts) { execute *MRSK.app.start, raise_on_non_zero_exit: false }
end
desc "Stop app on servers"
task :stop do
on(MRSK.config.hosts) { execute *MRSK.app.stop, raise_on_non_zero_exit: false }
end
desc "Start app on servers (use VERSION=<git-hash> to designate which version)"
task restart: %i[ stop start ]
desc "Display information about app containers"
task :info do
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.info) + "\n\n" }
end
desc "Execute a custom task on servers passed in as CMD='bin/rake some:task'"
task :exec do
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.exec(ENV["CMD"])) + "\n\n" }
end
desc "Start Rails Console on primary host"
task :console do
puts "Launching Rails console on #{MRSK.config.primary_host}..."
exec app.console
end
namespace :exec do
desc "Execute Rails command on servers, like CMD='runner \"puts %(Hello World)\""
task :rails do
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.exec("bin/rails", ENV["CMD"])) + "\n\n" }
end
desc "Execute a custom task on the first defined server"
task :once do
on(MRSK.config.primary_host) { puts capture(*MRSK.app.exec(ENV["CMD"])) }
end
namespace :once do
desc "Execute Rails command on the first defined server, like CMD='runner \"puts %(Hello World)\""
task :rails do
on(MRSK.config.primary_host) { puts capture(*MRSK.app.exec("bin/rails", ENV["CMD"])) }
end
end
end
desc "List all the app containers currently on servers"
task :containers do
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.list_containers) + "\n\n" }
end
desc "Show last 100 log lines from app on servers"
task :logs do
# FIXME: Catch when app containers aren't running
on(MRSK.config.hosts) { |host| puts "App Host: #{host}\n" + capture(*MRSK.app.logs) + "\n\n" }
end
desc "Remove app containers and images from servers"
task remove: %w[ remove:containers remove:images ]
namespace :remove do
desc "Remove app containers from servers"
task :containers do
on(MRSK.config.hosts) { execute *MRSK.app.remove_containers }
end
desc "Remove app images from servers"
task :images do
on(MRSK.config.hosts) { execute *MRSK.app.remove_images }
end
end
end
end

View File

@@ -1,52 +0,0 @@
require_relative "setup"
namespace :mrsk do
namespace :build do
desc "Deliver a newly built app image to servers"
task deliver: %i[ push pull ]
desc "Build locally and push app image to registry"
task :push do
run_locally do
begin
debug "Using builder: #{MRSK.builder.name}"
info "Building image may take a while (run with VERBOSE=1 for progress logging)"
execute *MRSK.builder.push
rescue SSHKit::Command::Failed => e
error "Missing compatible builder, so creating a new one first"
execute *MRSK.builder.create
execute *MRSK.builder.push
end
end unless ENV["VERSION"]
end
desc "Pull app image from the registry onto servers"
task :pull do
on(MRSK.config.hosts) { execute *MRSK.builder.pull }
end
desc "Create a local build setup"
task :create do
run_locally do
debug "Using builder: #{MRSK.builder.name}"
execute *MRSK.builder.create
end
end
desc "Remove local build setup"
task :remove do
run_locally do
debug "Using builder: #{MRSK.builder.name}"
execute *MRSK.builder.remove
end
end
desc "Show the name of the configured builder"
task :info do
run_locally do
puts "Builder: #{MRSK.builder.name} (#{MRSK.builder.target.class.name})"
puts capture(*MRSK.builder.info)
end
end
end
end

View File

@@ -1,37 +0,0 @@
require_relative "setup"
namespace :mrsk do
desc "Deploy app for the first time to a fresh server"
task fresh: %w[ server:bootstrap registry:login build:deliver traefik:run app:stop app:run ]
desc "Push the latest version of the app, ensure Traefik is running, then restart app"
task deploy: %w[ registry:login build:deliver traefik:run app:stop app:run prune ]
desc "Rollback to VERSION=x that was already run as a container on servers"
task rollback: %w[ app:restart ]
desc "Display information about Traefik and app containers"
task info: %w[ traefik:info app:info ]
desc "Create config stub in config/deploy.yml"
task :init do
require "fileutils"
if (deploy_file = Rails.root.join("config/deploy.yml")).exist?
puts "Config file already exists in config/deploy.yml (remove first to create a new one)"
else
FileUtils.cp_r Pathname.new(File.expand_path("templates/deploy.yml", __dir__)), deploy_file
puts "Created configuration file in config/deploy.yml"
end
if (binstub = Rails.root.join("bin/mrsk")).exist?
puts "Binstub already exists in bin/mrsk (remove first to create a new one)"
else
FileUtils.cp_r Pathname.new(File.expand_path("templates/mrsk", __dir__)), binstub
puts "Created binstub file in bin/mrsk"
end
end
desc "Remove Traefik, app, and registry session from servers"
task remove: %w[ traefik:remove app:remove registry:logout ]
end

View File

@@ -1,18 +0,0 @@
require_relative "setup"
namespace :mrsk do
desc "Prune unused images and stopped containers"
task prune: %w[ prune:containers prune:images ]
namespace :prune do
desc "Prune unused images older than 30 days"
task :images do
on(MRSK.config.hosts) { MRSK.verbosity(:debug) { execute *MRSK.prune.images } }
end
desc "Prune stopped containers for the service older than 3 days"
task :containers do
on(MRSK.config.hosts) { MRSK.verbosity(:debug) { execute *MRSK.prune.containers } }
end
end
end

View File

@@ -1,16 +0,0 @@
require_relative "setup"
namespace :mrsk do
namespace :registry do
desc "Login to the registry locally and remotely"
task :login do
run_locally { execute *MRSK.registry.login }
on(MRSK.config.hosts) { execute *MRSK.registry.login }
end
desc "Logout of the registry remotely"
task :logout do
on(MRSK.config.hosts) { execute *MRSK.registry.logout }
end
end
end

View File

@@ -1,11 +0,0 @@
require_relative "setup"
namespace :mrsk do
namespace :server do
desc "Setup Docker on the remote servers"
task :bootstrap do
# FIXME: Detect when apt-get is not available and use the appropriate alternative
on(MRSK.config.hosts) { execute "apt-get install docker.io -y" }
end
end
end

View File

@@ -1,6 +0,0 @@
require "sshkit"
require "sshkit/dsl"
include SSHKit::DSL
MRSK = Mrsk::Commander.new config_file: Rails.root.join("config/deploy.yml"), verbose: ENV["VERBOSE"]

View File

@@ -1,24 +0,0 @@
# Name of your application will be used for uniquely configuring Traefik and app containers.
# Your Dockerfile should set LABEL service=the-same-value to ensure image pruning works.
service: my-app
# Name of the container image
image: user/my-app
# All the servers targeted for deploy. You can reference a single server for a command by using SERVERS=192.168.0.1
servers:
- 192.168.0.1
# The following envs are made available to the container when started
env:
# Remember never to put passwords or tokens directly into this file, use encrypted credentials
# REDIS_URL: redis://x/y
# Where your images will be hosted
registry:
# Specify the registry server, if you're not using Docker Hub
# server: registry.digitalocean.com / ghcr.io / ...
# Set credentials with bin/rails credentials:edit
username: my-user
password: my-password-should-go-in-credentials

View File

@@ -1,8 +0,0 @@
#!/bin/bash
if [ "${*}" == "" ]; then
# Improve so list matches
exec bin/rake -T mrsk
else
exec bin/rake "mrsk:$@"
fi

View File

@@ -1,41 +0,0 @@
require_relative "setup"
namespace :mrsk do
namespace :traefik do
desc "Run Traefik on servers"
task :run do
on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.run, raise_on_non_zero_exit: false }
end
desc "Start existing Traefik on servers"
task :start do
on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.start, raise_on_non_zero_exit: false }
end
desc "Stop Traefik on servers"
task :stop do
on(MRSK.config.role(:web).hosts) { execute *MRSK.traefik.stop, raise_on_non_zero_exit: false }
end
desc "Restart Traefik on servers"
task restart: %i[ stop start ]
desc "Display information about Traefik containers from servers"
task :info do
on(MRSK.config.role(:web).hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*MRSK.traefik.info) + "\n\n" }
end
desc "Show last 100 log lines from Traefik on servers"
task :logs do
on(MRSK.config.hosts) { |host| puts "Traefik Host: #{host}\n" + capture(*MRSK.traefik.logs) + "\n\n" }
end
desc "Remove Traefik container and image from servers"
task remove: %i[ stop ] do
on(MRSK.config.role(:web).hosts) do
execute *MRSK.traefik.remove_container
execute *MRSK.traefik.remove_image
end
end
end
end

View File

@@ -6,11 +6,13 @@ Gem::Specification.new do |spec|
spec.authors = [ "David Heinemeier Hansson" ]
spec.email = "dhh@hey.com"
spec.homepage = "https://github.com/rails/mrsk"
spec.summary = "Deploy Docker containers with zero downtime to any host."
spec.summary = "Deploy Rails apps in containers to servers running Docker with zero downtime."
spec.license = "MIT"
spec.files = Dir["lib/**/*", "MIT-LICENSE", "README.md"]
spec.executables = %w[ mrsk ]
spec.add_dependency "railties", ">= 7.0.0"
spec.add_dependency "activesupport", ">= 7.0"
spec.add_dependency "sshkit", "~> 1.21"
spec.add_dependency "thor", "~> 1.2"
end