Compare commits
15 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
50c4bb83cb | ||
|
|
b2875ad056 | ||
|
|
8ec94f105c | ||
|
|
90f4212a68 | ||
|
|
648894f9a9 | ||
|
|
dc68639dfa | ||
|
|
244cf8b3b7 | ||
|
|
f25f506d77 | ||
|
|
c29a177a7a | ||
|
|
ec5fad5bea | ||
|
|
c671acf68f | ||
|
|
4f2cb5e184 | ||
|
|
63a065237a | ||
|
|
0f4e1888d9 | ||
|
|
d4d3308c34 |
@@ -1,7 +1,7 @@
|
||||
PATH
|
||||
remote: .
|
||||
specs:
|
||||
mrsk (0.6.2)
|
||||
mrsk (0.6.4)
|
||||
activesupport (>= 7.0)
|
||||
dotenv (~> 2.8)
|
||||
sshkit (~> 1.21)
|
||||
|
||||
25
README.md
25
README.md
@@ -1,6 +1,6 @@
|
||||
# MRSK
|
||||
|
||||
MRSK deploys web apps in containers to servers running Docker with zero downtime. It uses the dynamic reverse-proxy Traefik to hold requests while the new application container is started and the old one is stopped. It works seamlessly across multiple hosts, using SSHKit to execute commands.
|
||||
MRSK deploys web apps in containers to servers running Docker with zero downtime. It uses the dynamic reverse-proxy Traefik to hold requests while the new application container is started and the old one is stopped. It works seamlessly across multiple hosts, using SSHKit to execute commands. It was built for Rails applications, but works with any type of web app that can be bundled with Docker.
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -43,6 +43,16 @@ This will:
|
||||
|
||||
Voila! All the servers are now serving the app on port 80. If you're just running a single server, you're ready to go. If you're running multiple servers, you need to put a load balancer in front of them.
|
||||
|
||||
## Vision
|
||||
|
||||
In the past decade+, there's been an explosion in commercial offerings that make deploying web apps easier. Heroku kicked it off with an incredible offering that stayed ahead of the competition seemingly forever. These days we have excellent alternatives like Fly.io and Render. And hosted Kubernetes is making things easier too on AWS, GCP, Digital Ocean, and elsewhere. But these are all offerings that have you renting computers in the cloud at a premium. If you want to run on our own hardware, or even just have a clear migration path to do so, you need to carefully consider how locked in you get to these commercial platforms. Preferably before the bills swallow your business whole!
|
||||
|
||||
MRSK seeks to bring the advance in ergonomics pioneered by these commercial offerings to deploying web apps anywhere. Whether that's low-cost cloud options without the managed-service markup from the likes of Digital Ocean, Hetzner, OVH, etc, or it's your own colocated metal. To MRSK, it's all the same. Feed the config file a list of IP addresses with vanilla Ubuntu servers that have seen no prep beyond an added SSH key, and you'll be running in literally minutes.
|
||||
|
||||
This structure also gives you enormous portability. You can have your web app deployed on several clouds at ease like this. Or you can buy the baseline with your own hardware, then deploy to a cloud before a big seasonal spike to get more capacity. When you're not locked into a single provider from a tooling perspective, there's a lot of compelling options available.
|
||||
|
||||
Ultimately, MRSK is meant to compress the complexity of going to production using open source tooling that isn't tied to any commercial offering. Not to zero, though. You're probably still better off with a fully managed service if basic Linux or Docker is still difficult, but from an early stage when those concepts are familiar.
|
||||
|
||||
## Why not just run Capistrano, Kubernetes or Docker Swarm?
|
||||
|
||||
MRSK basically is Capistrano for Containers, which allow us to use vanilla servers as the hosts. No need to ensure that the servers have just the right version of Ruby or other dependencies you need. That all lives in the Docker image now. You can boot a brand new Ubuntu (or whatever) server, add it to the deploy servers of MRSK, and it'll be auto-provisioned with Docker, and run right away. Docker's layer caching also allows for quicker deployments with less mucking about on the server. And the images built for MRSK can be used for CI or later introspection.
|
||||
@@ -253,11 +263,11 @@ This build secret can then be referenced in the Dockerfile:
|
||||
# Copy Gemfiles
|
||||
COPY Gemfile Gemfile.lock ./
|
||||
|
||||
# Install dependencies, including private repositories via access token (then remove git configs with exposed GITHUB_TOKEN)
|
||||
# Install dependencies, including private repositories via access token (then remove bundle cache with exposed GITHUB_TOKEN)
|
||||
RUN --mount=type=secret,id=GITHUB_TOKEN \
|
||||
BUNDLE_GITHUB__COM=x-access-token:$(cat /run/secrets/GITHUB_TOKEN) \
|
||||
bundle install && \
|
||||
find /usr/local/bundle/cache/bundler/git -name "config" -delete
|
||||
rm -rf /usr/local/bundle/cache
|
||||
```
|
||||
|
||||
### Using command arguments for Traefik
|
||||
@@ -266,12 +276,13 @@ You can customize the traefik command line:
|
||||
|
||||
```yaml
|
||||
traefik:
|
||||
accesslog: true
|
||||
accesslog.format: json
|
||||
metrics.prometheus: true
|
||||
metrics.prometheus.buckets: 0.1,0.3,1.2,5.0
|
||||
args:
|
||||
accesslog: true
|
||||
accesslog.format: json
|
||||
```
|
||||
|
||||
This will start the traefik container with `--accesslog=true accesslog.format=json`.
|
||||
|
||||
### Configuring build args for new images
|
||||
|
||||
Build arguments that aren't secret can also be configured:
|
||||
|
||||
@@ -107,10 +107,14 @@ class Mrsk::Cli::Main < Mrsk::Cli::Base
|
||||
desc "envify", "Create .env by evaluating .env.erb (or .env.staging.erb -> .env.staging when using -d staging)"
|
||||
def envify
|
||||
if destination = options[:destination]
|
||||
File.write(".env.#{destination}", ERB.new(IO.read(Pathname.new(File.expand_path(".env.#{destination}.erb")))).result)
|
||||
env_template_path = ".env.#{destination}.erb"
|
||||
env_path = ".env.#{destination}"
|
||||
else
|
||||
File.write(".env", ERB.new(IO.read(Pathname.new(File.expand_path(".env.erb")))).result)
|
||||
env_template_path = ".env.erb"
|
||||
env_path = ".env"
|
||||
end
|
||||
|
||||
File.write(env_path, ERB.new(File.read(env_template_path)).result, perm: 0600)
|
||||
end
|
||||
|
||||
desc "remove", "Remove Traefik, app, and registry session from servers"
|
||||
|
||||
@@ -5,19 +5,27 @@ class Mrsk::Commands::Builder::Base < Mrsk::Commands::Base
|
||||
docker :pull, config.absolute_image
|
||||
end
|
||||
|
||||
def build_args
|
||||
argumentize "--build-arg", args, redacted: true
|
||||
end
|
||||
|
||||
def build_secrets
|
||||
argumentize "--secret", secrets.collect { |secret| [ "id", secret ] }
|
||||
end
|
||||
|
||||
def build_tags
|
||||
[ "-t", config.absolute_image, "-t", config.latest_image ]
|
||||
def build_options
|
||||
[ *build_tags, *build_labels, *build_args, *build_secrets ]
|
||||
end
|
||||
|
||||
private
|
||||
def build_tags
|
||||
[ "-t", config.absolute_image, "-t", config.latest_image ]
|
||||
end
|
||||
|
||||
def build_labels
|
||||
argumentize "--label", { service: config.service }
|
||||
end
|
||||
|
||||
def build_args
|
||||
argumentize "--build-arg", args, redacted: true
|
||||
end
|
||||
|
||||
def build_secrets
|
||||
argumentize "--secret", secrets.collect { |secret| [ "id", secret ] }
|
||||
end
|
||||
|
||||
def args
|
||||
(config.builder && config.builder["args"]) || {}
|
||||
end
|
||||
|
||||
@@ -12,9 +12,7 @@ class Mrsk::Commands::Builder::Multiarch < Mrsk::Commands::Builder::Base
|
||||
"--push",
|
||||
"--platform", "linux/amd64,linux/arm64",
|
||||
"--builder", builder_name,
|
||||
*build_tags,
|
||||
*build_args,
|
||||
*build_secrets,
|
||||
*build_options,
|
||||
"."
|
||||
end
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ class Mrsk::Commands::Builder::Native < Mrsk::Commands::Builder::Base
|
||||
|
||||
def push
|
||||
combine \
|
||||
docker(:build, *build_tags, *build_args, *build_secrets, "."),
|
||||
docker(:build, *build_options, "."),
|
||||
docker(:push, config.absolute_image)
|
||||
end
|
||||
|
||||
|
||||
@@ -16,9 +16,7 @@ class Mrsk::Commands::Builder::Native::Remote < Mrsk::Commands::Builder::Native
|
||||
"--push",
|
||||
"--platform", platform,
|
||||
"--builder", builder_name,
|
||||
*build_tags,
|
||||
*build_args,
|
||||
*build_secrets,
|
||||
*build_options,
|
||||
"."
|
||||
end
|
||||
|
||||
|
||||
@@ -6,11 +6,10 @@ class Mrsk::Commands::Prune < Mrsk::Commands::Base
|
||||
PRUNE_CONTAINERS_AFTER = 3.days.in_hours.to_i
|
||||
|
||||
def images
|
||||
docker :image, :prune, "-f", "--filter", "until=#{PRUNE_IMAGES_AFTER}h"
|
||||
docker :image, :prune, "--all", "--force", "--filter", "label=service=#{config.service}", "--filter", "until=#{PRUNE_IMAGES_AFTER}h"
|
||||
end
|
||||
|
||||
def containers
|
||||
docker :image, :prune, "-f", "--filter", "until=#{PRUNE_IMAGES_AFTER}h"
|
||||
docker :container, :prune, "-f", "--filter", "label=service=#{config.service}", "--filter", "'until=#{PRUNE_CONTAINERS_AFTER}h'"
|
||||
docker :container, :prune, "--force", "--filter", "label=service=#{config.service}", "--filter", "until=#{PRUNE_CONTAINERS_AFTER}h"
|
||||
end
|
||||
end
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
module Mrsk
|
||||
VERSION = "0.6.2"
|
||||
VERSION = "0.6.4"
|
||||
end
|
||||
|
||||
@@ -8,50 +8,68 @@ class CommandsBuilderTest < ActiveSupport::TestCase
|
||||
test "target multiarch by default" do
|
||||
builder = new_builder_command
|
||||
assert_equal "multiarch", builder.name
|
||||
assert_equal [:docker, :buildx, :build, "--push", "--platform", "linux/amd64,linux/arm64", "--builder", "mrsk-app-multiarch", "-t", "dhh/app:123", "-t", "dhh/app:latest", "."], builder.push
|
||||
assert_equal \
|
||||
"docker buildx build --push --platform linux/amd64,linux/arm64 --builder mrsk-app-multiarch -t dhh/app:123 -t dhh/app:latest --label service=app .",
|
||||
builder.push.join(" ")
|
||||
end
|
||||
|
||||
test "target native when multiarch is off" do
|
||||
builder = new_builder_command(builder: { "multiarch" => false })
|
||||
assert_equal "native", builder.name
|
||||
assert_equal [:docker, :build, "-t", "dhh/app:123", "-t", "dhh/app:latest", ".", "&&", :docker, :push, "dhh/app:123"], builder.push
|
||||
assert_equal \
|
||||
"docker build -t dhh/app:123 -t dhh/app:latest --label service=app . && docker push dhh/app:123",
|
||||
builder.push.join(" ")
|
||||
end
|
||||
|
||||
test "target multiarch remote when local and remote is set" do
|
||||
builder = new_builder_command(builder: { "local" => { }, "remote" => { } })
|
||||
assert_equal "multiarch/remote", builder.name
|
||||
assert_equal [:docker, :buildx, :build, "--push", "--platform", "linux/amd64,linux/arm64", "--builder", "mrsk-app-multiarch-remote", "-t", "dhh/app:123", "-t", "dhh/app:latest", "."], builder.push
|
||||
assert_equal \
|
||||
"docker buildx build --push --platform linux/amd64,linux/arm64 --builder mrsk-app-multiarch-remote -t dhh/app:123 -t dhh/app:latest --label service=app .",
|
||||
builder.push.join(" ")
|
||||
end
|
||||
|
||||
test "target native remote when only remote is set" do
|
||||
builder = new_builder_command(builder: { "remote" => { "arch" => "amd64" } })
|
||||
assert_equal "native/remote", builder.name
|
||||
assert_equal [:docker, :buildx, :build, "--push", "--platform", "linux/amd64", "--builder", "mrsk-app-native-remote", "-t", "dhh/app:123", "-t", "dhh/app:latest", "."], builder.push
|
||||
assert_equal \
|
||||
"docker buildx build --push --platform linux/amd64 --builder mrsk-app-native-remote -t dhh/app:123 -t dhh/app:latest --label service=app .",
|
||||
builder.push.join(" ")
|
||||
end
|
||||
|
||||
test "build args" do
|
||||
builder = new_builder_command(builder: { "args" => { "a" => 1, "b" => 2 } })
|
||||
assert_equal [ "--build-arg", "a=1", "--build-arg", "b=2" ], builder.target.build_args
|
||||
assert_equal \
|
||||
"-t dhh/app:123 -t dhh/app:latest --label service=app --build-arg a=1 --build-arg b=2",
|
||||
builder.target.build_options.join(" ")
|
||||
end
|
||||
|
||||
test "build secrets" do
|
||||
builder = new_builder_command(builder: { "secrets" => ["token_a", "token_b"] })
|
||||
assert_equal [ "--secret", "id=token_a", "--secret", "id=token_b" ], builder.target.build_secrets
|
||||
assert_equal \
|
||||
"-t dhh/app:123 -t dhh/app:latest --label service=app --secret id=token_a --secret id=token_b",
|
||||
builder.target.build_options.join(" ")
|
||||
end
|
||||
|
||||
test "native push with build args" do
|
||||
builder = new_builder_command(builder: { "multiarch" => false, "args" => { "a" => 1, "b" => 2 } })
|
||||
assert_equal [ :docker, :build, "-t", "dhh/app:123", "-t", "dhh/app:latest", "--build-arg", "a=1", "--build-arg", "b=2", ".", "&&", :docker, :push, "dhh/app:123" ], builder.push
|
||||
assert_equal \
|
||||
"docker build -t dhh/app:123 -t dhh/app:latest --label service=app --build-arg a=1 --build-arg b=2 . && docker push dhh/app:123",
|
||||
builder.push.join(" ")
|
||||
end
|
||||
|
||||
test "multiarch push with build args" do
|
||||
builder = new_builder_command(builder: { "args" => { "a" => 1, "b" => 2 } })
|
||||
assert_equal [ :docker, :buildx, :build, "--push", "--platform", "linux/amd64,linux/arm64", "--builder", "mrsk-app-multiarch", "-t", "dhh/app:123", "-t", "dhh/app:latest", "--build-arg", "a=1", "--build-arg", "b=2", "." ], builder.push
|
||||
assert_equal \
|
||||
"docker buildx build --push --platform linux/amd64,linux/arm64 --builder mrsk-app-multiarch -t dhh/app:123 -t dhh/app:latest --label service=app --build-arg a=1 --build-arg b=2 .",
|
||||
builder.push.join(" ")
|
||||
end
|
||||
|
||||
test "native push with with build secrets" do
|
||||
builder = new_builder_command(builder: { "multiarch" => false, "secrets" => [ "a", "b" ] })
|
||||
assert_equal [ :docker, :build, "-t", "dhh/app:123", "-t", "dhh/app:latest", "--secret", "id=a", "--secret", "id=b", ".", "&&", :docker, :push, "dhh/app:123" ], builder.push
|
||||
assert_equal \
|
||||
"docker build -t dhh/app:123 -t dhh/app:latest --label service=app --secret id=a --secret id=b . && docker push dhh/app:123",
|
||||
builder.push.join(" ")
|
||||
end
|
||||
|
||||
private
|
||||
|
||||
27
test/commands/prune_test.rb
Normal file
27
test/commands/prune_test.rb
Normal file
@@ -0,0 +1,27 @@
|
||||
require "test_helper"
|
||||
|
||||
class CommandsPruneTest < ActiveSupport::TestCase
|
||||
setup do
|
||||
@config = {
|
||||
service: "app", image: "dhh/app", registry: { "username" => "dhh", "password" => "secret" }, servers: [ "1.1.1.1" ],
|
||||
traefik: { "args" => { "accesslog.format" => "json", "metrics.prometheus.buckets" => "0.1,0.3,1.2,5.0" } }
|
||||
}
|
||||
end
|
||||
|
||||
test "images" do
|
||||
assert_equal \
|
||||
"docker image prune --all --force --filter label=service=app --filter until=168h",
|
||||
new_command.images.join(" ")
|
||||
end
|
||||
|
||||
test "containers" do
|
||||
assert_equal \
|
||||
"docker container prune --force --filter label=service=app --filter until=72h",
|
||||
new_command.containers.join(" ")
|
||||
end
|
||||
|
||||
private
|
||||
def new_command
|
||||
Mrsk::Commands::Prune.new(Mrsk::Configuration.new(@config, version: "123"))
|
||||
end
|
||||
end
|
||||
Reference in New Issue
Block a user