diff --git a/README.md b/README.md index e3626f4f..eba1eb96 100644 --- a/README.md +++ b/README.md @@ -14,27 +14,27 @@ servers: - 192.168.0.2 registry: username: registry-user-name - password: <%= ENV["MRSK_REGISTRY_PASSWORD"] %> + password: <%= ENV.fetch("MRSK_REGISTRY_PASSWORD") %> ``` Now you're ready to deploy a multi-arch image to the servers: ``` -export MRSK_REGISTRY_PASSWORD=your-real-registry-pw -mrsk deploy +MRSK_REGISTRY_PASSWORD=pw mrsk deploy ``` This will: -1. Install Docker on any server that might be missing it (using apt-get) -2. Log into the registry both locally and remotely -3. Build the image using the standard Dockerfile in the root of the application. -4. Push the image to the registry. -5. Pull the image from the registry on the servers. -6. Ensure Traefik is running and accepting traffic on port 80. -7. Stop any containers running a previous versions of the app. -8. Start a new container with the version of the app that matches the current git version hash. -9. Prune unused images and stopped containers to ensure servers don't fill up. +1. Connect to the servers over SSH (using root by default, authenticated by your loaded ssh key) +2. Install Docker on any server that might be missing it (using apt-get) +3. Log into the registry both locally and remotely +4. Build the image using the standard Dockerfile in the root of the application. +5. Push the image to the registry. +6. Pull the image from the registry on the servers. +7. Ensure Traefik is running and accepting traffic on port 80. +8. Stop any containers running a previous versions of the app. +9. Start a new container with the version of the app that matches the current git version hash. +10. Prune unused images and stopped containers to ensure servers don't fill up. Voila! All the servers are now serving the app on port 80. If you're just running a single server, you're ready to go. If you're running multiple servers, you need to put a load balancer in front of them. @@ -48,13 +48,13 @@ Kubernetes is a beast. Running it yourself on your own hardware is not for the f ### Using another registry than Docker Hub -The default registry for Docker is Docker Hub. If you'd like to use a different one, just configure the server, like so: +The default registry is Docker Hub, but you can change it using `registry/server`: ```yaml registry: server: registry.digitalocean.com username: registry-user-name - password: <%= ENV["MRSK_REGISTRY_PASSWORD"] %> + password: <%= ENV.fetch("MRSK_REGISTRY_PASSWORD") %> ``` ### Using a different SSH user than root @@ -65,9 +65,9 @@ The default SSH user is root, but you can change it using `ssh_user`: ssh_user: app ``` -### Adding custom env variables +### Using env variables -You can inject custom env variables into the app containers using `env`: +You can inject env variables into the app containers using `env`: ```yaml env: @@ -75,9 +75,9 @@ env: REDIS_URL: redis://redis1:6379/1 ``` -### Adding secret custom env variables +### Using secret env variables -If you have custom env variables that are secret, you can divide the `env` block into `clear` and `secret`: +If you have env variables that are secret, you can divide the `env` block into `clear` and `secret`: ```yaml env: @@ -96,7 +96,7 @@ If the referenced secret ENVs are missing, the configuration will be halted with Note: Marking an ENV as secret currently only redacts its value in the output for MRSK. The ENV is still injected in the clear into the container at runtime. -### Adding volumes +### Using volumes You can add custom volumes into the app containers using `volumes`: @@ -105,9 +105,9 @@ volumes: - "/local/path:/container/path" ``` -### Splitting servers into different roles +### Using different roles for servers -If your application uses separate hosts for running jobs or other roles beyond the default web running, you can specify these hosts and their custom entrypoint command like so: +If your application uses separate hosts for running jobs or other roles beyond the default web running, you can specify these hosts in a dedicated role with a new entrypoint command like so: ```yaml servers: @@ -121,23 +121,35 @@ servers: cmd: bin/jobs ``` -Traefik will only be installed and run on the servers in the `web` role (and on all servers if no roles are defined). +Note: Traefik will only by default be installed and run on the servers in the `web` role (and on all servers if no roles are defined). If you need Traefik on hosts in other roles than `web`, add `traefik: true`: -### Adding custom container labels +```yaml +servers: + web: + - 192.168.0.1 + - 192.168.0.2 + web2: + traefik: true + hosts: + - 192.168.0.3 + - 192.168.0.4 +``` -You can specialize the default Traefik rules by setting custom labels on the containers that are being started: +### Using container labels + +You can specialize the default Traefik rules by setting labels on the containers that are being started: ``` labels: traefik.http.routers.hey.rule: '''Host(`app.hey.com`)''' ``` -(Note: The extra quotes are needed to ensure the rule is passed in correctly!) +Note: The extra quotes are needed to ensure the rule is passed in correctly! This allows you to run multiple applications on the same server sharing the same Traefik instance and port. See https://doc.traefik.io/traefik/routing/routers/#rule for a full list of available routing rules. -The labels can even be applied on a per-role basis: +The labels can also be applied on a per-role basis: ```yaml servers: @@ -150,41 +162,52 @@ servers: - 192.168.0.4 cmd: bin/jobs labels: - my-custom-label: "50" + my-label: "50" ``` -### Configuring remote builder for native multi-arch +### Using remote builder for native multi-arch -If you're developing on ARM64 (like Apple Silicon), but you want to deploy on AMD64 (x86 64-bit), you have to use multi-archecture images. By default, MRSK will setup a local buildx configuration that allows for this through QEMU emulation. This can be slow, especially on the first build. +If you're developing on ARM64 (like Apple Silicon), but you want to deploy on AMD64 (x86 64-bit), you can use multi-archecture images. By default, MRSK will setup a local buildx configuration that does this through QEMU emulation. But this can be quite slow, especially on the first build. -If you want to speed up this process by using a remote AMD64 host to natively build the AMD64 part of the image, while natively building the ARM64 part locally, you can do so using builder options like follows: +If you want to speed up this process by using a remote AMD64 host to natively build the AMD64 part of the image, while natively building the ARM64 part locally, you can do so using builder options: ```yaml builder: local: arch: arm64 - host: unix:///Users/dhh/.docker/run/docker.sock + host: unix:///Users/<%= `whoami`.strip %>/.docker/run/docker.sock remote: arch: amd64 host: ssh://root@192.168.0.1 ``` -Note: You must have Docker running on the remote host being used as a builder. +Note: You must have Docker running on the remote host being used as a builder. This instance should only be shared for builds using the same registry and credentials. -With that configuration in place, you can setup the local/remote configuration using `mrsk build create`. If you wish to remove the contexts and buildx instances again, you can run `mrsk build remove`. If you had already built using the standard emulation setup, run `mrsk build remove` before doing `mrsk build remote`. +### Using remote builder for single-arch -### Configuring native builder when multi-arch isn't needed +If you're developing on ARM64 (like Apple Silicon), want to deploy on AMD64 (x86 64-bit), but don't need to run the image locally (or on other ARM64 hosts), you can configure a remote builder that just targets AMD64. This is a bit faster than building with multi-arch, as there's nothing to build locally. -If you're developing on the same architecture as the one you're deploying on, you can speed up the build a lot by forgoing a multi-arch image. This can be done by configuring the builder like so: +```yaml +builder: + remote: + arch: amd64 + host: ssh://root@192.168.0.1 +``` + +### Using native builder when multi-arch isn't needed + +If you're developing on the same architecture as the one you're deploying on, you can speed up the build by forgoing both multi-arch and remote building: ```yaml builder: multiarch: false ``` -### Configuring build secrets for new images +This is also a good option if you're running MRSK from a CI server that shares architecture with the deployment servers. -Some images need a secret passed in during build time, like a GITHUB_TOKEN to give access to private gem repositories. This can be done by having the secret in ENV, then referencing it like so in the configuration: +### Using build secrets for new images + +Some images need a secret passed in during build time, like a GITHUB_TOKEN to give access to private gem repositories. This can be done by having the secret in ENV, then referencing it in the builder configuration: ```yaml builder: @@ -192,13 +215,13 @@ builder: - GITHUB_TOKEN ``` -This build secret can then be used in the Dockerfile: +This build secret can then be referenced in the Dockerfile: ``` -# Install application gems +# Copy Gemfiles COPY Gemfile Gemfile.lock ./ -# Private repositories need an access token during the build +# Install dependencies, including private repositories via access token RUN --mount=type=secret,id=GITHUB_TOKEN \ BUNDLE_GITHUB__COM=x-access-token:$(cat /run/secrets/GITHUB_TOKEN) \ bundle install @@ -206,7 +229,7 @@ RUN --mount=type=secret,id=GITHUB_TOKEN \ ### Configuring build args for new images -Build arguments that aren't secret can be configured like so: +Build arguments that aren't secret can also be configured: ```yaml builder: @@ -224,26 +247,27 @@ FROM ruby:$RUBY_VERSION-slim as base ## Commands -### Remote execution +### Running remote execution and runners -If you need to execute commands inside the Rails containers, you can use `mrsk app exec`, `mrsk app exec --once`, `mrsk app runner`, and `mrsk app runner --once`. Examples: +If you need to execute commands inside the Rails containers, you can use `mrsk app exec` and `mrsk app runner`. Examples: ```bash # Runs command on all servers mrsk app exec 'ruby -v' -App Host: xxx.xxx.xxx.xxx +App Host: 192.168.0.1 ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] -App Host: xxx.xxx.xxx.xxx +App Host: 192.168.0.2 ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] -# Runs command on first server -mrsk app exec --once 'cat .ruby-version' +# Runs command on primary server +mrsk app exec --primary 'cat .ruby-version' +App Host: 192.168.0.1 3.1.3 # Runs Rails command on all servers mrsk app exec 'bin/rails about' -App Host: xxx.xxx.xxx.xxx +App Host: 192.168.0.1 About your application's environment Rails version 7.1.0.alpha Ruby version ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] @@ -255,7 +279,7 @@ Environment production Database adapter sqlite3 Database schema version 20221231233303 -App Host: xxx.xxx.xxx.xxx +App Host: 192.168.0.2 About your application's environment Rails version 7.1.0.alpha Ruby version ruby 3.1.3p185 (2022-11-24 revision 1a6b16756e) [x86_64-linux] @@ -267,50 +291,50 @@ Environment production Database adapter sqlite3 Database schema version 20221231233303 -# Runs Rails runner on first server -mrsk app runner 'puts Rails.application.config.time_zone' +# Run Rails runner on primary server +mrsk app runner -p 'puts Rails.application.config.time_zone' UTC ``` -### Running a Rails console on the primary host +### Running a Rails console If you need to interact with the production console for the app, you can use `mrsk app console`, which will start a Rails console session on the primary host. You can start the console on a different host using `mrsk app console --host 192.168.0.2`. Be mindful that this is a live wire! Any changes made to the production database will take effect immeditately. -### Inspecting +### Running details to see state of containers -You can see the state of your servers by running `mrsk details`. It'll show something like this: +You can see the state of your servers by running `mrsk details`: ``` -Traefik Host: xxx.xxx.xxx.xxx +Traefik Host: 192.168.0.1 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 6195b2a28c81 traefik "/entrypoint.sh --pr…" 30 minutes ago Up 19 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp traefik -Traefik Host: 164.92.105.119 +Traefik Host: 192.168.0.2 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES de14a335d152 traefik "/entrypoint.sh --pr…" 30 minutes ago Up 19 minutes 0.0.0.0:80->80/tcp, :::80->80/tcp traefik -App Host: 164.90.145.60 +App Host: 192.168.0.1 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES badb1aa51db3 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 13 minutes ago Up 13 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123 -App Host: 164.92.105.119 +App Host: 192.168.0.2 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1d3c91ed1f55 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 13 minutes ago Up 13 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123 ``` You can also see just info for app containers with `mrsk app details` or just for Traefik with `mrsk traefik details`. -### Rollback +### Running rollback to fix a bad deploy If you've discovered a bad deploy, you can quickly rollback by reactivating the old, paused container image. You can see what old containers are available for rollback by running `mrsk app containers`. It'll give you a presentation similar to `mrsk app details`, but include all the old containers as well. Showing something like this: ``` -App Host: 164.92.105.119 +App Host: 192.168.0.1 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 1d3c91ed1f51 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 19 minutes ago Up 19 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123 539f26b28369 registry.digitalocean.com/user/app:e5d9d7c2b898289dfbc5f7f1334140d984eedae4 "/rails/bin/docker-e…" 31 minutes ago Exited (1) 27 minutes ago chat-e5d9d7c2b898289dfbc5f7f1334140d984eedae4 -App Host: 164.90.145.60 +App Host: 192.168.0.2 CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES badb1aa51db4 registry.digitalocean.com/user/app:6ef8a6a84c525b123c5245345a8483f86d05a123 "/rails/bin/docker-e…" 19 minutes ago Up 19 minutes 3000/tcp chat-6ef8a6a84c525b123c5245345a8483f86d05a123 6f170d1172ae registry.digitalocean.com/user/app:e5d9d7c2b898289dfbc5f7f1334140d984eedae4 "/rails/bin/docker-e…" 31 minutes ago Exited (1) 27 minutes ago chat-e5d9d7c2b898289dfbc5f7f1334140d984eedae4 @@ -320,7 +344,7 @@ From the example above, we can see that `e5d9d7c2b898289dfbc5f7f1334140d984eedae Note that by default old containers are pruned after 3 days when you run `mrsk deploy`. -### Removing +### Running removal to clean up servers If you wish to remove the entire application, including Traefik, containers, images, and registry session, you can run `mrsk remove`. This will leave the servers clean. diff --git a/lib/mrsk/cli/app.rb b/lib/mrsk/cli/app.rb index bef85e81..d71987a1 100644 --- a/lib/mrsk/cli/app.rb +++ b/lib/mrsk/cli/app.rb @@ -49,16 +49,16 @@ class Mrsk::Cli::App < Mrsk::Cli::Base desc "console", "Start Rails Console on primary host (or specific host set by --hosts)" def console run_locally do - puts "Launching Rails console on #{MRSK.primary_host}..." - exec MRSK.app.console(host: MRSK.primary_host) + info "Launching Rails console on #{MRSK.primary_host}" + exec MRSK.app.console(host: MRSK.primary_host) end end desc "bash", "Start a bash session on primary host (or specific host set by --hosts)" def bash run_locally do - puts "Launching bash session on #{MRSK.primary_host}..." - exec MRSK.app.bash(host: MRSK.primary_host) + info "Launching bash session on #{MRSK.primary_host}" + exec MRSK.app.bash(host: MRSK.primary_host) end end @@ -77,26 +77,36 @@ class Mrsk::Cli::App < Mrsk::Cli::Base on(MRSK.hosts) { |host| puts_by_host host, capture_with_info(*MRSK.app.current_container_id) } end - desc "logs", "Show last 100 log lines from app on servers" + desc "logs", "Show lines from app on servers" option :since, aliases: "-s", desc: "Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)" option :lines, type: :numeric, aliases: "-n", desc: "Number of log lines to pull from each server" option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)" + option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)" def logs # FIXME: Catch when app containers aren't running - since = options[:since] - lines = options[:lines] - grep = options[:grep] + grep = options[:grep] - on(MRSK.hosts) do |host| - begin - puts_by_host host, capture_with_info(*MRSK.app.logs(since: since, lines: lines, grep: grep)) - rescue SSHKit::Command::Failed - puts_by_host host, "Nothing found" + if options[:follow] + run_locally do + info "Following logs on #{MRSK.primary_host}..." + info MRSK.app.follow_logs(host: MRSK.primary_host, grep: grep) + exec MRSK.app.follow_logs(host: MRSK.primary_host, grep: grep) + end + else + since = options[:since] + lines = options[:lines] + + on(MRSK.hosts) do |host| + begin + puts_by_host host, capture_with_info(*MRSK.app.logs(since: since, lines: lines, grep: grep)) + rescue SSHKit::Command::Failed + puts_by_host host, "Nothing found" + end end end end - + desc "remove", "Remove app containers and images from servers" option :only, default: "", desc: "Use 'containers' or 'images'" def remove diff --git a/lib/mrsk/cli/build.rb b/lib/mrsk/cli/build.rb index 89730f66..5f063439 100644 --- a/lib/mrsk/cli/build.rb +++ b/lib/mrsk/cli/build.rb @@ -13,13 +13,11 @@ class Mrsk::Cli::Build < Mrsk::Cli::Base run_locally do begin - debug "Using builder: #{MRSK.builder.name}" - info "Building image may take a while (run with --verbose for progress logging)" unless verbose - execute *MRSK.builder.push + MRSK.verbosity(:debug) { execute *MRSK.builder.push } rescue SSHKit::Command::Failed => e error "Missing compatible builder, so creating a new one first" execute *MRSK.builder.create - execute *MRSK.builder.push + MRSK.verbosity(:debug) { execute *MRSK.builder.push } end end end @@ -48,7 +46,7 @@ class Mrsk::Cli::Build < Mrsk::Cli::Base desc "details", "Show the name of the configured builder" def details run_locally do - puts "Builder: #{MRSK.builder.name} (#{MRSK.builder.target.class.name})" + puts "Builder: #{MRSK.builder.name}" puts capture(*MRSK.builder.info) end end diff --git a/lib/mrsk/cli/traefik.rb b/lib/mrsk/cli/traefik.rb index 6d85bf3c..e60d548f 100644 --- a/lib/mrsk/cli/traefik.rb +++ b/lib/mrsk/cli/traefik.rb @@ -6,6 +6,13 @@ class Mrsk::Cli::Traefik < Mrsk::Cli::Base on(MRSK.traefik_hosts) { execute *MRSK.traefik.run, raise_on_non_zero_exit: false } end + desc "reboot", "Reboot Traefik on servers (stop container, remove container, start new container)" + def reboot + invoke :stop + invoke :remove_container + invoke :boot + end + desc "start", "Start existing Traefik on servers" def start on(MRSK.traefik_hosts) { execute *MRSK.traefik.start, raise_on_non_zero_exit: false } @@ -27,18 +34,44 @@ class Mrsk::Cli::Traefik < Mrsk::Cli::Base on(MRSK.traefik_hosts) { |host| puts_by_host host, capture_with_info(*MRSK.traefik.info), type: "Traefik" } end - desc "logs", "Show last 100 log lines from Traefik on servers" + desc "logs", "Show log lines from Traefik on servers" + option :since, aliases: "-s", desc: "Show logs since timestamp (e.g. 2013-01-02T13:23:37Z) or relative (e.g. 42m for 42 minutes)" + option :lines, type: :numeric, aliases: "-n", desc: "Number of log lines to pull from each server" + option :grep, aliases: "-g", desc: "Show lines with grep match only (use this to fetch specific requests by id)" + option :follow, aliases: "-f", desc: "Follow logs on primary server (or specific host set by --hosts)" def logs - on(MRSK.traefik_hosts) { |host| puts_by_host host, capture(*MRSK.traefik.logs), type: "Traefik" } + grep = options[:grep] + + if options[:follow] + run_locally do + info "Following logs on #{MRSK.primary_host}..." + info MRSK.traefik.follow_logs(host: MRSK.primary_host, grep: grep) + exec MRSK.traefik.follow_logs(host: MRSK.primary_host, grep: grep) + end + else + since = options[:since] + lines = options[:lines] + + on(MRSK.traefik_hosts) do |host| + puts_by_host host, capture(*MRSK.traefik.logs(since: since, lines: lines, grep: grep)), type: "Traefik" + end + end end desc "remove", "Remove Traefik container and image from servers" def remove invoke :stop + invoke :remove_container + invoke :remove_image + end - on(MRSK.traefik_hosts) do - execute *MRSK.traefik.remove_container - execute *MRSK.traefik.remove_image - end + desc "remove_container", "Remove Traefik container from servers" + def remove_container + on(MRSK.traefik_hosts) { execute *MRSK.traefik.remove_container } + end + + desc "remove_container", "Remove Traefik image from servers" + def remove_image + on(MRSK.traefik_hosts) { execute *MRSK.traefik.remove_image } end end diff --git a/lib/mrsk/commands/app.rb b/lib/mrsk/commands/app.rb index 73d0990a..1f36a1fa 100644 --- a/lib/mrsk/commands/app.rb +++ b/lib/mrsk/commands/app.rb @@ -60,6 +60,14 @@ class Mrsk::Commands::App < Mrsk::Commands::Base *command end + def follow_logs(host:, grep: nil) + run_over_ssh pipe( + current_container_id, + "xargs docker logs -t -n 10 -f 2>&1", + ("grep '#{grep}'" if grep) + ).join(" "), host: host + end + def console(host:) exec_over_ssh "bin/rails", "c", host: host end @@ -82,7 +90,7 @@ class Mrsk::Commands::App < Mrsk::Commands::Base private def exec_over_ssh(*command, host:) - "ssh -t #{config.ssh_user}@#{host} '#{run_exec(*command, interactive: true).join(" ")}'" + run_over_ssh run_exec(*command, interactive: true).join(" "), host: host end def service_filter diff --git a/lib/mrsk/commands/base.rb b/lib/mrsk/commands/base.rb index d2a0ae89..e7305701 100644 --- a/lib/mrsk/commands/base.rb +++ b/lib/mrsk/commands/base.rb @@ -23,5 +23,9 @@ module Mrsk::Commands def docker(*args) args.compact.unshift :docker end + + def run_over_ssh(command, host:) + "ssh -t #{config.ssh_user}@#{host} '#{command}'" + end end end diff --git a/lib/mrsk/commands/builder.rb b/lib/mrsk/commands/builder.rb index ce18fc63..e24768d1 100644 --- a/lib/mrsk/commands/builder.rb +++ b/lib/mrsk/commands/builder.rb @@ -2,10 +2,9 @@ require "mrsk/commands/base" class Mrsk::Commands::Builder < Mrsk::Commands::Base delegate :create, :remove, :push, :pull, :info, to: :target - delegate :native?, :multiarch?, :remote?, to: :name def name - target.class.to_s.demodulize.downcase.inquiry + target.class.to_s.remove("Mrsk::Commands::Builder::").underscore end def target @@ -14,6 +13,8 @@ class Mrsk::Commands::Builder < Mrsk::Commands::Base native when config.builder && config.builder["local"] && config.builder["remote"] multiarch_remote + when config.builder && config.builder["remote"] + native_remote else multiarch end @@ -23,6 +24,10 @@ class Mrsk::Commands::Builder < Mrsk::Commands::Base @native ||= Mrsk::Commands::Builder::Native.new(config) end + def native_remote + @native ||= Mrsk::Commands::Builder::Native::Remote.new(config) + end + def multiarch @multiarch ||= Mrsk::Commands::Builder::Multiarch.new(config) end @@ -33,5 +38,6 @@ class Mrsk::Commands::Builder < Mrsk::Commands::Base end require "mrsk/commands/builder/native" +require "mrsk/commands/builder/native/remote" require "mrsk/commands/builder/multiarch" require "mrsk/commands/builder/multiarch/remote" diff --git a/lib/mrsk/commands/builder/base.rb b/lib/mrsk/commands/builder/base.rb index d08e8a87..792ebee1 100644 --- a/lib/mrsk/commands/builder/base.rb +++ b/lib/mrsk/commands/builder/base.rb @@ -17,10 +17,10 @@ class Mrsk::Commands::Builder::Base < Mrsk::Commands::Base private def args - config.builder["args"] || {} + (config.builder && config.builder["args"]) || {} end def secrets - config.builder["secrets"] || [] + (config.builder && config.builder["secrets"]) || [] end end diff --git a/lib/mrsk/commands/builder/multiarch.rb b/lib/mrsk/commands/builder/multiarch.rb index 82ef4ee5..bc87a86f 100644 --- a/lib/mrsk/commands/builder/multiarch.rb +++ b/lib/mrsk/commands/builder/multiarch.rb @@ -12,7 +12,8 @@ class Mrsk::Commands::Builder::Multiarch < Mrsk::Commands::Builder::Base def push docker :buildx, :build, "--push", - "--platform linux/amd64,linux/arm64", + "--platform", "linux/amd64,linux/arm64", + "--builder", builder_name, "-t", config.absolute_image, *build_args, *build_secrets, @@ -27,6 +28,6 @@ class Mrsk::Commands::Builder::Multiarch < Mrsk::Commands::Builder::Base private def builder_name - "mrsk-#{config.service}" + "mrsk-#{config.service}-multiarch" end end diff --git a/lib/mrsk/commands/builder/multiarch/remote.rb b/lib/mrsk/commands/builder/multiarch/remote.rb index 2bbd3c3a..b49de328 100644 --- a/lib/mrsk/commands/builder/multiarch/remote.rb +++ b/lib/mrsk/commands/builder/multiarch/remote.rb @@ -15,8 +15,16 @@ class Mrsk::Commands::Builder::Multiarch::Remote < Mrsk::Commands::Builder::Mult end private + def builder_name + super + "-remote" + end + + def builder_name_with_arch(arch) + "#{builder_name}-#{arch}" + end + def create_local_buildx - docker :buildx, :create, "--use", "--name", builder_name, builder_name_with_arch(local["arch"]), "--platform", "linux/#{local["arch"]}" + docker :buildx, :create, "--name", builder_name, builder_name_with_arch(local["arch"]), "--platform", "linux/#{local["arch"]}" end def append_remote_buildx @@ -50,9 +58,4 @@ class Mrsk::Commands::Builder::Multiarch::Remote < Mrsk::Commands::Builder::Mult def remote config.builder["remote"] end - - private - def builder_name_with_arch(arch) - "#{builder_name}-#{arch}" - end end diff --git a/lib/mrsk/commands/builder/native/remote.rb b/lib/mrsk/commands/builder/native/remote.rb new file mode 100644 index 00000000..6d2aeeea --- /dev/null +++ b/lib/mrsk/commands/builder/native/remote.rb @@ -0,0 +1,71 @@ +require "mrsk/commands/builder/native" + +class Mrsk::Commands::Builder::Native::Remote < Mrsk::Commands::Builder::Native + def create + combine \ + create_context, + create_buildx + end + + def remove + combine \ + remove_context, + remove_buildx + end + + def push + docker :buildx, :build, + "--push", + "--platform", platform, + "--builder", builder_name, + "-t", config.absolute_image, + *build_args, + *build_secrets, + "." + end + + def info + combine \ + docker(:context, :ls), + docker(:buildx, :ls) + end + + + private + def arch + config.builder["remote"]["arch"] + end + + def host + config.builder["remote"]["host"] + end + + def builder_name + "mrsk-#{config.service}-native-remote" + end + + def builder_name_with_arch + "#{builder_name}-#{arch}" + end + + def platform + "linux/#{arch}" + end + + def create_context + docker :context, :create, + builder_name_with_arch, "--description", "'#{builder_name} #{arch} native host'", "--docker", "'host=#{host}'" + end + + def remove_context + docker :context, :rm, builder_name_with_arch + end + + def create_buildx + docker :buildx, :create, "--name", builder_name, builder_name_with_arch, "--platform", platform + end + + def remove_buildx + docker :buildx, :rm, builder_name + end +end diff --git a/lib/mrsk/commands/traefik.rb b/lib/mrsk/commands/traefik.rb index 6e7461bc..a2d395d7 100644 --- a/lib/mrsk/commands/traefik.rb +++ b/lib/mrsk/commands/traefik.rb @@ -8,7 +8,8 @@ class Mrsk::Commands::Traefik < Mrsk::Commands::Base "-p 80:80", "-v /var/run/docker.sock:/var/run/docker.sock", "traefik", - "--providers.docker" + "--providers.docker", + "--log.level=DEBUG" end def start @@ -23,8 +24,17 @@ class Mrsk::Commands::Traefik < Mrsk::Commands::Base docker :ps, "--filter", "name=traefik" end - def logs - docker :logs, "traefik", "-n", "100", "-t" + def logs(since: nil, lines: nil, grep: nil) + pipe \ + docker(:logs, "traefik", (" --since #{since}" if since), (" -n #{lines}" if lines), "-t", "2>&1"), + ("grep '#{grep}'" if grep) + end + + def follow_logs(host:, grep: nil) + run_over_ssh pipe( + docker(:logs, "traefik", "-t", "-n", "10", "-f", "2>&1"), + ("grep '#{grep}'" if grep) + ).join(" "), host: host end def remove_container diff --git a/test/commands/builder_test.rb b/test/commands/builder_test.rb index 16c8f1ca..14aee153 100644 --- a/test/commands/builder_test.rb +++ b/test/commands/builder_test.rb @@ -8,15 +8,27 @@ class CommandsBuilderTest < ActiveSupport::TestCase end test "target multiarch by default" do - assert new_builder_command.multiarch? + builder = new_builder_command + assert_equal "multiarch", builder.name + assert_equal [:docker, :buildx, :build, "--push", "--platform", "linux/amd64,linux/arm64", "--builder", "mrsk-app-multiarch", "-t", "dhh/app:123", "."], builder.push end test "target native when multiarch is off" do - assert new_builder_command(builder: { "multiarch" => false }).native? + builder = new_builder_command(builder: { "multiarch" => false }) + assert_equal "native", builder.name + assert_equal [:docker, :build, "-t", "dhh/app:123", ".", "&&", :docker, :push, "dhh/app:123"], builder.push end test "target multiarch remote when local and remote is set" do - assert new_builder_command(builder: { "local" => { }, "remote" => { } }).remote? + builder = new_builder_command(builder: { "local" => { }, "remote" => { } }) + assert_equal "multiarch/remote", builder.name + assert_equal [:docker, :buildx, :build, "--push", "--platform", "linux/amd64,linux/arm64", "--builder", "mrsk-app-multiarch-remote", "-t", "dhh/app:123", "."], builder.push + end + + test "target native remote when only remote is set" do + builder = new_builder_command(builder: { "remote" => { "arch" => "amd64" } }) + assert_equal "native/remote", builder.name + assert_equal [:docker, :buildx, :build, "--push", "--platform", "linux/amd64", "--builder", "mrsk-app-native-remote", "-t", "dhh/app:123", "."], builder.push end test "build args" do @@ -36,7 +48,7 @@ class CommandsBuilderTest < ActiveSupport::TestCase test "multiarch push with build args" do builder = new_builder_command(builder: { "args" => { "a" => 1, "b" => 2 } }) - assert_equal [ :docker, :buildx, :build, "--push", "--platform linux/amd64,linux/arm64", "-t", "dhh/app:123", "--build-arg", "a=1", "--build-arg", "b=2", "." ], builder.push + assert_equal [ :docker, :buildx, :build, "--push", "--platform", "linux/amd64,linux/arm64", "--builder", "mrsk-app-multiarch", "-t", "dhh/app:123", "--build-arg", "a=1", "--build-arg", "b=2", "." ], builder.push end test "native push with with build secrets" do diff --git a/test/configuration_role_test.rb b/test/configuration/role_test.rb similarity index 100% rename from test/configuration_role_test.rb rename to test/configuration/role_test.rb