Fuzzing Artifacts Archive Specification


                    Fuzzing Artifacts Archive 1.0
                  fuzzing-artifacts.tar[.xz|.bz2|.gz]


Abstract

   This specification describes the format of the fuzzing artifacts
   archive for usage with CI FUZZ on https://app.code-intelligence.com
   or Code Intelligence on-premise solutions.


Table of Contents

   1. Archive Contents
   2. Fuzz targets
      2.1 Fuzz targets for coverage measurement
   3. Metadata
      3.1. General properties
      3.2. Fuzzer properties
   Appendix A. Schema of cifuzz.yaml
   Appendix B. Example of an exception policy JSON file
   Appendix C. Schema of an exception policy JSON file


1. Archive Contents

   The fuzzing artifacts archive MUST be a Tar archive (.tar), or a
   [xz|bz2|gz]-compressed Tar archive (.tar.xz, .tar.bz2, .tar.gz).

   It MUST contain the following files and directories:

     cifuzz.yaml
     work_dir/

   All of the aforementioned directories MAY be empty. The archive MAY
   contain arbitrary other files and directories. It SHOULD however be
   limited in size to some degree, as it might be copied and
   transferred several times during fuzzing.


2. Fuzz targets

   Every fuzz target is a binary in the fuzzing artifact archive and
   can be dependent on libraries (see 'library_paths' in Section 3.2.)
   and other artifacts in the same archive. Every fuzz target needs an
   entry in the fuzz target list 'fuzzers' in cifuzz.yaml.


2.1 Fuzz targets for coverage measurement

   In order to measure the coverage of a fuzzing run, an additional
   fuzz target for coverage measurement has to be configured. It has
   to have the same name as the fuzz target it is measuring coverage
   for and must use a coverage engine, currently only "LLVM_COV".

   A fuzz target for coverage measurement can use the same binary as
   a fuzz target for fuzzing, if the binary is instrumented for
   fuzzing and coverage measurement.

   For measuring Java coverage ("JAVA_LIBFUZZER"), no extra fuzz
   target for coverage measurment is needed.


3. Metadata

   The metadata file cifuzz.yaml contains all information necessary to
   start a fuzzing run using the artifact. For a complete schema of
   cifuzz.yaml, see Appendix A.

   All paths in cifuzz.yaml are given relative to the archive's root
   directory and are written WITHOUT a leading slash (/).

   A typical cifuzz.yaml might look like this:

     run_environment:
         docker: ubuntu:rolling
     fuzzers:
         - target: .code-intelligence/fuzz_targets/fuzz_target.cpp
           engine: LIBFUZZER
           build_dir: "/home/ci/.local/share/code-intelligence/\
             projects/blub-060bf9a1/llvm_cov"
           path: libfuzzer/fuzz_target
           library_paths:
             - cifuzz-libs
           max_run_time: 1800
         - target: .code-intelligence/fuzz_targets/fuzz_target.cpp
           engine: LLVM_COV
           build_dir: "/home/ci/.local/share/code-intelligence/\
             projects/blub-060bf9a1/llvm_cov"
           path: llvm_cov/fuzz_target
           library_paths:
             - cifuzz-libs
           max_run_time: 86400

   Aside: Per convention, a fuzz target with file name <fuzz_target> and
   engine <engine> usually has the path /<engine>/<fuzz_target>, and a
   fuzz target with name <fuzz_target>, engine <engine> and using
   sanitizer <sanitizer> usually has the path
   /<engine>/<sanitizer>/<fuzz_target>. However, this is not necessary,
   and path has to be given in all cases.


3.1. General properties

   There are currently only two general properties that can be set in
   cifuzz.yaml, 'run_environment' and 'code_revision':

   'run_environment' specifies which environment to run the fuzzing
     in. It currently only supports the subkey 'docker', which gives a
     string—the docker container to run fuzzing in. Currently, every
     container can be used as a run environment, but in the future this
     may be limited to a set of containers deemed secure for usage.

   'code_revision' specifies the version of the code according to the
     source control management in use. Currently, only the key 'git' is
     supported, giving a object with two keys, 'commit' and 'branch',
     that give the current git commit hash and the current branch name
     if applicable, respectively.


3.2. Fuzzer properties

   Each fuzzer can take the following keys:

   'target' [required] specifies the name of the fuzz target.

   'engine' [required] specifies the fuzzing engine to be used. One of:
       "LIBFUZZER", "AFL", "AFL_LLVM_MODE", "HONGGFUZZ", "LLVM_COV",
       "UNIT_TEST_RUNNER", "JAVA_LIBFUZZER", "WEB_APP".

   'path' [required] specifies the path of the fuzzing binary.

   'sanitizer' specifies the sanitizer in LIBFUZZER mode. One of:
       "UNKNOWN_SANITIZER", "ADDRESS", "MEMORY", "THREAD"

   'build_dir' specifies the directory the fuzzers where built in.
     This is used in order to differentiate functions from the
     system under test from other functions (e.g. language or
     framework) in stacktrace. Omitting 'build_dir' will thus lead
     to empty stack traces.

   'fuzzer_run_args' specifies a list of arguments to be given to the
     fuzzer.

   'library_paths' specifies the lists of paths to libraries needed to
     run the fuzzer.

   'seeds' specifies the path to directory containing the initial
     corpora in the archive.

   'dictionary' specifies the path to a dict file in the archive.

   'engine_options' specifies options for the fuzzing engine. It has two
     subkeys, 'flags' and 'env':
     - 'flags' is a list of strings to be given to the engine as command
       line arguments.
     - 'env' is a list of strings to be given to the engine as
       environment variables.

   'max_run_time' specifies the maximum run time in seconds.

   'parallel_executions' specifies the number of fuzzers to be run at
     the same time.


   Additional properties apply to web app fuzzing (out-of-process
   fuzzing) only:

   'exception_policy' specifies a path to an exception policy JSON file.
     See Appendix B for an example file and Appendix C for a JSON schema
     for exception policy JSON files.

   'application_url' specifies the URL that the application is reachable
     under.

   'web_services' lists the webservices.


Appendix A. Schema of cifuzz.yaml

   The following is the JSON Schema [http://json-schema.org/] of
   cifuzz.yaml. Note that even though we are using JSON Schema as a
   means of describing the format, the file format is still YAML.
   See Section 3. "Metadata" for an example file.

   {
     "title": "cifuzz.yaml",
     "format": "YAML",
     "description": "This is a schema that describes the content of cifuzz.yaml in the top level of a fuzzing artifact archive.",
     "type": "object",
     "required": ["run_environment", "fuzzers"],
     "properties": {
       "run_environment": {
         "type": "object",
         "required": ["docker"],
         "properties": {
           "docker": {
             "description": "The docker container to run the fuzzers in.",
             "examples": ["debian:stable", "ubuntu:rolling"],
             "type": "string"
           }
         }
       },
       "code_revision": {
         "type": "object",
         "properties": {
           "git": {
             "type": "object",
             "properties": {
               "commit": { "type": "string" },
               "branch": { "type": "string" }
             }
           }
         }
       },
       "fuzzers": {
         "type": "array",
         "items": {
           "type": "object",
           "required": ["target", "engine", "path"],
           "properties": {
             "target": {
               "description": "The name of the fuzz target. Should be the name of the source file.",
               "examples": [".code-intelligence/fuzz_targets/AllImagesParser.java"],
               "type": "string"
             },
             "engine": { "enum": ["LIBFUZZER", "AFL", "AFL_LLVM_MODE", "HONGGFUZZ", "LLVM_COV", "UNIT_TEST_RUNNER", "JAVA_LIBFUZZER", "WEB_APP"] },
             "sanitizer": { "enum": ["UNKNOWN_SANITIZER", "ADDRESS", "MEMORY", "THREAD"] },
             "build_dir": { "type": "string" },
             "path": {
               "description": "The path of the fuzzer binary in the fuzzing artifact archive.",
               "examples": ["java_libfuzzer/AllImagesParser.jar"],
               "type": "string"
             },
             "library_paths": { "type": "array", "items": { "type": "string" }},
             "seeds": { "type": "string" },
             "dictionary": { "type": "string" },
             "engine_options": {
               "type": "object",
               "properties": {
                 "flags": { "type": "array", "items": { "type": "string" }},
                 "env": { "type": "array", "items": { "type": "string" }}
               }
             },
             "max_run_time": { "type": "number" },
             "parallel_executions": { "type": "number" },
             "skip_initial_corpus": { "type": "boolean" },
             "exception_policy": { "type": "string" },
             "application_url": { "type": "string" },
             "web_services": { "type": "array", "items": { "type": "string" }},
           }
         }
       }
     }
   }


Appendix B. Example of an exception policy JSON file

   This is an example for an exception policy JSON file. It will
   instruct the fuzzer monitor to ignore all exceptions from
   responses with an HTTP return code of 4xx or 302 and all
   exceptions of the type java.lang.IllegalArgumentException if
   the stacktrace contains the string
   "org.springframework.web.util.UriComponentsBuilder".

     {
       "policy": [
         {
           "enabled": true,
           "responsePolicy": {
             "responseMatcher": "CODE_STARTS_WITH",
             "code": 4
           }
         },
         {
           "enabled": true,
           "responsePolicy": {
             "code": 302
           }
         },
         {
           "enabled": true,
           "exceptionPolicy": {
             "Exception": "java.lang.IllegalArgumentException",
             "exceptionStrategy": "STACKTRACE_CONTAINS",
             "content": "org.springframework.web.util.UriComponentsBuilder"
           }
         }
       ]
     }


Appendix C. Schema of an exception policy JSON file

   The following is the JSON Schema [http://json-schema.org/] of an
   exception policy JSON file given in the fuzz target metadata key
   'exception_policy'. Any matching policy _whitelists_ the given
   exception or response, meaning that it is ignored and not considered
   to be a finding. See Appendix B for an example file.

   {
     "title": "Exception policy",
     "format": "JSON",
     "description": "This is a schema that describes the content of a exception policy file in a fuzzing artifact archive.",
     "type": "object",
     "required": ["policy"],
     "properties": {
       "policy": {
         "type": "array",
         "items": {
           "anyOf": [
             {
               "type": "object",
               "required": ["enabled", "exceptionPolicy"],
               "properties": {
                 "enabled": { "type": "boolean" },
                 "exceptionPolicy": {
                   "description": "Policy for exceptions.",
                   "type": "object",
                   "properties": {
                     "Exception": {"type": "string"},
                     "content": {"type": "string"},
                     "exceptionMatcher": {
                       "enum": ["EXCEPTION_IS", "EXCEPTION_FROM_PACKAGE", "EXCEPTION_STARTS_WITH"]
                     },
                     "exceptionStrategy": {
                       "enum": ["MATCH_ALL", "MESSAGE_CONTAINS", "MESSAGE_EXCLUDES", "STACKTRACE_CONTAINS"]
                     }
                   }
                 }
               }
             },
             {
               "type": "object",
               "required": ["enabled", "responsePolicy"],
               "properties": {
                 "enabled": { "type": "boolean" },
                 "responsePolicy": {
                   "description": "Policy for API responses.",
                   "type": "object",
                   "properties": {
                     "code": {"type": "number"},
                     "content": {"type": "string"},
                     "responseMatcher": {
                       "enum": ["CODE_IS", "CODE_STARTS_WITH"]
                     },
                     "responseStrategy": {
                       "enum": ["MATCH_ALL", "BODY_CONTAINS", "BODY_EXCLUDES"]
                     }
                   }
                 }
               }
             },
             {
               "type": "object",
               "required": ["enabled", "zapPolicy"],
               "properties": {
                 "enabled": { "type": "boolean" },
                 "zapPolicy": {
                   "description": "Policy for API responses.",
                   "type": "object",
                   "properties": {
                     "alert": {"type": "number"},
                     "content": {"type": "string"},
                     "alertMatcher": {
                       "enum": ["ALERT_NAME_IS", "ALERT_NAME_CONTAINS"]
                     },
                     "alertStrategy": {
                       "enum": ["MATCH_ALL", "URL_IS", "URL_CONTAINS"]
                     }
                   }
                 }
               }
             }
           ]
         }
       }
     }
   }