Continuous Fuzz Testing Setup - legacy

Integrating CI Fuzz Into Your CI/CD Process

Legacy content

This guide describes CI/CD integration with the CI Fuzz server running the legacy Go backend. For the latest versions, see here.

Architecture of CI Fuzz Integration

When running a pipeline, your CI/CD platform triggers fuzzing on the CI Fuzz server. The fuzzing server checks out the source code, instruments it, builds and starts the configured fuzz tests. This begins with a regression test by checking previously generated inputs and crashes found in previous runs. When vulnerabilities or bugs are found, the pipeline is marked as failed and the detailed bug reports can be viewed and debugged in the dashboard. You can log in to the dashboard using your single sign on from your GitLab, Bitbucket or GitHub account. The CI Fuzz server can be either in cloud or deployed on-premise. 

Example With Jenkins and Bitbucket:

fuzzing with jenkins and bitbucket | CI Fuzz

How to Generate the Config Script for Your CI/CD Server

To enable continuous fuzz testing of your projects, use the CI configuration wizard in the UI which is accessible from the dashboard.

How to Generate the Fuzzing Confic Script for your CI/CD Server | CI Fuzz

In the wizard you select the type of continuous integration you are using, the test collection that you want to fuzz continuously and whether you want to be notified on additional channels if bugs are found in the continuous integration. Then generate the CI/CD script.

Run Fuzzing in Your CI/CD

The text then needs to be pasted into your specific CI/CD configuration file. When the script runs for the first time, it will pull a docker image with a tool called cictl, which handles communication with the fuzz server.

Next, create an Access Token for your CI/CD platform by clicking on your picture in the upper right corner and the Generate button.

create an Access Token for your CI/CD platform | CI Fuzz

Your CI/CD platform will need read and execute scopes.

Next, store this token into your CI/CD platform. Example with GitHub Actions:

store this token into your CI/CD platform. Example with GitHub Actions

Example with GitLab CI/CD:

The "Key" field should be the same as "Token name" in CI Fuzz.

The "Key" field should be the same as "Token name" in CI Fuzz.

Trigger the pipeline (for example by creating a pull/merge request) to start fuzzing. If the fuzzer finds bugs, the pipeline will fail, and you will see a link to findings in your CI/CD platform.

Example Finding From a Failing Pipeline in GitHub Actions

Fuzzing Example Finding From a Failing Pipeline in GitHub Actions

Recommended Settings

Code Intelligence recommends running fuzz tests for 5 minutes (the TIMEOUT variable in your CI/CD script set to 300) every time new code is pushed to a branch, or every time a pull/merge request is created. Five minutes is usually enough to find regressions and easily discoverable bugs. This is possible, because CI Fuzz remembers bugs and code coverage information from previous runs.

Additionally, Code Intelligence recommends setting up fuzzing for a few hours every night. Given enough time, CI Fuzz will discover new inputs that cause the program under test to behave differently, and possibly find new bugs. Even if it does not discover new bugs  or gain new code coverage during one such run, there is still a high chance it will find something next time, as inputs are generated randomly and are different every time.

To set up nightly fuzzing, set up a nightly build in your CI/CD, using a script generated as described above, and set the TIMEOUT variable accordingly.

CI Fuzz On-Prem Special Steps

You may need to change some URLs in the generated script to reflect the URLs of your On-Prem CI services.

Also, you will need to tell the "cictl" tool, which will be running on your CI/CD platform, to trust your On-Prem CI Fuzz API's server certificate, or the CA that signed it. This step is not needed if your CI Fuzz server's certificate is signed by a public CA.

Store the certificate so that it's accessible in the CI/CD script.

Example with GitLab CI/CD (the type "file" stores a path to the file in an environment variable):

CI Fuzz On-Prem ExampleThen add the following commands at the beginning of your CI/CD script:

head $CIFUZZ_CA_OR_SERVER_CERT #to see if it is reachable 
cp $CIFUZZ_CA_OR_SERVER_CERT /usr/local/share/ca-certificates/ci/on-prem-ci.crt

The extension must be crt.

If your CICD script will be running on Centos, the commands are a little bit different:

head $CIFUZZ_CA_OR_SERVER_CERT #to see if it is reachable 
cp $CIFUZZ_CA_OR_SERVER_CERT /etc/pki/ca-trust/source/anchors/on-prem-ci.crt

Trigger the CI/CD pipeline and see if fuzzing starts. If there are any issues, please contact our support.