SPIRV-Tools/docs/spirv-fuzz.md

Guide to writing a spirv-fuzz fuzzer pass

Writing a spirv-fuzz fuzzer pass usually requires two main contributions:

• A transformation, capturing a small semantics-preserving change that can be made to a SPIR-V module. This requires adding a protobuf message representing the transformation, and a corresponding class that implements the Transformation interface.
• A new fuzzer pass class, implementing the FuzzerPass interface, that knows how to walk a SPIR-V module and apply the new transformation in a randomized fashion.

In some cases, more than one kind of transformation is required for a single fuzzer pass, and in some cases the transformations that a new fuzzer pass requires have already been introduced by existing passes. But the most common case is to introduce a transformation and fuzzer pass together.

As an example, let's consider the TransformationSetSelectionControl transformation. In SPIR-V, an OpSelectionMerge instruction (which intuitively indicates the start of an if or switch statement in a function) has a selection control mask, that can be one of None, Flatten or DontFlatten. The details of these do not matter much for this little tutorial, but in brief, this parameter provides a hint to the shader compiler as to whether it would be profitable to attempt to flatten a piece of conditional code so that all of its statements are executed in a predicated fashion.

As the selection control mask is just a hint, changing the value of this mask should have no semantic impact on the module. The TransformationSelectionControl transformation specifies a new value for a given selection control mask.

Adding a new protobuf message

Take a look at the Transformation message in spvtoolsfuzz.proto. This has a oneof field that can be any one of the different spirv-fuzz transformations. Observe that one of the options is TransformationSetSelectionControl. When adding a transformation you first need to add an option for your transformation to the end of the oneof declaration.

Now look at the TransformationSetSelectionControl message. If adding your own transformation you need to add a new message for your transformation, and it should be placed alphabetically with respect to other transformations.

The fields of TransformationSetSelectionControl provide just enough information to (a) determine whether a given example of this transformation is actually applicable, and (b) apply the transformation in the case that it is applicable. The details of the transformation message will vary a lot between transformations. In this case, the message has a block_id field, specifying a block that must end with OpSelectionMerge, and a selection_control field, which is the new value for the selection control mask of the OpSelectionMerge instruction.

Adding a new transformation class

If your transformation is called TransformationSomeThing, you need to add transformation_some_thing.h and transformation_some_thing.cpp to source/fuzz and the corresponding CMakeLists.txt file. So for TransformationSetSelectionControl we have transformation_selection_control.h and transformation_selection_control.cpp, and we will use this as an example to illustrate the expected contents of these files.

The header file contains the specification of a class, TransformationSetSelectionControl, that implements the Transformation interface (from transformation.h).

A transformation class should always have a single field, which should be the associated protobuf message; in our case:

 private:
  protobufs::TransformationSetSelectionControl message_;

and two public constructors, one that takes a protobuf message; in our case:

  explicit TransformationSetSelectionControl(
      const protobufs::TransformationSetSelectionControl& message);

and one that takes a parameter for each protobuf message field; in our case:

  TransformationSetSelectionControl(uint32_t block_id);

The first constructor allows an instance of the class to be created from a corresponding protobuf message. The second should provide the ingredients necessary to populate a protobuf message.

The class should also override the IsApplicable, Apply and ToMessage methods from Transformation.

See transformation_set_selection_control.h for an example.

The IsApplicable method should have a comment in the header file describing the conditions for applicability in simple terms. These conditions should be implemented in the body of this method in the .cpp file.

In the case of TransformationSetSelectionControl, IsApplicable involves checking that block_id is indeed the id of a block that has an OpSelectionMerge instruction, and that selection_control is a valid selection mask.

The Apply method should have a comment in the header file summarising the result of applying the transformation. It should be implemented in the .cpp file, and you should assume that IsApplicable holds whenever Apply is invoked.

Writing tests for the transformation class

Whenever you add a transformation class, TransformationSomeThing, you should add an associated test file, transformation_some_thing_test.cpp, under test/fuzz, adding it to the associated CMakeLists.txt file.

For example test/fuzz/transformation_set_selection_control_test.cpp contains tests for TransformationSetSelectionControl. Your tests should aim to cover one example from each scenario where the transformation is inapplicable, and check that it is indeed deemed inapplicable, and then check that the transformation does the right thing when applied in a few different ways.

For example, the tests for TransformationSetSelectionControl check that a transformation of this kind is inapplicable if the block_id field of the transformation is not a block, or does not end in OpSelectionMerge, or if the selection_control mask has an illegal value. It also checks that applying a sequence of valid transformations to a SPIR-V shader leads to a shader with appropriately modified selection controls.

Adding a new fuzzer pass class

A fuzzer pass traverses a SPIR-V module looking for places to apply a certain kind of transformation, and randomly decides at which of these points to actually apply the transformation. It might be necessary to apply other transformations in order to apply a given transformation (for example, if a transformation requires a certain type to be present in the module, said type can be added if not already present via another transformation).

A fuzzer pass implements the FuzzerPass interface, and overrides its Apply method. If your fuzzer pass is named FuzzerPassSomeThing then it should be represented by fuzzer_pass_some_thing.h and fuzzer_pass_some_thing.cpp, under source/fuzz; these should be added to the associated CMakeLists.txt file.

Have a look at the source filed for FuzzerPassAdjustSelectionControls. This pass considers every block that ends with OpSelectionMerge. It decides randomly whether to adjust the selection control of this merge instruction via:

if (!GetFuzzerContext()->ChoosePercentage(
        GetFuzzerContext()->GetChanceOfAdjustingSelectionControl())) {
  continue;
}

The GetChanceOfAddingSelectionControl() method has been added to FuzzerContext specifically to support this pass, and returns a percentage between 0 and 100. It returns the chance_of_adjusting_selection_control_ of FuzzerContext, which is randomly initialized to lie with the interval defined by kChanceOfAdjustingSelectionControl in fuzzer_context.cpp. For any pass you write, you will need to add an analogous GetChanceOf... method to FuzzerContext, backed by an appropriate field, and you will need to decide on lower and upper bounds for this field and specify these via a kChanceOf... constant.