cli: [docs] add detailed description of Modes of Operation
This commit is contained in:
Oleksii Oleksenko
Родитель
1c68fddce0
Коммит
2471cb83b0
|
|
@ -90,7 +90,7 @@ multiple Python files:
|
|||
`Input`, etc)
|
||||
* `isa\_loader.py` - defines the `InstructionSet` class, used to load an
|
||||
ISA's specifications from a JSON file provided via the
|
||||
[command-line interface](cli.md).
|
||||
[command-line interface](user/cli.md).
|
||||
* `service.py` - defines logging, statistical, and other services to all other
|
||||
modules within revizor.
|
||||
|
||||
|
|
|
|||
|
|
@ -83,7 +83,7 @@ To minimize this interference, we recommend the following system configuration:
|
|||
* Disable Turbo Boost (BIOS option);
|
||||
* Boot the kernel on a single core (add `-maxcpus=1` to [Linux boot parameters]((https://wiki.ubuntu.com/Kernel/KernelBootParameters))).
|
||||
|
||||
If you skip these steps, Revizor may produce false positives, especially if you use a low value for [`executor_sample_sizes`](config.md#executor-configuration) for measurements.
|
||||
If you skip these steps, Revizor may produce false positives, especially if you use a low value for [`executor_sample_sizes`](user/config.md#executor-configuration) for measurements.
|
||||
However, a large sample size (> 300-400) usually mitigates this issue.
|
||||
|
||||
## Quick Start
|
||||
|
|
@ -187,7 +187,7 @@ This command should terminate with no violations.
|
|||
|
||||
### Full-Scale Fuzzing Campaign
|
||||
|
||||
To start a full-scale test, write your own configuration file (see description [here](config.md) and an example config [here](https://github.com/microsoft/sca-fuzzer/tree/main/demo/big-fuzz.yaml)), and launch the fuzzer.
|
||||
To start a full-scale test, write your own configuration file (see description [here](user/config.md) and an example config [here](https://github.com/microsoft/sca-fuzzer/tree/main/demo/big-fuzz.yaml)), and launch the fuzzer.
|
||||
|
||||
Below is a example launch command, which will start a 24-hour fuzzing session, with 100 input classes per test case, and which uses [big-fuzz.yaml](https://github.com/microsoft/sca-fuzzer/tree/main/demo/big-fuzz.yaml) configuration:
|
||||
```shell
|
||||
|
|
@ -207,9 +207,9 @@ rvzr minimize -s base.json -c violation-<timestamp>/minimize.yaml -g violation-<
|
|||
```
|
||||
|
||||
The result of minimization will be stored in `violation-<timestamp>/minimized.asm`.
|
||||
If the result is still too complicated, try [other minimization passes](minimization.md).
|
||||
If the result is still too complicated, try [other minimization passes](user/minimization.md).
|
||||
|
||||
The further analysis is manual; you can find an example in [this guide](fuzzing-guide.md).
|
||||
The further analysis is manual; you can find an example in [this guide](user/fuzzing-guide.md).
|
||||
|
||||
## Need Help with Revizor?
|
||||
|
||||
|
|
|
|||
|
|
@ -9,25 +9,20 @@ For information on configuration files, see the [configuration documentation](co
|
|||
|
||||
## Modes
|
||||
|
||||
Revizor can run in one of multiple "modes":
|
||||
|
||||
* **Fuzzing mode** is revizor's main form of execution.
|
||||
In this mode, revizor generates random test cases, tests them on the target CPU and the model,
|
||||
and checks for contract violations.
|
||||
* **Template fuzzing mode** is a variant of fuzzing mode that uses a template to generate test cases.
|
||||
* **Reproduce mode** is a variant of fuzzing mode that attempts to reproduce a violation found in a previous run.
|
||||
* **Minimize mode** accepts a test case and attempts to simplify it by applying a series of passes.
|
||||
|
||||
The command line options depend on the selected mode of operation (see [modes page](modes.md) for their descriptions).
|
||||
To select a mode on the command-line, begin your command with:
|
||||
|
||||
```shell
|
||||
rvzr MODE # ... arguments go here
|
||||
|
||||
# Where MODE can be:
|
||||
# fuzz for fuzzing mode
|
||||
# tfuzz for template fuzzing mode
|
||||
# reproduce for reproduce mode
|
||||
# minimize for test case minimization mode
|
||||
# fuzz fuzzing mode
|
||||
# tfuzz template fuzzing mode
|
||||
# reproduce reproduce mode
|
||||
# minimize test case minimization mode
|
||||
# analyse stand-alone trace analysis mode
|
||||
# generate stand-alone generation mode
|
||||
# download_spec call the script that downloads the instruction set specification
|
||||
```
|
||||
|
||||
## Fuzzing Mode
|
||||
|
|
@ -104,4 +99,77 @@ The following command-line arguments are supported in `reproduce` mode:
|
|||
|
||||
## Minimize Mode
|
||||
|
||||
Minimize mode is described in detail in the [minimization documentation](minimization.md).
|
||||
The following command-line arguments are supported in `minimize` mode.
|
||||
See also the [minimization documentation](minimization.md) for a list of available minimization passes.
|
||||
|
||||
```
|
||||
-h, --help show this help message and exit
|
||||
-c CONFIG, --config CONFIG
|
||||
Path to the configuration file (YAML) that will be used during fuzzing.
|
||||
-I INCLUDE_DIR, --include-dir INCLUDE_DIR
|
||||
Path to the directory containing configuration files that included by the main configuration file (received
|
||||
via --config).
|
||||
-s INSTRUCTION_SET, --instruction-set INSTRUCTION_SET
|
||||
Path to the instruction set specification (JSON) file.
|
||||
--testcase TESTCASE, -t TESTCASE
|
||||
Path to the test case program that needs to be minimized.
|
||||
-i NUM_INPUTS, --num-inputs NUM_INPUTS
|
||||
Number of inputs to the program that will be used during minimization.
|
||||
--testcase-outfile TESTCASE_OUTFILE, -o TESTCASE_OUTFILE
|
||||
Output path for the minimized test case program.
|
||||
--input-outdir INPUT_OUTDIR
|
||||
Output directory for storing minimized inputs.
|
||||
--num-attempts NUM_ATTEMPTS
|
||||
Number of attempts to minimize the test case.
|
||||
--enable-<pass> Enable a specific pass during minimization.
|
||||
```
|
||||
|
||||
## Stand-alone Trace Analysis Mode
|
||||
|
||||
The following command-line arguments are supported in `analyse` mode:
|
||||
|
||||
```
|
||||
-h, --help show this help message and exit
|
||||
-c CONFIG, --config CONFIG
|
||||
Path to the configuration file (YAML) that will be used during fuzzing.
|
||||
-I INCLUDE_DIR, --include-dir INCLUDE_DIR
|
||||
Path to the directory containing configuration files that included by the main configuration file (received
|
||||
via --config).
|
||||
-s INSTRUCTION_SET, --instruction-set INSTRUCTION_SET
|
||||
Path to the instruction set specification (JSON) file.
|
||||
--ctraces CTRACES
|
||||
--htraces HTRACES
|
||||
```
|
||||
|
||||
## Stand-alone Generation Mode
|
||||
|
||||
The following command-line arguments are supported in `generate` mode:
|
||||
|
||||
```
|
||||
-h, --help show this help message and exit
|
||||
-c CONFIG, --config CONFIG
|
||||
Path to the configuration file (YAML) that will be used during fuzzing.
|
||||
-I INCLUDE_DIR, --include-dir INCLUDE_DIR
|
||||
Path to the directory containing configuration files that included by the main configuration file (received
|
||||
via --config).
|
||||
-s INSTRUCTION_SET, --instruction-set INSTRUCTION_SET
|
||||
Path to the instruction set specification (JSON) file.
|
||||
-r SEED, --seed SEED Add seed to generate test case.
|
||||
-n NUM_TEST_CASES, --num-test-cases NUM_TEST_CASES
|
||||
Number of test cases.
|
||||
-i NUM_INPUTS, --num-inputs NUM_INPUTS
|
||||
Number of inputs per test case.
|
||||
-w WORKING_DIRECTORY, --working-directory WORKING_DIRECTORY
|
||||
--permit-overwrite Permit overwriting existing files.
|
||||
```
|
||||
|
||||
## Download Instruction Set Specification
|
||||
|
||||
The following command-line arguments are supported in `download_spec` mode:
|
||||
|
||||
```
|
||||
-h, --help show this help message and exit
|
||||
-a ARCHITECTURE, --architecture ARCHITECTURE The ISA to download the specification for (e.g., x86-64)
|
||||
--outfile OUTFILE, -o OUTFILE The destination file to save the downloaded specification.
|
||||
--extensions [EXTENSIONS ...] List of ISA extensions to include in the specification (e.g., SSE, VTX)
|
||||
```
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
# Modes of Operation
|
||||
|
||||
Revizor supports several modes of operation, each targeting a different use cases.
|
||||
The selection of the mode is described in the [CLI documentation](cli.md).
|
||||
Below is a brief description of each mode.
|
||||
|
||||
|
||||
| Mode | CLI Key | Use Case | Description |
|
||||
| ---------------- | ------------- | ------------------------ | ---------------------------------------------------------------------------------------------------------- |
|
||||
| Fuzzing | fuzz | General Testing | Test a CPU against a contract model. Test cases generated randomly |
|
||||
| Template Fuzzing | tfuzz | Targeted Testing | Test a CPU against a contract model. Test cases generated based on a template |
|
||||
| Reproduce | reproduce | Reproducing a Violation | Reproduce a violation found by fuzzing OR run a manually-written test case |
|
||||
| Minimization | minimize | Violation Simplification | Simplify a test case by applying a series of simplification passes to the test case program and its inputs |
|
||||
| Trace Analysis | analyse | Stand-alone Analysis | Analyze pre-recorded traces for violations |
|
||||
| Generation | generate | Stand-alone Generation | Only generate test cases, without testing them |
|
||||
| ISA Spec Install | download_spec | Tool Installation | Call a script that downloads the instruction set specification |
|
||||
|
||||
|
||||
## Fuzzing and Template Fuzzing Modes
|
||||
|
||||
Two main modes of operation in revizor are fuzzing and template fuzzing.
|
||||
These modes are used to test a CPU against a contract model.
|
||||
In both modes, revizor generates test cases and executes them on the target CPU and the model, records the corresponding traces, and checks if the hardware traces contain the same (or less) information as the contract traces.
|
||||
|
||||
In the fuzzing mode, test cases are generated randomly, with the instruction set and size of test cases defined by the config file.
|
||||
This mode is used for broad testing of the CPU.
|
||||
|
||||
In the template fuzzing mode, test cases are generated based on a template:
|
||||
The generator takes an assembly template as an input, and produces a test case by expanding the `random_instructions` macro in the template.
|
||||
This mode is used to narrow down the fuzzing space and focus on specific scenarios, such as testing microarchitectural patches or certain interactions between actors.
|
||||
|
||||
## Reproduce
|
||||
|
||||
In this mode, Revizor loads a test case from a set of files and runs a single round of the fuzzer with this test case.
|
||||
The test case is usually a violation previously found in the (template) fuzzing mode, but it can also be written manually.
|
||||
|
||||
There are three main use cases for this mode:
|
||||
|
||||
1. **Analysis of the violation**: to understand the root cause of the violation, the user may manually modify the test case and re-run it in the reproduce mode to see if the violation is still present.
|
||||
2. **Reproducibility check**: to check if a violation is reproducible on different CPUs, or on different configurations of the same CPU (e.g., after a microcode patch has been applied).
|
||||
3. **Manual testing**: to test a manually-written test case.
|
||||
|
||||
## Minimization
|
||||
|
||||
In this mode, Revizor takes a test case that causes a violation and applies a series of simplification passes to the test case program and its inputs.
|
||||
The goal is to reduce the test case to its minimal form to simplify the root cause analysis of the violation.
|
||||
Revizor supports an extensive list of passes, described in the [minimization documentation](minimization.md).
|
||||
|
||||
## Stand-alone Interfaces
|
||||
|
||||
The `analyse` and `generate` modes are used to perform stand-alone access to modules of Revizor.
|
||||
In the `analyse` mode, the user can analyze pre-recorded traces for violations.
|
||||
In the `generate` mode, the user can generate test cases without testing them.
|
||||
|
||||
## ISA Spec Install
|
||||
|
||||
The `download_spec` mode isn't used for testing, but rather for tool installation.
|
||||
It provides an interface to download, parse, and store the instruction specifications for the tested ISA in the JSON format.
|
||||
|
|
@ -40,10 +40,11 @@ nav:
|
|||
- Home: index.md
|
||||
- Quick Start: quick-start.md
|
||||
- User Documentation:
|
||||
- Command Line Interface: cli.md
|
||||
- Configuration Options: config.md
|
||||
- Fuzzing Guide: fuzzing-guide.md
|
||||
- Minimization Passes: minimization.md
|
||||
- Command Line Interface: user/cli.md
|
||||
- Modes of Operation: user/modes.md
|
||||
- Configuration Options: user/config.md
|
||||
- Minimization Passes: user/minimization.md
|
||||
- Fuzzing Guide: user/fuzzing-guide.md
|
||||
- Developer Documentation:
|
||||
- Developer Guide: development.md
|
||||
- How Revizor Works: how-revizor-works.md
|
||||
|
|
|
|||
|
|
@ -22,7 +22,7 @@ class DocumentationTest(unittest.TestCase):
|
|||
Test if the documentation contains all the config options.
|
||||
"""
|
||||
# get the text of the config documentation
|
||||
with open(DOC_DIR / "config.md", "r") as f:
|
||||
with open(DOC_DIR / "user/config.md", "r") as f:
|
||||
doc_text = f.read()
|
||||
|
||||
# get a list of config options
|
||||
|
|
@ -41,7 +41,7 @@ class DocumentationTest(unittest.TestCase):
|
|||
Test if the documentation contains all possible values for the config options.
|
||||
"""
|
||||
# get the text of the config documentation
|
||||
with open(DOC_DIR / "config.md", "r") as f:
|
||||
with open(DOC_DIR / "user/config.md", "r") as f:
|
||||
doc_text = f.readlines()
|
||||
|
||||
# build a map of config options to their possible values in the doc
|
||||
|
|
|
|||
Загрузка…
Ссылка в новой задаче