зеркало из https://github.com/Azure/counterfit.git
Public internal CF updates to public (#51)
### Description
#### 1. Counterfit as a package
Update Counterfit to support import as a module. User can now run interact with different attacks and models directly from Python.
* Update to allow import of Counterfit as a package.
``` python
import counterfit
import counterfit.targets as cf_targets
digits_target = cf_targets.Digits()
digits_target.load()
cf_attack = counterfit.Counterfit.build_attack(digits_target, 'hop_skip_jump')
results = counterfit.Counterfit.run_attack(cf_attack)
```
#### 2. Update Counterfit CLI
* Update Counterfit Command Line Interface (CLI) to simplify the usage process
* The `interact` command has been replaced with the more idiomatic `set_target` command.
* The `use` command has been replaced with the more idiomatic `set_attack` command.
* The `set` command has been replaced with the more idiomatic `set_params` command.
* The `load` command has been deprecated. Frameworks and attacks are now automatically loaded by the CLI.
``` bash
$ counterfit
__ _____ __
_________ __ ______ / /____ _____/ __(_) /_
/ ___/ __ \/ / / / __ \/ __/ _ \/ ___/ /_/ / __/
/ /__/ /_/ / /_/ / / / / /_/ __/ / / __/ / /
\___/\____/\__,_/_/ /_/\__/\___/_/ /_/ /_/\__/
Version: 1.1.0
counterfit> list targets
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name ┃ Model Type ┃ Data Type ┃ Input Shape ┃ # Samples ┃ Endpoint ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ cart_pole │ closed-box │ tabular │ (1080000,) │ 0 │ cartpole_dqn_10000.pt.gz │
│ cart_pole_initstate │ closed-box │ tabular │ (4,) │ 0 │ cartpole_dqn_10000.pt.gz │
│ creditfraud │ closed-box │ tabular │ (30,) │ 0 │ creditfraud/creditfraud_sklearn_pipeline.pkl │
│ digits_keras │ closed-box │ image │ (28, 28, 1) │ 0. │ digits_keras/mnist_model.h5 │
│ digits_mlp │ closed-box │ image │ (1, 28, 28) │ 0 │ digits_mlp/mnist_sklearn_pipeline.pkl │
│ movie_reviews │ closed-box │ text │ (1,) │ 0. │ movie_reviews/movie_reviews_sentiment_analysis.pt │
│ satellite │ closed-box │ image │ (3, 256, 256) │ 0 │ satellite/satellite-image-params-airplane-stadium.h5 │
└─────────────────────┴────────────┴───────────┴───────────────┴───────────┴──────────────────────────────────────────────────────┘
counterfit> set_target satellite
satellite> set_attack hop_skip_jump
[+] success: Using fb58020f
satellite>HopSkipJump:fb58020f> show info
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Attack Field ┃ Description ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ Name │ hop_skip_jump │
│ Type │ closed-box │
│ Category │ evasion │
│ Tags │ image, tabular │
│ Framework │ art │
│ Docs │ Implementation of the HopSkipJump attack from Jianbo et al. (2019). This is a powerful closed-box attack that only requires final class prediction, and │
│ │ is an advanced version of the boundary attack. | Paper link: https://arxiv.org/abs/1904.02144 │
└──────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
satellite>HopSkipJump:fb58020f> run
HopSkipJump: 0%| | 0/1 [00:00<?, ?it/sFailed to draw a random image that is adversarial, attack failed.
HopSkipJump: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:04<00:00, 4.88s/it]
[+] success: Attack completed fb58020f
```
#### 3. Update to internal architecture
* Internal Counterfit organization is updated to be more extensible and easier to understand.
* Update targets and attacks naming convention
* Adds framework for integration tests before pull-requests can be merged to `main`
* Add support for Cart Pole targets to attack Reinforcement Learning (RL) models.
This commit is contained in:
Gary
GitHub
Родитель
289c0e91b6
Коммит
353dca2b71
|
|
@ -0,0 +1,2 @@
|
|||
# Owners for the Counterfit repository. Approval necessary to merge PRs.
|
||||
* @Azure/trustworthy-ml-admin
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
#
|
||||
# Test to run the Corrupted Replay Attack (CRA).
|
||||
#
|
||||
# Usage:
|
||||
#
|
||||
# conda activate counterfit
|
||||
# ./cra_test.exp "$(uname)" 120
|
||||
|
||||
# Get os from cli input
|
||||
set os [lindex $argv 0]
|
||||
|
||||
# Get seconds from cli input
|
||||
set timeout [lindex $argv 1]
|
||||
|
||||
# Get terminal.py path from cli input
|
||||
set termpy [lindex $argv 2]
|
||||
|
||||
if { $os == "Linux" } {
|
||||
spawn xvfb-run -a python $termpy
|
||||
} else {
|
||||
spawn python $termpy
|
||||
}
|
||||
|
||||
expect "counterfit>"
|
||||
send "set_target cart_pole\r"
|
||||
send "set_attack hop_skip_jump\r"
|
||||
send "set_params --max_eval 100 --init_eval 10 --init_size 10\r"
|
||||
send "run\r"
|
||||
|
||||
expect eof;
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
#
|
||||
# Test to run the Initial State Perturbation Attack.
|
||||
#
|
||||
# Usage:
|
||||
#
|
||||
# conda activate counterfit
|
||||
# ./ispa_test.exp "$(uname)" 120
|
||||
|
||||
# Get os from cli input
|
||||
set os [lindex $argv 0]
|
||||
|
||||
# Get seconds from cli input
|
||||
set timeout [lindex $argv 1]
|
||||
|
||||
# Get terminal.py path from cli input
|
||||
set termpy [lindex $argv 2]
|
||||
|
||||
if { $os == "Linux" } {
|
||||
spawn xvfb-run -a python $termpy
|
||||
} else {
|
||||
spawn python $termpy
|
||||
}
|
||||
|
||||
expect "counterfit>"
|
||||
send "set_target creditfraud\r"
|
||||
expect "creditfraud>"
|
||||
send "set_attack hop_skip_jump\r"
|
||||
expect "success:"
|
||||
send "run\r"
|
||||
expect "success:"
|
||||
send "show results\r"
|
||||
expect "1/1"
|
||||
send "exit counterfit\r"
|
||||
send "y\r"
|
||||
|
||||
expect eof;
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
#
|
||||
# Test to run the Initial State Perturbation Attack.
|
||||
#
|
||||
# Usage:
|
||||
#
|
||||
# conda activate counterfit
|
||||
# ./ispa_test.exp "$(uname)" 120
|
||||
|
||||
# Get os from cli input
|
||||
set os [lindex $argv 0]
|
||||
|
||||
# Get seconds from cli input
|
||||
set timeout [lindex $argv 1]
|
||||
|
||||
# Get terminal.py path from cli input
|
||||
set termpy [lindex $argv 2]
|
||||
|
||||
if { $os == "Linux" } {
|
||||
spawn xvfb-run -a python $termpy
|
||||
} else {
|
||||
spawn python $termpy
|
||||
}
|
||||
|
||||
expect "counterfit>"
|
||||
send "set_target digits_keras\r"
|
||||
expect "digits_keras>"
|
||||
send "set_attack hop_skip_jump\r"
|
||||
expect "success:"
|
||||
send "run\r"
|
||||
expect "success:"
|
||||
send "show results\r"
|
||||
expect "1/1"
|
||||
send "exit counterfit\r"
|
||||
send "y\r"
|
||||
expect eof;
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
#
|
||||
# Test the counterfit installation.
|
||||
#
|
||||
# Usage:
|
||||
#
|
||||
# conda activate counterfit
|
||||
# ./install_test.exp "$(uname)" 120
|
||||
|
||||
# Get os from cli input
|
||||
set os [lindex $argv 0]
|
||||
|
||||
# Get seconds from cli input
|
||||
set timeout [lindex $argv 1]
|
||||
|
||||
# Get terminal.py path from cli input
|
||||
set termpy [lindex $argv 2]
|
||||
|
||||
if { $os == "Linux" } {
|
||||
spawn xvfb-run -a python $termpy
|
||||
} else {
|
||||
spawn python $termpy
|
||||
}
|
||||
|
||||
expect "counterfit>"
|
||||
send "exit counterfit\r"
|
||||
send "y\r"
|
||||
|
||||
expect eof;
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
#
|
||||
# Test to run the Initial State Perturbation Attack.
|
||||
#
|
||||
# Usage:
|
||||
#
|
||||
# conda activate counterfit
|
||||
# ./ispa_test.exp "$(uname)" 120
|
||||
|
||||
# Get os from cli input
|
||||
set os [lindex $argv 0]
|
||||
|
||||
# Get seconds from cli input
|
||||
set timeout [lindex $argv 1]
|
||||
|
||||
# Get terminal.py path from cli input
|
||||
set termpy [lindex $argv 2]
|
||||
|
||||
if { $os == "Linux" } {
|
||||
spawn xvfb-run -a python $termpy
|
||||
} else {
|
||||
spawn python $termpy
|
||||
}
|
||||
|
||||
expect "counterfit>"
|
||||
send "set_target cart_pole_initstate\r"
|
||||
send "set_attack hop_skip_jump\r"
|
||||
send "set --max_eval 100 --init_eval 10 --init_size 10\r"
|
||||
send "run\r"
|
||||
|
||||
expect eof;
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
#
|
||||
# Test to run the Initial State Perturbation Attack.
|
||||
#
|
||||
# Usage:
|
||||
#
|
||||
# conda activate counterfit
|
||||
# ./ispa_test.exp "$(uname)" 120
|
||||
|
||||
# Get os from cli input
|
||||
set os [lindex $argv 0]
|
||||
|
||||
# Get seconds from cli input
|
||||
set timeout [lindex $argv 1]
|
||||
|
||||
# Get terminal.py path from cli input
|
||||
set termpy [lindex $argv 2]
|
||||
|
||||
if { $os == "Linux" } {
|
||||
spawn xvfb-run -a python $termpy
|
||||
} else {
|
||||
spawn python $termpy
|
||||
}
|
||||
|
||||
expect "counterfit>"
|
||||
send "set_target satellite\r"
|
||||
expect "satellite>"
|
||||
send "set_attack hop_skip_jump\r"
|
||||
expect "success:"
|
||||
send "run\r"
|
||||
expect "success:"
|
||||
send "exit counterfit\r"
|
||||
send "y\r"
|
||||
|
||||
expect eof;
|
||||
|
|
@ -5,9 +5,9 @@ on:
|
|||
push:
|
||||
branches: [main]
|
||||
|
||||
# Run once a week (see https://crontab.guru)
|
||||
# Run at 4am everyday (see https://crontab.guru)
|
||||
schedule:
|
||||
- cron: "0 0 * * 0"
|
||||
- cron: "0 4 * * *"
|
||||
|
||||
env:
|
||||
PYTHON_VERSION: 3.8.8
|
||||
|
|
@ -20,14 +20,12 @@ jobs:
|
|||
runs-on: ${{ matrix.os }}
|
||||
strategy:
|
||||
matrix:
|
||||
os: [ubuntu-latest, macos-latest]
|
||||
os: [ubuntu-latest]
|
||||
steps:
|
||||
- name: Set up git repository
|
||||
uses: actions/checkout@v2
|
||||
with:
|
||||
ref: main
|
||||
uses: actions/checkout@v3.1.0
|
||||
|
||||
- name: Install dependencies
|
||||
- name: Install OS dependencies
|
||||
run: |
|
||||
if [ "${RUNNER_OS}" == "Linux" ]; then
|
||||
sudo apt update
|
||||
|
|
@ -46,17 +44,14 @@ jobs:
|
|||
exit 1
|
||||
fi
|
||||
|
||||
- name: Populate action files
|
||||
- name: Populate acceptance tests
|
||||
run: |
|
||||
if [ "${RUNNER_OS}" == "Linux" ]; then
|
||||
cp .github/files/environment-Linux.yml ./environment.yml
|
||||
elif [ "${RUNNER_OS}" == "macOS" ]; then
|
||||
cp .github/files/environment-Darwin.yml ./environment.yml
|
||||
else
|
||||
echo "${RUNNER_OS} not supported"
|
||||
exit 1
|
||||
fi
|
||||
cp .github/files/*.exp .
|
||||
cp "${GITHUB_WORKSPACE}/.github/helpers/quality-gates/install_test.exp" ./install_test.exp
|
||||
cp "${GITHUB_WORKSPACE}/.github/helpers/quality-gates/cra_test.exp" ./cra_test.exp
|
||||
cp "${GITHUB_WORKSPACE}/.github/helpers/quality-gates/ispa_test.exp" ./ispa_test.exp
|
||||
cp "${GITHUB_WORKSPACE}/.github/helpers/quality-gates/satellite_hsj_test.exp" ./satellite_hsj_test.exp
|
||||
cp "${GITHUB_WORKSPACE}/.github/helpers/quality-gates/creditfraud_hsj_test.exp" ./creditfraud_hsj_test.exp
|
||||
cp "${GITHUB_WORKSPACE}/.github/helpers/quality-gates/digits_hsj_test.exp" ./digits_hsj_test.exp
|
||||
|
||||
- name: Setup Anaconda
|
||||
uses: conda-incubator/setup-miniconda@v2
|
||||
|
|
@ -64,9 +59,8 @@ jobs:
|
|||
activate-environment: counterfit
|
||||
auto-update-conda: true
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
environment-file: environment.yml
|
||||
|
||||
- name: Setup conda
|
||||
- name: Setup Conda
|
||||
run: |
|
||||
if [ "${RUNNER_OS}" == "Linux" ]; then
|
||||
echo 'conda activate "counterfit"' >> "${HOME}/.profile"
|
||||
|
|
@ -77,61 +71,112 @@ jobs:
|
|||
exit 1
|
||||
fi
|
||||
|
||||
- name: Setup test environment
|
||||
- name: Install Pycld2
|
||||
run: |
|
||||
pip install https://github.com/aboSamoor/pycld2/zipball/e3ac86ed4d4902e912691c1531d0c5645382a726
|
||||
|
||||
- name: Setup counterfit
|
||||
shell: bash -l {0}
|
||||
run: |
|
||||
pip install -r requirements.txt
|
||||
echo "PYTHON_PATH=$(which python)" >> $GITHUB_ENV
|
||||
|
||||
if [ "${RUNNER_OS}" == "Linux" ]; then
|
||||
source "${HOME}/.profile"
|
||||
conda activate counterfit
|
||||
cd "${GITHUB_WORKSPACE}/"
|
||||
pip install --no-input .[dev]
|
||||
elif [ "${RUNNER_OS}" == "macOS" ]; then
|
||||
conda activate counterfit
|
||||
cd "${GITHUB_WORKSPACE}/"
|
||||
pip install --no-input .[dev]
|
||||
else
|
||||
echo "${RUNNER_OS} not supported"
|
||||
exit 1
|
||||
fi
|
||||
- name: Test counterfit install
|
||||
shell: bash -l {0}
|
||||
run: |
|
||||
if [ "${RUNNER_OS}" == "Linux" ]; then
|
||||
source "${HOME}/.profile"
|
||||
conda activate counterfit
|
||||
expect -d ./install_test.exp Linux ${{ env.SECONDS }}
|
||||
expect -d ./install_test.exp Linux ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
elif [ "${RUNNER_OS}" == "macOS" ]; then
|
||||
conda activate counterfit
|
||||
expect -d ./install_test.exp Darwin ${{ env.SECONDS }}
|
||||
expect -d ./install_test.exp Darwin ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
else
|
||||
echo "${RUNNER_OS} not supported"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Run Initial State Perturbation Attack test for n minutes
|
||||
- name: Test Initial State Perturbation Attack for n minutes
|
||||
shell: bash -l {0}
|
||||
run: |
|
||||
if [ "${RUNNER_OS}" == "Linux" ]; then
|
||||
source "${HOME}/.profile"
|
||||
conda activate counterfit
|
||||
expect -d ./ispa_test.exp Linux ${{ env.SECONDS }}
|
||||
expect -d ./ispa_test.exp Linux ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
elif [ "${RUNNER_OS}" == "macOS" ]; then
|
||||
conda activate counterfit
|
||||
expect -d ./ispa_test.exp Darwin ${{ env.SECONDS }}
|
||||
expect -d ./ispa_test.exp Darwin ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
else
|
||||
echo "${RUNNER_OS} not supported"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Run Corrupted Replay Attack (CRA) test for n minutes
|
||||
- name: Test Corrupted Replay Attack (CRA) for n minutes
|
||||
shell: bash -l {0}
|
||||
run: |
|
||||
if [ "${RUNNER_OS}" == "Linux" ]; then
|
||||
source "${HOME}/.profile"
|
||||
conda activate counterfit
|
||||
expect -d ./cra_test.exp Linux ${{ env.SECONDS }}
|
||||
expect -d ./cra_test.exp Linux ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
elif [ "${RUNNER_OS}" == "macOS" ]; then
|
||||
conda activate counterfit
|
||||
expect -d ./cra_test.exp Darwin ${{ env.SECONDS }}
|
||||
expect -d ./cra_test.exp Darwin ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
else
|
||||
echo "${RUNNER_OS} not supported"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Test Hop Skip Jump Attack (HSJ) attack on Satellite target for n minutes
|
||||
shell: bash -l {0}
|
||||
run: |
|
||||
if [ "${RUNNER_OS}" == "Linux" ]; then
|
||||
source "${HOME}/.profile"
|
||||
conda activate counterfit
|
||||
expect -d ./satellite_hsj_test.exp Linux ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
elif [ "${RUNNER_OS}" == "macOS" ]; then
|
||||
conda activate counterfit
|
||||
expect -d ./satellite_hsj_test.exp Darwin ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
else
|
||||
echo "${RUNNER_OS} not supported"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Test Hop Skip Jump Attack (HSJ) attack on CreditFraud target for n minutes
|
||||
shell: bash -l {0}
|
||||
run: |
|
||||
if [ "${RUNNER_OS}" == "Linux" ]; then
|
||||
source "${HOME}/.profile"
|
||||
conda activate counterfit
|
||||
expect -d ./creditfraud_hsj_test.exp Linux ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
elif [ "${RUNNER_OS}" == "macOS" ]; then
|
||||
conda activate counterfit
|
||||
expect -d ./creditfraud_hsj_test.exp Darwin ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
else
|
||||
echo "${RUNNER_OS} not supported"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Test Hop Skip Jump Attack (HSJ) attack on Digits Keras
|
||||
shell: bash -l {0}
|
||||
run: |
|
||||
source "${HOME}/.profile"
|
||||
conda activate counterfit
|
||||
expect -d ./digits_hsj_test.exp Linux ${{ env.SECONDS }} "${GITHUB_WORKSPACE}/examples/terminal/terminal.py"
|
||||
|
||||
- name: Report failure
|
||||
uses: nashmaniac/create-issue-action@v1.1
|
||||
# Only report failures of pushes (PRs have are visible through the Checks section)
|
||||
# to the default branch
|
||||
# Only report failures of pushes (PRs have are visible through the Checks section) to the default branch
|
||||
if: failure() && github.event_name == 'main' && github.ref == 'refs/heads/main'
|
||||
with:
|
||||
title: 🐛 Coverage report failed for ${{ github.sha }}
|
||||
|
|
|
|||
|
|
@ -1,3 +1,7 @@
|
|||
# Counterfit
|
||||
results
|
||||
prof
|
||||
|
||||
# Byte-compiled / optimized / DLL files
|
||||
*__pycache__/
|
||||
*.py[cod]
|
||||
|
|
@ -116,3 +120,9 @@ counterfit/targets/*/results
|
|||
counterfit/docs/source/_autosummary
|
||||
counterfit/docs/build/doctrees/_autosummary
|
||||
docs/build/doctrees
|
||||
|
||||
# build folder
|
||||
build
|
||||
|
||||
# dev env
|
||||
cf-venv
|
||||
|
|
|
|||
|
|
@ -0,0 +1,17 @@
|
|||
repos:
|
||||
- repo: https://github.com/pre-commit/pre-commit-hooks
|
||||
rev: v3.2.0
|
||||
hooks:
|
||||
- id: trailing-whitespace
|
||||
- id: end-of-file-fixer
|
||||
- id: check-yaml
|
||||
- id: check-added-large-files
|
||||
- id: requirements-txt-fixer
|
||||
- id: no-commit-to-branch
|
||||
args: [--branch, main]
|
||||
- id: double-quote-string-fixer
|
||||
- repo: https://github.com/pycqa/flake8
|
||||
rev: '5.0.4'
|
||||
hooks:
|
||||
- id: flake8
|
||||
args: ['--max-line-length=120']
|
||||
178
README.md
178
README.md
|
|
@ -18,15 +18,15 @@
|
|||
```
|
||||
|
||||
## About
|
||||
|
||||
Counterfit is a command-line tool and generic automation layer for assessing the security of machine learning systems.
|
||||
Counterfit is a generic automation layer for assessing the security of machine learning systems. It brings several existing adversarial frameworks under one tool, or allows users to create their own.
|
||||
|
||||
### Requirements
|
||||
|
||||
- Python 3.7 or 3.8
|
||||
- Ubuntu 18.04+
|
||||
- Python 3.8
|
||||
- Windows is supported by Counterfit, but not necessarily officially supported by each individual framework.
|
||||
- On Windows the [Visual C++ 2019 redistributable](https://support.microsoft.com/en-us/help/2977003/the-latest-supported-visual-c-downloads) is required
|
||||
|
||||
## Getting Started
|
||||
## Quick Start
|
||||
|
||||
Choose one of these methods to get started quickly:
|
||||
|
||||
|
|
@ -52,38 +52,100 @@ az container exec --resource-group RESOURCE_GROUP --name counterfit --exec-comma
|
|||
|
||||
4. Within the container, launch Counterfit.
|
||||
|
||||
```
|
||||
python counterfit.py
|
||||
```
|
||||
### Option 2: Set up an Anaconda Python environment and install locally
|
||||
|
||||
### Option 2: Setup an Anaconda Python environment and install locally
|
||||
|
||||
1. Install [Anaconda Python](https://www.anaconda.com/products/individual) and [git](https://git-scm.com/downloads).
|
||||
2. Clone this repository.
|
||||
|
||||
```
|
||||
git clone https://github.com/Azure/counterfit.git
|
||||
```
|
||||
|
||||
3. Open an Anaconda shell and create a virtual environment and dependencies.
|
||||
|
||||
```
|
||||
#### Installation with Python virtual environment
|
||||
```bash
|
||||
sudo apt install python3.8 python3.8-venv
|
||||
python -m venv counterfit
|
||||
git clone -b main https://github.com/Azure/counterfit.git
|
||||
cd counterfit
|
||||
conda create --yes -n counterfit python=3.8.8
|
||||
pip install .[dev]
|
||||
python -c "import nltk; nltk.download('stopwords')"
|
||||
```
|
||||
|
||||
#### Installation with Conda
|
||||
|
||||
```bash
|
||||
conda update -c conda-forge --all -y
|
||||
conda create --yes -n counterfit python=3.8.0
|
||||
conda activate counterfit
|
||||
pip install -r requirements.txt
|
||||
export PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
|
||||
git clone -b main https://github.com/Azure/counterfit.git
|
||||
cd counterfit
|
||||
pip install .[dev]
|
||||
python -c "import nltk; nltk.download('stopwords')"
|
||||
```
|
||||
|
||||
4. Launch Counterfit.
|
||||
To start the Counterfit terminal, run `counterfit` from your Windows or Linux shell.
|
||||
```bash
|
||||
$ counterfit
|
||||
|
||||
__ _____ __
|
||||
_________ __ ______ / /____ _____/ __(_) /_
|
||||
/ ___/ __ \/ / / / __ \/ __/ _ \/ ___/ /_/ / __/
|
||||
/ /__/ /_/ / /_/ / / / / /_/ __/ / / __/ / /
|
||||
\___/\____/\__,_/_/ /_/\__/\___/_/ /_/ /_/\__/
|
||||
|
||||
Version: 1.1.0
|
||||
|
||||
|
||||
counterfit>
|
||||
```
|
||||
python counterfit.py
|
||||
|
||||
Alternatively, you can also import the counterfit module from within you Python code.
|
||||
```python
|
||||
import counterfit
|
||||
import counterfit.targets as targets
|
||||
|
||||
|
||||
target = targets.CreditFraud()
|
||||
target.load()
|
||||
attack_name = 'hop_skip_jump'
|
||||
new_attack = counterfit.Counterfit.build_attack(target, attack_name)
|
||||
results = counterfit.Counterfit.run_attack(new_attack)
|
||||
```
|
||||
|
||||
See the [Counterfit examples README.md](examples/README.md) for more information.
|
||||
|
||||
Notes:
|
||||
- Windows requires C++ build tools
|
||||
- If textattack has been installed, it will initialize by downloading nltk data
|
||||
|
||||
|
||||
## Attack Support
|
||||
|
||||
Each of the Counterfit targets supports a different data type (i.e., text,
|
||||
tabular, and image). For an attack to be compatible, it has to be able to work
|
||||
on that type of data as well.
|
||||
|
||||
For example, Hop Skip Jump, is an evasion and closed-box attack that can be used
|
||||
for image and tabular data types. As such, it will be able to be used against
|
||||
Digits Keras (because it accepts images as input) but not Movie Reviews (because
|
||||
it accepts text as input). It's important to ensure that the target supports the
|
||||
specific attack before running an attack.
|
||||
|
||||
To get a full view of the attack and targets, run the `list targets` and `list
|
||||
attacks` command.
|
||||
|
||||
- **Text Targets**: movie_reviews
|
||||
- **Text Attacks**: a2t_yoo_2021, bae_garg_2019, bert_attack_li_2020, checklist_ribeiro_2020, clare_li_2020, deepwordbug_gao_2018, faster_genetic_algorithm_jia_2019, genetic_algorithm_alzantot_2018, hotflip_ebrahimi_2017, iga_wang_2019, input_reduction_feng_2018, kuleshov_2017, morpheus_tan_2020, pruthi_2019, pso_zang_2020, pwws_ren_2019, seq2sick_cheng_2018_blackbox, textbugger_li_2018, textfooler_jin_2019,
|
||||
|
||||
|
||||
- **Image Targets**: digits_keras, digits_mlp, satellite
|
||||
- **Image Attacks**: boundary, carlini, copycat_cnn, deepfool, elastic_net, functionally_equivalent_extraction, hop_skip_jump, knockoff_nets, label_only_boundary_distance, mi_face, newtonfool, pixel_threshold, projected_gradient_descent_numpy, saliency_map, simba, spatial_transformation, universal_perturbation, virtual_adversarial, wasserstein, ApplyLambda, Blur, Brightness, ChangeAspectRatio, ClipImageSize, ColorJitter, Contrast, ConvertColor, Crop, EncodingQuality, Grayscale, HFlip, MemeFormat, Opacity, OverlayEmoji, OverlayOntoScreenshot, OverlayStripes, OverlayText, Pad, PadSquare, PerspectiveTransform, Pixelization, RandomEmojiOverlay, RandomNoise, Resize, Rotate, Saturation, Scale, Sharpen, ShufflePixels, VFlip
|
||||
|
||||
|
||||
- **Tabular Targets**: cart_pole, cart_pole_initstate, creditfraud
|
||||
- **Tabular Attacks**: boundary, carlini, deepfool, elastic_net, functionally_equivalent_extraction, hop_skip_jump, knockoff_nets, label_only_boundary_distance, mi_face, newtonfool, projected_gradient_descent_numpy, saliency_map, spatial_transformation
|
||||
|
||||
|
||||
## Acknowledgments
|
||||
Counterfit leverages excellent open source projects, including,
|
||||
|
||||
- [Adversarial Robustness Toolbox](https://github.com/Trusted-AI/adversarial-robustness-toolbox)
|
||||
- [TextAttack](https://github.com/QData/TextAttack)
|
||||
- [Augly](https://github.com/facebookresearch/AugLy)
|
||||
|
||||
Counterfit leverages excellent open source projects, including, [Adversarial Robustness Toolbox](https://github.com/Trusted-AI/adversarial-robustness-toolbox), [TextAttack](https://github.com/QData/TextAttack), and [Augly](https://github.com/facebookresearch/AugLy)
|
||||
|
||||
## Contributing
|
||||
|
||||
|
|
@ -108,67 +170,5 @@ Use of Microsoft trademarks or logos in modified versions of this project must n
|
|||
Any use of third-party trademarks or logos are subject to those third-party's policies.
|
||||
|
||||
## Contact Us
|
||||
For comments or questions about how to leverage Counterfit, please contact <counterfithelpline@microsoft.com>.
|
||||
|
||||
For comments or questions about how to leverage Counterfit, please contact <counterfithelpline@microsoft.com>.
|
||||
|
||||
# Version 1.0
|
||||
|
||||
First and foremost, the ATML team would like the thank everyone for their support over the last few months. Counterfit recieved a very warm welcome from the community. What started as some simple red team tooling has become a place for collaboration, experiementatation, and of course security assessments. While verson 0.1 was useful, unless a user was familiar with the code, it was admitedly difficult to use beyond it's basic functionality. Users of Counterfit should know that their frustrations with the tool were also our frustrations. While our internal version may have different targets, custom algos, reporting, the public version of Counterfit is ultimately the base of our internal version. For those unfamiliar with infosec, this is a common practice that creates a shared experience. These shared experiences will allow us to communicate and come to a common understanding of risk in the ML space.
|
||||
|
||||
Let's checkout the new digs. We will cover the changes at a high-level and get into details later,
|
||||
|
||||
- Frameworks are a first-class concept.
|
||||
- New logging capabilities
|
||||
- Options structure
|
||||
- New attacks from art, textattack
|
||||
- New attacks via Augly
|
||||
- Various command functionality
|
||||
- Running via run_pyscript
|
||||
- New reporting structure
|
||||
- Python Rich integration
|
||||
- docs and tests
|
||||
|
||||
# Frameworks are a first-class concept
|
||||
|
||||
Frameworks are the drivers behind Counterfit and they provide the functionality for Counterfit. Counterfit now takes a back seat and offloads the majority of work to the framework responsible for an attack. Frameworks are not loaded on start, rather by using the `load` command Like other objects in Counterfit, frameworks are built around their folder structure within the project. Each framework has its own folder under `counterfit/frameworks`.In order to be loaded by Counterfit, a framework should inherit from `counterfit.core.frameworks.Framework`. A framework should also define a number of core functions. These include `load()`, `build()`, `run()`, `check_success()`, `pre_attack_proccessing()`, `post_attack_processing()`. Everything begins and ends with a framework and so in order to add a new framework it is important to be familiar with some Counterfit internals.
|
||||
|
||||
# Python Rich integration
|
||||
|
||||
Thanks to Python Rich, Counterfit has a lot more colors and is generally better looking. Rich requires that everything is string or a "renderable". Be aware of this when using the `logging` module.
|
||||
|
||||
## Options structure
|
||||
|
||||
During `framework.load()` a framework author has the opportunity to set options for an attack via `attack_default_params`. Counterfit uses these to populate the `set` command arguments. Every attack will reflect its own unique options that can be changed with the `set` command, it will also loosely enforce some typing on the arguments. It is advised to handle any options issues in the framework rather than in `set`.
|
||||
|
||||
## Logging structure
|
||||
|
||||
Counterfit injects its own options into the options structure. Options related to logging being `enable_logging` and `logger`. Technically logging is always enabled, and only collects the number of queries sent. To set a logger other than the default logger, use `set --logger json`.
|
||||
|
||||
## New attacks from art, textattack
|
||||
|
||||
Because frameworks are first class concept, Counterfit no longer wraps attacks, rather it depends on the framework code to handle the majority of the attack life-cycle. This means that Counterfit can support the full menu of attacks that the orginial frameworks provided. For example, where Counterfit v0.1 only supported blackbox evasion attacks from the Adversarial Robustness Toolbox, Counterfit v1.0 supports MIFace (blackbox-inversion), KnockOffNets(blackbox-extraction), CariliniWagner(whitebox-evasion), and several others out of the box.
|
||||
|
||||
## New attacks via Augly
|
||||
|
||||
Augly is a powerful data augmentation framework built by Facebook. While not explicilty "adversarial", Counterfit uses Augly to include a new bug class for testing - common corruptions. In terms of implementation, Augly is a good example of how to both use a "config" only load and wrap a class to create a custom attack.
|
||||
|
||||
## Various command functionality
|
||||
|
||||
Most commands remain the same in functionality, however some arguments may have changed.
|
||||
|
||||
- set: Arguments are part of argparse
|
||||
- show attacks: Access historical attacks
|
||||
- reload: frameworks, targets, and commands
|
||||
- exit: target, attack, or counterfit.
|
||||
|
||||
## New reporting structure
|
||||
|
||||
Counterfit comes with some basic reporting functionality, but if there are attacks or datatypes Counterfit does not support for reporting, a user can override them in the framework via `post_attack_processing()`.
|
||||
|
||||
## Running Counterfit via run_pyscript
|
||||
|
||||
The core code and the terminal commands have been decoupled. It is possible to use the cmd2 `run_pyscript` to automate scans.
|
||||
|
||||
## Docs and Tests
|
||||
|
||||
Tests are implement via Pyest and make docs with `counterfit\docs\make html`. Use the `docs` command to start a local server for browsing.
|
||||
|
|
|
|||
|
|
@ -6,9 +6,8 @@ This project uses GitHub Issues to track bugs and feature requests. Please searc
|
|||
issues before filing new issues to avoid duplicates. For new issues, file your bug or
|
||||
feature request as a new Issue.
|
||||
|
||||
For help and questions about using this project, please **REPO MAINTAINER: INSERT INSTRUCTIONS HERE
|
||||
FOR HOW TO ENGAGE REPO OWNERS OR COMMUNITY FOR HELP. COULD BE A STACK OVERFLOW TAG OR OTHER
|
||||
CHANNEL. WHERE WILL YOU HELP PEOPLE?**.
|
||||
For help and questions about using this project, [look under the existing issues with the "help" label](https://github.com/Azure/counterfit/labels/question),
|
||||
or create one if your question has not been answered.
|
||||
|
||||
## Microsoft Support Policy
|
||||
|
||||
|
|
|
|||
|
|
@ -1,40 +0,0 @@
|
|||
import sys
|
||||
import os
|
||||
import warnings
|
||||
import argparse
|
||||
|
||||
from counterfit.core.config import Config
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.core.terminal import Terminal
|
||||
|
||||
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3" # make tensorflow quiet
|
||||
warnings.filterwarnings("ignore", category=FutureWarning)
|
||||
warnings.filterwarnings("ignore", category=DeprecationWarning)
|
||||
warnings.filterwarnings("ignore", category=UserWarning)
|
||||
|
||||
if sys.version_info < (3, 7):
|
||||
sys.exit("[!] Python 3.7+ is required")
|
||||
|
||||
|
||||
def main(args):
|
||||
# create the terminal
|
||||
terminal = Terminal()
|
||||
|
||||
# import targets and frameworks
|
||||
CFState.state()._init_state()
|
||||
|
||||
# load commands last. Choices depend on targets and attacks.
|
||||
terminal.load_commands()
|
||||
|
||||
print(Config.start_banner)
|
||||
|
||||
# run the terminal loop
|
||||
sys.exit(terminal.cmdloop())
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument(
|
||||
"-d", "--debug", action="store_true", help="enable debug messages")
|
||||
args = parser.parse_args()
|
||||
main(args)
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
import os
|
||||
import warnings
|
||||
|
||||
# make tensorflow quiet
|
||||
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
|
||||
os.environ['TF_ENABLE_ONEDNN_OPTS'] = '0'
|
||||
warnings.filterwarnings('ignore')
|
||||
|
||||
from . import core, data, frameworks, reporting, targets
|
||||
from .core.attacks import CFAttack
|
||||
from .core.core import Counterfit
|
||||
from .core.frameworks import CFFramework
|
||||
from .core.logger import CFLogger
|
||||
from .core.options import CFOptions
|
||||
from .core.output import CFPrint
|
||||
from .core.targets import CFTarget
|
||||
|
||||
|
||||
__version__ = '1.1.0'
|
||||
name = 'counterfit'
|
||||
|
|
@ -1,9 +0,0 @@
|
|||
import os
|
||||
|
||||
for module in os.listdir(os.path.dirname(__file__)):
|
||||
if module == "__init__.py" or module[-3:] != ".py":
|
||||
continue
|
||||
else:
|
||||
__import__("counterfit.commands." + module[:-3], locals(), globals())
|
||||
|
||||
del module
|
||||
|
|
@ -1,27 +0,0 @@
|
|||
import argparse
|
||||
from cmd2 import with_category
|
||||
from cmd2 import with_argparser
|
||||
import subprocess
|
||||
from subprocess import DEVNULL, STDOUT
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("-p", "--port", default="2718")
|
||||
parser.add_argument("-a", "--address", default="127.0.0.1")
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_docs(self, args: argparse.Namespace):
|
||||
"""Start a local web server that hosts the documentation
|
||||
|
||||
Args:
|
||||
port (str): the port to open.
|
||||
address (str): the ip address to host on.
|
||||
"""
|
||||
try:
|
||||
subprocess.Popen(["python", f"docs/server.py --port {args.port} --address {args.address}"],
|
||||
stdout=DEVNULL, stderr=STDOUT)
|
||||
|
||||
CFPrint.success("started server on http://127.0.0.1:5000/index.html")
|
||||
except Exception as e:
|
||||
CFPrint.error(f"Failed to start server: {e}")
|
||||
|
|
@ -1,42 +0,0 @@
|
|||
import os
|
||||
import argparse
|
||||
from rich.prompt import Prompt
|
||||
from cmd2 import with_argparser
|
||||
from cmd2 import with_category
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("option", choices=["counterfit", "target", "attack"])
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_exit(self, args: str) -> None:
|
||||
"""Exit Counterfit
|
||||
|
||||
Args:
|
||||
option (str): the object to exit.
|
||||
"""
|
||||
if args.option == "target":
|
||||
CFState.state().active_target = None
|
||||
return
|
||||
|
||||
elif args.option == "attack":
|
||||
CFState.state().active_target.active_attack = None
|
||||
return
|
||||
|
||||
elif args.option == "counterfit":
|
||||
while True:
|
||||
answer = Prompt.ask("[yellow][*][/yellow] Are you sure?", choices=["y", "n"])
|
||||
|
||||
print()
|
||||
if answer == "y":
|
||||
CFPrint.success("Come again soon!\n")
|
||||
os._exit(0)
|
||||
elif answer == "n":
|
||||
return False
|
||||
else:
|
||||
return True
|
||||
|
||||
else:
|
||||
CFPrint.info("Argument not recognized")
|
||||
|
|
@ -1,25 +0,0 @@
|
|||
import argparse
|
||||
from cmd2 import with_category
|
||||
from cmd2 import with_argparser
|
||||
from counterfit.core.state import CFState
|
||||
|
||||
def get_targets():
|
||||
return CFState.state().list_targets()
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("target", choices=get_targets())
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_interact(self, args: argparse.Namespace) -> None:
|
||||
"""Sets the the active target.
|
||||
|
||||
Args:
|
||||
target (str): The target to interact with.
|
||||
"""
|
||||
|
||||
# Load the target
|
||||
target = CFState.state().load_target(args.target)
|
||||
|
||||
# Set it as the active target
|
||||
CFState.state().set_active_target(target)
|
||||
|
|
@ -1,21 +0,0 @@
|
|||
import argparse
|
||||
from cmd2 import Cmd2ArgumentParser, with_argparser, with_category
|
||||
|
||||
from counterfit.core.state import CFState
|
||||
|
||||
parser = Cmd2ArgumentParser()
|
||||
parser.add_argument("framework", nargs='+',
|
||||
choices=CFState.state().get_frameworks().keys())
|
||||
parser.add_argument("-f", "--force-no-config", action="store_true",
|
||||
help="Force loading framework without using config")
|
||||
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_load(self, args: argparse.Namespace) -> None:
|
||||
"""Loads a framework.
|
||||
|
||||
"""
|
||||
|
||||
for framework in args.framework:
|
||||
CFState.state().load_framework(framework, args.force_no_config)
|
||||
|
|
@ -1,95 +0,0 @@
|
|||
import argparse
|
||||
import random
|
||||
|
||||
from cmd2 import with_argparser
|
||||
from cmd2 import with_category
|
||||
from rich.table import Table
|
||||
|
||||
from counterfit.core.output import CFPrint
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.report.report_generator import get_target_data_type_obj
|
||||
from counterfit.report import report_generator
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("-i", "--index", type=str, default=None,
|
||||
help="Send the selected samples to the target model. Python list and range accepted. Examples: 0, [0,1,2,3], range(5)")
|
||||
parser.add_argument("-r", "--random", action="store_true",
|
||||
help="Send a randomly selected sample to the target model")
|
||||
parser.add_argument("-a", "--attack_result", action="store_true",
|
||||
help="Send the result of the active_attack to the target model")
|
||||
|
||||
|
||||
def predict_table(heading1, sample_index, samples, results, labels=None):
|
||||
table = Table(header_style="bold magenta")
|
||||
table.add_column(heading1)
|
||||
table.add_column("Sample")
|
||||
if labels is not None:
|
||||
table.add_column("Label")
|
||||
table.add_column("Output Scores")
|
||||
|
||||
if labels is None:
|
||||
for idx, sample, result in zip(sample_index, samples, results):
|
||||
table.add_row(str(idx), str(sample), result)
|
||||
else:
|
||||
for idx, sample, label, result in zip(sample_index, samples, labels, results):
|
||||
table.add_row(str(idx), str(sample), str(label), result)
|
||||
|
||||
CFPrint.output(table)
|
||||
|
||||
|
||||
@with_category("Counterfit Commands")
|
||||
@with_argparser(parser)
|
||||
def do_predict(self, args: argparse.Namespace) -> None:
|
||||
"""Predict a single sample for the active target.
|
||||
"""
|
||||
|
||||
target = CFState.state().get_active_target()
|
||||
if not target:
|
||||
CFPrint.warn(
|
||||
"No active target. Try 'interact' <target>")
|
||||
return
|
||||
heading1 = "Sample Index"
|
||||
if args.index is not None: # default behavior
|
||||
sample_index = eval(args.index)
|
||||
samples = target.get_samples(sample_index)
|
||||
prefix = 'initial'
|
||||
elif args.attack_result:
|
||||
active_attack = target.active_attack
|
||||
if not active_attack:
|
||||
CFPrint.warn("No active attack. Try 'use' <attack>")
|
||||
return
|
||||
|
||||
elif active_attack.attack_status == "complete":
|
||||
heading1 = "Attack ID"
|
||||
samples = active_attack.results
|
||||
sample_index = [target.active_attack.attack_id] * len(samples)
|
||||
prefix = 'adversarial'
|
||||
else:
|
||||
CFPrint.warn(
|
||||
"Attack not complete or no results. Please run attack using 'run'")
|
||||
return
|
||||
|
||||
elif args.random:
|
||||
sample_index = random.randint(0, len(target.X) - 1)
|
||||
samples = target.get_samples(sample_index)
|
||||
prefix = "random"
|
||||
|
||||
else:
|
||||
CFPrint.warn("No index sample.")
|
||||
return
|
||||
|
||||
result = target.predict(samples) # results is list of probability scores
|
||||
labels = target.outputs_to_labels(result)
|
||||
target_datatype = target.target_data_type
|
||||
target_data_type_obj = get_target_data_type_obj(target_datatype)
|
||||
samples = target_data_type_obj.printable(target, samples, prefix)
|
||||
results = report_generator.printable_numpy(result)
|
||||
if not hasattr(sample_index, "__iter__"):
|
||||
sample_index = [sample_index]
|
||||
|
||||
predict_table(heading1,
|
||||
sample_index=sample_index,
|
||||
samples=samples,
|
||||
results=results,
|
||||
labels=labels
|
||||
)
|
||||
|
|
@ -1,39 +0,0 @@
|
|||
import argparse
|
||||
from cmd2 import with_argparser
|
||||
from cmd2 import with_category
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("-c", "--commands", action="store_true",
|
||||
help="Reload commands")
|
||||
parser.add_argument("-t", "--target", action="store_true",
|
||||
help="Reload the active target")
|
||||
parser.add_argument("-f", "--framework", help="Reload a framework",
|
||||
choices=list(CFState.state().get_frameworks().keys()))
|
||||
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_reload(self, args: argparse.Namespace) -> None:
|
||||
"""Reload a Counterfit object
|
||||
"""
|
||||
|
||||
if args.target:
|
||||
if not CFState.state().get_active_target():
|
||||
CFPrint.warn("Not interacting with a target.")
|
||||
else:
|
||||
CFState.state().reload_target()
|
||||
CFPrint.success("Successfully reloaded target")
|
||||
|
||||
elif args.commands:
|
||||
self.load_commands()
|
||||
CFPrint.success("Successfully reloaded commands")
|
||||
|
||||
elif args.framework:
|
||||
CFState.state().reload_framework(args.framework)
|
||||
CFPrint.success(f"Successfully reloaded {args.framework}")
|
||||
|
||||
else:
|
||||
CFPrint.failed("Argument not recognized")
|
||||
|
|
@ -1,33 +0,0 @@
|
|||
import argparse
|
||||
from cmd2 import with_argparser
|
||||
from cmd2 import with_category
|
||||
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("-v", "--verbose", action="store_true", help="print a summary ", default=False)
|
||||
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_run(self, args: argparse.Namespace) -> None:
|
||||
"""Run an attack
|
||||
"""
|
||||
|
||||
target_to_scan = CFState.state().get_active_target()
|
||||
if not target_to_scan:
|
||||
CFPrint.warn("Active target not set. Try 'interact <target>''")
|
||||
return
|
||||
|
||||
active_attack = CFState.state().active_target.active_attack
|
||||
if not active_attack:
|
||||
CFPrint.warn("No attack specified. Try 'use <attack>''")
|
||||
return
|
||||
|
||||
attack_id = CFState.state().active_target.get_active_attack()
|
||||
attack_name = active_attack.attack_name
|
||||
CFPrint.info(
|
||||
f"Running attack {attack_name} with id {attack_id} on {target_to_scan.target_name})\n")
|
||||
|
||||
CFState.state().run_attack(target_to_scan.target_name, attack_id)
|
||||
|
|
@ -1,41 +0,0 @@
|
|||
import argparse
|
||||
from cmd2 import with_argparser
|
||||
from cmd2 import with_category
|
||||
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("-p", "--parameters", action="store_true",
|
||||
help="Save the parameters for an attack")
|
||||
parser.add_argument("-r", "--results", action="store_true",
|
||||
help="Save the results and metadata for an attack")
|
||||
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_save(self, args: argparse.Namespace) -> None:
|
||||
"""Save results or parameters to disk.
|
||||
|
||||
Args:
|
||||
results (bool): save the results from `cfattack.results`.
|
||||
parameters (bool): Save the parameters used for the attack.
|
||||
"""
|
||||
|
||||
if not CFState.state().active_target:
|
||||
CFPrint.warn(
|
||||
"\n [!] Not interacting with a target. Set the active target with `interact`.\n")
|
||||
return
|
||||
|
||||
results_folder = CFState.state().active_target.active_attack.get_results_folder()
|
||||
|
||||
if args.parameters:
|
||||
CFState.state().active_target.active_attack.options.save_options(
|
||||
f"{results_folder}/params.json")
|
||||
CFPrint.success(f"Successfully wrote {results_folder}/params.json")
|
||||
|
||||
if args.results:
|
||||
CFState.state().active_target.active_attack.save_run_summary(
|
||||
f"{results_folder}/run_summary.json")
|
||||
CFPrint.success(
|
||||
f"Successfully wrote {results_folder}/run_summary.json")
|
||||
|
|
@ -1,96 +0,0 @@
|
|||
import argparse
|
||||
from cmd2 import with_argparser
|
||||
from cmd2 import with_category
|
||||
|
||||
from collections import defaultdict
|
||||
|
||||
from counterfit.core.output import CFPrint
|
||||
from counterfit.core.utils import set_id
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.report.report_generator import get_target_data_type_obj, get_scan_summary, printable_scan_summary
|
||||
from counterfit.commands.use import list_attacks
|
||||
from counterfit.commands.set import get_sample_index
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("-a", "--attacks", nargs='+', required=True, choices=list_attacks())
|
||||
parser.add_argument("-o", "--options",
|
||||
choices=["default", "random", "optimize"], default="default")
|
||||
parser.add_argument("-n", "--num_iters", type=int, default=1)
|
||||
parser.add_argument("-i", "--sample_index", type=str, default="0")
|
||||
parser.add_argument("-v", "--verbose", action="store_true")
|
||||
parser.add_argument("-s", "--summary", action="store_true", help="also summarize scans by class label")
|
||||
|
||||
# TODO Decouple setting options from build such that options are passed in at build time rather than during build time.
|
||||
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_scan(self, args):
|
||||
"""[summary]
|
||||
|
||||
Args:
|
||||
args.attacks (str): The list of attacks to run
|
||||
args.options (str): How attack parameters are selected
|
||||
"""
|
||||
|
||||
target_to_scan = CFState.state().get_active_target()
|
||||
if not target_to_scan:
|
||||
CFPrint.warn("Active target not set. Try 'interact <target>")
|
||||
return
|
||||
else:
|
||||
CFPrint.success(
|
||||
f"Scanning Target: {target_to_scan.target_name} ({target_to_scan.target_id})")
|
||||
|
||||
sample_index = get_sample_index(args.sample_index)
|
||||
if sample_index is None:
|
||||
return
|
||||
|
||||
scan_id = set_id()
|
||||
# loop over loaded attacks
|
||||
scans_by_attack = defaultdict(list)
|
||||
scans_by_label = defaultdict(list)
|
||||
for attack in args.attacks:
|
||||
for run in range(args.num_iters):
|
||||
attack_id = CFState.state().build_new_attack(
|
||||
target_name=CFState.state().active_target.target_name,
|
||||
attack_name=attack,
|
||||
scan_id=scan_id
|
||||
)
|
||||
if attack_id is None:
|
||||
CFPrint.warn(f"Attack not found: {attack}. Load <Framework>")
|
||||
return
|
||||
else:
|
||||
# create an attack
|
||||
CFState.state().active_target.set_active_attack(attack_id)
|
||||
# set options for this attack
|
||||
active_attack = CFState.state().active_target.active_attack
|
||||
active_attack.options.set_options({'sample_index': sample_index})
|
||||
# run the attack
|
||||
CFState.state().run_attack(target_to_scan.target_name, attack_id)
|
||||
|
||||
# display intermediate results
|
||||
if args.verbose:
|
||||
current_datatype = target_to_scan.target_data_type
|
||||
|
||||
current_dt_report_gen = get_target_data_type_obj(current_datatype)
|
||||
|
||||
summary = current_dt_report_gen.get_run_summary(active_attack)
|
||||
current_dt_report_gen.print_run_summary(summary)
|
||||
|
||||
# update the status and show success
|
||||
current_datatype = target_to_scan.target_data_type
|
||||
|
||||
current_dt_report_gen = get_target_data_type_obj(current_datatype)
|
||||
|
||||
summary = current_dt_report_gen.get_run_summary(active_attack)
|
||||
scans_by_attack[attack].append(summary)
|
||||
for lab in summary['initial_label']:
|
||||
scans_by_label[lab].append(summary)
|
||||
|
||||
# print scan summary
|
||||
summary_by_attack = {k : get_scan_summary(v) for k, v in scans_by_attack.items()}
|
||||
summary_by_label = {k : get_scan_summary(v) for k, v in scans_by_label.items()}
|
||||
|
||||
printable_scan_summary(summary_by_attack, summary_by_label if args.summary else None)
|
||||
|
||||
|
|
@ -1,219 +0,0 @@
|
|||
import argparse
|
||||
from cmd2 import with_argparser
|
||||
from cmd2 import with_category
|
||||
from rich.table import Table
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.core.output import CFPrint
|
||||
from typing import Union
|
||||
|
||||
|
||||
def set_table(default_options, current_options, new_options):
|
||||
default_options_list = CFState.state(
|
||||
).active_target.active_attack.options.default_options_list
|
||||
|
||||
cfattack_options_list = CFState.state(
|
||||
).active_target.active_attack.options.cfattack_options_list
|
||||
|
||||
table = Table(header_style="bold magenta")
|
||||
table.add_column("Parameter (type)")
|
||||
table.add_column("Default")
|
||||
table.add_column("Current")
|
||||
table.add_column("New")
|
||||
|
||||
# print attack params first
|
||||
table.add_row("Algorithm Parameters")
|
||||
table.add_row("--------------------", "--", "--", "--")
|
||||
for option in default_options_list:
|
||||
default_value = default_options.get(option)
|
||||
current_value = current_options.get(option)
|
||||
new_value = new_options.get(option, "-")
|
||||
|
||||
if new_value != current_value:
|
||||
table.add_row(f"{option} ({str(type(default_value).__name__)})",
|
||||
str(default_value), str(current_value), str(new_value))
|
||||
else:
|
||||
table.add_row(f"{option} ({str(type(default_value).__name__)})",
|
||||
str(default_value), str(current_value), " ")
|
||||
|
||||
# print cfspecific options next
|
||||
table.add_row()
|
||||
table.add_row("Attack Options")
|
||||
table.add_row("--------------------", "--", "--", "--")
|
||||
for option in cfattack_options_list:
|
||||
default_value = default_options.get(option)
|
||||
current_value = current_options.get(option)
|
||||
new_value = new_options.get(option, "-")
|
||||
|
||||
if "sample_index" == option:
|
||||
parameter_type = "int or expr"
|
||||
else:
|
||||
parameter_type = str(type(default_value).__name__)
|
||||
|
||||
if new_value != current_value:
|
||||
table.add_row(f"{option} ({parameter_type})",
|
||||
str(default_value), str(current_value), str(new_value))
|
||||
else:
|
||||
table.add_row(f"{option} ({parameter_type})",
|
||||
str(default_value), str(current_value), " ")
|
||||
|
||||
CFPrint.output(table)
|
||||
|
||||
|
||||
def get_options() -> list:
|
||||
# dynamically get the list of options
|
||||
if not CFState.state().active_target:
|
||||
options = {}
|
||||
elif not CFState.state().active_target.active_attack:
|
||||
options = {}
|
||||
elif CFState.state().active_target.active_attack and hasattr(CFState.state().active_target.active_attack, 'options'):
|
||||
options = CFState.state().active_target.active_attack.options.get_all_options()
|
||||
else:
|
||||
options = {}
|
||||
return options
|
||||
|
||||
def get_sample_index(sample_index: str) -> Union[list, int, range, None]:
|
||||
try:
|
||||
sample_index = eval(sample_index)
|
||||
except Exception as e:
|
||||
CFPrint.failed(f"Error parsing '--sample_index {sample_index}: {e}")
|
||||
return None
|
||||
|
||||
if type(sample_index) is tuple:
|
||||
sample_index = list(sample_index)
|
||||
|
||||
if type(sample_index) not in (range, int, list):
|
||||
CFPrint.failed(f"Error parsing '--sample_index {sample_index}: expression must result in a 'list', 'range' or 'int'")
|
||||
return None
|
||||
|
||||
if type(sample_index) is list:
|
||||
if any([type(el) is not int for el in sample_index]):
|
||||
CFPrint.failed(f"Error parsing '--sample_index {sample_index}': list must only contain integers")
|
||||
return None
|
||||
|
||||
return sample_index
|
||||
|
||||
NoneType = type(None)
|
||||
def get_clip_values(clip_values: str) -> Union[tuple,NoneType]:
|
||||
try:
|
||||
clip_values = eval(clip_values)
|
||||
except Exception as e:
|
||||
CFPrint.failed(f"Error parsing '--clip_values {clip_values}': {e}")
|
||||
return None
|
||||
|
||||
if clip_values is None:
|
||||
return "None"
|
||||
|
||||
if type(clip_values) not in (tuple,):
|
||||
CFPrint.failed(f"Error parsing '--clip_values {clip_values}: expression must result in a 'tuple' or 'None'")
|
||||
return None
|
||||
|
||||
return clip_values
|
||||
|
||||
|
||||
def parse_numeric(argname: str, val_str: str) -> Union[int, float, None]:
|
||||
# simple check for "inf" as a shortcut to float('inf)
|
||||
if type(val_str) is str and val_str == "inf":
|
||||
return float('inf')
|
||||
try:
|
||||
val = eval(val_str)
|
||||
except Exception as e:
|
||||
CFPrint.failed(f"Error parsing --'{argname} {val_str}': {e}")
|
||||
return None
|
||||
|
||||
if type(val) not in (int, float):
|
||||
CFPrint.failed(f"Error parsing '--{argname} {val_str}': expression must result in a 'int' or 'float'")
|
||||
return None
|
||||
|
||||
return val
|
||||
|
||||
|
||||
def parse_boolean(argname: str, val_str: str) -> Union[bool, None]:
|
||||
if val_str.lower() in ("true", "t", "yes", "y", "1"):
|
||||
return True
|
||||
elif val_str.lower() in ("false", "f", "no", "n", "0"):
|
||||
return False
|
||||
else:
|
||||
CFPrint.failed(f"Error parsing '--{argname} {val_str}': must be 'true' or 'false'")
|
||||
return None
|
||||
|
||||
|
||||
# dynamic option add
|
||||
parser = argparse.ArgumentParser()
|
||||
for option, value in get_options().items():
|
||||
if "sample_index" == option or "clip_values" == option:
|
||||
parser.add_argument(f"--{option}", type=str, default=str(value))
|
||||
elif type(value) in (float, int):
|
||||
parser.add_argument(f"--{option}", type=str, default=str(value))
|
||||
elif type(value) == bool:
|
||||
parser.add_argument(f"--{option}", type=str, default=str(value))
|
||||
else:
|
||||
parser.add_argument(
|
||||
f"--{option}", type=type(value), default=value)
|
||||
|
||||
|
||||
def update_options(target, partial_options):
|
||||
default_options = target.active_attack.options.previous_options[0]
|
||||
current_options = target.active_attack.options.get_all_options()
|
||||
new_options = current_options.copy()
|
||||
|
||||
new_options = {}
|
||||
for option, val in partial_options.items():
|
||||
if type(val) is bool and val: # toggle boolean values
|
||||
val = not current_options.get(option)
|
||||
new_options[option] = val
|
||||
|
||||
target.active_attack.options.set_options(new_options)
|
||||
|
||||
return default_options, current_options, new_options
|
||||
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_set(self, args: argparse.Namespace) -> None:
|
||||
"""Set parameters of the active attack on the active target using
|
||||
--param1 val1 --param2 val2.
|
||||
|
||||
For infinity, use 'inf' or 'float("inf")'.
|
||||
|
||||
This command replaces built-in "set" command, which is renamed to "setg".
|
||||
"""
|
||||
target = CFState.state().get_active_target()
|
||||
if not target:
|
||||
CFPrint.warn("No active target. Try 'interact <target>'")
|
||||
return
|
||||
|
||||
if not target.get_active_attack():
|
||||
CFPrint.warn("No active attack. Try 'use <attack>'")
|
||||
return
|
||||
|
||||
default_options = target.active_attack.options.previous_options[0]
|
||||
|
||||
for argname, argval in args.__dict__.items():
|
||||
if argname == 'clip_values':
|
||||
clip_values = get_clip_values(args.clip_values)
|
||||
if clip_values is None:
|
||||
return
|
||||
if clip_values=="None": # None is a valid type we handle separately
|
||||
clip_values = None
|
||||
args.clip_values = clip_values
|
||||
elif argname == 'sample_index':
|
||||
sample_index = get_sample_index(args.sample_index)
|
||||
if sample_index is None:
|
||||
return
|
||||
args.sample_index = sample_index
|
||||
elif type(default_options.get(argname)) in (float, int) and type(argval) is str:
|
||||
# parse numeric type
|
||||
argval = parse_numeric(argname, argval)
|
||||
if argval is None:
|
||||
return
|
||||
args.__dict__[argname] = argval
|
||||
elif type(default_options.get(argname)) is bool:
|
||||
# parse boolean type
|
||||
argval = parse_boolean(argname, argval)
|
||||
if argval is None:
|
||||
return
|
||||
args.__dict__[argname] = argval
|
||||
|
||||
default_options, current_options, new_options = update_options(target, args.__dict__)
|
||||
|
||||
set_table(default_options, current_options, new_options)
|
||||
|
|
@ -1,175 +0,0 @@
|
|||
import cmd2
|
||||
import argparse
|
||||
from rich.table import Table
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
# show
|
||||
base_parser = argparse.ArgumentParser()
|
||||
base_subparsers = base_parser.add_subparsers(title="subcommands", help="show information about models and targets")
|
||||
|
||||
# show info
|
||||
parser_info = base_subparsers.add_parser("info", help="show information about the model and active attack")
|
||||
parser_info.add_argument("--attack", default=None, help="attack to show info (defaults to active attack)")
|
||||
|
||||
# show options
|
||||
parser_options = base_subparsers.add_parser("options", help="show configuration options for the active attack")
|
||||
parser_options.add_argument("--attack", default=None, help="attack to show info (defaults to active attack)")
|
||||
|
||||
# show results
|
||||
parser_results = base_subparsers.add_parser("results", help="show results from the active attack")
|
||||
|
||||
# show attacks
|
||||
parser_attacks = base_subparsers.add_parser("attacks", help="show the list of attacks run against this target")
|
||||
|
||||
|
||||
def show_results(self, args):
|
||||
# default to the active attack
|
||||
if not CFState.state().active_target:
|
||||
CFPrint.warn("No active target.")
|
||||
return
|
||||
|
||||
if not CFState.state().active_target.active_attack:
|
||||
CFPrint.warn("No active acttack.")
|
||||
return
|
||||
|
||||
cfattack = CFState.state().active_target.active_attack
|
||||
if cfattack.attack_status != 'complete':
|
||||
CFPrint.warn("No completed attacks. Try 'run'.")
|
||||
return
|
||||
|
||||
# Get the framework responsible for the attack
|
||||
framework = CFState.state().frameworks.get(cfattack.framework_name)
|
||||
|
||||
# Give the framework an opportunity to process the results, generate reports, etc
|
||||
framework.post_attack_processing(cfattack)
|
||||
|
||||
|
||||
def show_options(self, args):
|
||||
# default to the active attack
|
||||
if not CFState.state().active_target:
|
||||
CFPrint.warn("No active target.")
|
||||
return
|
||||
|
||||
if not CFState.state().active_target.active_attack:
|
||||
CFPrint.warn("No active acttack.")
|
||||
return
|
||||
|
||||
current_options = CFState.state().active_target.active_attack.options.get_all_options()
|
||||
default_options = CFState.state(
|
||||
).active_target.active_attack.options.previous_options[0]
|
||||
|
||||
default_options_list = CFState.state(
|
||||
).active_target.active_attack.options.default_options_list
|
||||
|
||||
cfattack_options_list = CFState.state(
|
||||
).active_target.active_attack.options.cfattack_options_list
|
||||
|
||||
table = Table(header_style="bold magenta")
|
||||
table.add_column("Attack Options (type)")
|
||||
table.add_column("Default")
|
||||
table.add_column("Current")
|
||||
|
||||
# print attack params first
|
||||
table.add_row("Algo Parameters")
|
||||
table.add_row("--------------------", "--", "--")
|
||||
for option in default_options_list:
|
||||
default_value = default_options.get(option)
|
||||
current_value = current_options.get(option)
|
||||
table.add_row(f"{option} ({str(type(default_value).__name__)})",
|
||||
str(default_value), str(current_value))
|
||||
|
||||
# print cfspecific options next
|
||||
table.add_row()
|
||||
table.add_row("CFAttack Options")
|
||||
table.add_row("--------------------", "--", "--")
|
||||
for option in cfattack_options_list:
|
||||
default_value = default_options.get(option)
|
||||
current_value = current_options.get(option)
|
||||
table.add_row(f"{option} ({str(type(default_value).__name__)})",
|
||||
str(default_value), str(current_value))
|
||||
|
||||
CFPrint.output(table)
|
||||
|
||||
def show_info(self, args):
|
||||
table = Table(header_style="bold magenta")
|
||||
table.add_column("Attack Info")
|
||||
table.add_column("-----------")
|
||||
|
||||
if CFState.state().active_target and hasattr(CFState.state().active_target, 'active_attack'):
|
||||
cfattack = CFState.state().active_target.active_attack
|
||||
if cfattack is None:
|
||||
CFPrint.warn("No active attack")
|
||||
return
|
||||
framework = CFState.state().frameworks.get(cfattack.framework_name)
|
||||
attack = framework.attacks.get(cfattack.attack_name)
|
||||
else:
|
||||
CFPrint.warn("No active attack")
|
||||
return
|
||||
|
||||
table.add_row("Attack name", str(attack.attack_name))
|
||||
table.add_row("Attack type", str(attack.attack_type))
|
||||
table.add_row("Attack category", str(attack.attack_category))
|
||||
table.add_row("Attack tags", ", ".join(attack.attack_data_tags))
|
||||
table.add_row("Attack framework", str(attack.framework_name))
|
||||
table.add_row("Docs", str(attack.attack_class.__doc__))
|
||||
CFPrint.output(table)
|
||||
|
||||
|
||||
def show_attacks(self, args):
|
||||
# Active attacks
|
||||
table = Table(header_style="bold magenta")
|
||||
table.add_column("Attack id")
|
||||
table.add_column("Name")
|
||||
table.add_column("Status")
|
||||
table.add_column("Success")
|
||||
|
||||
if CFState.state().active_target:
|
||||
for k, v in CFState.state().active_target.attacks.items():
|
||||
if k == CFState.state().active_target.active_attack.attack_id:
|
||||
row_style = "cyan"
|
||||
id = "*" + k
|
||||
else:
|
||||
row_style = None
|
||||
id = k
|
||||
|
||||
if v.success is None:
|
||||
success = "N/A"
|
||||
else:
|
||||
sample_index = v.options.sample_index
|
||||
if type(sample_index) is int:
|
||||
sample_index = [sample_index]
|
||||
success = str(dict(zip(list(sample_index), v.success)))
|
||||
|
||||
table.add_row(
|
||||
id,
|
||||
v.attack_name,
|
||||
str(v.attack_status),
|
||||
success,
|
||||
style=row_style
|
||||
|
||||
)
|
||||
|
||||
CFPrint.output(table)
|
||||
|
||||
|
||||
parser_options.set_defaults(func=show_options)
|
||||
parser_attacks.set_defaults(func=show_attacks)
|
||||
parser_info.set_defaults(func=show_info)
|
||||
parser_results.set_defaults(func=show_results)
|
||||
|
||||
|
||||
@cmd2.with_argparser(base_parser)
|
||||
@cmd2.with_category("Counterfit Commands")
|
||||
def do_show(self, args):
|
||||
"""'show info' describes the active attack.
|
||||
'show options' lists attack parameters.
|
||||
'show sample' displays the target data
|
||||
"""
|
||||
|
||||
func = getattr(args, "func", None)
|
||||
|
||||
if func is not None:
|
||||
func(self, args)
|
||||
else:
|
||||
self.do_help("show")
|
||||
|
|
@ -1,52 +0,0 @@
|
|||
import argparse
|
||||
from typing import List
|
||||
from cmd2 import with_argparser
|
||||
from cmd2 import with_category
|
||||
from counterfit.core.output import CFPrint
|
||||
from counterfit.core import utils
|
||||
from counterfit.core.state import CFState
|
||||
|
||||
|
||||
# return a list of attack names
|
||||
def list_attacks() -> List[str]:
|
||||
# dynamically get the list of attacks
|
||||
attack_names = list(CFState.state().get_attacks().keys()) # new attack
|
||||
|
||||
# add existing attack if there's an active target
|
||||
if CFState.state().active_target and hasattr(CFState.state().active_target, 'attacks'):
|
||||
# existing attack
|
||||
attack_names += list(CFState.state().active_target.attacks.keys())
|
||||
return attack_names
|
||||
|
||||
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("attack", choices=list_attacks(),
|
||||
help="The attack to use, either <attack name> or <attack id>")
|
||||
|
||||
|
||||
@with_argparser(parser)
|
||||
@with_category("Counterfit Commands")
|
||||
def do_use(self, args: argparse.Namespace) -> None:
|
||||
"""Select an attack to use on the active target.
|
||||
Use 'interact' to select a target first.
|
||||
"""
|
||||
|
||||
if not CFState.state().active_target:
|
||||
CFPrint.warn(
|
||||
"Not interacting with any targets. Try interacting with a target.")
|
||||
return
|
||||
|
||||
if args.attack in CFState.state().active_target.attacks: # existing attack
|
||||
attack_id = args.attack
|
||||
else:
|
||||
scan_id = utils.set_id()
|
||||
attack_id = CFState.state().build_new_attack(
|
||||
target_name=CFState.state().active_target.target_name,
|
||||
attack_name=args.attack,
|
||||
scan_id=scan_id
|
||||
)
|
||||
|
||||
if attack_id is None:
|
||||
CFPrint.warn("Attack not found: {}".format(args.attack))
|
||||
else:
|
||||
CFState.state().active_target.set_active_attack(attack_id)
|
||||
|
|
@ -1 +1 @@
|
|||
__all__=[]
|
||||
from .core import Counterfit
|
||||
|
|
|
|||
|
|
@ -1,126 +1,52 @@
|
|||
import os
|
||||
import orjson
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import os
|
||||
import uuid
|
||||
|
||||
from counterfit.core.output import CFPrint
|
||||
from counterfit.logging.logging import get_attack_logger_obj
|
||||
from counterfit.report.report_generator import get_target_data_type_obj
|
||||
import orjson
|
||||
from counterfit.core.logger import CFLogger, get_attack_logger_obj
|
||||
from counterfit.core.options import CFOptions
|
||||
from counterfit.core.targets import CFTarget
|
||||
|
||||
class CFAttackOptions:
|
||||
"""A container class for all settable options for a `CFAttack.attack`. Default parameters should
|
||||
be loaded during `framework.load()` and added to `attack_default_params`. Additionally, a
|
||||
number of global Counterfit options are injected.
|
||||
"""
|
||||
def __init__(self, **kwargs):
|
||||
self.previous_options = []
|
||||
for key, value in kwargs.items():
|
||||
if key == "targeted":
|
||||
value = False
|
||||
setattr(self, "target_labels", 0)
|
||||
if key == "verbose":
|
||||
value = False
|
||||
setattr(self, key, value)
|
||||
|
||||
# Save the default options as the first value in previous_options
|
||||
self.save_previous_options()
|
||||
|
||||
def get_default_options(self) -> dict:
|
||||
"""Get the default options.
|
||||
|
||||
Returns:
|
||||
dict: A dictionary of default options for `CFAttack.attack`
|
||||
"""
|
||||
default_options = {}
|
||||
for option in self.default_options_list:
|
||||
default_options[option] = getattr(self, option)
|
||||
|
||||
return default_options
|
||||
|
||||
def get_current_options(self) -> dict:
|
||||
"""Get the current options.
|
||||
|
||||
Returns:
|
||||
dict: A dictionary of current options for `CFAttack`
|
||||
"""
|
||||
parameters = {}
|
||||
for param in self.default_options_list:
|
||||
parameters[param] = getattr(self, param)
|
||||
return parameters
|
||||
|
||||
def get_all_options(self) -> dict:
|
||||
"""Get all options.
|
||||
|
||||
Returns:
|
||||
dict: A dictionary of all available options for `CFAttack`
|
||||
"""
|
||||
available_options = {}
|
||||
all_options_list = self.default_options_list + self.cfattack_options_list
|
||||
for option in all_options_list:
|
||||
available_options[option] = getattr(self, option)
|
||||
return available_options
|
||||
|
||||
def save_previous_options(self):
|
||||
"""Save the previous options.
|
||||
"""
|
||||
current_options = self.get_all_options()
|
||||
self.previous_options.append(current_options)
|
||||
|
||||
def set_options(self, options_to_update: dict) -> None:
|
||||
"""Sets an option. Take in a dictionary of options and applies attack specific checks before saving them as attributes.
|
||||
|
||||
Args:
|
||||
params_to_update (dict): A dictionary of `{option:value}`
|
||||
"""
|
||||
|
||||
# Save the previous options
|
||||
self.save_previous_options()
|
||||
|
||||
for k, v in options_to_update.items():
|
||||
setattr(self, k, v)
|
||||
|
||||
def save_options(self, filename: str = None):
|
||||
"""Saves the current options to a json file.
|
||||
|
||||
Args:
|
||||
filename (str, optional): the output path. Defaults to None.
|
||||
"""
|
||||
options = self.get_all_options()
|
||||
data = orjson.dumps(
|
||||
options,
|
||||
option=orjson.OPT_SERIALIZE_NUMPY | orjson.OPT_APPEND_NEWLINE
|
||||
)
|
||||
|
||||
if not filename:
|
||||
filename = f"{self.attack_id}_params.json"
|
||||
|
||||
with open(filename, "w") as paramfile:
|
||||
paramfile.write(data.decode())
|
||||
|
||||
class CFAttack(object):
|
||||
class CFAttack:
|
||||
"""
|
||||
The base class for all attacks in all frameworks.
|
||||
"""
|
||||
|
||||
def __init__(self, attack_id, attack_name, attack, default_params, framework_name, target, scan_id=None):
|
||||
# Parent framework
|
||||
self.framework_name = framework_name
|
||||
name: str
|
||||
target: CFTarget
|
||||
framework: None
|
||||
attack: CFAttack
|
||||
attack_id: str
|
||||
options: CFOptions
|
||||
scan_id: str
|
||||
|
||||
# Target information
|
||||
def __init__(
|
||||
self,
|
||||
name,
|
||||
target,
|
||||
framework,
|
||||
attack,
|
||||
options,
|
||||
scan_id=None):
|
||||
|
||||
|
||||
# Parent framework
|
||||
self.name = name
|
||||
self.attack_id = uuid.uuid4().hex[:8]
|
||||
self.scan_id = scan_id
|
||||
self.target = target
|
||||
self.target_data_type = self.set_datatype()
|
||||
self.framework = framework
|
||||
self.attack = attack
|
||||
self.options = options
|
||||
|
||||
# Attack information
|
||||
self.attack_id = attack_id
|
||||
self.attack_name = attack_name
|
||||
self.attack = attack
|
||||
self.created_on = datetime.datetime.utcnow().strftime("%a, %d %b %Y %H:%M:%S GMT")
|
||||
self.created_on = datetime.datetime.utcnow().strftime("%m%d%y_%H:%M:%S_UTC")
|
||||
self.attack_status = "pending"
|
||||
|
||||
# Scan group information
|
||||
self.scan_id = scan_id
|
||||
|
||||
# Algo parameters
|
||||
self.default_params = default_params
|
||||
self.samples = None
|
||||
self.initial_labels = None
|
||||
self.initial_outputs = None
|
||||
|
|
@ -128,50 +54,29 @@ class CFAttack(object):
|
|||
# Attack results
|
||||
self.final_outputs = None
|
||||
self.final_labels = None
|
||||
self.results = None
|
||||
self.results: list = None
|
||||
self.success = None
|
||||
self.elapsed_time = None
|
||||
|
||||
# reporting
|
||||
self.run_summary = None
|
||||
self.logger: CFLogger = None
|
||||
|
||||
# CFAttack options
|
||||
self.cfattack_options = {
|
||||
"sample_index": 0,
|
||||
"logger": "default",
|
||||
}
|
||||
|
||||
self.options = self.init_options()
|
||||
|
||||
def init_options(self):
|
||||
all_options = {**self.default_params, **self.cfattack_options}
|
||||
return CFAttackOptions(
|
||||
**all_options,
|
||||
default_options_list=list(self.default_params.keys()),
|
||||
cfattack_options_list=list(self.cfattack_options.keys()))
|
||||
|
||||
def set_datatype(self):
|
||||
datatype = get_target_data_type_obj(self.target.target_data_type)
|
||||
self.target_data_type = datatype
|
||||
|
||||
def prepare_attack(self):
|
||||
# Set the datatype.
|
||||
# TODO I don't think this is needed.
|
||||
self.set_datatype()
|
||||
curr_log = self.options.cf_options["logger"]["current"]
|
||||
self.logger = self.set_logger(logger=curr_log)
|
||||
self.target.logger = self.logger
|
||||
|
||||
# Set the samples
|
||||
self.samples = self.target.get_samples(self.options.sample_index)
|
||||
# Get the samples.
|
||||
curr_idx = self.options.cf_options["sample_index"]["current"]
|
||||
self.samples = self.target.get_samples(curr_idx)
|
||||
|
||||
# Send a request to the target for the selected sample
|
||||
self.initial_outputs, self.initial_labels = self.target.get_sample_labels(
|
||||
self.samples)
|
||||
|
||||
# Set the logger
|
||||
self.set_logger(self.options.logger)
|
||||
|
||||
def get_default_params(self):
|
||||
return self.default_params
|
||||
|
||||
outputs, labels = self.target.get_sample_labels(self.samples)
|
||||
self.initial_outputs = outputs
|
||||
self.initial_labels = labels
|
||||
|
||||
def set_results(self, results: object) -> None:
|
||||
self.results = results
|
||||
|
||||
|
|
@ -181,29 +86,29 @@ class CFAttack(object):
|
|||
def set_success(self, success: bool = False) -> None:
|
||||
self.success = success
|
||||
|
||||
def set_logger(self, logger="default"):
|
||||
def set_logger(self, logger: str) -> CFLogger:
|
||||
new_logger = get_attack_logger_obj(logger)
|
||||
results_folder = self.get_results_folder()
|
||||
self.logger = new_logger(filepath=results_folder)
|
||||
self.target.logger = self.logger
|
||||
logger = new_logger(attack_id=self.attack_id, ts=self.created_on)
|
||||
return logger
|
||||
|
||||
def set_elapsed_time(self, start_time, end_time):
|
||||
def set_elapsed_time(self, start_time, end_time) -> float:
|
||||
self.elapsed_time = end_time - start_time
|
||||
|
||||
def get_results_folder(self):
|
||||
module_path = "/".join(self.target.__module__.split(".")[:-1])
|
||||
def get_results_folder(self) -> str:
|
||||
results_folder = self.target.get_results_folder()
|
||||
|
||||
if "results" not in os.listdir(module_path):
|
||||
os.mkdir(f"{module_path}/results")
|
||||
if self.attack_id not in os.listdir(f"{module_path}/results"):
|
||||
os.mkdir(f"{module_path}/results/{self.attack_id}")
|
||||
if not os.path.exists(results_folder):
|
||||
os.mkdir(results_folder)
|
||||
|
||||
results_folder = f"{module_path}/results/{self.attack_id}"
|
||||
return results_folder
|
||||
scan_folder = os.path.join(results_folder, self.attack_id)
|
||||
if not os.path.exists(scan_folder):
|
||||
os.mkdir(scan_folder)
|
||||
|
||||
def save_run_summary(self, filename=None, verbose=False):
|
||||
return scan_folder
|
||||
|
||||
def save_run_summary(self, filename: str=None, verbose: bool =False) -> None:
|
||||
run_summary = {
|
||||
"sample_index": self.options.sample_index,
|
||||
"sample_index": self.options.cf_options['sample_index'],
|
||||
"initial_labels": self.initial_labels,
|
||||
"final_labels": self.final_labels,
|
||||
"elapsed_time": self.elapsed_time,
|
||||
|
|
@ -215,14 +120,12 @@ class CFAttack(object):
|
|||
if verbose:
|
||||
run_summary["input_samples"] = self.samples
|
||||
|
||||
data = orjson.dumps(
|
||||
run_summary,
|
||||
option=orjson.OPT_SERIALIZE_NUMPY | orjson.OPT_APPEND_NEWLINE
|
||||
)
|
||||
options = orjson.OPT_SERIALIZE_NUMPY | orjson.OPT_APPEND_NEWLINE
|
||||
data = orjson.dumps(run_summary, option=options)
|
||||
|
||||
if not filename:
|
||||
results_folder = self.get_results_folder()
|
||||
filename = f"{results_folder}/run_summary.json"
|
||||
|
||||
with open(filename, "w") as summary_file:
|
||||
summary_file.write(data.decode())
|
||||
with open(filename, "wb") as summary_file:
|
||||
summary_file.write(data)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,201 @@
|
|||
import glob
|
||||
import importlib
|
||||
import inspect
|
||||
import os
|
||||
import time
|
||||
import traceback
|
||||
|
||||
import yaml
|
||||
from counterfit.core.attacks import CFAttack
|
||||
from counterfit.core.frameworks import CFFramework
|
||||
from counterfit.core.options import CFOptions
|
||||
from counterfit.core.output import CFPrint
|
||||
from counterfit.core.targets import CFTarget
|
||||
|
||||
|
||||
class Counterfit:
|
||||
|
||||
# Frameworks
|
||||
@classmethod
|
||||
def get_frameworks(cls) -> dict:
|
||||
"""Imports the available frameworks from the frameworks folder. Adds
|
||||
the loaded frameworks into self.frameworks. Frameworks contain the
|
||||
methods required for managing the attacks that are found within the
|
||||
framework.
|
||||
|
||||
Args:
|
||||
frameworks_path (str): A folder path where frameworks are kept.
|
||||
"""
|
||||
frameworks = {}
|
||||
cf_frameworks = importlib.import_module("counterfit.frameworks")
|
||||
for framework in cf_frameworks.CFFramework.__subclasses__():
|
||||
framework_name = framework.__module__.split(".")[-1]
|
||||
|
||||
frameworks[framework_name] = {}
|
||||
frameworks[framework_name]["attacks"] = {}
|
||||
frameworks[framework_name]["module"] = framework
|
||||
|
||||
framework_path = os.path.dirname(inspect.getfile(framework))
|
||||
|
||||
for attack in glob.glob(f"{framework_path}/attacks/*.yml"):
|
||||
with open(attack, 'r') as f:
|
||||
data = yaml.safe_load(f)
|
||||
|
||||
if data["attack_name"] not in frameworks[framework_name]["attacks"].keys():
|
||||
frameworks[framework_name]["attacks"][data['attack_name']] = data
|
||||
|
||||
return frameworks
|
||||
|
||||
@classmethod
|
||||
def build_target(
|
||||
cls,
|
||||
data_type: str,
|
||||
endpoint: str,
|
||||
output_classes: list,
|
||||
classifier: str,
|
||||
input_shape: tuple,
|
||||
load_func: object,
|
||||
predict_func: object,
|
||||
X: list) -> CFTarget:
|
||||
try:
|
||||
target = CFTarget(
|
||||
data_type=data_type,
|
||||
endpoint=endpoint,
|
||||
output_classes=output_classes,
|
||||
classifier=classifier,
|
||||
input_shape=input_shape,
|
||||
load=load_func,
|
||||
predict=predict_func,
|
||||
X=X)
|
||||
except Exception as error:
|
||||
CFPrint.failed(f"Failed to build target: {error}")
|
||||
try:
|
||||
target.load()
|
||||
except Exception as error:
|
||||
CFPrint.failed(f"Failed to load target: {error}")
|
||||
CFPrint.success(f"Successfully created target")
|
||||
return target
|
||||
|
||||
@classmethod
|
||||
def build_attack(
|
||||
cls,
|
||||
target: CFTarget,
|
||||
attack: str,
|
||||
scan_id: str = None) -> CFAttack:
|
||||
"""Build a new CFAttack.
|
||||
|
||||
Search through the loaded frameworks for the attack and create a new
|
||||
CFAttack object for use.
|
||||
|
||||
Args:
|
||||
target_name (CFTarget, required): The target object.
|
||||
attack_name (str, required): The attack name.
|
||||
scan_id (str, Optional): A unique value
|
||||
|
||||
Returns:
|
||||
CFAttack: A new CFAttack object.
|
||||
"""
|
||||
# Resolve the attack
|
||||
framework: CFFramework
|
||||
try:
|
||||
# Look for framework based on name.
|
||||
for k, v in cls.get_frameworks().items():
|
||||
if attack in list(v["attacks"].keys()):
|
||||
framework = v["module"]()
|
||||
attack = v["attacks"][attack]
|
||||
except Exception as error:
|
||||
msg = f"Failed to load framework or resolve {attack}: {error}"
|
||||
CFPrint.failed(msg)
|
||||
traceback.print_exc()
|
||||
# Ensure the attack is compatible with the target
|
||||
if target.data_type not in attack["attack_data_tags"]:
|
||||
msg = f"Target data type ({target.data_type}) is not compatible"
|
||||
msg += f" with the attack chosen ({attack['attack_data_tags']})"
|
||||
CFPrint.failed(msg)
|
||||
return False
|
||||
|
||||
|
||||
# Have the framework build the attack.
|
||||
try:
|
||||
new_attack = framework.build(
|
||||
target=target,
|
||||
attack=attack["attack_class"]) # The dotted path of the attack.
|
||||
except Exception as error:
|
||||
CFPrint.failed(f"Framework failed to build attack: {error}")
|
||||
traceback.print_exc()
|
||||
return
|
||||
|
||||
# Create a CFAttack object
|
||||
try:
|
||||
cfattack = CFAttack(
|
||||
name=attack["attack_class"],
|
||||
target=target,
|
||||
framework=framework,
|
||||
attack=new_attack,
|
||||
options=CFOptions(attack["attack_parameters"])
|
||||
)
|
||||
|
||||
except Exception as error:
|
||||
CFPrint.failed(f"Failed to build CFAttack: {error}")
|
||||
traceback.print_exc()
|
||||
|
||||
return cfattack
|
||||
|
||||
@classmethod
|
||||
def run_attack(cls, attack: CFAttack) -> bool:
|
||||
"""Run a prepared attack. Get the appropriate framework and execute
|
||||
the attack.
|
||||
|
||||
Args:
|
||||
attack_id (str, required): The attack id to run.
|
||||
|
||||
Returns:
|
||||
Attack: A new Attack object with an updated cfattack_class.
|
||||
Additional properties set in this function include, attack_id (str)
|
||||
and the parent framework (str). The framework string is added to
|
||||
prevent the duplication of code in run_attack.
|
||||
"""
|
||||
# Set the initial values for the attack. Samples, logger, etc.
|
||||
attack.prepare_attack()
|
||||
|
||||
# Run the attack
|
||||
attack.set_status("running")
|
||||
|
||||
# Start timing the attack for the elapsed_time metric
|
||||
start_time = time.time()
|
||||
|
||||
# Run the attack
|
||||
try:
|
||||
results = attack.framework.run(attack)
|
||||
except Exception as error:
|
||||
# postprocessing steps for failed attacks
|
||||
success = [False] * len(attack.initial_labels)
|
||||
|
||||
msg = f"Failed to run {attack.attack_id} ({attack.name}): {error}"
|
||||
CFPrint.failed(msg)
|
||||
return False
|
||||
|
||||
# postprocessing steps for successful attacks
|
||||
|
||||
# Set the elapsed time metric
|
||||
attack.set_elapsed_time(start_time, time.time())
|
||||
|
||||
# Set the results the attack returns
|
||||
# Results are attack and framework specific.
|
||||
attack.set_results(results)
|
||||
|
||||
# Determine the success of the attack
|
||||
success = attack.framework.check_success(attack)
|
||||
|
||||
# Set the success value
|
||||
attack.set_success(success)
|
||||
|
||||
# Give the framework an opportunity to process the results, generate reports, etc
|
||||
attack.framework.post_attack_processing(attack)
|
||||
|
||||
# Mark the attack as complete
|
||||
attack.set_status("complete")
|
||||
|
||||
# Let the user know the attack has completed successfully.
|
||||
CFPrint.success(f"Attack completed {attack.attack_id}")
|
||||
return True
|
||||
|
|
@ -1,30 +1,8 @@
|
|||
import json
|
||||
from abc import abstractmethod
|
||||
from pydoc import locate
|
||||
|
||||
from collections import namedtuple, defaultdict
|
||||
from counterfit.core.attacks import CFAttack
|
||||
|
||||
|
||||
class Framework:
|
||||
class CFFramework:
|
||||
"""Base class for all frameworks. This class enforces standard functions used by Counterfit.
|
||||
"""
|
||||
def __init__(self, loaded_status=False):
|
||||
self.loaded_status = loaded_status
|
||||
self.attacks = defaultdict()
|
||||
|
||||
@abstractmethod
|
||||
def load(self, config_path: str = None) -> None:
|
||||
"""Should load the attack classes for a framework.
|
||||
|
||||
Args:
|
||||
config_path (str, optional): path to a frameworks `config.json`. Defaults to None.
|
||||
"""
|
||||
if config_path:
|
||||
self.load_from_config(config_path)
|
||||
else:
|
||||
self.load()
|
||||
self.loaded_status = True
|
||||
|
||||
@abstractmethod
|
||||
def build(self, target, attack):
|
||||
|
|
@ -35,81 +13,14 @@ class Framework:
|
|||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
def pre_attack_processing(self, cfattack: CFAttack):
|
||||
pass
|
||||
def pre_attack_processing(self, cfattack):
|
||||
raise NotImplementedError('pre_attack_processing() not implemented')
|
||||
|
||||
@abstractmethod
|
||||
def post_attack_processing(self, cfattack: CFAttack):
|
||||
pass
|
||||
def post_attack_processing(self, cfattack):
|
||||
raise NotImplementedError('post_attack_processing() not implemented')
|
||||
|
||||
@abstractmethod
|
||||
def set_parameters(self, attack, parameters: dict) -> None:
|
||||
raise NotImplementedError
|
||||
|
||||
def get_attack(self, attack_name: str) -> object:
|
||||
"""Get an attack stored in `framework.attacks`
|
||||
|
||||
Args:
|
||||
attack_name (str): The name (key value) of the attack.
|
||||
|
||||
Returns:
|
||||
object: The attack.
|
||||
"""
|
||||
attack = self.attacks.get(attack_name)
|
||||
return attack
|
||||
|
||||
def load_from_config(self, config_path: str) -> None:
|
||||
"""Loads a framework from a config file. Uses `pydoc.locate` to find the attack class.
|
||||
|
||||
Args:
|
||||
config_path (str): The path to a config.json file.
|
||||
"""
|
||||
with open(config_path, 'r') as config:
|
||||
attacks_to_load = json.load(config)
|
||||
|
||||
for attack_name, properties in attacks_to_load.items():
|
||||
if attack_name not in self.attacks.keys():
|
||||
attack_class_to_load = properties.get("attack_class")
|
||||
attack_class = locate(f"{attack_class_to_load}.{attack_name}")
|
||||
attack_type = properties.get("attack_type")
|
||||
attack_category = properties.get("attack_category")
|
||||
attack_data_tags = properties.get("attack_data_tags")
|
||||
attack_params = properties.get("attack_parameters")
|
||||
|
||||
self.add_attack(
|
||||
attack_name=attack_name,
|
||||
attack_class=attack_class,
|
||||
attack_type=attack_type,
|
||||
attack_category=attack_category,
|
||||
attack_data_tags=attack_data_tags,
|
||||
attack_default_params=attack_params,
|
||||
)
|
||||
self.loaded_status = True
|
||||
|
||||
def add_attack(self, attack_name: str, attack_type: str, attack_category: str,
|
||||
attack_data_tags: list, attack_class: object, attack_default_params: dict) -> None:
|
||||
"""Adds an attacks the the framework, is called during load.
|
||||
|
||||
Args:
|
||||
attack_name (str): The name of the attack.
|
||||
attack_type (str): The type of the attack (Extraction, Evasion, Inference, Inversion, Poisoning, Custom...)
|
||||
attack_category (str): Whitebox or blackbox attacks. Some attacks only work with local models (i.e access to gradients)
|
||||
attack_data_tags (list): A list of the data modalities an attack works with (image, tabular, video, etc)
|
||||
attack_class (object): The attack that will get built and run.
|
||||
attack_default_params (dict): Any settable parameters for the attack (max_iters, norm, query_budget, etc).
|
||||
"""
|
||||
|
||||
AttackEntry = namedtuple(
|
||||
"attack_entry", "attack_name, attack_type, attack_category, attack_data_tags, attack_class, attack_default_params, framework_name")
|
||||
|
||||
framework_name = self.__module__.split(".")[-1]
|
||||
|
||||
new_attack = AttackEntry(
|
||||
attack_name,
|
||||
attack_type,
|
||||
attack_category,
|
||||
attack_data_tags,
|
||||
attack_class,
|
||||
attack_default_params,
|
||||
framework_name=framework_name)
|
||||
self.attacks[attack_name] = new_attack
|
||||
# TODO. Bring back the set_parameters() function for all frameworks.
|
||||
# @abstractmethod
|
||||
# def set_parameters(self, cfattack):
|
||||
# raise NotImplementedError('set_parameters() not implemented')
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
import pathlib
|
||||
|
||||
import orjson
|
||||
|
||||
|
||||
class CFLogger:
|
||||
""" Base class for all loggers.
|
||||
"""
|
||||
num_queries: int
|
||||
|
||||
def __init__(self) -> None:
|
||||
pass
|
||||
|
||||
|
||||
class BasicLogger(CFLogger):
|
||||
""" The default logger. Only logs the number of queries against a model.
|
||||
"""
|
||||
def __init__(self, **kwargs):
|
||||
super(CFLogger).__init__()
|
||||
self.num_queries = 0
|
||||
|
||||
def log(self, item):
|
||||
self.num_queries += 1
|
||||
|
||||
|
||||
class JSONLogger(CFLogger):
|
||||
"""Logs queries to a json file saved to disk.
|
||||
"""
|
||||
def __init__(self, **kwargs):
|
||||
super(CFLogger).__init__()
|
||||
attack_id = kwargs["attack_id"]
|
||||
ts = kwargs['ts']
|
||||
self.num_queries = 0
|
||||
self.filepath = pathlib.Path(f"attack_logs/{attack_id}_{ts}_logs.json")
|
||||
self.logs = []
|
||||
self.filepath.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def log(self, item):
|
||||
options = orjson.OPT_SERIALIZE_NUMPY | orjson.OPT_APPEND_NEWLINE
|
||||
data = orjson.dumps(item,option=options)
|
||||
with self.filepath.open('a+b') as fd:
|
||||
fd.write(data)
|
||||
self.logs.append(item)
|
||||
self.num_queries += 1
|
||||
|
||||
|
||||
def get_attack_logger_obj(logger_type: str) -> CFLogger:
|
||||
""" Factory method to get the requested logger.
|
||||
"""
|
||||
|
||||
attack_logger_obj_map = {
|
||||
'basic': BasicLogger,
|
||||
'json': JSONLogger
|
||||
}
|
||||
|
||||
if logger_type not in attack_logger_obj_map:
|
||||
raise KeyError(
|
||||
f'Logger is not supported {logger_type}...Please provide one of: {list(attack_logger_obj_map.keys())}...')
|
||||
|
||||
return attack_logger_obj_map[logger_type]
|
||||
|
|
@ -1,10 +0,0 @@
|
|||
from abc import ABC
|
||||
class CFAttackLogger(ABC):
|
||||
"""Base class for all loggers.
|
||||
|
||||
Returns:
|
||||
[type]: [description]
|
||||
"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
pass
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
import optuna
|
||||
import numpy as np
|
||||
from counterfit.core import Counterfit
|
||||
from counterfit import CFTarget, CFAttack
|
||||
from rich import print
|
||||
|
||||
# Optimization function
|
||||
def max_abs_change(cfattack: CFAttack) -> np.ndarray:
|
||||
batch_size = len(cfattack.results)
|
||||
i_0 = np.atleast_2d(cfattack.samples)
|
||||
i_f = np.array(cfattack.results)
|
||||
i_0 = i_0.reshape(batch_size, -1).astype(float)
|
||||
i_f = i_f.reshape(batch_size, -1).astype(float)
|
||||
|
||||
max_abs_change = np.atleast_1d(abs(i_f - i_0).max(axis=-1))
|
||||
|
||||
return max_abs_change
|
||||
|
||||
def num_queries(cfattack: CFAttack):
|
||||
return cfattack.logger.num_queries
|
||||
|
||||
def optimize(scan_id: str, target: CFTarget, attack: str, num_iters: int=2, verbose=False) -> optuna.study.Study:
|
||||
def objective(trial):
|
||||
cfattack = Counterfit.build_attack(target, attack)
|
||||
params = {}
|
||||
for k, v in cfattack.options.attack_parameters.items():
|
||||
if v.get("optimize"):
|
||||
if v["optimize"].get("uniform"):
|
||||
params[k] = trial.suggest_int(k, v["optimize"]["uniform"]["min"], v["optimize"]["uniform"]["max"])
|
||||
# params["logger"] = "json"
|
||||
cfattack.options.update(params)
|
||||
|
||||
if verbose:
|
||||
print(f'[green][+][/green] Trial parameters: {trial.params}\n')
|
||||
|
||||
try:
|
||||
Counterfit.run_attack(cfattack)
|
||||
return max_abs_change(cfattack), cfattack.logger.num_queries
|
||||
except Exception as error:
|
||||
print(error)
|
||||
return 1, 50000, 0
|
||||
|
||||
sampler = optuna.samplers.TPESampler()
|
||||
study = optuna.create_study(study_name=scan_id, sampler=sampler, directions=["minimize", "minimize"])
|
||||
study.optimize(objective, n_trials=num_iters, gc_after_trial=True)
|
||||
|
||||
return study
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
import orjson
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
|
||||
class CFOptions:
|
||||
"""A container class for all settable options for a `CFAttack.attack`. Default parameters should
|
||||
be loaded during `framework.load()` and added to `attack_default_params`. Additionally, a
|
||||
number of global Counterfit options are injected.
|
||||
"""
|
||||
attack_parameters: dict
|
||||
cf_options: dict
|
||||
|
||||
def __init__(self, attack_parameters):
|
||||
self.attack_parameters = attack_parameters
|
||||
self.cf_options = {
|
||||
"sample_index": {
|
||||
"default": 0,
|
||||
"current": 0,
|
||||
"previous": [],
|
||||
"docs": "Sample index to attack"
|
||||
},
|
||||
"optimize": {
|
||||
"default": False,
|
||||
"current": False,
|
||||
"previous": [],
|
||||
"docs": "Use Optuna to optimize attack parameters"
|
||||
},
|
||||
"logger": {
|
||||
"default": "basic",
|
||||
"current": "basic",
|
||||
"previous": [],
|
||||
"docs": "Logger to log queries with"
|
||||
}
|
||||
}
|
||||
|
||||
if 'targeted' in self.attack_parameters.keys():
|
||||
self.attack_parameters['target_labels'] = {"docs": "target labels for a targeted attack", "default": 0}
|
||||
for k, v in self.attack_parameters.items():
|
||||
v.update({"current": v["default"], "previous": []})
|
||||
|
||||
|
||||
def update(self, parameters_to_update: dict) -> None:
|
||||
"""Sets an option. Take in a dictionary of options and applies attack specific checks before saving them as attributes.
|
||||
|
||||
Args:
|
||||
params_to_update (dict): A dictionary of `{option:value}`
|
||||
"""
|
||||
|
||||
for k, v in parameters_to_update.items():
|
||||
if k in self.attack_parameters.keys():
|
||||
self.attack_parameters[k]["previous"].append(self.attack_parameters[k]["current"])
|
||||
self.attack_parameters[k]["current"] = v
|
||||
elif k in self.cf_options.keys():
|
||||
self.cf_options[k]["previous"].append(self.cf_options[k]["current"])
|
||||
self.cf_options[k]["current"] = v
|
||||
else:
|
||||
CFPrint.warn("Option not found")
|
||||
|
||||
def get_all(self):
|
||||
return {**self.attack_parameters, **self.cf_options}
|
||||
|
||||
def save_options(self, filename: str = None):
|
||||
"""Saves the current options to a json file.
|
||||
|
||||
Args:
|
||||
filename (str, optional): the output path. Defaults to None.
|
||||
"""
|
||||
options = {**self.attack_parameters, **self.cf_options}
|
||||
dump_options = orjson.OPT_SERIALIZE_NUMPY | orjson.OPT_APPEND_NEWLINE
|
||||
data = orjson.dumps(options, option=dump_options)
|
||||
|
||||
if not filename:
|
||||
filename = f"{self.attack_id}_params.json"
|
||||
|
||||
with open(filename, "w") as paramfile:
|
||||
paramfile.write(data.decode())
|
||||
|
|
@ -10,7 +10,7 @@ class CFPrint:
|
|||
Args:
|
||||
message (str): The message to print
|
||||
"""
|
||||
console.print(f"[cyan][-][/cyan] {message}")
|
||||
console.print(f"[cyan][-] info: [/cyan] {message}")
|
||||
|
||||
@staticmethod
|
||||
def success(message: str):
|
||||
|
|
@ -19,7 +19,7 @@ class CFPrint:
|
|||
Args:
|
||||
message (str): The message to print
|
||||
"""
|
||||
console.print(f"[green][+][/green] {message}")
|
||||
console.print(f"[green][+] success: [/green] {message}")
|
||||
|
||||
@staticmethod
|
||||
def warn(message):
|
||||
|
|
@ -28,7 +28,7 @@ class CFPrint:
|
|||
Args:
|
||||
message (str): The message to print
|
||||
"""
|
||||
console.print(f"[yellow][*][/yellow] {message}")
|
||||
console.print(f"[yellow][*] warning: [/yellow] {message}")
|
||||
|
||||
@staticmethod
|
||||
def failed(message):
|
||||
|
|
@ -37,7 +37,7 @@ class CFPrint:
|
|||
Args:
|
||||
message (str): The message to print
|
||||
"""
|
||||
console.print(f"[red][!][/red] {message}")
|
||||
console.print(f"[red][!] failed: [/red] {message}")
|
||||
|
||||
@staticmethod
|
||||
def output(message):
|
||||
|
|
|
|||
|
|
@ -0,0 +1,122 @@
|
|||
from abc import ABC
|
||||
from abc import abstractmethod
|
||||
import json
|
||||
|
||||
import numpy as np
|
||||
from rich.table import Table
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
class CFReportGenerator(ABC):
|
||||
@abstractmethod
|
||||
def printable(target, batch):
|
||||
raise NotImplementedError()
|
||||
|
||||
@abstractmethod
|
||||
def get_run_summary(target, cfattack):
|
||||
raise NotImplementedError()
|
||||
|
||||
def printable_numpy(batch):
|
||||
o = np.get_printoptions()
|
||||
np.set_printoptions(
|
||||
threshold=30, precision=2, floatmode="maxprec_equal", formatter=dict(float=lambda x: f"{x:4.2f}")
|
||||
)
|
||||
result = [str(np.array(row)).replace("\n", " ") for row in batch]
|
||||
np.set_printoptions(
|
||||
threshold=o["threshold"], precision=o["precision"], floatmode=o["floatmode"], formatter=o["formatter"]
|
||||
)
|
||||
return result
|
||||
|
||||
def get_scan_summary(list_of_runs):
|
||||
# summarize by attack -- what is the best
|
||||
# - success rate
|
||||
# - average time
|
||||
# - best result (highest confidence confusion)
|
||||
# - attack_id for best parameters
|
||||
# - attack_name
|
||||
|
||||
total_successes = sum([s["successes"] for s in list_of_runs])
|
||||
total_runs = sum([s["batch_size"] for s in list_of_runs])
|
||||
times = [s["elapsed_time"] for s in list_of_runs]
|
||||
queries = [s["queries"] for s in list_of_runs]
|
||||
best_attack = None
|
||||
best_id = None
|
||||
best_score = None
|
||||
best_params = None
|
||||
best_queries = None
|
||||
for s in list_of_runs:
|
||||
for conf, il, fl in zip(s['final_confidence'], s['initial_label'], s['final_label']):
|
||||
if (s['targeted'] and s['target_label'] == fl) or (s['targeted'] == False and fl != il):
|
||||
if best_score is None or conf > best_score or (conf == best_score and s['queries'] < best_queries):
|
||||
best_score = conf
|
||||
best_id = s["attack_id"]
|
||||
best_attack = s["attack_name"]
|
||||
best_params = s["parameters"]
|
||||
best_queries = s["queries"]
|
||||
return {
|
||||
"total_runs": total_runs,
|
||||
"total_successes": total_successes,
|
||||
"avg_time": np.mean(times),
|
||||
"min_time": np.min(times),
|
||||
"max_time": np.max(times),
|
||||
"avg_queries": int(np.mean(queries)),
|
||||
"min_queries": np.min(queries),
|
||||
"max_queries": np.max(queries),
|
||||
"best_attack_name": best_attack,
|
||||
"best_attack_id": best_id,
|
||||
"best_attack_score": best_score,
|
||||
"best_params": best_params,
|
||||
}
|
||||
|
||||
def printable_scan_summary(summaries_by_attack, summaries_by_label=None):
|
||||
"""Print scan summaries in the command line console
|
||||
|
||||
Args:
|
||||
summaries_by_attack ([dict]): Dictionary contains summary details per attack
|
||||
summaries_by_label ([dict], optional): Dictionary contains summary details per label. Defaults to None.
|
||||
"""
|
||||
CFPrint.output(
|
||||
"\n =============== \n <SCAN SUMMARY> \n ===============\n\n")
|
||||
table = Table(header_style="bold magenta")
|
||||
table.add_column("Attack Name")
|
||||
table.add_column("Total Runs")
|
||||
table.add_column("Successes (%)")
|
||||
table.add_column("Best Score (attack_id)")
|
||||
table.add_column("Best Parameters", width=110)
|
||||
|
||||
for name, summary in summaries_by_attack.items():
|
||||
frac = summary["total_successes"] / summary["total_runs"]
|
||||
successes = f"{summary['total_successes']} ({frac:>.1%})"
|
||||
best = (
|
||||
f"{summary['best_attack_score']:0.1f} ({summary['best_attack_id']})"
|
||||
if summary["best_attack_score"]
|
||||
else "N/A"
|
||||
)
|
||||
best_params = json.dumps(summary["best_params"])
|
||||
table.add_row(str(name), str(summary["total_runs"]), str(
|
||||
successes), str(best), str(best_params))
|
||||
st = Table(header_style="bold magenta")
|
||||
CFPrint.output(table)
|
||||
|
||||
if summaries_by_label is not None:
|
||||
st.add_column("Class Label")
|
||||
st.add_column("Total Runs")
|
||||
st.add_column("Successes (%)")
|
||||
st.add_column("Best Score (Attack)")
|
||||
for name, summary in sorted(summaries_by_label.items()):
|
||||
frac = summary["total_successes"] / summary["total_runs"]
|
||||
successes = f"{summary['total_successes']} ({frac:>.1%})"
|
||||
best = (
|
||||
f"{summary['best_attack_score']:0.1f} ({summary['best_attack_name']})"
|
||||
if summary["best_attack_score"]
|
||||
else "N/A"
|
||||
)
|
||||
st.add_row(str(name), str(
|
||||
summary["total_runs"]), str(successes), str(best))
|
||||
|
||||
CFPrint.output(st)
|
||||
output = ""
|
||||
times_str = f"{summary['min_time']:>4.1f}/{summary['avg_time']:>4.1f}/{summary['max_time']:>4.1f}"
|
||||
queries_str = f"{summary['min_queries']:>5d}/{summary['avg_queries']:>5d}/{summary['max_queries']:>5d}"
|
||||
output += f"[+] Time[sec] (min/avg/max) {times_str} \n"
|
||||
output += f"\n[+] Queries (min/avg/max) {queries_str} \n"
|
||||
CFPrint.output(output)
|
||||
|
|
@ -1,404 +0,0 @@
|
|||
# state.py
|
||||
# This file keeps track of all load targets.
|
||||
|
||||
import importlib
|
||||
import sys
|
||||
import time
|
||||
from counterfit.core.config import Config
|
||||
from counterfit.core import utils
|
||||
from counterfit.core.output import CFPrint
|
||||
from counterfit.core.attacks import CFAttack
|
||||
from counterfit.core.frameworks import Framework
|
||||
from counterfit.core.targets import Target
|
||||
|
||||
class CFState:
|
||||
"""Singleton class responsible for managing targets and attacks. Instantiation of a class is restricted to one object.
|
||||
"""
|
||||
__instance__ = None
|
||||
|
||||
def __init__(self):
|
||||
if CFState.__instance__ is None:
|
||||
CFState.__instance__ = self
|
||||
else:
|
||||
raise Exception("You cannot create another CFState class")
|
||||
|
||||
self.frameworks = {}
|
||||
self.targets = {}
|
||||
self.scans = {}
|
||||
self.active_target = None
|
||||
|
||||
@staticmethod
|
||||
def state():
|
||||
"""Static method to fetch the current instance.
|
||||
"""
|
||||
if not CFState.__instance__:
|
||||
CFState.__instance__ = CFState()
|
||||
return CFState.__instance__
|
||||
|
||||
def _init_state(self) -> None:
|
||||
"""Initializes Counterfit
|
||||
"""
|
||||
self.import_frameworks(frameworks_path="counterfit/frameworks")
|
||||
self.import_targets(targets_path="counterfit/targets")
|
||||
|
||||
# Frameworks
|
||||
def import_frameworks(self, frameworks_path: str) -> None:
|
||||
"""Imports the available frameworks from the frameworks folder. Adds the loaded frameworks into self.frameworks. Frameworks contain the methods required for managing the attacks that are found within the framework.
|
||||
|
||||
Args:
|
||||
frameworks_path (str): A folder path where frameworks are kept.
|
||||
"""
|
||||
available_frameworks = utils.import_subclass(
|
||||
frameworks_path, Framework)
|
||||
|
||||
for k, v in available_frameworks.items():
|
||||
self.add_framework(k, v())
|
||||
|
||||
def add_framework(self, framework_name: str, framework: Framework = None) -> None:
|
||||
"""Adds a framework to Counterfit.
|
||||
|
||||
Args:
|
||||
framework_name (str): Name of the framework
|
||||
framework (Framework, optional): The Framework object to add. Defaults to None.
|
||||
"""
|
||||
if not framework:
|
||||
framework = Framework()
|
||||
self.frameworks[framework_name] = framework
|
||||
|
||||
def load_framework(self, framework: str, force_no_config: bool = False) -> None:
|
||||
"""Load a framework by name. By loading a framework, the attacks under the framework are loaded and ready for use. Attacks are not loaded during import_frameworks as loading could be slow, or a particular framework could cause errors and prevent all frameworks from loading.
|
||||
|
||||
Args:
|
||||
framework (str): The framework name
|
||||
force_no_config (bool, optional): [description]. Defaults to False.
|
||||
"""
|
||||
|
||||
framework_to_load = self.frameworks.get(framework)
|
||||
|
||||
try:
|
||||
if force_no_config:
|
||||
framework_to_load.load()
|
||||
else:
|
||||
try:
|
||||
config_path = f"{Config.frameworks_path}/{framework}/config.json"
|
||||
framework_to_load.load(config_path=config_path)
|
||||
CFPrint.success(f"{framework} successfully loaded!")
|
||||
except Exception:
|
||||
CFPrint.info(f"unable to load from {config_path}. Initialize framework from default config.")
|
||||
# load without config
|
||||
framework_to_load.load()
|
||||
CFPrint.success(
|
||||
f"{framework} successfully loaded with defaults (no config file provided)")
|
||||
|
||||
except Exception as e:
|
||||
CFPrint.failed(f"Could not load {framework}: {e}\n")
|
||||
|
||||
def get_frameworks(self) -> dict:
|
||||
"""Get all available frameworks
|
||||
|
||||
Args:
|
||||
None
|
||||
|
||||
Returns:
|
||||
dict: all frameworks.
|
||||
"""
|
||||
return self.frameworks
|
||||
|
||||
def reload_framework(self, framework_name):
|
||||
"""Reimports the framework.
|
||||
|
||||
Args:
|
||||
framework_name (str): framework to reload
|
||||
"""
|
||||
|
||||
if framework_name not in self.get_frameworks().keys():
|
||||
CFPrint.failed(f"{framework_name} not found")
|
||||
else:
|
||||
# Get the framework to reload
|
||||
framework_to_reload = self.get_frameworks().get(framework_name)
|
||||
|
||||
# Get the class we want to instantiate after module reload
|
||||
framework_class = framework_to_reload.__class__.__name__
|
||||
|
||||
# Reload the module
|
||||
reloaded_module = importlib.reload(
|
||||
sys.modules[framework_to_reload.__module__])
|
||||
|
||||
# Reload the framework
|
||||
reloaded_framework = reloaded_module.__dict__.get(
|
||||
framework_class)()
|
||||
|
||||
# Delete the old class
|
||||
del self.frameworks[framework_name]
|
||||
|
||||
# Replace the old module with the new one
|
||||
self.frameworks[framework_name] = reloaded_framework
|
||||
|
||||
# Load the attacks.
|
||||
self.load_framework(framework_name)
|
||||
|
||||
# Targets
|
||||
def import_targets(self, targets_path: str) -> None:
|
||||
"""Imports available targets from the targets folder. Adds the loaded frameworks to CFState.targets.
|
||||
Targets contain the data and methods to interact with a target machine learning system.
|
||||
|
||||
Args:
|
||||
targets_path (str): Folder path to where targets are kept
|
||||
"""
|
||||
available_targets = utils.import_subclass(
|
||||
targets_path, Target)
|
||||
|
||||
for k, v in available_targets.items():
|
||||
self.add_target(k, v())
|
||||
|
||||
def add_target(self, target_name: str, target: Target = None):
|
||||
"""Add a target to Counterfit
|
||||
|
||||
Args:
|
||||
target_name (str): The name of the target
|
||||
target (Target, optional): The Target object to add. Defaults to None.
|
||||
"""
|
||||
if not target:
|
||||
target = Target()
|
||||
self.targets[target_name] = target
|
||||
|
||||
def load_target(self, target: str) -> Target:
|
||||
"""Load a target by name. Targets are not loaded during import to prevent loading unnecessary targets
|
||||
during a session. This function calls `target.load()`.
|
||||
|
||||
Args:
|
||||
The target name
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
target_to_load = self.targets.get(target, None)
|
||||
|
||||
if not target_to_load.loaded_status:
|
||||
try:
|
||||
target_to_load.load()
|
||||
CFPrint.success(
|
||||
f"{target_to_load.target_name} ({target_to_load.target_id}) successfully loaded!")
|
||||
target_to_load.set_loaded_status(True)
|
||||
|
||||
except Exception as e:
|
||||
CFPrint.failed(
|
||||
f"Could not load {target_to_load.target_name}: {e}\n")
|
||||
else:
|
||||
CFPrint.warn(
|
||||
f"{target} is already loaded, use 'reload' to reload a target")
|
||||
|
||||
return target_to_load
|
||||
|
||||
def reload_target(self):
|
||||
"""Reloads the active target.
|
||||
"""
|
||||
if not self.active_target:
|
||||
CFPrint.failed("No active target")
|
||||
return
|
||||
|
||||
else:
|
||||
# Get the framework to reload
|
||||
target_name = self.active_target.target_name
|
||||
target_to_reload = self.targets[target_name]
|
||||
|
||||
# Get the attacks
|
||||
attacks = target_to_reload.attacks
|
||||
active_attack = target_to_reload.active_attack
|
||||
|
||||
# Get the class we want to instantiate after module reload
|
||||
target_class = target_to_reload.__class__.__name__
|
||||
|
||||
# Reload the module
|
||||
reloaded_module = importlib.reload(
|
||||
sys.modules[target_to_reload.__module__])
|
||||
|
||||
# Reload the target
|
||||
reloaded_target = reloaded_module.__dict__.get(
|
||||
target_class)()
|
||||
|
||||
# Delete the old class
|
||||
del self.targets[target_name]
|
||||
|
||||
# Replace the old module with the new one
|
||||
self.targets[target_name] = reloaded_target
|
||||
|
||||
# Load the attacks.
|
||||
target_to_load = self.load_target(target_name)
|
||||
|
||||
# Set it as the active target
|
||||
self.set_active_target(target_to_load)
|
||||
|
||||
# Replace the history
|
||||
self.active_target.attacks = attacks
|
||||
self.active_target.active_attack = active_attack
|
||||
|
||||
def list_targets(self) -> list:
|
||||
"""Get a list of targets
|
||||
|
||||
Returns:
|
||||
list: A list of imported targets
|
||||
"""
|
||||
|
||||
return list(self.targets.keys())
|
||||
|
||||
def get_attacks(self) -> dict:
|
||||
"""Get all available attacks
|
||||
|
||||
Returns:
|
||||
dict: all available attacks from all frameworks.
|
||||
"""
|
||||
all_attacks = {}
|
||||
frameworks = self.get_frameworks()
|
||||
for framework_name, framework in frameworks.items():
|
||||
for k, v in sorted(framework.attacks.items()):
|
||||
all_attacks[k] = v
|
||||
|
||||
return all_attacks
|
||||
|
||||
def get_active_target(self) -> Target:
|
||||
"""Get the active target
|
||||
|
||||
Returns:
|
||||
Target: The active target.
|
||||
"""
|
||||
return self.active_target
|
||||
|
||||
def add_attack_to_target(self, target_name: str, cfattack: CFAttack) -> None:
|
||||
"""After a CFAttack object has been built, add it to the target for tracking.
|
||||
|
||||
Args:
|
||||
target_name (str): The target name
|
||||
cfattack (CFAttack): The CFAttack object to add.
|
||||
"""
|
||||
target = self.targets.get(target_name)
|
||||
target.add_attack(cfattack)
|
||||
|
||||
def build_new_attack(self, target_name: str, attack_name: str, scan_id: str = None) -> CFAttack:
|
||||
"""Build a new CFAttack. Search through the loaded frameworks for the attack and create a new CFAttack object for use.
|
||||
|
||||
Args:
|
||||
target_name (str, required): The target name.
|
||||
attack_name (str, required): The attack name.
|
||||
scan_id (str, Optional): A unique value
|
||||
|
||||
Returns:
|
||||
CFAttack: A new CFAttack object.
|
||||
"""
|
||||
|
||||
# Get the target to run the attack against.
|
||||
target = self.targets.get(target_name)
|
||||
|
||||
# Hunt through frameworks for the attack.
|
||||
for framework_name, framework in self.frameworks.items():
|
||||
for attack in framework.attacks:
|
||||
if attack_name != attack:
|
||||
continue
|
||||
else:
|
||||
attack = framework.attacks.get(attack)
|
||||
|
||||
# Have the framework build the attack.
|
||||
new_attack = framework.build(
|
||||
target=target,
|
||||
attack=attack.attack_class
|
||||
)
|
||||
|
||||
cfattack = CFAttack(
|
||||
attack_id=utils.set_id(),
|
||||
attack_name=attack_name,
|
||||
attack=new_attack,
|
||||
default_params=attack.attack_default_params,
|
||||
framework_name=framework_name,
|
||||
target=target,
|
||||
scan_id=scan_id
|
||||
) # This is a mutable runtime friendly structure.
|
||||
|
||||
self.add_attack_to_target(target_name, cfattack)
|
||||
|
||||
CFPrint.success(
|
||||
f"New {attack_name} ({cfattack.attack_id}) created")
|
||||
return cfattack.attack_id
|
||||
|
||||
def run_attack(self, target_name: str, attack_id: str) -> bool:
|
||||
"""Run a prepared attack. Get the appropriate framework and execute the attack.
|
||||
|
||||
Args:
|
||||
attack_id (str, required): The attack id to run.
|
||||
|
||||
Returns:
|
||||
Attack: A new Attack object with an updated cfattack_class. Additional properties set in this function include, attack_id (str)
|
||||
and the parent framework (str). The framework string is added to prevent the duplication of code in run_attack.
|
||||
"""
|
||||
|
||||
# Get the cfattack object to run
|
||||
cfattack = self.targets.get(target_name).attacks.get(attack_id)
|
||||
|
||||
# Get the framework responsible for the attack
|
||||
framework = self.frameworks.get(cfattack.framework_name)
|
||||
|
||||
# Make sure there is a cfattack object
|
||||
if not cfattack:
|
||||
CFPrint.failed("Attack id not found")
|
||||
return False
|
||||
|
||||
# Set the initial values for the attack. Samples, logger, etc.
|
||||
CFPrint.info("Preparing attack...")
|
||||
cfattack.prepare_attack()
|
||||
|
||||
# Run the attack
|
||||
CFPrint.info("Running attack...")
|
||||
cfattack.set_status("running")
|
||||
|
||||
# Give the framework an opportunity to preprocess any thing in the attack.
|
||||
framework.pre_attack_processing(cfattack)
|
||||
|
||||
# Start timing the attack for the elapsed_time metric
|
||||
start_time = time.time()
|
||||
|
||||
try:
|
||||
# Get the results of the attack
|
||||
results = framework.run(cfattack)
|
||||
except Exception as e:
|
||||
# postprocessing steps for failed attacks
|
||||
success = [False] * len(cfattack.initial_labels)
|
||||
|
||||
# Let the user know the attack failed
|
||||
CFPrint.failed(
|
||||
f"Failed to run {cfattack.attack_id} ({cfattack.attack_name}): {e}")
|
||||
|
||||
finally:
|
||||
# postprocessing steps for successful attacks
|
||||
# Set the results that the attack returns
|
||||
cfattack.set_results(results)
|
||||
|
||||
# Determine the success of the attack
|
||||
success = framework.check_success(cfattack)
|
||||
|
||||
# Stop the timer
|
||||
end_time = time.time()
|
||||
|
||||
# Set the elapsed time metric
|
||||
cfattack.set_elapsed_time(start_time, end_time)
|
||||
|
||||
# Set the success value
|
||||
cfattack.set_success(success)
|
||||
|
||||
# Give the framework an opportunity to process the results, generate reports, etc
|
||||
framework.post_attack_processing(cfattack)
|
||||
|
||||
# Mark the attack as complete
|
||||
cfattack.set_status("complete")
|
||||
|
||||
# Let the user know the attack has completed successfully.
|
||||
CFPrint.success(
|
||||
f"Attack completed {cfattack.attack_id} ({cfattack.attack_name})")
|
||||
return True
|
||||
|
||||
|
||||
def set_active_target(self, target: Target) -> None:
|
||||
"""Set the active target with the target name provided.
|
||||
|
||||
Args:
|
||||
target (Target): The target object to set as the active target
|
||||
"""
|
||||
self.active_target = target
|
||||
|
|
@ -2,27 +2,33 @@ import datetime
|
|||
import inspect
|
||||
import os
|
||||
import pathlib
|
||||
from typing import Union
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from collections import defaultdict
|
||||
from abc import abstractmethod
|
||||
from collections import defaultdict, namedtuple
|
||||
from typing import Any, Tuple, Union
|
||||
|
||||
import numpy as np
|
||||
# from counterfit.core.attacks import CFAttack
|
||||
from counterfit.core.output import CFPrint
|
||||
from counterfit.core.attacks import CFAttack
|
||||
from counterfit.core.utils import set_id
|
||||
|
||||
class Target(ABC):
|
||||
|
||||
class CFTarget:
|
||||
"""Base class for all targets.
|
||||
"""
|
||||
target_name: str
|
||||
active_attack: Any
|
||||
target_id: str
|
||||
attacks: dict
|
||||
logger: None
|
||||
|
||||
def __init__(self):
|
||||
def __init__(self, **kwargs):
|
||||
self.target_id = set_id()
|
||||
self.loaded_status = False
|
||||
self.active_attack = None
|
||||
self.logger = None
|
||||
self.attacks = defaultdict()
|
||||
|
||||
for k, v in kwargs.items():
|
||||
setattr(self, k, v)
|
||||
|
||||
@abstractmethod
|
||||
def load(self):
|
||||
"""Loads data, models, etc in preparation. Is called by `interact`.
|
||||
|
|
@ -33,7 +39,7 @@ class Target(ABC):
|
|||
raise NotImplementedError
|
||||
|
||||
@abstractmethod
|
||||
def predict(self, x):
|
||||
def predict(self, x: np.ndarray) -> np.ndarray:
|
||||
"""The predict interface to the target model.
|
||||
|
||||
Raises:
|
||||
|
|
@ -41,66 +47,13 @@ class Target(ABC):
|
|||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def set_loaded_status(self, status: bool = False) -> None:
|
||||
"""Set the loaded status of a framework
|
||||
|
||||
Args:
|
||||
status (bool, optional): [description]. Defaults to False.
|
||||
"""
|
||||
self.loaded_status = status
|
||||
|
||||
def add_attack(self, attack: CFAttack) -> None:
|
||||
"""Add a CFAttack to the target.
|
||||
|
||||
Args:
|
||||
attack (CFAttack): The CFAttack object
|
||||
"""
|
||||
self.attacks[attack.attack_id] = attack
|
||||
|
||||
def set_active_attack(self, attack_id: str) -> None:
|
||||
"""Sets the active attack
|
||||
|
||||
Args:
|
||||
attack_id (str): The attack_id of the attack to use.
|
||||
"""
|
||||
active_attack = self.attacks.get(attack_id, None)
|
||||
if not active_attack:
|
||||
CFPrint.failed(f"{attack_id} not found")
|
||||
else:
|
||||
CFPrint.success(f"Using {attack_id}")
|
||||
self.active_attack = active_attack
|
||||
|
||||
def get_active_attack(self) -> None:
|
||||
"""Get the active attack
|
||||
"""
|
||||
if self.active_attack is None:
|
||||
return None
|
||||
return self.active_attack.attack_id
|
||||
|
||||
def get_attacks(self, scan_id: str = None) -> dict:
|
||||
"""Get all of the attacks
|
||||
|
||||
Args:
|
||||
scan_id (str, optional): The scan_id to filter on. Defaults to None.
|
||||
|
||||
Returns:
|
||||
dict: [description]
|
||||
"""
|
||||
if scan_id:
|
||||
scans_by_scan_id = {}
|
||||
for attack_id, attack in self.attacks.items():
|
||||
if attack.scan_id == scan_id:
|
||||
scans_by_scan_id[attack_id] = attack
|
||||
return scans_by_scan_id
|
||||
else:
|
||||
return self.attacks
|
||||
|
||||
|
||||
def get_samples(self, sample_index: Union[int, list, range] = 0) -> np.ndarray:
|
||||
"""This function helps to directly set sample_index and samples for a target not depending on attack.
|
||||
"""This function helps to directly set sample_index and samples for a
|
||||
target not depending on attack.
|
||||
|
||||
Args:
|
||||
sample_index (int, list or range, optional): [single or multiple indices]. Defaults to 0.
|
||||
sample_index (int, list or range, optional): [single or multiple
|
||||
indices]. Defaults to 0.
|
||||
|
||||
Returns:
|
||||
np.ndarray: [description]
|
||||
|
|
@ -115,7 +68,7 @@ class Target(ABC):
|
|||
else:
|
||||
# multiple index (numpy)
|
||||
out = np.array([self.X[i] for i in sample_index])
|
||||
batch_shape = (-1,) + self.target_input_shape
|
||||
batch_shape = (-1,) + self.input_shape
|
||||
elif type(self.X[sample_index]) is str:
|
||||
# single index (string)
|
||||
# array of strings (textattack)
|
||||
|
|
@ -125,11 +78,11 @@ class Target(ABC):
|
|||
# single index (array)
|
||||
# array of arrays (art)
|
||||
out = np.atleast_2d(self.X[sample_index])
|
||||
batch_shape = (-1,) + self.target_input_shape
|
||||
batch_shape = (-1,) + self.input_shape
|
||||
|
||||
return out.reshape(batch_shape)
|
||||
|
||||
def predict_wrapper(self, x, **kwargs):
|
||||
def predict_wrapper(self, x: np.ndarray, **kwargs: dict):
|
||||
output = self.predict(x)
|
||||
if self.logger:
|
||||
labels = self.outputs_to_labels(output)
|
||||
|
|
@ -143,41 +96,41 @@ class Target(ABC):
|
|||
}
|
||||
|
||||
self.logger.log(log_entry)
|
||||
|
||||
except Exception as e:
|
||||
print(e)
|
||||
continue
|
||||
raise
|
||||
|
||||
return output
|
||||
|
||||
def fullpath(self, file: str) -> str:
|
||||
"""A conveiance function
|
||||
"""A convenience function.
|
||||
|
||||
Args:
|
||||
file (str): The file tp get the full path for.
|
||||
file (str): The file to get the full path for.
|
||||
|
||||
Returns:
|
||||
str: The full path of the file
|
||||
"""
|
||||
basedir = pathlib.Path(os.path.abspath(
|
||||
inspect.getfile(self.__class__))).parent.resolve()
|
||||
curr_file_path = os.path.abspath(inspect.getfile(self.__class__))
|
||||
basedir = pathlib.Path(curr_file_path).parent.resolve()
|
||||
return os.path.join(basedir, file)
|
||||
|
||||
def get_sample_labels(self, samples):
|
||||
"""A covienance function to get outputs and labels for a target query.
|
||||
def get_sample_labels(self, samples: np.ndarray) -> Tuple[str, str]:
|
||||
"""
|
||||
|
||||
Args:
|
||||
samples ([type]): [description]
|
||||
|
||||
Returns:
|
||||
[type]: [description]
|
||||
Tuple[str, str]: The (output, labels).
|
||||
"""
|
||||
output = self.predict_wrapper(samples)
|
||||
labels = self.outputs_to_labels(output)
|
||||
return output, labels
|
||||
|
||||
def outputs_to_labels(self, output):
|
||||
"""Default multiclass label selector via argmax. User can override this function if, for example, one wants to choose a specific threshold
|
||||
"""Default multiclass label selector via argmax. User can override this
|
||||
function if, for example, one wants to choose a specific threshold
|
||||
Args:
|
||||
output ([type]): [description]
|
||||
|
||||
|
|
@ -185,4 +138,21 @@ class Target(ABC):
|
|||
[type]: [description]
|
||||
"""
|
||||
output = np.atleast_2d(output)
|
||||
return [self.target_output_classes[i] for i in np.argmax(output, axis=1)]
|
||||
return [self.output_classes[i] for i in np.argmax(output, axis=1)]
|
||||
|
||||
def get_results_folder(self, folder="results"):
|
||||
return os.path.join(os.curdir, folder)
|
||||
|
||||
def get_data_type_obj(self):
|
||||
target_data_types = {
|
||||
'text': "TextReportGenerator",
|
||||
'image': "ImageReportGenerator",
|
||||
'tabular': "TabularReportGenerator"
|
||||
}
|
||||
if self.data_type not in target_data_types.keys():
|
||||
options = list(target_data_types.keys())
|
||||
msg = f"{self.data_type} not supported. Choose one of {options}"
|
||||
CFPrint.failed(msg)
|
||||
return
|
||||
|
||||
return target_data_types[self.data_type]
|
||||
|
|
|
|||
|
|
@ -1,110 +0,0 @@
|
|||
import os
|
||||
import pkgutil
|
||||
import inspect
|
||||
import types
|
||||
|
||||
from cmd2 import Cmd
|
||||
from cmd2 import ansi
|
||||
from typing import Any
|
||||
|
||||
from counterfit.core.state import CFState
|
||||
from counterfit.core.config import Config
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
|
||||
class Terminal(Cmd):
|
||||
"""Terminal class responsible for setting the CLI and loading commands
|
||||
"""
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(startup_script=".counterfit",
|
||||
allow_cli_args=False, include_ipy=True)
|
||||
|
||||
# rename the built-in "set" attribute to "setg"
|
||||
setattr(Cmd, "do_setg", Cmd.do_set)
|
||||
delattr(Cmd, "do_set")
|
||||
|
||||
self.prompt = "counterfit> "
|
||||
|
||||
def _set_prompt(self):
|
||||
"""Set the terminal prompt
|
||||
"""
|
||||
if not CFState.state().active_target:
|
||||
self.prompt = "counterfit> "
|
||||
|
||||
else:
|
||||
if not CFState.state().active_target.active_attack:
|
||||
self.prompt = f"{CFState.state().active_target.target_name}> "
|
||||
else:
|
||||
self.prompt = f"{CFState.state().active_target.target_name}>{CFState.state().active_target.active_attack.attack_id}> "
|
||||
|
||||
def default(self, command: str) -> None:
|
||||
"""Executed when the command given isn't a recognized command implemented by a do_* method.
|
||||
|
||||
Args:
|
||||
command (str): command object with parsed input
|
||||
"""
|
||||
CFPrint.warn("Command does not exist.\n")
|
||||
return
|
||||
|
||||
def precmd(self, line):
|
||||
"""Run code prior to exe
|
||||
|
||||
Args:
|
||||
line ([type]): [description]
|
||||
|
||||
Returns:
|
||||
[type]: [description]
|
||||
"""
|
||||
print()
|
||||
return line
|
||||
|
||||
def postcmd(self, stop, line):
|
||||
"""Hook method executed just after a command dispatch is finished.
|
||||
"""
|
||||
print()
|
||||
|
||||
# Set the prompt to reflect interaction changes
|
||||
self._set_prompt()
|
||||
|
||||
# Trigger new choices to populate
|
||||
self.load_commands()
|
||||
return stop
|
||||
|
||||
def load_commands(self):
|
||||
"""Loads all the commands that exists under counterfit.commands sub-package
|
||||
"""
|
||||
commands_full_dir_path = os.path.join(
|
||||
os.getcwd(), Config.commands_path)
|
||||
for module_finder, package_name, ispkg in pkgutil.iter_modules([commands_full_dir_path]):
|
||||
if not ispkg:
|
||||
current_module = module_finder.find_module(
|
||||
package_name).load_module()
|
||||
for member in inspect.getmembers(current_module, inspect.isfunction):
|
||||
if "do_" in member[0] or "complete_" in member[0]:
|
||||
setattr(self, member[0],
|
||||
types.MethodType(member[1], self))
|
||||
if "finish_" in member[0]:
|
||||
continue
|
||||
del current_module
|
||||
|
||||
def pexcept(self, msg: Any, *, end: str = '\n', apply_style: bool = True) -> None:
|
||||
"""Print an exception
|
||||
Args:
|
||||
msg (Any): [description]
|
||||
end (str, optional): [description]. Defaults to '\n'.
|
||||
apply_style (bool, optional): [description]. Defaults to True.
|
||||
"""
|
||||
if isinstance(msg, Exception):
|
||||
final_msg = f"EXCEPTION of type '{type(msg).__name__}' occurred with message: {msg}"
|
||||
else:
|
||||
final_msg = str(msg)
|
||||
|
||||
if apply_style:
|
||||
final_msg = ansi.style_error(final_msg)
|
||||
|
||||
if not self.debug and 'debug' in self.settables:
|
||||
warning = "\n [!] To enable full traceback, run the following command: 'setg debug true'"
|
||||
final_msg += ansi.style_warning(warning)
|
||||
|
||||
self.perror(final_msg, end=end, apply_style=False)
|
||||
|
|
@ -6,9 +6,9 @@ import io
|
|||
import mimetypes
|
||||
import os
|
||||
import uuid
|
||||
|
||||
import numpy as np
|
||||
|
||||
import pathlib
|
||||
from PIL import Image
|
||||
|
||||
def param_floats_to_ints(parameters: dict) -> dict:
|
||||
"""Convert floats to integers
|
||||
|
|
@ -92,15 +92,20 @@ def get_predict_folder(target):
|
|||
return results_folder
|
||||
|
||||
|
||||
def is_img_save_in_azure_storage():
|
||||
def install_path(path) -> pathlib.Path:
|
||||
return pathlib.Path(os.path.join("counterfit", path)).resolve()
|
||||
|
||||
|
||||
def is_img_save_in_azure_storage() -> bool:
|
||||
"""Checks if Azure environment variables are set.
|
||||
|
||||
Returns:
|
||||
bool: Returns True if Azure environment variables are set, otherwise False
|
||||
"""
|
||||
return True if "AzureStorageAccountName" and "AzureStorageContainerName" in os.environ else False
|
||||
return True if "AzureStorageAccountName" and "AzureStorageContainerName" in os.environ else False
|
||||
|
||||
def get_azure_storage_sas_uri(filename):
|
||||
|
||||
def get_azure_storage_sas_uri(filename: str) -> str:
|
||||
"""Generate Azure Storage SAS URI based on Azure Storage Account, Azure Storage SAS token, and file path
|
||||
|
||||
Args:
|
||||
|
|
@ -116,14 +121,29 @@ def get_azure_storage_sas_uri(filename):
|
|||
azure_storage_sas_uri = f"https://{azure_storage_account_name}.blob.core.windows.net/{filename}?{azure_storage_sas_token}"
|
||||
return azure_storage_sas_uri
|
||||
|
||||
def get_image_in_bytes(image, format='png'):
|
||||
|
||||
def get_image_in_bytes(image:Image, format: str='png') -> bytes:
|
||||
""" Convert a PIL.Image to bytes
|
||||
|
||||
Args:
|
||||
image (PIL.Image): The input image.
|
||||
format (str): The input file format (default 'png').
|
||||
|
||||
Returns:
|
||||
bytes: the image as raw bytes.
|
||||
"""
|
||||
buf = io.BytesIO()
|
||||
image.save(buf, format=format) # In the above code, we save the image Image object into BytesIO object buffer
|
||||
im = buf.getvalue()
|
||||
return im
|
||||
|
||||
def get_mime_type(url):
|
||||
# Get MIME type based on a given url name (ex., "input_example.json" -> application/json, image/jpeg, image/png)
|
||||
|
||||
def get_mime_type(url: str) -> str:
|
||||
""" Get MIME type based on a given url name
|
||||
|
||||
Example:
|
||||
"input_example.json" -> application/json, image/jpeg, image/png
|
||||
"""
|
||||
content_type = mimetypes.guess_type(url)
|
||||
if not content_type[0]:
|
||||
raise ValueError(f'Invalid MIME type detected for the URL {url}. \
|
||||
|
|
|
|||
|
|
@ -0,0 +1 @@
|
|||
from . import *
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
import numpy as np
|
||||
from PIL.Image import Image
|
||||
|
||||
class ImageDataType():
|
||||
@staticmethod
|
||||
def is_channels_last(input_shape):
|
||||
if input_shape[-1] == 3:
|
||||
channels_last = True
|
||||
elif input_shape[-1] == 1:
|
||||
channels_last = True
|
||||
else:
|
||||
channels_last = False
|
||||
return channels_last
|
||||
|
||||
@staticmethod
|
||||
def convert_to_uint8(array, factor=None):
|
||||
if not factor:
|
||||
return np.uint8(array)
|
||||
else:
|
||||
return np.uint8(array * factor)
|
||||
|
||||
@staticmethod
|
||||
def get_channels(input_shape):
|
||||
if input_shape[-1] == 3 or input_shape[0] == 3:
|
||||
return "RGB"
|
||||
else:
|
||||
return "L"
|
||||
|
||||
@staticmethod
|
||||
def transpose_array(array):
|
||||
return np.transpose(array)
|
||||
|
||||
@staticmethod
|
||||
def convert_to_pil(array):
|
||||
pil_image = Image.fromarray(array)
|
||||
return pil_image
|
||||
|
||||
@staticmethod
|
||||
def convert_to_array(image: Image):
|
||||
image_array = np.array(image)
|
||||
return image_array
|
||||
|
||||
@staticmethod
|
||||
def grayscale_to_pil(array: np.ndarray) -> Image:
|
||||
pil_image = Image.fromarray((np.squeeze(array) * 255).astype(np.uint8))
|
||||
return pil_image
|
||||
|
|
@ -0,0 +1,7 @@
|
|||
# framework imports
|
||||
|
||||
from counterfit.core.frameworks import CFFramework
|
||||
|
||||
from .art import art
|
||||
from .augly import augly
|
||||
from .textattack import textattack
|
||||
|
|
@ -1,351 +1,48 @@
|
|||
import importlib
|
||||
import inspect
|
||||
import re
|
||||
import os
|
||||
from counterfit.core.output import CFPrint
|
||||
|
||||
import numpy as np
|
||||
from pydoc import locate
|
||||
import tensorflow as tf
|
||||
from collections import defaultdict
|
||||
import importlib
|
||||
import inspect
|
||||
import pathlib
|
||||
import glob
|
||||
import yaml
|
||||
import re
|
||||
|
||||
from rich.table import Table
|
||||
|
||||
from art.utils import compute_success_array
|
||||
from art.attacks.poisoning.perturbations import add_pattern_bd
|
||||
from art.attacks.poisoning import PoisoningAttackBackdoor
|
||||
from art.estimators.classification import KerasClassifier
|
||||
from art.attacks.inference.membership_inference import (
|
||||
MembershipInferenceBlackBox)
|
||||
from art.attacks.evasion.fast_gradient import FastGradientMethod
|
||||
|
||||
from counterfit.core.attacks import CFAttack
|
||||
from counterfit.report.report_generator import get_target_data_type_obj
|
||||
from counterfit.core.frameworks import Framework
|
||||
from counterfit.core.targets import Target
|
||||
from counterfit.core.output import CFPrint
|
||||
from counterfit.core.frameworks import CFFramework
|
||||
from counterfit.core.targets import CFTarget
|
||||
|
||||
# These are some random things that are used for loading and pulling default params.
|
||||
# These should be hot-patched during self.build or self.run
|
||||
tf.compat.v1.disable_eager_execution()
|
||||
from .utils import attack_factory
|
||||
from art.utils import compute_success_array, random_targets
|
||||
from scipy.stats import entropy
|
||||
from art.utils import clip_and_round
|
||||
|
||||
attacks_still_wip = set([
|
||||
'AdversarialPatch', # error
|
||||
'AdversarialPatchNumpy', # error
|
||||
'BasicIterativeMethod', # error
|
||||
'BrendelBethgeAttack', # error
|
||||
'CarliniWagnerASR', # no ASR models
|
||||
'DPatch', # error
|
||||
'DecisionTreeAttack', # error
|
||||
'FeatureAdversariesNumpy', # error
|
||||
'FeatureAdversariesPyTorch', # error
|
||||
'FeatureAdversariesTensorFlowV2', # error
|
||||
'GeoDA', # error
|
||||
'HighConfidenceLowUncertainty', # error: requires GPR models
|
||||
'LowProFool', # error
|
||||
# 'MIFace', # THIS ACTUALLY WORKS, but counerfit can't deal with it currently
|
||||
'MalwareGDTensorFlow', # error
|
||||
'OverTheAirFlickeringPyTorch', # error
|
||||
'RobustDPatch', # error
|
||||
'ShadowAttack', # error
|
||||
'SquareAttack', # error
|
||||
'TargetedUniversalPerturbation', # error
|
||||
'ThresholdAttack', # error
|
||||
'ZooAttack', # error
|
||||
])
|
||||
|
||||
attack_tags = {
|
||||
"AdversarialPatch": ["image"],
|
||||
"AdversarialPatchNumpy": ["image"],
|
||||
"BasicIterativeMethod": ["image", "tabular"],
|
||||
"BrendelBethgeAttack": ["image", "tabular"],
|
||||
"BoundaryAttack": ["image", "tabular"],
|
||||
"CarliniL0Method": ["image", "tabular"],
|
||||
"CarliniLInfMethod": ["image", "tabular"],
|
||||
"CarliniWagnerASR": ["image", "tabular"],
|
||||
"CopycatCNN": ["image"],
|
||||
"DPatch": ["image"],
|
||||
"DecisionTreeAttack": ["image", "tabular"],
|
||||
"DeepFool": ["image", "tabular"],
|
||||
"ElasticNet": ["image", "tabular"],
|
||||
"FeatureAdversariesNumpy": ["image", "tabular"],
|
||||
"FeatureAdversariesPyTorch": ["image", "tabular"],
|
||||
"FeatureAdversariesTensorFlowV2": ["image", "tabular"],
|
||||
"FunctionallyEquivalentExtraction": ["image", "tabular"],
|
||||
"GeoDA": ["image", "tabular"],
|
||||
"HopSkipJump": ["image", "tabular"],
|
||||
"KnockoffNets": ["image", "tabular"],
|
||||
"LabelOnlyDecisionBoundary": ["image", "tabular"],
|
||||
"LowProFool": ["image", "tabular"],
|
||||
"MIFace": ["image", "tabular"],
|
||||
"MalwareGDTensorFlow": ["image", "tabular"],
|
||||
"NewtonFool": ["image", "tabular"],
|
||||
"OverTheAirFlickeringPyTorch": ["image", "tabular"],
|
||||
"ProjectedGradientDescentCommon": ["image", "tabular"],
|
||||
"RobustDPatch": ["image", "tabular"],
|
||||
"SaliencyMapMethod": ["image", "tabular"],
|
||||
"ShadowAttack": ["image", "tabular"],
|
||||
"ShapeShifter": ["image", "tabular"],
|
||||
"ProjectedGradientDescentCommon": ["image", "tabular"],
|
||||
"SimBA": ["image"],
|
||||
"SpatialTransformation": ["image", "tabular"],
|
||||
"SquareAttack": ["image"],
|
||||
"TargetedUniversalPerturbation": ["image", "tabular"],
|
||||
"ThresholdAttack": ["image"],
|
||||
"UniversalPerturbation": ["image"],
|
||||
"Wasserstein": ["image"],
|
||||
"VirtualAdversarialMethod": ["image"],
|
||||
"ZooAttack": ["image"],
|
||||
}
|
||||
|
||||
|
||||
attack_types = {
|
||||
"AdversarialPatch": "WhiteBox",
|
||||
"AdversarialPatchNumpy": "WhiteBox",
|
||||
"BasicIterativeMethod": "WhiteBox",
|
||||
"BrendelBethgeAttack": "WhiteBox",
|
||||
"BoundaryAttack": "BlackBox",
|
||||
"CarliniL0Method": "WhiteBox",
|
||||
"CarliniLInfMethod": "WhiteBox",
|
||||
"CarliniWagnerASR": "WhiteBox",
|
||||
"CopycatCNN": "BlackBox",
|
||||
"DPatch": "WhiteBox",
|
||||
"DecisionTreeAttack": "WhiteBox",
|
||||
"DeepFool": "WhiteBox",
|
||||
"ElasticNet": "WhiteBox",
|
||||
"FeatureAdversariesNumpy": "WhiteBox",
|
||||
"FeatureAdversariesPyTorch": "WhiteBox",
|
||||
"FeatureAdversariesTensorFlowV2": "WhiteBox",
|
||||
"FunctionallyEquivalentExtraction": "BlackBox",
|
||||
"GeoDA": "WhiteBox",
|
||||
"HopSkipJump": "BlackBox",
|
||||
"KnockoffNets": "BlackBox",
|
||||
"LabelOnlyDecisionBoundary": "WhiteBox",
|
||||
"LowProFool": "WhiteBox",
|
||||
"MIFace": "WhiteBox",
|
||||
"MalwareGDTensorFlow": "WhiteBox",
|
||||
"NewtonFool": "WhiteBox",
|
||||
"OverTheAirFlickeringPyTorch": "WhiteBox",
|
||||
"ProjectedGradientDescentCommon": "WhiteBox",
|
||||
"RobustDPatch": "WhiteBox",
|
||||
"SaliencyMapMethod": "WhiteBox",
|
||||
"ShadowAttack": "WhiteBox",
|
||||
"ShapeShifter": "WhiteBox",
|
||||
"ProjectedGradientDescentCommon": "WhiteBox",
|
||||
"SimBA": "WhiteBox",
|
||||
"SpatialTransformation": "WhiteBox",
|
||||
"SquareAttack": "BlackBox",
|
||||
"TargetedUniversalPerturbation": "WhiteBox",
|
||||
"ThresholdAttack": "BlackBox",
|
||||
"UniversalPerturbation": "WhiteBox",
|
||||
"Wasserstein": "WhiteBox",
|
||||
"VirtualAdversarialMethod": "WhiteBox",
|
||||
"ZooAttack": "BlackBox",
|
||||
}
|
||||
|
||||
|
||||
def wrapper(*args, **kwargs):
|
||||
"""
|
||||
This function returns a wrapped estimator for art. It takes anything art asks for a instantiates a class.
|
||||
"""
|
||||
estimators = args[0]
|
||||
|
||||
class OneWrapperToRuleThemAll(*estimators):
|
||||
def __init__(self, *args, **kwargs):
|
||||
for k, v in kwargs.items():
|
||||
self.__dict__[k] = v
|
||||
|
||||
self.postprocessing_defences = []
|
||||
self.preprocessing_operations = []
|
||||
|
||||
def fit(self):
|
||||
pass
|
||||
|
||||
def loss_gradient(self, **kwargs):
|
||||
pass
|
||||
|
||||
def predict(self, x, **kwargs):
|
||||
return np.array(self.predict_wrapper(x, **kwargs))
|
||||
|
||||
def compute_loss(self):
|
||||
pass
|
||||
|
||||
def get_activations(self):
|
||||
pass
|
||||
|
||||
def class_gradient(self):
|
||||
pass
|
||||
|
||||
def input_shape(self):
|
||||
return self.w_input_shape
|
||||
|
||||
def compute_loss_and_decoded_output(self):
|
||||
pass
|
||||
|
||||
def sample_rate(self):
|
||||
pass
|
||||
|
||||
def to_training_mode(self):
|
||||
pass
|
||||
|
||||
def native_label_is_pytorch_format(self):
|
||||
pass
|
||||
|
||||
def perturbation(self):
|
||||
pass
|
||||
|
||||
return OneWrapperToRuleThemAll(estimators, **kwargs)
|
||||
|
||||
|
||||
class ArtFramework(Framework):
|
||||
class ArtFramework(CFFramework):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
|
||||
self.classifiers = defaultdict()
|
||||
|
||||
def load(self):
|
||||
temp_model = tf.keras.models.load_model("counterfit/targets/digits_keras/mnist_model.h5")
|
||||
temp_classifier = KerasClassifier(model=temp_model, channels_first=False)
|
||||
adv_crafter = FastGradientMethod(temp_classifier, eps=0.1)
|
||||
meminf_attack = MembershipInferenceBlackBox(temp_classifier, attack_model_type="nn")
|
||||
backdoor = PoisoningAttackBackdoor(add_pattern_bd)
|
||||
|
||||
def some_func():
|
||||
return True
|
||||
|
||||
self.load_classifiers()
|
||||
art_attacks = importlib.import_module("art.attacks")
|
||||
@classmethod
|
||||
def get_attacks(cls, framework_path=f"{pathlib.Path(__file__).parent.resolve()}/attacks"):
|
||||
attacks = {}
|
||||
files = glob.glob(f"{framework_path}/*.yml")
|
||||
|
||||
# Hunt for the different attack categories
|
||||
for attack_type in art_attacks.Attack.__subclasses__():
|
||||
if attack_type.__name__ not in attacks.keys():
|
||||
attacks[attack_type.__name__] = []
|
||||
for attack in files:
|
||||
with open(attack, 'r') as f:
|
||||
data = yaml.safe_load(f)
|
||||
|
||||
attacks[data['attack_name']] = data
|
||||
|
||||
for attack_class in attack_type.__subclasses__():
|
||||
if attack_class.__subclasses__():
|
||||
for subclass in attack_class.__subclasses__():
|
||||
attacks[attack_type.__name__].append(subclass)
|
||||
new_attack = subclass
|
||||
else:
|
||||
attacks[attack_type.__name__].append(attack_class)
|
||||
new_attack = attack_class
|
||||
return attacks
|
||||
|
||||
# filter out attacks that are still being worked on
|
||||
if new_attack.__name__ in attacks_still_wip:
|
||||
continue
|
||||
|
||||
try:
|
||||
if new_attack.__name__ not in self.attacks.keys():
|
||||
|
||||
if new_attack.__name__ in attack_tags.keys():
|
||||
attack_data_tags = attack_tags.get(
|
||||
new_attack.__name__)
|
||||
else:
|
||||
attack_data_tags = ["unknown"]
|
||||
|
||||
attack_category = attack_types.get(new_attack.__name__, "unknown")
|
||||
|
||||
if "ImperceptibleASR" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(new_attack._estimator_requirements), masker=locate(
|
||||
'art.attacks.evasion.imperceptible_asr.imperceptible_asr.PsychoacousticMasker')())
|
||||
elif "PoisoningAttackAdversarialEmbedding" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements), backdoor=backdoor, feature_layer="test", target="")
|
||||
elif "PoisoningAttackCleanLabelBackdoor" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements), backdoor=backdoor, proxy_classifier=temp_classifier, target="")
|
||||
elif "FrameSaliencyAttack" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
(temp_classifier), attacker=adv_crafter))
|
||||
elif "FeatureAdversariesPyTorch" == new_attack.__name__:
|
||||
loaded_attack = new_attack(
|
||||
wrapper(new_attack._estimator_requirements), delta=0.1, step_size=1)
|
||||
elif "FeatureAdversariesTensorFlowV2" == new_attack.__name__:
|
||||
loaded_attack = new_attack(
|
||||
wrapper(new_attack._estimator_requirements), delta=0.1, step_size=1)
|
||||
elif "FeatureAdversariesNumpy" == new_attack.__name__:
|
||||
loaded_attack = new_attack(
|
||||
wrapper(new_attack._estimator_requirements), delta=0.1, layer=1)
|
||||
elif "MalwareGDTensorFlow" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements), embedding_weights=np.array([1, 2]), param_dic={})
|
||||
elif "ShapeShifter" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements, random_transform=some_func))
|
||||
elif "AttributeInferenceBaseline" == new_attack.__name__:
|
||||
loaded_attack = new_attack(
|
||||
wrapper(temp_classifier))
|
||||
elif "AttributeInferenceMembership" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
temp_classifier), membership_attack=meminf_attack)
|
||||
elif "AutoProjectedGradientDescent" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements, loss_type=None))
|
||||
elif "SquareAttack" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements), adv_criterion=np.array([1.0]), loss=some_func())
|
||||
elif "MembershipInferenceBlackBox" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements))
|
||||
elif "LabelOnlyDecisionBoundary" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements), distance_threshold_tau=1.0)
|
||||
elif "FunctionallyEquivalentExtraction" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements,
|
||||
_nb_classes=10,
|
||||
input_shape=(28, 28, 1)),
|
||||
num_neurons=10)
|
||||
elif "ZooAttack" == new_attack.__name__:
|
||||
loaded_attack = new_attack(wrapper(
|
||||
new_attack._estimator_requirements,
|
||||
clip_values=(0.0, 255.0),
|
||||
input_shape=(1, 28, 28),
|
||||
nb_classes=10,
|
||||
channels_first=True))
|
||||
else:
|
||||
loaded_attack = new_attack(
|
||||
wrapper(
|
||||
new_attack._estimator_requirements,
|
||||
input_shape=(1, 28, 28),
|
||||
channels_first=True, _channels_first=True,
|
||||
clip_values=(0.0, 255.0), _clip_values=(0.0, 255.0),
|
||||
nb_classes=10, _nb_classes=10
|
||||
))
|
||||
|
||||
default_params = {}
|
||||
for i in loaded_attack.attack_params:
|
||||
default_params[i] = loaded_attack.__dict__.get(i)
|
||||
|
||||
for estimator in new_attack._estimator_requirements:
|
||||
try:
|
||||
for prop in estimator.estimator_params:
|
||||
if prop == "clip_values":
|
||||
default_params["clip_values"] = (
|
||||
0.0, 1.0)
|
||||
if prop == "channels_first":
|
||||
default_params["channels_first"] = False
|
||||
except Exception as e:
|
||||
continue
|
||||
|
||||
# Add the attack to the framework
|
||||
self.add_attack(
|
||||
attack_name=new_attack.__name__,
|
||||
attack_type=attack_type.__name__,
|
||||
attack_category=attack_category,
|
||||
attack_data_tags=attack_data_tags,
|
||||
attack_class=loaded_attack,
|
||||
attack_default_params=default_params)
|
||||
|
||||
except Exception as e:
|
||||
# print(new_attack.__name__, e)
|
||||
continue
|
||||
self.loaded_status = True
|
||||
|
||||
def load_classifiers(self):
|
||||
@classmethod
|
||||
def get_classifiers(cls):
|
||||
"""
|
||||
Load ART classifiers
|
||||
"""
|
||||
classifiers = {}
|
||||
base_import = importlib.import_module(f"art.estimators.classification")
|
||||
|
||||
for classifier in base_import.__dict__.values():
|
||||
if inspect.isclass(classifier):
|
||||
if len(classifier.__subclasses__()) > 0:
|
||||
|
|
@ -359,72 +56,117 @@ class ArtFramework(Framework):
|
|||
else:
|
||||
classifier_name = re.findall(
|
||||
r"\w+", str(subclass).split(".")[-1])[0]
|
||||
self.classifiers[classifier_name] = subclass
|
||||
classifiers[classifier_name] = subclass
|
||||
return classifiers
|
||||
|
||||
def build(self, target: Target, attack: object) -> CFAttack:
|
||||
@classmethod
|
||||
def build(cls, target: CFTarget, attack: str) -> object:
|
||||
"""
|
||||
Build the attack.
|
||||
|
||||
Initialize parameters.
|
||||
Set samples.
|
||||
"""
|
||||
# Return the correct estimator for the target selected.
|
||||
# Keep an empty classifier around for extraction attacks.
|
||||
classifier = self.get_classifier(target)
|
||||
|
||||
# Build the classifier
|
||||
# Pass to helper factory function for attack creation.
|
||||
loaded_attack = attack_factory(attack)
|
||||
|
||||
# Return the correct ART estimator for the target selected.
|
||||
# Keep an empty classifier around for extraction attacks.
|
||||
classifier = cls.set_classifier(target)
|
||||
|
||||
# Build the blackbox classifier
|
||||
if "BlackBox" in classifier.__name__:
|
||||
target_classifier = classifier(
|
||||
target.predict_wrapper,
|
||||
target.target_input_shape,
|
||||
len(target.target_output_classes)
|
||||
target.input_shape,
|
||||
len(target.output_classes)
|
||||
)
|
||||
|
||||
# Build the classifier if log_probs is present
|
||||
elif "QueryEfficientGradient" in classifier.__name__:
|
||||
class QEBBWrapper:
|
||||
def __init__(self) -> None:
|
||||
class ModelWrapper:
|
||||
predict = target.predict_wrapper
|
||||
|
||||
self.model = ModelWrapper()
|
||||
self.clip_values = loaded_attack.estimator.clip_values
|
||||
self.nb_classes = loaded_attack.estimator.nb_classes
|
||||
self.predict = target.predict_wrapper
|
||||
|
||||
def class_gradient(x, label, target=target, num_basis=1, sigma=1.5, clip_values=(0., 255.), round_samples=0.0):
|
||||
epsilon_map = sigma * np.random.normal(size=([num_basis] + list(target.input_shape)))
|
||||
grads = []
|
||||
|
||||
y = target.predict_wrapper(x[0])
|
||||
|
||||
for i in range(len(x)):
|
||||
minus = clip_and_round(
|
||||
np.repeat(x[i], num_basis, axis=0) - epsilon_map,
|
||||
clip_values,
|
||||
round_samples,
|
||||
)
|
||||
plus = clip_and_round(
|
||||
np.repeat(x[i], num_basis, axis=0) + epsilon_map,
|
||||
clip_values,
|
||||
round_samples,
|
||||
)
|
||||
|
||||
new_y_minus = np.array([entropy(y[i], p) for p in target.predict_wrapper(minus)])
|
||||
new_y_plus = np.array([entropy(y[i], p) for p in target.predict_wrapper(plus)])
|
||||
query_efficient_grad = 2 * np.mean(
|
||||
np.multiply(
|
||||
epsilon_map.reshape(num_basis, -1),
|
||||
(new_y_plus - new_y_minus).reshape(num_basis, -1) / (2 * sigma),
|
||||
).reshape([-1] + list(target.input_shape)),
|
||||
axis=0,
|
||||
)
|
||||
grads.append(query_efficient_grad)
|
||||
# grads_array = self._apply_preprocessing_gradient(x, np.array(grads))
|
||||
|
||||
return np.array(grads)
|
||||
|
||||
temp_classifier = QEBBWrapper()
|
||||
setattr(temp_classifier, "class_gradient", class_gradient)
|
||||
|
||||
target_classifier = classifier(
|
||||
classifier=temp_classifier,
|
||||
num_basis = 3,
|
||||
sigma = 1.5
|
||||
)
|
||||
|
||||
setattr(target_classifier, "class_gradient", class_gradient)
|
||||
|
||||
# Everything else takes a model file.
|
||||
else:
|
||||
target_classifier = classifier(
|
||||
model=target.model
|
||||
)
|
||||
|
||||
# 100% build rate. Same % run rate.
|
||||
attack._estimator = target_classifier
|
||||
loaded_attack._estimator = target_classifier
|
||||
|
||||
return attack
|
||||
return loaded_attack
|
||||
|
||||
def run(self, cfattack: CFAttack):
|
||||
@classmethod
|
||||
def run(cls, cfattack: CFAttack):
|
||||
|
||||
# Give the framework an opportunity to preprocess any thing in the attack.
|
||||
cls.pre_attack_processing(cfattack)
|
||||
|
||||
# Find the appropriate "run" function
|
||||
attack_attributes = set(cfattack.attack.__dir__())
|
||||
|
||||
# ART has its own set_params function. Use it.
|
||||
cfattack.attack.set_params(**cfattack.options.get_current_options())
|
||||
attack_attributes = cfattack.attack.__dir__()
|
||||
|
||||
# Run the attack. Each attack type has it's own execution function signature.
|
||||
if "infer" in attack_attributes:
|
||||
results = cfattack.attack.infer(
|
||||
np.array(cfattack.samples, dtype=np.float32), y=np.array(cfattack.target.target_output_classes, dtype=np.float32))
|
||||
|
||||
results = cfattack.attack.infer(cfattack.samples, np.array(cfattack.target.output_classes).astype(np.int64))
|
||||
elif "reconstruct" in attack_attributes:
|
||||
results = cfattack.attack.reconstruct(
|
||||
np.array(cfattack.samples, dtype=np.float32))
|
||||
|
||||
elif "generate" in attack_attributes:
|
||||
if "CarliniWagnerASR" == cfattack.attack_name:
|
||||
y = cfattack.target.target_output_classes
|
||||
elif "FeatureAdversariesNumpy" in attack_attributes:
|
||||
y = cfattack.samples
|
||||
elif "FeatureAdversariesPyTorch" in attack_attributes:
|
||||
y = cfattack.samples
|
||||
elif "FeatureAdversariesTensorFlowV2" in attack_attributes:
|
||||
y = cfattack.samples
|
||||
else:
|
||||
y = None
|
||||
|
||||
if "ZooAttack" == cfattack.attack_name:
|
||||
# patch ZooAttack
|
||||
cfattack.attack.estimator.channels_first = True
|
||||
|
||||
results = cfattack.attack.generate(
|
||||
np.array(cfattack.samples, dtype=np.float32), y=y)
|
||||
original_inputs = np.array(cfattack.samples, dtype=np.float32)
|
||||
results = cfattack.attack.generate(x=original_inputs)
|
||||
|
||||
elif "poison" in attack_attributes:
|
||||
results = cfattack.attack.poison(
|
||||
|
|
@ -437,7 +179,7 @@ class ArtFramework(Framework):
|
|||
elif "extract" in attack_attributes:
|
||||
# Returns a thieved classifier
|
||||
training_shape = (
|
||||
len(cfattack.target.X), *cfattack.target.target_input_shape)
|
||||
len(cfattack.target.X), *cfattack.target.input_shape)
|
||||
|
||||
samples_to_query = cfattack.target.X.reshape(
|
||||
training_shape).astype(np.float32)
|
||||
|
|
@ -449,33 +191,45 @@ class ArtFramework(Framework):
|
|||
print("Not found!")
|
||||
return results
|
||||
|
||||
def post_attack_processing(self, cfattack: CFAttack):
|
||||
attack_attributes = set(cfattack.attack.__dir__())
|
||||
@classmethod
|
||||
def pre_attack_processing(cls, cfattack: CFAttack):
|
||||
cls.set_parameters(cfattack)
|
||||
|
||||
if "generate" in attack_attributes:
|
||||
current_datatype = cfattack.target.target_data_type
|
||||
current_dt_report_gen = get_target_data_type_obj(current_datatype)
|
||||
summary = current_dt_report_gen.get_run_summary(cfattack)
|
||||
current_dt_report_gen.print_run_summary(summary)
|
||||
@staticmethod
|
||||
def post_attack_processing(cfattack: CFAttack):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def set_classifier(cls, target: CFTarget):
|
||||
|
||||
# Match the target.classifier attribute with an ART classifier type
|
||||
classifiers = cls.get_classifiers()
|
||||
|
||||
# If no classifer attribute has been set, assume a closed-box.
|
||||
if not hasattr(target, "classifier"):
|
||||
|
||||
# If the target model returns log_probs, return the relevant estimator
|
||||
if hasattr(target, "log_probs"):
|
||||
if target.log_probs == True:
|
||||
return classifiers.get("QueryEfficientGradientEstimationClassifier")
|
||||
|
||||
elif "extract" in attack_attributes:
|
||||
# Override default reporting for the attack type
|
||||
extract_table = Table(header_style="bold magenta")
|
||||
# Add columns to extraction table
|
||||
extract_table.add_column("Success")
|
||||
extract_table.add_column("Copy Cat Accuracy")
|
||||
extract_table.add_column("Elapsed time")
|
||||
extract_table.add_column("Total Queries")
|
||||
# Else return a plain BB estimator
|
||||
else:
|
||||
return classifiers.get("BlackBoxClassifierNeuralNetwork")
|
||||
|
||||
# Add data to extraction table
|
||||
success = cfattack.success[0] # Starting value
|
||||
thieved_accuracy = cfattack.results
|
||||
elapsed_time = cfattack.elapsed_time
|
||||
num_queries = cfattack.logger.num_queries
|
||||
extract_table.add_row(str(success), str(
|
||||
thieved_accuracy), str(elapsed_time), str(num_queries))
|
||||
|
||||
CFPrint.output(extract_table)
|
||||
|
||||
# Else resolve the correct classifier
|
||||
else:
|
||||
for classifier in classifiers.keys():
|
||||
# This translation is necessary as ART modules use the blackbox/whitebox naming convention for
|
||||
# closed-box/open-box attacks.
|
||||
target_classifier_name = target.classifier.lower()
|
||||
if target_classifier_name == 'closed-box':
|
||||
target_classifier_name = 'blackbox'
|
||||
if target_classifier_name == 'open-box':
|
||||
target_classifier_name = 'whitebox'
|
||||
if target_classifier_name in classifier.lower():
|
||||
return classifiers.get(classifier, None)
|
||||
|
||||
def check_success(self, cfattack: CFAttack) -> bool:
|
||||
attack_attributes = set(cfattack.attack.__dir__())
|
||||
|
|
@ -486,42 +240,24 @@ class ArtFramework(Framework):
|
|||
elif "extract" in attack_attributes:
|
||||
return self.extraction_success(cfattack)
|
||||
|
||||
def get_classifier(self, target: Target):
|
||||
# this code attempts to match the .target_classifier attribute of a target with an ART
|
||||
if not getattr(target, "target_classifier"):
|
||||
return self.classifiers.get("BlackBoxClassifierNeuralNetwork")
|
||||
elif target.target_classifier == None:
|
||||
return self.classifiers.get("BlackBoxClassifierNeuralNetwork")
|
||||
else:
|
||||
for classifier in self.classifiers.keys():
|
||||
if target.target_classifier.lower() in classifier.lower():
|
||||
return self.classifiers.get(classifier)
|
||||
|
||||
def evasion_success(self, cfattack: CFAttack):
|
||||
if cfattack.options.__dict__.get("targeted") == True:
|
||||
labels = cfattack.options.target_labels
|
||||
labels = cfattack.options.attack_parameters['target_labels']
|
||||
targeted = True
|
||||
else:
|
||||
labels = cfattack.initial_labels
|
||||
targeted = False
|
||||
|
||||
success = compute_success_array(
|
||||
cfattack.attack._estimator,
|
||||
cfattack.samples,
|
||||
labels,
|
||||
cfattack.results,
|
||||
targeted
|
||||
)
|
||||
success = compute_success_array(cfattack.attack._estimator, cfattack.samples, labels, cfattack.results, targeted)
|
||||
|
||||
final_outputs, final_labels = cfattack.target.get_sample_labels(
|
||||
cfattack.results)
|
||||
final_outputs, final_labels = cfattack.target.get_sample_labels(cfattack.results)
|
||||
cfattack.final_labels = final_labels
|
||||
cfattack.final_outputs = final_outputs
|
||||
return success
|
||||
|
||||
def extraction_success(self, cfattack: CFAttack):
|
||||
training_shape = (
|
||||
len(cfattack.target.X), *cfattack.target.target_input_shape)
|
||||
len(cfattack.target.X), *cfattack.target.input_shape)
|
||||
training_data = cfattack.target.X.reshape(training_shape)
|
||||
|
||||
victim_preds = np.atleast_1d(np.argmax(
|
||||
|
|
@ -537,3 +273,12 @@ class ArtFramework(Framework):
|
|||
return [True]
|
||||
else:
|
||||
return [False]
|
||||
|
||||
@classmethod
|
||||
def set_parameters(cls, cfattack) -> None:
|
||||
# ART has its own set_params function. Use it.
|
||||
attack_params = {}
|
||||
for k, v in cfattack.options.attack_parameters.items():
|
||||
attack_params[k] = v["current"]
|
||||
cfattack.attack.set_params(**attack_params)
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,19 @@
|
|||
attack_category: inference
|
||||
attack_class: art.attacks.inference.membership_inference.black_box_rule_based.MembershipInferenceBlackBoxRuleBased
|
||||
attack_data_tags: []
|
||||
attack_docs: "\n Implementation of a simple, rule-based open-box membership inference\
|
||||
\ attack.\n\n This implementation uses the simple rule: if the model's prediction\
|
||||
\ for a sample is correct, then it is a\n member. Otherwise, it is not a member.\n\
|
||||
\ "
|
||||
attack_name: black_box_rule_based
|
||||
attack_parameters:
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,88 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.boundary.BoundaryAttack
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Implementation of the boundary attack from Brendel et al. (2018).\
|
||||
\ This is a powerful closed-box attack that\n only requires final class prediction.\n\
|
||||
\n | Paper link: https://arxiv.org/abs/1712.04248\n "
|
||||
attack_name: boundary
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 64
|
||||
docs: The size of the batch used by the estimator during inference.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
delta:
|
||||
default: 0.01
|
||||
docs: Initial step size for the orthogonal step.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
epsilon:
|
||||
default: 0.01
|
||||
docs: Initial step size for the step towards the target.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
init_size:
|
||||
default: 100
|
||||
docs: Maximum number of trials for initial generation of adversarial examples.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
max_iter:
|
||||
default: 5000
|
||||
docs: Maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
num_trial:
|
||||
default: 25
|
||||
docs: Maximum number of trials per iteration.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
sample_size:
|
||||
default: 20
|
||||
docs: Number of samples per trial.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
step_adapt:
|
||||
default: 0.667
|
||||
docs: Factor by which the step sizes are multiplied or divided, must be in the
|
||||
range (0, 1).
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
targeted:
|
||||
default: null
|
||||
docs: Should the attack target one specific class.
|
||||
optimize: {}
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,82 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.carlini.CarliniLInfMethod
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n This is a modified version of the L_2 optimized attack of Carlini\
|
||||
\ and Wagner (2016). It controls the L_Inf\n norm, i.e. the maximum perturbation\
|
||||
\ applied to each pixel.\n "
|
||||
attack_name: carlini
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 128
|
||||
docs: Size of the batch on which adversarial samples are generated.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
confidence:
|
||||
default: 0.0
|
||||
docs: 'Confidence of adversarial examples: a higher value produces examples that
|
||||
are farther away,'
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
eps:
|
||||
default: 0.3
|
||||
docs: An upper bound for the L_0 norm of the adversarial perturbation.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
learning_rate:
|
||||
default: 0.01
|
||||
docs: The initial learning rate for the attack algorithm. Smaller values produce
|
||||
better
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
max_doubling:
|
||||
default: 5
|
||||
docs: Maximum number of doubling steps in the line search optimization.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
max_halving:
|
||||
default: 5
|
||||
docs: Maximum number of halving steps in the line search optimization.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
max_iter:
|
||||
default: 10
|
||||
docs: The maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
targeted:
|
||||
default: null
|
||||
docs: Should the attack target one specific class.
|
||||
optimize: {}
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
attack_category: inversion
|
||||
attack_class: art.attacks.extraction.copycat_cnn.CopycatCNN
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Implementation of the Copycat CNN attack from Rodrigues Correia-Silva\
|
||||
\ et al. (2018).\n\n | Paper link: https://arxiv.org/abs/1806.05476\n "
|
||||
attack_name: copycat_cnn
|
||||
attack_parameters:
|
||||
batch_size_fit:
|
||||
default: 1
|
||||
docs: Size of batches for fitting the thieved classifier.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
batch_size_query:
|
||||
default: 1
|
||||
docs: Size of batches for querying the victim classifier.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
nb_epochs:
|
||||
default: 10
|
||||
docs: Number of epochs to use for training.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
nb_stolen:
|
||||
default: 1
|
||||
docs: Number of queries submitted to the victim classifier to steal it.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
use_probability:
|
||||
default: false
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,55 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.deepfool.DeepFool
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Implementation of the attack from Moosavi-Dezfooli et al. (2015).\n\
|
||||
\n | Paper link: https://arxiv.org/abs/1511.04599\n "
|
||||
attack_name: deepfool
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 1
|
||||
docs: Batch size
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
epsilon:
|
||||
default: 1.0e-06
|
||||
docs: Overshoot parameter.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
max_iter:
|
||||
default: 100
|
||||
docs: The maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
nb_grads:
|
||||
default: 10
|
||||
docs: The number of class gradients (top nb_grads w.r.t. prediction) to compute.
|
||||
This way only the
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,89 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.elastic_net.ElasticNet
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n The elastic net attack of Pin-Yu Chen et al. (2018).\n\n |\
|
||||
\ Paper link: https://arxiv.org/abs/1709.04114\n "
|
||||
attack_name: elastic_net
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 1
|
||||
docs: Internal size of batches on which adversarial samples are generated.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
beta:
|
||||
default: 0.001
|
||||
docs: Hyperparameter trading off L2 minimization for L1 minimization.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
binary_search_steps:
|
||||
default: 9
|
||||
docs: Number of times to adjust constant with binary search (positive value).
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
confidence:
|
||||
default: 0.0
|
||||
docs: 'Confidence of adversarial examples: a higher value produces examples that
|
||||
are farther'
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
decision_rule:
|
||||
default: EN
|
||||
docs: Decision rule. 'EN' means Elastic Net rule, 'L1' means L1 rule, 'L2' means
|
||||
L2 rule.
|
||||
optimize:
|
||||
choice:
|
||||
- EN
|
||||
initial_const:
|
||||
default: 0.001
|
||||
docs: The initial trade-off constant `c` to use to tune the relative importance
|
||||
of distance
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
learning_rate:
|
||||
default: 0.01
|
||||
docs: The initial learning rate for the attack algorithm. Smaller values produce
|
||||
better
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
max_iter:
|
||||
default: 100
|
||||
docs: The maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
targeted:
|
||||
default: null
|
||||
docs: Should the attack target one specific class.
|
||||
optimize: {}
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,28 @@
|
|||
attack_category: inversion
|
||||
attack_class: art.attacks.extraction.functionally_equivalent_extraction.FunctionallyEquivalentExtraction
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n This module implements the Functionally Equivalent Extraction\
|
||||
\ attack for neural networks with two dense layers,\n ReLU activation at the\
|
||||
\ first layer and logits output after the second layer.\n\n | Paper link: https://arxiv.org/abs/1909.01838\n\
|
||||
\ "
|
||||
attack_name: functionally_equivalent_extraction
|
||||
attack_parameters:
|
||||
channels_first:
|
||||
default: false
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,71 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.hop_skip_jump.HopSkipJump
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Implementation of the HopSkipJump attack from Jianbo et al. (2019).\
|
||||
\ This is a powerful closed-box attack that\n only requires final class prediction,\
|
||||
\ and is an advanced version of the boundary attack.\n\n | Paper link: https://arxiv.org/abs/1904.02144\n\
|
||||
\ "
|
||||
attack_name: hop_skip_jump
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 64
|
||||
docs: The size of the batch used by the estimator during inference.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 1000
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
curr_iter:
|
||||
default: 0
|
||||
docs: Refer to attack file.
|
||||
init_eval:
|
||||
default: 100
|
||||
docs: Initial number of evaluations for estimating gradient.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 1000
|
||||
min: 1
|
||||
init_size:
|
||||
default: 100
|
||||
docs: Maximum number of trials for initial generation of adversarial examples.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
max_eval:
|
||||
default: 1000
|
||||
docs: Maximum number of evaluations for estimating gradient.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 10000
|
||||
min: 1
|
||||
max_iter:
|
||||
default: 50
|
||||
docs: Maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 1000
|
||||
min: 1
|
||||
norm:
|
||||
default: 2
|
||||
docs: 'Order of the norm. Possible values: "inf", np.inf or 2.'
|
||||
optimize:
|
||||
choice:
|
||||
inf: "inf"
|
||||
targeted:
|
||||
default: false
|
||||
docs: Should the attack target one specific class.
|
||||
optimize:
|
||||
bool:
|
||||
"true": true
|
||||
"false": false
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,73 @@
|
|||
attack_category: inversion
|
||||
attack_class: art.attacks.extraction.knockoff_nets.KnockoffNets
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Implementation of the Knockoff Nets attack from Orekondy et al.\
|
||||
\ (2018).\n\n | Paper link: https://arxiv.org/abs/1812.02766\n "
|
||||
attack_name: knockoff_nets
|
||||
attack_parameters:
|
||||
batch_size_fit:
|
||||
default: 1
|
||||
docs: Size of batches for fitting the thieved classifier.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
batch_size_query:
|
||||
default: 1
|
||||
docs: Size of batches for querying the victim classifier.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
nb_epochs:
|
||||
default: 10
|
||||
docs: Number of epochs to use for training.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
nb_stolen:
|
||||
default: 1
|
||||
docs: Number of queries submitted to the victim classifier to steal it.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
reward:
|
||||
default: all
|
||||
docs: Reward type, in ['cert', 'div', 'loss', 'all'].
|
||||
optimize:
|
||||
choice:
|
||||
- all
|
||||
sampling_strategy:
|
||||
default: random
|
||||
docs: Sampling strategy, either `random` or `adaptive`.
|
||||
optimize:
|
||||
choice:
|
||||
- random
|
||||
use_probability:
|
||||
default: false
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
attack_category: inference
|
||||
attack_class: art.attacks.inference.membership_inference.label_only_boundary_distance.LabelOnlyDecisionBoundary
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Implementation of Label-Only Inference Attack based on Decision\
|
||||
\ Boundary.\n\n | Paper link: https://arxiv.org/abs/2007.14321 (Choquette-Choo\
|
||||
\ et al.) and https://arxiv.org/abs/2007.15528 (Li\n and Zhang)\n\n You only\
|
||||
\ need to call ONE of the calibrate methods, depending on which attack you want\
|
||||
\ to launch.\n "
|
||||
attack_name: label_only_boundary_distance
|
||||
attack_parameters:
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
distance_threshold_tau:
|
||||
default: 1.0
|
||||
docs: Threshold distance for decision boundary. Samples with boundary distances
|
||||
larger
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,64 @@
|
|||
attack_category: inference
|
||||
attack_class: art.attacks.inference.model_inversion.mi_face.MIFace
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Implementation of the MIFace algorithm from Fredrikson et al.\
|
||||
\ (2015). While in that paper the attack is demonstrated\n specifically against\
|
||||
\ face recognition models, it is applicable more broadly to classifiers with continuous\
|
||||
\ features\n which expose class gradients.\n\n | Paper link: https://dl.acm.org/doi/10.1145/2810103.2813677\n\
|
||||
\ "
|
||||
attack_name: mi_face
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 1
|
||||
docs: Size of internal batches.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
learning_rate:
|
||||
default: 0.1
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
max_iter:
|
||||
default: 10000
|
||||
docs: Maximum number of gradient descent iterations for the model inversion.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
threshold:
|
||||
default: 0.99
|
||||
docs: Threshold for descent stopping criterion.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
window_length:
|
||||
default: 100
|
||||
docs: Length of window for checking whether descent should be aborted.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.newtonfool.NewtonFool
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Implementation of the attack from Uyeong Jang et al. (2017).\n\
|
||||
\n | Paper link: http://doi.acm.org/10.1145/3134600.3134635\n "
|
||||
attack_name: newtonfool
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 1
|
||||
docs: Size of the batch on which adversarial samples are generated.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
eta:
|
||||
default: 0.01
|
||||
docs: The eta coefficient.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
max_iter:
|
||||
default: 100
|
||||
docs: The maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,64 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.pixel_threshold.PixelAttack
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n This attack was originally implemented by Vargas et al. (2019).\
|
||||
\ It is generalisation of One Pixel Attack originally\n implemented by Su et\
|
||||
\ al. (2019).\n\n | One Pixel Attack Paper link: https://arxiv.org/abs/1710.08864\n\
|
||||
\ | Pixel Attack Paper link: https://arxiv.org/abs/1906.06026\n "
|
||||
attack_name: pixel_threshold
|
||||
attack_parameters:
|
||||
channels_first:
|
||||
default: false
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
es:
|
||||
default: 1
|
||||
docs: Indicates whether the attack uses CMAES (0) or DE (1) as Evolutionary Strategy.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
max_iter:
|
||||
default: 100
|
||||
docs: Sets the Maximum iterations to run the Evolutionary Strategies for optimisation.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
targeted:
|
||||
default: false
|
||||
docs: Indicates whether the attack is targeted (True) or untargeted (False).
|
||||
optimize: {}
|
||||
th:
|
||||
default: null
|
||||
docs: threshold value of the Pixel/ Threshold attack. th=None indicates finding
|
||||
a minimum threshold.
|
||||
optimize: {}
|
||||
verbose:
|
||||
default: false
|
||||
docs: Indicates whether to print verbose messages of ES used.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
verbose_es:
|
||||
default: false
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: unknown
|
||||
|
|
@ -0,0 +1,100 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.projected_gradient_descent.projected_gradient_descent_numpy.ProjectedGradientDescentCommon
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Common class for different variations of implementation of the\
|
||||
\ Projected Gradient Descent attack. The attack is an\n iterative method in which,\
|
||||
\ after each iteration, the perturbation is projected on an lp-ball of specified\
|
||||
\ radius (in\n addition to clipping the values of the adversarial sample so that\
|
||||
\ it lies in the permitted data range). This is the\n attack proposed by Madry\
|
||||
\ et al. for adversarial training.\n\n | Paper link: https://arxiv.org/abs/1706.06083\n\
|
||||
\ "
|
||||
attack_name: projected_gradient_descent_numpy
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 32
|
||||
docs: Size of the batch on which adversarial samples are generated.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
eps:
|
||||
default: 0.3
|
||||
docs: Maximum perturbation that the attacker can introduce.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
eps_step:
|
||||
default: 0.1
|
||||
docs: Attack step size (input variation) at each iteration.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
max_iter:
|
||||
default: 100
|
||||
docs: The maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
minimal:
|
||||
default: false
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
norm:
|
||||
default: .inf
|
||||
docs: The norm of the adversarial perturbation supporting "inf", np.inf, 1 or
|
||||
2.
|
||||
optimize:
|
||||
choice:
|
||||
- inf
|
||||
num_random_init:
|
||||
default: 0
|
||||
docs: Number of random initialisations within the epsilon ball. For num_random_init=0
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
random_eps:
|
||||
default: false
|
||||
docs: When True, epsilon is drawn randomly from truncated normal distribution.
|
||||
The literature
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
targeted:
|
||||
default: null
|
||||
docs: Indicates whether the attack is targeted (True) or untargeted (False).
|
||||
optimize: {}
|
||||
tensor_board:
|
||||
default: false
|
||||
docs: 'Activate summary writer for TensorBoard: Default is `False` and deactivated
|
||||
summary writer.'
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.saliency_map.SaliencyMapMethod
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Implementation of the Jacobian-based Saliency Map Attack (Papernot\
|
||||
\ et al. 2016).\n\n | Paper link: https://arxiv.org/abs/1511.07528\n "
|
||||
attack_name: saliency_map
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 1
|
||||
docs: Size of the batch on which adversarial samples are generated.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
gamma:
|
||||
default: 1.0
|
||||
docs: Maximum fraction of features being perturbed (between 0 and 1).
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
theta:
|
||||
default: 0.1
|
||||
docs: Amount of Perturbation introduced to each modified feature per step (can
|
||||
be positive or negative).
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,76 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.simba.SimBA
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n This class implements the closed-box attack `SimBA`.\n\n | Paper\
|
||||
\ link: https://arxiv.org/abs/1905.07121\n "
|
||||
attack_name: simba
|
||||
attack_parameters:
|
||||
attack:
|
||||
default: dct
|
||||
docs: 'attack type: pixel (px) or DCT (dct) attacks'
|
||||
optimize:
|
||||
choice:
|
||||
- dct
|
||||
batch_size:
|
||||
default: 1
|
||||
docs: Batch size (but, batch process unavailable in this implementation)
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
channels_first:
|
||||
default: false
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
epsilon:
|
||||
default: 0.1
|
||||
docs: Overshoot parameter.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
freq_dim:
|
||||
default: 4
|
||||
docs: dimensionality of 2D frequency space (DCT).
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
max_iter:
|
||||
default: 3000
|
||||
docs: The maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
order:
|
||||
default: random
|
||||
docs: 'order of pixel attacks: random or diagonal (diag)'
|
||||
optimize:
|
||||
choice:
|
||||
- random
|
||||
stride:
|
||||
default: 1
|
||||
docs: stride for block order (DCT).
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
targeted:
|
||||
default: null
|
||||
docs: perform targeted attack
|
||||
optimize: {}
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.spatial_transformation.SpatialTransformation
|
||||
attack_data_tags:
|
||||
- image
|
||||
- tabular
|
||||
attack_docs: "\n Implementation of the spatial transformation attack using translation\
|
||||
\ and rotation of inputs. The attack conducts\n closed-box queries to the target\
|
||||
\ model in a grid search over possible translations and rotations to find optimal\n\
|
||||
\ attack parameters.\n\n | Paper link: https://arxiv.org/abs/1712.02779\n\
|
||||
\ "
|
||||
attack_name: spatial_transformation
|
||||
attack_parameters:
|
||||
channels_first:
|
||||
default: false
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
max_rotation:
|
||||
default: 0.0
|
||||
docs: The maximum rotation in either direction in degrees. The value is expected
|
||||
to be in the
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
max_translation:
|
||||
default: 0.0
|
||||
docs: The maximum translation in any direction as percentage of image size. The
|
||||
value is
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
num_rotations:
|
||||
default: 1
|
||||
docs: The number of rotations to search on grid spacing.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
num_translations:
|
||||
default: 1
|
||||
docs: The number of translations to search on grid spacing per direction.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,74 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.universal_perturbation.UniversalPerturbation
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Implementation of the attack from Moosavi-Dezfooli et al. (2016).\
|
||||
\ Computes a fixed perturbation to be applied to all\n future inputs. To this\
|
||||
\ end, it can use any adversarial attack method.\n\n | Paper link: https://arxiv.org/abs/1610.08401\n\
|
||||
\ "
|
||||
attack_name: universal_perturbation
|
||||
attack_parameters:
|
||||
attacker:
|
||||
default: deepfool
|
||||
docs: 'Adversarial attack name. Default is ''deepfool''. Supported names: ''carlini'',
|
||||
''carlini_inf'','
|
||||
optimize:
|
||||
choice:
|
||||
- deepfool
|
||||
attacker_params:
|
||||
default: null
|
||||
docs: Parameters specific to the adversarial attack. If this parameter is not
|
||||
specified,
|
||||
optimize: {}
|
||||
batch_size:
|
||||
default: 32
|
||||
docs: Batch size for model evaluations in UniversalPerturbation.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
delta:
|
||||
default: 0.2
|
||||
docs: desired accuracy
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
eps:
|
||||
default: 10.0
|
||||
docs: Attack step size (input variation).
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
max_iter:
|
||||
default: 20
|
||||
docs: The maximum number of iterations for computing universal perturbation.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
norm:
|
||||
default: .inf
|
||||
docs: 'The norm of the adversarial perturbation. Possible values: "inf", np.inf,
|
||||
2.'
|
||||
optimize:
|
||||
choice:
|
||||
- inf
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.virtual_adversarial.VirtualAdversarialMethod
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n This attack was originally proposed by Miyato et al. (2016) and\
|
||||
\ was used for virtual adversarial training.\n\n | Paper link: https://arxiv.org/abs/1507.00677\n\
|
||||
\ "
|
||||
attack_name: virtual_adversarial
|
||||
attack_parameters:
|
||||
batch_size:
|
||||
default: 1
|
||||
docs: Size of the batch on which adversarial samples are generated.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
eps:
|
||||
default: 0.1
|
||||
docs: Attack step (max input variation).
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
finite_diff:
|
||||
default: 1.0e-06
|
||||
docs: The finite difference parameter.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
max_iter:
|
||||
default: 10
|
||||
docs: The maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,121 @@
|
|||
attack_category: evasion
|
||||
attack_class: art.attacks.evasion.wasserstein.Wasserstein
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Implements ``Wasserstein Adversarial Examples via Projected Sinkhorn\
|
||||
\ Iterations`` as evasion attack.\n\n | Paper link: https://arxiv.org/abs/1902.07906\n\
|
||||
\ "
|
||||
attack_name: wasserstein
|
||||
attack_parameters:
|
||||
ball:
|
||||
default: wasserstein
|
||||
docs: 'The ball of the adversarial perturbation. Possible values: `inf`, `1`,
|
||||
`2` or `wasserstein`.'
|
||||
optimize:
|
||||
choice:
|
||||
- wasserstein
|
||||
batch_size:
|
||||
default: 1
|
||||
docs: Size of batches.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
conjugate_sinkhorn_max_iter:
|
||||
default: 400
|
||||
docs: The maximum number of iterations for the conjugate sinkhorn optimizer.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
eps:
|
||||
default: 0.3
|
||||
docs: Maximum perturbation that the attacker can introduce.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
eps_factor:
|
||||
default: 1.1
|
||||
docs: Factor to increase the epsilon.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
eps_iter:
|
||||
default: 10
|
||||
docs: Number of iterations to increase the epsilon.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
eps_step:
|
||||
default: 0.1
|
||||
docs: Attack step size (input variation) at each iteration.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
kernel_size:
|
||||
default: 5
|
||||
docs: Kernel size for computing the cost matrix.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
max_iter:
|
||||
default: 400
|
||||
docs: The maximum number of iterations.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
norm:
|
||||
default: wasserstein
|
||||
docs: 'The norm of the adversarial perturbation. Possible values: `inf`, `1`,
|
||||
`2` or `wasserstein`.'
|
||||
optimize:
|
||||
choice:
|
||||
- inf
|
||||
p:
|
||||
default: 2
|
||||
docs: The p-wasserstein distance.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
projected_sinkhorn_max_iter:
|
||||
default: 400
|
||||
docs: The maximum number of iterations for the projected sinkhorn optimizer.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
regularization:
|
||||
default: 3000.0
|
||||
docs: Entropy regularization.
|
||||
optimize:
|
||||
discrete:
|
||||
max: 1.0
|
||||
min: 0.01
|
||||
targeted:
|
||||
default: null
|
||||
docs: Indicates whether the attack is targeted (True) or untargeted (False).
|
||||
optimize: {}
|
||||
verbose:
|
||||
default: true
|
||||
docs: Show progress bars.
|
||||
optimize:
|
||||
bool:
|
||||
- true
|
||||
- false
|
||||
attack_type: open-box
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
attack_category: inference
|
||||
attack_class: art.attacks.inference.attribute_inference.white_box_decision_tree.AttributeInferenceWhiteBoxDecisionTree
|
||||
attack_data_tags: []
|
||||
attack_docs: "\n A variation of the method proposed by of Fredrikson et al. in:\n\
|
||||
\ https://dl.acm.org/doi/10.1145/2810103.2813677\n\n Assumes the availability\
|
||||
\ of the attacked model's predictions for the samples under attack, in addition\
|
||||
\ to access to\n the model itself and the rest of the feature values. If this\
|
||||
\ is not available, the true class label of the samples\n may be used as a proxy.\
|
||||
\ Also assumes that the attacked feature is discrete or categorical, with limited\
|
||||
\ number of\n possible values. For example: a boolean feature.\n\n | Paper\
|
||||
\ link: https://dl.acm.org/doi/10.1145/2810103.2813677\n "
|
||||
attack_name: white_box_decision_tree
|
||||
attack_parameters:
|
||||
attack_feature:
|
||||
default: 0
|
||||
docs: The index of the feature to be attacked.
|
||||
optimize:
|
||||
uniform:
|
||||
max: 200
|
||||
min: 1
|
||||
clip_values:
|
||||
default:
|
||||
- 0.0
|
||||
- 1.0
|
||||
docs: Refer to attack file.
|
||||
optimize:
|
||||
uniform:
|
||||
- 0.0
|
||||
- 1.0
|
||||
attack_type: unknown
|
||||
Разница между файлами не показана из-за своего большого размера
Загрузить разницу
|
|
@ -0,0 +1,107 @@
|
|||
import re
|
||||
import yaml
|
||||
import argparse
|
||||
|
||||
from frameworks.art.utils import attack_factory, get_default_params, load_attacks, attack_tags, attacks_still_wip, attack_types
|
||||
|
||||
def generate_configs():
|
||||
|
||||
# Get available attack in ART
|
||||
attacks = load_attacks()
|
||||
|
||||
for k, v in attacks.items():
|
||||
if k in attacks_still_wip:
|
||||
continue
|
||||
|
||||
loaded_attack = attack_factory(v)
|
||||
default_params = get_default_params(loaded_attack)
|
||||
|
||||
attack_docs = re.findall(r"\:param\s(\w+)\:\s+(\w+.*)", loaded_attack.__init__.__doc__)
|
||||
|
||||
docs = {}
|
||||
for i in attack_docs:
|
||||
if i[0] in default_params.keys():
|
||||
docs[i[0]] = i[1]
|
||||
else:
|
||||
continue
|
||||
|
||||
params = {}
|
||||
for k, v in default_params.items():
|
||||
optimize = {}
|
||||
type_v = type(v)
|
||||
if k == "norm":
|
||||
optimize["choice"] = ["inf"]
|
||||
elif type_v is bool:
|
||||
optimize["bool"] = {"true": True, "false": False}
|
||||
elif type_v is str:
|
||||
optimize["choice"] = [v]
|
||||
elif type_v is int:
|
||||
optimize["uniform"] = {"min": 1, "max": 200}
|
||||
elif type_v is float:
|
||||
optimize["discrete"] = {"min": 0.01, "max": 1.0}
|
||||
elif type_v is tuple:
|
||||
optimize["uniform"] = [*v]
|
||||
v = [*v]
|
||||
params[k] = {
|
||||
"docs": docs.get(k, "Refer to attack file."),
|
||||
"default": v,
|
||||
"optimize": optimize
|
||||
}
|
||||
|
||||
# category
|
||||
attack_attributes = loaded_attack.__dir__()
|
||||
|
||||
if "generate" in attack_attributes:
|
||||
attack_category = "evasion"
|
||||
elif "infer" in attack_attributes:
|
||||
attack_category = "inference"
|
||||
elif "reconstruct" in attack_attributes:
|
||||
attack_category = "inversion"
|
||||
elif "extract" in attack_attributes:
|
||||
attack_category = "inversion"
|
||||
elif "poison" or "poison_estimator" in attack_attributes:
|
||||
attack_category = "poison"
|
||||
else:
|
||||
attack_category = "unknown"
|
||||
|
||||
# data tags
|
||||
if loaded_attack.__class__.__name__ in attack_tags.keys():
|
||||
attack_data_tags = attack_tags[loaded_attack.__class__.__name__]
|
||||
else:
|
||||
attack_data_tags = []
|
||||
|
||||
|
||||
# type
|
||||
if loaded_attack.__class__.__name__ in attack_types.keys():
|
||||
attack_type = attack_types[loaded_attack.__class__.__name__]
|
||||
else:
|
||||
attack_type = "unknown"
|
||||
|
||||
|
||||
attack = {
|
||||
"attack_name": loaded_attack.__module__.split(".")[-1],
|
||||
"attack_docs": loaded_attack.__doc__,
|
||||
"attack_class": f"{loaded_attack.__module__}.{loaded_attack.__class__.__name__}",
|
||||
"attack_type": attack_type,
|
||||
"attack_category": attack_category,
|
||||
"attack_data_tags": attack_data_tags,
|
||||
"attack_parameters": params
|
||||
}
|
||||
|
||||
filename = loaded_attack.__module__.split(".")[-1]
|
||||
|
||||
with open(f"counterfit/frameworks/art/attacks/{filename}.yml", "w") as f:
|
||||
yaml.safe_dump(attack, f, sort_keys=True, indent=2)
|
||||
|
||||
def main(args):
|
||||
generate_configs()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--generate", "-g", help="Generate base attack configurations for ART")
|
||||
args = parser.parse_args()
|
||||
|
||||
main(args)
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,237 @@
|
|||
import pydoc
|
||||
import numpy as np
|
||||
|
||||
attacks_still_wip = set([
|
||||
'AdversarialPatch', # error
|
||||
'AdversarialPatchNumpy', # error
|
||||
'BasicIterativeMethod', # error
|
||||
'BrendelBethgeAttack', # error
|
||||
'CarliniWagnerASR', # no ASR models
|
||||
'DPatch', # error
|
||||
'DecisionTreeAttack', # error
|
||||
'FeatureAdversariesNumpy', # error
|
||||
'FeatureAdversariesPyTorch', # error
|
||||
'FeatureAdversariesTensorFlowV2', # error
|
||||
'GeoDA', # error
|
||||
'HighConfidenceLowUncertainty', # error: requires GPR models
|
||||
'LowProFool', # error
|
||||
'MalwareGDTensorFlow', # error
|
||||
'OverTheAirFlickeringPyTorch', # error
|
||||
'RobustDPatch', # error
|
||||
'ShadowAttack', # error
|
||||
# 'SquareAttack', # error
|
||||
'TargetedUniversalPerturbation', # error
|
||||
'ThresholdAttack', # error
|
||||
# 'ZooAttack', # error
|
||||
'AdversarialPatchTensorFlowV2',
|
||||
'AdversarialPatchPyTorch',
|
||||
'AutoProjectedGradientDescent',
|
||||
"AutoAttack",
|
||||
"FrameSaliencyAttack",
|
||||
"ImperceptibleASRPyTorch",
|
||||
"ShapeShifter",
|
||||
'PoisoningAttackAdversarialEmbedding',
|
||||
"PoisoningAttackBackdoor",
|
||||
"PoisoningAttackCleanLabelBackdoor",
|
||||
"PoisoningAttackSVM",
|
||||
"FeatureCollisionAttack",
|
||||
"BullseyePolytopeAttackPyTorch",
|
||||
"AttributeInferenceBlackBox",
|
||||
"AttributeInferenceBaseline",
|
||||
"AttributeInferenceBaselineTrueLabel",
|
||||
"AttributeInferenceWhiteBoxLifestyleDecisionTree",
|
||||
"AttributeInferenceMembership",
|
||||
# "MembershipInferenceBlackBox",
|
||||
"DatabaseReconstruction"
|
||||
|
||||
])
|
||||
|
||||
attack_types = {
|
||||
"AdversarialPatch": "open-box",
|
||||
"AdversarialPatchNumpy": "open-box",
|
||||
"BasicIterativeMethod": "open-box",
|
||||
"BrendelBethgeAttack": "open-box",
|
||||
"BoundaryAttack": "closed-box",
|
||||
"CarliniL0Method": "open-box",
|
||||
"CarliniLInfMethod": "open-box",
|
||||
"CarliniWagnerASR": "open-box",
|
||||
"CopycatCNN": "closed-box",
|
||||
"DPatch": "open-box",
|
||||
"DecisionTreeAttack": "open-box",
|
||||
"DeepFool": "open-box",
|
||||
"ElasticNet": "open-box",
|
||||
"FeatureAdversariesNumpy": "open-box",
|
||||
"FeatureAdversariesPyTorch": "open-box",
|
||||
"FeatureAdversariesTensorFlowV2": "open-box",
|
||||
"FunctionallyEquivalentExtraction": "closed-box",
|
||||
"GeoDA": "open-box",
|
||||
"HopSkipJump": "closed-box",
|
||||
"KnockoffNets": "closed-box",
|
||||
"LabelOnlyDecisionBoundary": "open-box",
|
||||
"LowProFool": "open-box",
|
||||
"MIFace": "open-box",
|
||||
"MalwareGDTensorFlow": "open-box",
|
||||
"NewtonFool": "open-box",
|
||||
"OverTheAirFlickeringPyTorch": "open-box",
|
||||
"ProjectedGradientDescentCommon": "open-box",
|
||||
"RobustDPatch": "open-box",
|
||||
"SaliencyMapMethod": "open-box",
|
||||
"ShadowAttack": "open-box",
|
||||
"ShapeShifter": "open-box",
|
||||
"ProjectedGradientDescentCommon": "open-box",
|
||||
"SimBA": "open-box",
|
||||
"SpatialTransformation": "open-box",
|
||||
"SquareAttack": "closed-box",
|
||||
"TargetedUniversalPerturbation": "open-box",
|
||||
"ThresholdAttack": "closed-box",
|
||||
"UniversalPerturbation": "open-box",
|
||||
"Wasserstein": "open-box",
|
||||
"VirtualAdversarialMethod": "open-box",
|
||||
"ZooAttack": "closed-box",
|
||||
}
|
||||
|
||||
attack_tags = {
|
||||
"AdversarialPatch": ["image"],
|
||||
"AdversarialPatchNumpy": ["image"],
|
||||
"BasicIterativeMethod": ["image", "tabular"],
|
||||
"BrendelBethgeAttack": ["image", "tabular"],
|
||||
"BoundaryAttack": ["image", "tabular"],
|
||||
"CarliniL0Method": ["image", "tabular"],
|
||||
"CarliniLInfMethod": ["image", "tabular"],
|
||||
"CarliniWagnerASR": ["image", "tabular"],
|
||||
"CopycatCNN": ["image"],
|
||||
"DPatch": ["image"],
|
||||
"DecisionTreeAttack": ["image", "tabular"],
|
||||
"DeepFool": ["image", "tabular"],
|
||||
"ElasticNet": ["image", "tabular"],
|
||||
"FeatureAdversariesNumpy": ["image", "tabular"],
|
||||
"FeatureAdversariesPyTorch": ["image", "tabular"],
|
||||
"FeatureAdversariesTensorFlowV2": ["image", "tabular"],
|
||||
"FunctionallyEquivalentExtraction": ["image", "tabular"],
|
||||
"GeoDA": ["image", "tabular"],
|
||||
"HopSkipJump": ["image", "tabular"],
|
||||
"KnockoffNets": ["image", "tabular"],
|
||||
"LabelOnlyDecisionBoundary": ["image", "tabular"],
|
||||
"LowProFool": ["image", "tabular"],
|
||||
"MIFace": ["image", "tabular"],
|
||||
"MalwareGDTensorFlow": ["image", "tabular"],
|
||||
"NewtonFool": ["image", "tabular"],
|
||||
"OverTheAirFlickeringPyTorch": ["image", "tabular"],
|
||||
"ProjectedGradientDescentCommon": ["image", "tabular"],
|
||||
"RobustDPatch": ["image", "tabular"],
|
||||
"SaliencyMapMethod": ["image", "tabular"],
|
||||
"ShadowAttack": ["image", "tabular"],
|
||||
"ShapeShifter": ["image", "tabular"],
|
||||
"ProjectedGradientDescentCommon": ["image", "tabular"],
|
||||
"SimBA": ["image"],
|
||||
"SpatialTransformation": ["image", "tabular"],
|
||||
"SquareAttack": ["image"],
|
||||
"TargetedUniversalPerturbation": ["image", "tabular"],
|
||||
"ThresholdAttack": ["image"],
|
||||
"UniversalPerturbation": ["image"],
|
||||
"Wasserstein": ["image"],
|
||||
"VirtualAdversarialMethod": ["image"],
|
||||
"ZooAttack": ["image"],
|
||||
}
|
||||
|
||||
def wrapper(*args, **kwargs):
|
||||
"""
|
||||
This function returns a wrapped estimator for art. It takes anything art asks for a instantiates a class.
|
||||
"""
|
||||
estimators = args[0]
|
||||
|
||||
class OneWrapperToRuleThemAll(*estimators):
|
||||
def __init__(self, *args, **kwargs):
|
||||
for k, v in kwargs.items():
|
||||
self.__dict__[k] = v
|
||||
|
||||
self.postprocessing_defences = []
|
||||
self.preprocessing_operations = []
|
||||
|
||||
def fit(self):
|
||||
pass
|
||||
|
||||
def loss_gradient(self, **kwargs):
|
||||
pass
|
||||
|
||||
def predict(self, x, **kwargs):
|
||||
return np.array(self.predict_wrapper(x, **kwargs))
|
||||
|
||||
def compute_loss(self):
|
||||
pass
|
||||
|
||||
def get_activations(self):
|
||||
pass
|
||||
|
||||
def class_gradient(self):
|
||||
pass
|
||||
|
||||
def input_shape(self):
|
||||
return self.w_input_shape
|
||||
|
||||
def compute_loss_and_decoded_output(self):
|
||||
pass
|
||||
|
||||
def sample_rate(self):
|
||||
pass
|
||||
|
||||
def to_training_mode(self):
|
||||
pass
|
||||
|
||||
def native_label_is_pytorch_format(self):
|
||||
pass
|
||||
|
||||
def perturbation(self):
|
||||
pass
|
||||
|
||||
return OneWrapperToRuleThemAll(estimators, **kwargs)
|
||||
|
||||
|
||||
def attack_factory(attack_module: str) -> object:
|
||||
"""Take an an attack loaded with pydoc.locate and instantiate it. Used in self.build()
|
||||
|
||||
Args:
|
||||
attack (object): pydoc.locate("art.attacks.evasion.hop_skip_jump")
|
||||
|
||||
Returns:
|
||||
object: Returns an instantiated attack class.
|
||||
"""
|
||||
attack = pydoc.locate(attack_module)
|
||||
loaded_attack = attack(
|
||||
wrapper(
|
||||
attack._estimator_requirements,
|
||||
input_shape=(1, 28, 28),
|
||||
channels_first=True, _channels_first=True,
|
||||
clip_values=(0.0, 255.0), _clip_values=(0.0, 255.0),
|
||||
nb_classes=10, _nb_classes=10
|
||||
))
|
||||
|
||||
return loaded_attack
|
||||
|
||||
def get_default_params(loaded_attack: object) -> dict:
|
||||
"""Gathers the default parameters for the attack.
|
||||
|
||||
Args:
|
||||
loaded_attack (object): An attack object returned by self.attack_factory.
|
||||
|
||||
Returns:
|
||||
dict: the default parameters for the attack.
|
||||
"""
|
||||
default_params = {}
|
||||
for i in loaded_attack.attack_params:
|
||||
default_params[i] = loaded_attack.__dict__.get(i)
|
||||
|
||||
for estimator in loaded_attack._estimator_requirements:
|
||||
try:
|
||||
for prop in estimator.estimator_params:
|
||||
if prop == "clip_values":
|
||||
default_params["clip_values"] = (0.0, 1.0)
|
||||
if prop == "channels_first":
|
||||
default_params["channels_first"] = False
|
||||
except Exception as e:
|
||||
continue
|
||||
|
||||
return default_params
|
||||
|
||||
|
||||
|
||||
|
|
@ -0,0 +1,23 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.ApplyLambda
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Apply a user-defined lambda on an image\n\n @param\
|
||||
\ image: PIL Image to be augmented\n\n @param metadata: if set to be a list,\
|
||||
\ metadata about the function execution\n including its name, the source\
|
||||
\ & dest width, height, etc. will be appended to\n the inputted list.\
|
||||
\ If set to None, no metadata will be appended or returned\n\n @param bboxes:\
|
||||
\ a list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: ApplyLambda
|
||||
attack_parameters:
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Blur
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Blurs the image\n\n @param image: PIL Image to be augmented\n\
|
||||
\n @param metadata: if set to be a list, metadata about the function execution\n\
|
||||
\ including its name, the source & dest width, height, etc. will be appended\
|
||||
\ to\n the inputted list. If set to None, no metadata will be appended\
|
||||
\ or returned\n\n @param bboxes: a list of bounding boxes can be passed in\
|
||||
\ here if desired. If\n provided, this list will be modified in place\
|
||||
\ such that each bounding box is\n transformed according to this function\n\
|
||||
\n @param bbox_format: signifies what bounding box format was used in `bboxes`.\
|
||||
\ Must\n specify `bbox_format` if `bboxes` is provided. Supported bbox_format\
|
||||
\ values\n are \"pascal_voc\", \"pascal_voc_norm\", \"coco\", and \"\
|
||||
yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: Blur
|
||||
attack_parameters:
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
radius:
|
||||
default: 2.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Brightness
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Alters the brightness of the image\n\n @param image:\
|
||||
\ PIL Image to be augmented\n\n @param metadata: if set to be a list, metadata\
|
||||
\ about the function execution\n including its name, the source & dest\
|
||||
\ width, height, etc. will be appended to\n the inputted list. If set\
|
||||
\ to None, no metadata will be appended or returned\n\n @param bboxes: a\
|
||||
\ list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: Brightness
|
||||
attack_parameters:
|
||||
factor:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.ChangeAspectRatio
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Alters the aspect ratio of the image\n\n @param image:\
|
||||
\ PIL Image to be augmented\n\n @param metadata: if set to be a list, metadata\
|
||||
\ about the function execution\n including its name, the source & dest\
|
||||
\ width, height, etc. will be appended to\n the inputted list. If set\
|
||||
\ to None, no metadata will be appended or returned\n\n @param bboxes: a\
|
||||
\ list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: ChangeAspectRatio
|
||||
attack_parameters:
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
ratio:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.ClipImageSize
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Scales the image up or down if necessary to fit in the given\
|
||||
\ min and max\n resolution\n\n @param image: PIL Image to be augmented\n\
|
||||
\n @param metadata: if set to be a list, metadata about the function execution\n\
|
||||
\ including its name, the source & dest width, height, etc. will be appended\
|
||||
\ to\n the inputted list. If set to None, no metadata will be appended\
|
||||
\ or returned\n\n @param bboxes: a list of bounding boxes can be passed in\
|
||||
\ here if desired. If\n provided, this list will be modified in place\
|
||||
\ such that each bounding box is\n transformed according to this function\n\
|
||||
\n @param bbox_format: signifies what bounding box format was used in `bboxes`.\
|
||||
\ Must\n specify `bbox_format` if `bboxes` is provided. Supported bbox_format\
|
||||
\ values\n are \"pascal_voc\", \"pascal_voc_norm\", \"coco\", and \"\
|
||||
yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: ClipImageSize
|
||||
attack_parameters:
|
||||
max_resolution:
|
||||
default: null
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
min_resolution:
|
||||
default: null
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.ColorJitter
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Color jitters the image\n\n @param image: PIL Image\
|
||||
\ to be augmented\n\n @param metadata: if set to be a list, metadata about\
|
||||
\ the function execution\n including its name, the source & dest width,\
|
||||
\ height, etc. will be appended to\n the inputted list. If set to None,\
|
||||
\ no metadata will be appended or returned\n\n @param bboxes: a list of bounding\
|
||||
\ boxes can be passed in here if desired. If\n provided, this list will\
|
||||
\ be modified in place such that each bounding box is\n transformed according\
|
||||
\ to this function\n\n @param bbox_format: signifies what bounding box format\
|
||||
\ was used in `bboxes`. Must\n specify `bbox_format` if `bboxes` is provided.\
|
||||
\ Supported bbox_format values\n are \"pascal_voc\", \"pascal_voc_norm\"\
|
||||
, \"coco\", and \"yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: ColorJitter
|
||||
attack_parameters:
|
||||
brightness_factor:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
contrast_factor:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
saturation_factor:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Contrast
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Alters the contrast of the image\n\n @param image:\
|
||||
\ PIL Image to be augmented\n\n @param metadata: if set to be a list, metadata\
|
||||
\ about the function execution\n including its name, the source & dest\
|
||||
\ width, height, etc. will be appended to\n the inputted list. If set\
|
||||
\ to None, no metadata will be appended or returned\n\n @param bboxes: a\
|
||||
\ list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: Contrast
|
||||
attack_parameters:
|
||||
factor:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.ConvertColor
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Converts the image in terms of color modes\n\n @param\
|
||||
\ image: PIL Image to be augmented\n\n @param metadata: if set to be a list,\
|
||||
\ metadata about the function execution\n including its name, the source\
|
||||
\ & dest width, height, etc. will be appended to\n the inputted list.\
|
||||
\ If set to None, no metadata will be appended or returned\n\n @param bboxes:\
|
||||
\ a list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: ConvertColor
|
||||
attack_parameters:
|
||||
colors:
|
||||
default: 256
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
dither:
|
||||
default: null
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
matrix:
|
||||
default: null
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
mode:
|
||||
default: null
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
palette:
|
||||
default: 0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,38 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Crop
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Crops the image\n\n @param image: PIL Image to be augmented\n\
|
||||
\n @param metadata: if set to be a list, metadata about the function execution\n\
|
||||
\ including its name, the source & dest width, height, etc. will be appended\
|
||||
\ to\n the inputted list. If set to None, no metadata will be appended\
|
||||
\ or returned\n\n @param bboxes: a list of bounding boxes can be passed in\
|
||||
\ here if desired. If\n provided, this list will be modified in place\
|
||||
\ such that each bounding box is\n transformed according to this function\n\
|
||||
\n @param bbox_format: signifies what bounding box format was used in `bboxes`.\
|
||||
\ Must\n specify `bbox_format` if `bboxes` is provided. Supported bbox_format\
|
||||
\ values\n are \"pascal_voc\", \"pascal_voc_norm\", \"coco\", and \"\
|
||||
yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: Crop
|
||||
attack_parameters:
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
x1:
|
||||
default: 0.25
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
x2:
|
||||
default: 0.75
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
y1:
|
||||
default: 0.25
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
y2:
|
||||
default: 0.75
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.EncodingQuality
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Changes the JPEG encoding quality level\n\n @param\
|
||||
\ image: PIL Image to be augmented\n\n @param metadata: if set to be a list,\
|
||||
\ metadata about the function execution\n including its name, the source\
|
||||
\ & dest width, height, etc. will be appended to\n the inputted list.\
|
||||
\ If set to None, no metadata will be appended or returned\n\n @param bboxes:\
|
||||
\ a list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: EncodingQuality
|
||||
attack_parameters:
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
quality:
|
||||
default: 50
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Grayscale
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Alters an image to be grayscale\n\n @param image: PIL\
|
||||
\ Image to be augmented\n\n @param metadata: if set to be a list, metadata\
|
||||
\ about the function execution\n including its name, the source & dest\
|
||||
\ width, height, etc. will be appended to\n the inputted list. If set\
|
||||
\ to None, no metadata will be appended or returned\n\n @param bboxes: a\
|
||||
\ list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: Grayscale
|
||||
attack_parameters:
|
||||
mode:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.HFlip
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Horizontally flips an image\n\n @param image: PIL Image\
|
||||
\ to be augmented\n\n @param metadata: if set to be a list, metadata about\
|
||||
\ the function execution\n including its name, the source & dest width,\
|
||||
\ height, etc. will be appended to\n the inputted list. If set to None,\
|
||||
\ no metadata will be appended or returned\n\n @param bboxes: a list of bounding\
|
||||
\ boxes can be passed in here if desired. If\n provided, this list will\
|
||||
\ be modified in place such that each bounding box is\n transformed according\
|
||||
\ to this function\n\n @param bbox_format: signifies what bounding box format\
|
||||
\ was used in `bboxes`. Must\n specify `bbox_format` if `bboxes` is provided.\
|
||||
\ Supported bbox_format values\n are \"pascal_voc\", \"pascal_voc_norm\"\
|
||||
, \"coco\", and \"yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: HFlip
|
||||
attack_parameters:
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.MemeFormat
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Creates a new image that looks like a meme, given text and\
|
||||
\ an image\n\n @param image: PIL Image to be augmented\n\n @param\
|
||||
\ metadata: if set to be a list, metadata about the function execution\n \
|
||||
\ including its name, the source & dest width, height, etc. will be appended\
|
||||
\ to\n the inputted list. If set to None, no metadata will be appended\
|
||||
\ or returned\n\n @param bboxes: a list of bounding boxes can be passed in\
|
||||
\ here if desired. If\n provided, this list will be modified in place\
|
||||
\ such that each bounding box is\n transformed according to this function\n\
|
||||
\n @param bbox_format: signifies what bounding box format was used in `bboxes`.\
|
||||
\ Must\n specify `bbox_format` if `bboxes` is provided. Supported bbox_format\
|
||||
\ values\n are \"pascal_voc\", \"pascal_voc_norm\", \"coco\", and \"\
|
||||
yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: MemeFormat
|
||||
attack_parameters:
|
||||
caption_height:
|
||||
default: 250
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
font_file:
|
||||
default: Raleway-ExtraBold.ttf
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
meme_bg_color:
|
||||
default: 250
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
opacity:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
text:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
text_color:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Opacity
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Alters the opacity of an image\n\n @param image: PIL\
|
||||
\ Image to be augmented\n\n @param metadata: if set to be a list, metadata\
|
||||
\ about the function execution\n including its name, the source & dest\
|
||||
\ width, height, etc. will be appended to\n the inputted list. If set\
|
||||
\ to None, no metadata will be appended or returned\n\n @param bboxes: a\
|
||||
\ list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: Opacity
|
||||
attack_parameters:
|
||||
level:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.OverlayEmoji
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Overlay an emoji onto the original image\n\n @param\
|
||||
\ image: PIL Image to be augmented\n\n @param metadata: if set to be a list,\
|
||||
\ metadata about the function execution\n including its name, the source\
|
||||
\ & dest width, height, etc. will be appended to\n the inputted list.\
|
||||
\ If set to None, no metadata will be appended or returned\n\n @param bboxes:\
|
||||
\ a list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: OverlayEmoji
|
||||
attack_parameters:
|
||||
emoji_path:
|
||||
default: smiling_face_with_heart_eyes.png
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
emoji_size:
|
||||
default: 0.15
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
opacity:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
x_pos:
|
||||
default: 0.4
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
y_pos:
|
||||
default: 0.8
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.OverlayOntoScreenshot
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Overlay the image onto a screenshot template so it looks like\
|
||||
\ it was\n screenshotted on Instagram\n\n @param image: PIL Image\
|
||||
\ to be augmented\n\n @param metadata: if set to be a list, metadata about\
|
||||
\ the function execution\n including its name, the source & dest width,\
|
||||
\ height, etc. will be appended to\n the inputted list. If set to None,\
|
||||
\ no metadata will be appended or returned\n\n @param bboxes: a list of bounding\
|
||||
\ boxes can be passed in here if desired. If\n provided, this list will\
|
||||
\ be modified in place such that each bounding box is\n transformed according\
|
||||
\ to this function\n\n @param bbox_format: signifies what bounding box format\
|
||||
\ was used in `bboxes`. Must\n specify `bbox_format` if `bboxes` is provided.\
|
||||
\ Supported bbox_format values\n are \"pascal_voc\", \"pascal_voc_norm\"\
|
||||
, \"coco\", and \"yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: OverlayOntoScreenshot
|
||||
attack_parameters:
|
||||
crop_src_to_fit:
|
||||
default: false
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
max_image_size_pixels:
|
||||
default: null
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
resize_src_to_match_template:
|
||||
default: true
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
template_bboxes_filepath:
|
||||
default: bboxes.json
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
template_filepath:
|
||||
default: web.png
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.OverlayStripes
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Overlay stripe pattern onto the image (by default, stripes\
|
||||
\ are horizontal)\n\n @param image: PIL Image to be augmented\n\n \
|
||||
\ @param metadata: if set to be a list, metadata about the function execution\n\
|
||||
\ including its name, the source & dest width, height, etc. will be appended\
|
||||
\ to\n the inputted list. If set to None, no metadata will be appended\
|
||||
\ or returned\n\n @param bboxes: a list of bounding boxes can be passed in\
|
||||
\ here if desired. If\n provided, this list will be modified in place\
|
||||
\ such that each bounding box is\n transformed according to this function\n\
|
||||
\n @param bbox_format: signifies what bounding box format was used in `bboxes`.\
|
||||
\ Must\n specify `bbox_format` if `bboxes` is provided. Supported bbox_format\
|
||||
\ values\n are \"pascal_voc\", \"pascal_voc_norm\", \"coco\", and \"\
|
||||
yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: OverlayStripes
|
||||
attack_parameters:
|
||||
line_angle:
|
||||
default: 0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
line_color:
|
||||
default: 0.5
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
line_density:
|
||||
default: 0.5
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
line_opacity:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
line_type:
|
||||
default: 0.5
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
line_width:
|
||||
default: 0.5
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.OverlayText
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Overlay text onto the image (by default, text is randomly\
|
||||
\ overlaid)\n\n @param image: PIL Image to be augmented\n\n @param\
|
||||
\ metadata: if set to be a list, metadata about the function execution\n \
|
||||
\ including its name, the source & dest width, height, etc. will be appended\
|
||||
\ to\n the inputted list. If set to None, no metadata will be appended\
|
||||
\ or returned\n\n @param bboxes: a list of bounding boxes can be passed in\
|
||||
\ here if desired. If\n provided, this list will be modified in place\
|
||||
\ such that each bounding box is\n transformed according to this function\n\
|
||||
\n @param bbox_format: signifies what bounding box format was used in `bboxes`.\
|
||||
\ Must\n specify `bbox_format` if `bboxes` is provided. Supported bbox_format\
|
||||
\ values\n are \"pascal_voc\", \"pascal_voc_norm\", \"coco\", and \"\
|
||||
yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: OverlayText
|
||||
attack_parameters:
|
||||
color:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
font_file:
|
||||
default: NotoNaskhArabic-Regular.ttf
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
font_size:
|
||||
default: 0.15
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
opacity:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
text:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
x_pos:
|
||||
default: 0.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
y_pos:
|
||||
default: 0.5
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Pad
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Pads the image\n\n @param image: PIL Image to be augmented\n\
|
||||
\n @param metadata: if set to be a list, metadata about the function execution\n\
|
||||
\ including its name, the source & dest width, height, etc. will be appended\
|
||||
\ to\n the inputted list. If set to None, no metadata will be appended\
|
||||
\ or returned\n\n @param bboxes: a list of bounding boxes can be passed in\
|
||||
\ here if desired. If\n provided, this list will be modified in place\
|
||||
\ such that each bounding box is\n transformed according to this function\n\
|
||||
\n @param bbox_format: signifies what bounding box format was used in `bboxes`.\
|
||||
\ Must\n specify `bbox_format` if `bboxes` is provided. Supported bbox_format\
|
||||
\ values\n are \"pascal_voc\", \"pascal_voc_norm\", \"coco\", and \"\
|
||||
yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: Pad
|
||||
attack_parameters:
|
||||
color:
|
||||
default: 0.25
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
h_factor:
|
||||
default: 0.25
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
w_factor:
|
||||
default: 0.25
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.PadSquare
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Pads the shorter edge of the image such that it is now square-shaped\n\
|
||||
\n @param image: PIL Image to be augmented\n\n @param metadata: if\
|
||||
\ set to be a list, metadata about the function execution\n including\
|
||||
\ its name, the source & dest width, height, etc. will be appended to\n \
|
||||
\ the inputted list. If set to None, no metadata will be appended or returned\n\
|
||||
\n @param bboxes: a list of bounding boxes can be passed in here if desired.\
|
||||
\ If\n provided, this list will be modified in place such that each bounding\
|
||||
\ box is\n transformed according to this function\n\n @param bbox_format:\
|
||||
\ signifies what bounding box format was used in `bboxes`. Must\n specify\
|
||||
\ `bbox_format` if `bboxes` is provided. Supported bbox_format values\n \
|
||||
\ are \"pascal_voc\", \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n \
|
||||
\ @returns: Augmented PIL Image\n "
|
||||
attack_name: PadSquare
|
||||
attack_parameters:
|
||||
color:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.PerspectiveTransform
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Apply a perspective transform to the image so it looks like\
|
||||
\ it was taken\n as a photo from another device (e.g. taking a picture from\
|
||||
\ your phone of a\n picture on a computer).\n\n @param image: PIL\
|
||||
\ Image to be augmented\n\n @param metadata: if set to be a list, metadata\
|
||||
\ about the function execution\n including its name, the source & dest\
|
||||
\ width, height, etc. will be appended to\n the inputted list. If set\
|
||||
\ to None, no metadata will be appended or returned\n\n @param bboxes: a\
|
||||
\ list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: PerspectiveTransform
|
||||
attack_parameters:
|
||||
dx:
|
||||
default: 0.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
dy:
|
||||
default: 0.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
seed:
|
||||
default: 42
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
sigma:
|
||||
default: 50.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Pixelization
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Pixelizes an image\n\n @param image: PIL Image to be\
|
||||
\ augmented\n\n @param metadata: if set to be a list, metadata about the\
|
||||
\ function execution\n including its name, the source & dest width, height,\
|
||||
\ etc. will be appended to\n the inputted list. If set to None, no metadata\
|
||||
\ will be appended or returned\n\n @param bboxes: a list of bounding boxes\
|
||||
\ can be passed in here if desired. If\n provided, this list will be\
|
||||
\ modified in place such that each bounding box is\n transformed according\
|
||||
\ to this function\n\n @param bbox_format: signifies what bounding box format\
|
||||
\ was used in `bboxes`. Must\n specify `bbox_format` if `bboxes` is provided.\
|
||||
\ Supported bbox_format values\n are \"pascal_voc\", \"pascal_voc_norm\"\
|
||||
, \"coco\", and \"yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: Pixelization
|
||||
attack_parameters:
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
ratio:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.RandomEmojiOverlay
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Transform that overlays a random emoji onto an image\n\n \
|
||||
\ @param image: PIL Image to be augmented\n\n @param metadata: if set\
|
||||
\ to be a list, metadata about the function execution\n including its\
|
||||
\ name, the source & dest width, height, etc. will be appended to\n the\
|
||||
\ inputted list. If set to None, no metadata will be appended or returned\n\n \
|
||||
\ @param bboxes: a list of bounding boxes can be passed in here if desired.\
|
||||
\ If\n provided, this list will be modified in place such that each bounding\
|
||||
\ box is\n transformed according to this function\n\n @param bbox_format:\
|
||||
\ signifies what bounding box format was used in `bboxes`. Must\n specify\
|
||||
\ `bbox_format` if `bboxes` is provided. Supported bbox_format values\n \
|
||||
\ are \"pascal_voc\", \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n \
|
||||
\ @returns: Augmented PIL Image\n "
|
||||
attack_name: RandomEmojiOverlay
|
||||
attack_parameters:
|
||||
emoji_directory:
|
||||
default: smileys
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
emoji_size:
|
||||
default: 0.15
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
opacity:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
x_pos:
|
||||
default: 0.4
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
y_pos:
|
||||
default: 0.4
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.RandomNoise
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Adds random noise to the image\n\n @param image: PIL\
|
||||
\ Image to be augmented\n\n @param metadata: if set to be a list, metadata\
|
||||
\ about the function execution\n including its name, the source & dest\
|
||||
\ width, height, etc. will be appended to\n the inputted list. If set\
|
||||
\ to None, no metadata will be appended or returned\n\n @param bboxes: a\
|
||||
\ list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: RandomNoise
|
||||
attack_parameters:
|
||||
mean:
|
||||
default: 0.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
seed:
|
||||
default: 42
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
var:
|
||||
default: 0.01
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Resize
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Resizes an image\n\n @param image: PIL Image to be\
|
||||
\ augmented\n\n @param metadata: if set to be a list, metadata about the\
|
||||
\ function execution\n including its name, the source & dest width, height,\
|
||||
\ etc. will be appended to\n the inputted list. If set to None, no metadata\
|
||||
\ will be appended or returned\n\n @param bboxes: a list of bounding boxes\
|
||||
\ can be passed in here if desired. If\n provided, this list will be\
|
||||
\ modified in place such that each bounding box is\n transformed according\
|
||||
\ to this function\n\n @param bbox_format: signifies what bounding box format\
|
||||
\ was used in `bboxes`. Must\n specify `bbox_format` if `bboxes` is provided.\
|
||||
\ Supported bbox_format values\n are \"pascal_voc\", \"pascal_voc_norm\"\
|
||||
, \"coco\", and \"yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: Resize
|
||||
attack_parameters:
|
||||
height:
|
||||
default: null
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
width:
|
||||
default: null
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,26 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Rotate
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Rotates the image\n\n @param image: PIL Image to be\
|
||||
\ augmented\n\n @param metadata: if set to be a list, metadata about the\
|
||||
\ function execution\n including its name, the source & dest width, height,\
|
||||
\ etc. will be appended to\n the inputted list. If set to None, no metadata\
|
||||
\ will be appended or returned\n\n @param bboxes: a list of bounding boxes\
|
||||
\ can be passed in here if desired. If\n provided, this list will be\
|
||||
\ modified in place such that each bounding box is\n transformed according\
|
||||
\ to this function\n\n @param bbox_format: signifies what bounding box format\
|
||||
\ was used in `bboxes`. Must\n specify `bbox_format` if `bboxes` is provided.\
|
||||
\ Supported bbox_format values\n are \"pascal_voc\", \"pascal_voc_norm\"\
|
||||
, \"coco\", and \"yolo\"\n\n @returns: Augmented PIL Image\n "
|
||||
attack_name: Rotate
|
||||
attack_parameters:
|
||||
degrees:
|
||||
default: 15.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Saturation
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Alters the saturation of an image\n\n @param image:\
|
||||
\ PIL Image to be augmented\n\n @param metadata: if set to be a list, metadata\
|
||||
\ about the function execution\n including its name, the source & dest\
|
||||
\ width, height, etc. will be appended to\n the inputted list. If set\
|
||||
\ to None, no metadata will be appended or returned\n\n @param bboxes: a\
|
||||
\ list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: Saturation
|
||||
attack_parameters:
|
||||
factor:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
attack_category: common-corruption
|
||||
attack_class: augly.image.transforms.Scale
|
||||
attack_data_tags:
|
||||
- image
|
||||
attack_docs: "\n Alters the resolution of an image\n\n @param image:\
|
||||
\ PIL Image to be augmented\n\n @param metadata: if set to be a list, metadata\
|
||||
\ about the function execution\n including its name, the source & dest\
|
||||
\ width, height, etc. will be appended to\n the inputted list. If set\
|
||||
\ to None, no metadata will be appended or returned\n\n @param bboxes: a\
|
||||
\ list of bounding boxes can be passed in here if desired. If\n provided,\
|
||||
\ this list will be modified in place such that each bounding box is\n \
|
||||
\ transformed according to this function\n\n @param bbox_format: signifies\
|
||||
\ what bounding box format was used in `bboxes`. Must\n specify `bbox_format`\
|
||||
\ if `bboxes` is provided. Supported bbox_format values\n are \"pascal_voc\"\
|
||||
, \"pascal_voc_norm\", \"coco\", and \"yolo\"\n\n @returns: Augmented PIL\
|
||||
\ Image\n "
|
||||
attack_name: Scale
|
||||
attack_parameters:
|
||||
factor:
|
||||
default: 0.5
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
interpolation:
|
||||
default: null
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
p:
|
||||
default: 1.0
|
||||
docs: Refer to attack file
|
||||
optimize: {}
|
||||
attack_type: closed-box
|
||||
Некоторые файлы не были показаны из-за слишком большого количества измененных файлов Показать больше
Загрузка…
Ссылка в новой задаче