From 9aee1ea6958fe48edbe289aaef895a902aeccc80 Mon Sep 17 00:00:00 2001 From: Raja Sekhar Rao Dheekonda <43563047+rdheekonda@users.noreply.github.com> Date: Wed, 19 Apr 2023 14:47:04 -0700 Subject: [PATCH] Users/rdheekonda/cleanup (#84) * cleanup readme and renamed static assets * fixed hyperlink in readme * moved hyperlink to the bottom --------- Co-authored-by: Raja Sekhar Rao Dheekonda --- README.md | 20 ++-- examples/DEMO1-satellite.md | 3 +- examples/DEMO2-creditfraud.md | 4 +- examples/DEMO3-moviereviews.md | 3 +- examples/DEMO4-digits.md | 3 +- examples/DEMO5-reinforcement-learning.md | 4 +- examples/DEMO6-digits-closed-box.md | 109 +++++++++--------- ...ci_terminal.png => azure_aci_terminal.png} | Bin ...eployment.png => azure_arm_deployment.png} | Bin ...tance.png => azure_container_instance.png} | Bin ...zure_delete_rg.png => azure_delete_rg.png} | Bin ...re_account_portal.PNG => azure_portal.png} | Bin ...terfit_resource_group.png => azure_rg.png} | Bin ...md.PNG => counterfit_command_terminal.png} | Bin 14 files changed, 74 insertions(+), 72 deletions(-) rename static/{counterfit_container_aci_terminal.png => azure_aci_terminal.png} (100%) rename static/{counterfit_arm_deployment.png => azure_arm_deployment.png} (100%) rename static/{counterfit_container_instance.png => azure_container_instance.png} (100%) rename static/{counterfit_azure_delete_rg.png => azure_delete_rg.png} (100%) rename static/{counterfit_azure_account_portal.PNG => azure_portal.png} (100%) rename static/{counterfit_resource_group.png => azure_rg.png} (100%) rename static/{counterfit_terminal_cmd.PNG => counterfit_command_terminal.png} (100%) diff --git a/README.md b/README.md index 25af4fc..03ee466 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ To run this tool, you need to have the following softwares installed: - Anaconda: https://www.anaconda.com/products/distribution 3. Git installed from the url https://git-scm.com/downloads to clone the `counterfit` repository. -> **Note: Support for running `Counterfit` on Windows and macOS is currently a work in progress. Please stay tuned for updates on when this feature will be available.** +> Note: Support for running `Counterfit` on Windows and macOS is currently a work in progress. Please stay tuned for updates on when this feature will be available.** ## **Operating System Support** @@ -80,7 +80,7 @@ Note: If you do not already have an Azure subscription, you can begin by obtaini 8. On the `Identity verification by card` page, please enter details of a valid credit card. Enter the card information and select Sign Up. 9. Once Azure account is created, select `Go to the Azure portal` and you should be able to see the image as shown below. -![Counterfit Azure Account Portal IMage](./static/counterfit_azure_account_portal.png) +![Azure Account Portal Image](./static/azure_portal.png) #### **To deploy and run Counterfit on Azure Cloud** @@ -90,7 +90,7 @@ Note: If you do not already have an Azure subscription, you can begin by obtaini 2. In the configuration blade, select your Subscription name (for new Azure account users, it will be `Azure subscription 1`), Resource group (Create new if you do not have one.), and Region from the drop-down menu as shown below. Do not change other field values. - ![Counterfit ARM Deployment](./static/counterfit_arm_deployment.png) + ![Azure ARM Deployment](./static/azure_arm_deployment.png) 3. The above deployment would take approximately 5-8 minutes approximately. This deployment involves creating Azure Storage Account resource for storing Counterfit generated original and adversarial images and Azure Container Instance resource for running Counterfit. @@ -106,15 +106,15 @@ Note: If you do not already have an Azure subscription, you can begin by obtaini + Once deployment is successful, go to the Azure Resource Group and select `counterfit` Azure Container Instance resource as shown below. - ![Counterfit Azure Resource Group](./static/counterfit_resource_group.png) - ![Counterfit Azure Container Instance](./static/counterfit_container_instance.png) + ![Azure Resource Group](./static/azure_rg.png) + ![Azure Container Instance](./static/azure_container_instance.png) + Once the above step is completed, it will take you to the `Container instance` page, click `Containers` under `Settings` section on the left side and click `Connect` from the menu and hit `Connect` button again. - ![Counterfit Azure Container Instance Terminal](./static/counterfit_container_aci_terminal.png) + ![Azure Container Instance Terminal](./static/azure_aci_terminal.png) 5. Within the container terminal, launch Counterfit using the command `counterfit` in the terminal. Once Counterfit is loaded, you should be able to see a banner as shown below - ![Counterfit Terminal](./static/counterfit_terminal_cmd.PNG) + ![Counterfit Command Terminal](./static/counterfit_command_terminal.png) #### **Steps to clean up the Azure Resources** @@ -124,7 +124,7 @@ Note: If you do not already have an Azure subscription, you can begin by obtaini 3. Confirm the deletion by typing the name of the resource group in the text box provided. 4. Click on the "Delete" button to delete the resource group, along with all the resources that were deployed as part of this process. -This will ensure that the Azure Container Instance and Storage Account that were deployed as part of this process are also deleted and you will not incur any further charges. Here's a screenshot to help guide you: ![Counterfit Azure Delete Resource Group](./static/counterfit_azure_delete_rg.PNG) +This will ensure that the Azure Container Instance and Storage Account that were deployed as part of this process are also deleted and you will not incur any further charges. Here's a screenshot to help guide you: ![Azure Delete Resource Group](./static/azure_delete_rg.png) #### **Steps to delete an Azure Account** @@ -134,7 +134,7 @@ Here's a link to the official Microsoft documentation on how to cancel an Azure #### **Using Windows with WSL** ->** Note: If you do not have WSL installed on your Windows machine, you can follow the installation instructions provided by Microsoft at https://learn.microsoft.com/en-us/windows/wsl/install. +>Note: If you do not have WSL installed on your Windows machine, you can follow the installation instructions provided by Microsoft at https://learn.microsoft.com/en-us/windows/wsl/install. To create a conda virtual environment for this tool, it is recommended to have miniconda/anaconda installed on your machine. This will allow you to easily manage and maintain dependencies for your project within a separate virtual environment. @@ -205,7 +205,7 @@ new_attack = counterfit.Counterfit.build_attack(target, attack_name) results = counterfit.Counterfit.run_attack(new_attack) ``` -See the [Counterfit examples README.md](examples/README.md) for more information. +See the [Counterfit examples README.md](examples/README.MD) for more information. Notes: - If textattack has been installed, it will initialize by downloading nltk data diff --git a/examples/DEMO1-satellite.md b/examples/DEMO1-satellite.md index e5f239f..801fa7c 100755 --- a/examples/DEMO1-satellite.md +++ b/examples/DEMO1-satellite.md @@ -1,5 +1,4 @@ **# DEMO 1: Satellite Images Red Team Operation -[[Demo Home]](./README.md) ## Objective Change the label of a satellite image (true label: `stadium`) into `airplane`. @@ -143,3 +142,5 @@ Change the label of a satellite image (true label: `stadium`) into `airplane`. 1. What minimum level of model access was required for this model? 2. How many queries were required to (a) change the decision of the ML model and (b) still remain convincing to a human? 3. For HopSkipJump, change `norm`, `max_iter` and `max_eval` parameters to tune the results of the algorithm. Are there tradeoffs between the resulting quality of the adversarial example and number of queries required to create it? + +[[Demo Home]](./README.MD) \ No newline at end of file diff --git a/examples/DEMO2-creditfraud.md b/examples/DEMO2-creditfraud.md index f6a408b..6bf2bea 100755 --- a/examples/DEMO2-creditfraud.md +++ b/examples/DEMO2-creditfraud.md @@ -1,5 +1,4 @@ # DEMO 2: Credit Card Fraud Integrity Attack Assessment -[[Demo Home]](./README.md) ## Objective Determine roughly how many queries are required to successfully evade `creditfraud` model with `HopSkipJump` @@ -197,4 +196,5 @@ Determine roughly how many queries are required to successfully evade `creditfra 2. What is the `sample_index` that was used during this tutorial? (Hint: you did not set it explicitly.) 3. Optimize the attack for Hop Skip Jump (HSJ). Try adjusting parameters such that you have a "1/1 Success", but in few "Total Queries". For `HopSkipJump`, you might play with `max_iter`, `max_eval` and `init_size`. -[[Demo Home]](../demo/README.md) +[[Demo Home]](./README.MD) + diff --git a/examples/DEMO3-moviereviews.md b/examples/DEMO3-moviereviews.md index 6c2f44c..e4eaeb3 100755 --- a/examples/DEMO3-moviereviews.md +++ b/examples/DEMO3-moviereviews.md @@ -1,5 +1,4 @@ # DEMO 3: Movie Reviews Sentiment Analysis -[[Demo Home]](./README.md) ## Objective Change the spelling of a few words to change the sentiment of an ML model that understands movie reviews. @@ -218,4 +217,4 @@ Change the spelling of a few words to change the sentiment of an ML model that u 1. How would you run this attack with the same parameters simultaneously for several different movie reviews? (Hint: how would you include many different values for `sample_index` during `run`?) 2. The first 10 samples include only positive reviews (`predict -i range(10)`), but the last 10 samples include both positive and negative reviews (`predict -i range(90,100)`). Can you find a negative review (`Label` is `0`) for which an attack changes it to a positive review? -[[Demo Home]](../demo/README.md) \ No newline at end of file +[[Demo Home]](./README.MD) diff --git a/examples/DEMO4-digits.md b/examples/DEMO4-digits.md index e05f5dc..0b0f049 100755 --- a/examples/DEMO4-digits.md +++ b/examples/DEMO4-digits.md @@ -1,5 +1,5 @@ # DEMO 4: Open-box attack of an on-disk -[[Demo Home]](./README.md) + ## Objective Compare open-box and API attacks to digits model @@ -224,3 +224,4 @@ Compare open-box and API attacks to digits model 1. What are the model access requirements for `HopSkipJump`? 2. Find a set of _best_ parameters for `HopSkipJump` that provides excellent visual quality with few "Total Queries". +[[Demo Home]](./README.MD) \ No newline at end of file diff --git a/examples/DEMO5-reinforcement-learning.md b/examples/DEMO5-reinforcement-learning.md index 9ae078e..49d7d7a 100644 --- a/examples/DEMO5-reinforcement-learning.md +++ b/examples/DEMO5-reinforcement-learning.md @@ -56,4 +56,6 @@ OpenAI gym, which we are using for our reinforcement learning target, normally h # else set the attack_id to the ID from the cart_pole --attack_id ``` -3. Use the ID of the attack you just ran, with the flag `init_attack_id` if you used `cart_pole_initstate`. Otherwise, use `attack_id` if you used `cart_pole`. \ No newline at end of file +3. Use the ID of the attack you just ran, with the flag `init_attack_id` if you used `cart_pole_initstate`. Otherwise, use `attack_id` if you used `cart_pole`. + +[[Demo Home]](./README.MD) \ No newline at end of file diff --git a/examples/DEMO6-digits-closed-box.md b/examples/DEMO6-digits-closed-box.md index 416dba4..64e553c 100644 --- a/examples/DEMO6-digits-closed-box.md +++ b/examples/DEMO6-digits-closed-box.md @@ -1,5 +1,5 @@ # DEMO 6: Closed-box attack on Digits model -[[Demo Home]](./README.md) + ## Objective Change the label of a digit image (true label: `5`) into `3`. @@ -52,21 +52,21 @@ Change the label of a digit image (true label: `5`) into `3`. 4. Look at a few digits ``` digits_mlp> predict -i range(10) - -┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ -┃ Sample Index ┃ Sample ┃ Label ┃ Output Scores ┃ -┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ -│ 0 │ counterfit/targets/results/predict/initial-digits_mlp-0325662d-sample-0.png │ 5 │ [0.00 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00] │ -│ 1 │ counterfit/targets/results/predict/initial-digits_mlp-5035cf68-sample-1.png │ 0 │ [1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ -│ 2 │ counterfit/targets/results/predict/initial-digits_mlp-bd4a80a3-sample-2.png │ 4 │ [0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00] │ -│ 3 │ counterfit/targets/results/predict/initial-digits_mlp-bcebd1b5-sample-3.png │ 1 │ [0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ -│ 4 │ counterfit/targets/results/predict/initial-digits_mlp-69c2acd8-sample-4.png │ 9 │ [0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00] │ -│ 5 │ counterfit/targets/results/predict/initial-digits_mlp-17d2153c-sample-5.png │ 2 │ [0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ -│ 6 │ counterfit/targets/results/predict/initial-digits_mlp-18a723d7-sample-6.png │ 1 │ [0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ -│ 7 │ counterfit/targets/results/predict/initial-digits_mlp-a3bb68e5-sample-7.png │ 3 │ [0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00] │ -│ 8 │ counterfit/targets/results/predict/initial-digits_mlp-4d8a919b-sample-8.png │ 1 │ [0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ -│ 9 │ counterfit/targets/results/predict/initial-digits_mlp-a5418d19-sample-9.png │ 4 │ [0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00] │ -└──────────────┴─────────────────────────────────────────────────────────────────────────────┴───────┴─────────────────────────────────────────────────────┘ + ┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ + ┃ Sample Index ┃ Sample ┃ Label ┃ Output Scores ┃ + ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ + │ 0 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-032566… │ 5 │ [0.00 0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00] │ + │ 1 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-5035cf… │ 0 │ [1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ + │ 2 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-bd4a80… │ 4 │ [0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00] │ + │ 3 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-bcebd1… │ 1 │ [0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ + │ 4 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-69c2ac… │ 9 │ [0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1.00] │ + │ 5 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-17d215… │ 2 │ [0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ + │ 6 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-18a723… │ 1 │ [0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ + │ 7 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-a3bb68… │ 3 │ [0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00] │ + │ 8 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-4d8a91… │ 1 │ [0.00 1.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00] │ + │ 9 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/predict/initial-digits_mlp-a5418d… │ 4 │ [0.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 0.00 0.00] │ + └──────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────┴─────────────────────────────────────────────────────┘ + ``` @@ -156,63 +156,62 @@ Change the label of a digit image (true label: `5`) into `3`. 8. Use `hop_skip_jump` with changed parameters. ``` digits_mlp>> set_attack hop_skip_jump - [+] success: Using 0abbe6ef + [+] success: Using 92784548 - digits_mlp>HopSkipJump:0abbe6ef> set_params --sample_index 0 --max_eval 1000 --max_iter 50 -┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ -┃ Parameter (type) ┃ Default ┃ Current ┃ New ┃ -┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ -│ Algo Parameters │ │ │ │ -│ -------------------- │ -- │ -- │ -- │ -│ batch_size (int) │ 64 │ 64 │ The size of the batch used by the estimator during inference. │ -│ clip_values (list) │ [0.0, 1.0] │ (0.0, 1.0) │ Refer to attack file. │ -│ curr_iter (int) │ 0 │ 0 │ Refer to attack file. │ -│ init_eval (int) │ 100 │ 100 │ Initial number of evaluations for estimating gradient. │ -│ init_size (int) │ 100 │ 100 │ Maximum number of trials for initial generation of adversarial examples. │ -│ max_eval (int) │ 1000 │ 1250 │ Maximum number of evaluations for estimating gradient. │ -│ max_iter (int) │ 50 │ 60 │ Maximum number of iterations. │ -│ norm (int) │ 2 │ 2 │ Order of the norm. Possible values: "inf", np.inf or 2. │ -│ targeted (bool) │ False │ False │ Should the attack target one specific class. │ -│ verbose (bool) │ True │ True │ Show progress bars. │ -│ target_labels (int) │ 0 │ 0 │ target labels for a targeted attack │ -│ │ │ │ │ -│ CFAttack Options │ │ │ │ -│ -------------------- │ -- │ -- │ -- │ -│ sample_index (int) │ 0 │ 0 │ Sample index to attack │ -│ optimize (bool) │ False │ False │ Use Optuna to optimize attack parameters │ -│ logger (str) │ basic │ basic │ Logger to log queries with │ -└──────────────────────┴────────────┴────────────┴──────────────────────────────────────────────────────────────────────────┘ + digits_mlp>HopSkipJump:92784548> set_params --sample_index 0 --max_eval 1000 --max_iter 50 + ┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓ + ┃ Parameter (type) ┃ Default ┃ Current ┃ New ┃ + ┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩ + │ Algo Parameters │ │ │ │ + │ -------------------- │ -- │ -- │ -- │ + │ batch_size (int) │ 64 │ 64 │ The size of the batch used by the estimator during inference. │ + │ clip_values (list) │ [0.0, 1.0] │ (0.0, 1.0) │ Refer to attack file. │ + │ curr_iter (int) │ 0 │ 0 │ Refer to attack file. │ + │ init_eval (int) │ 100 │ 100 │ Initial number of evaluations for estimating gradient. │ + │ init_size (int) │ 100 │ 100 │ Maximum number of trials for initial generation of adversarial examples. │ + │ max_eval (int) │ 1000 │ 1000 │ Maximum number of evaluations for estimating gradient. │ + │ max_iter (int) │ 50 │ 50 │ Maximum number of iterations. │ + │ norm (int) │ 2 │ 2 │ Order of the norm. Possible values: "inf", np.inf or 2. │ + │ targeted (bool) │ False │ False │ Should the attack target one specific class. │ + │ verbose (bool) │ True │ True │ Show progress bars. │ + │ target_labels (int) │ 0 │ 0 │ target labels for a targeted attack │ + │ │ │ │ │ + │ CFAttack Options │ │ │ │ + │ -------------------- │ -- │ -- │ -- │ + │ sample_index (int) │ 0 │ 0 │ Sample index to attack │ + │ optimize (bool) │ False │ False │ Use Optuna to optimize attack parameters │ + │ logger (str) │ basic │ basic │ Logger to log queries with │ + └──────────────────────┴────────────┴────────────┴──────────────────────────────────────────────────────────────────────────┘ ``` 9. Run the attack ``` - digits_mlp>HopSkipJump:0abbe6ef> run + digits_mlp>HopSkipJump:92784548> run HopSkipJump: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 2.53it/s] - [+] success: Attack completed 0abbe6ef + [+] success: Attack completed 92784548 ``` 10. Show results ``` - digits_mlp>HopSkipJump:0abbe6ef> show results - [-] info: Image has been saved in the location ./results/0abbe6ef/digits_mlp-bcae6586.png + digits_mlp>HopSkipJump:92784548> show results + [-] info: Image has been saved in the location ┏━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┓ ┃ Success ┃ Elapsed time ┃ Total Queries ┃ ┡━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━┩ - │ 1/1 │ 3.6 │ 32125 (9023.7 query/sec) │ + │ 1/1 │ 7.6 │ 24552 (3251.4 query/sec) │ └─────────┴──────────────┴──────────────────────────┘ - ┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓ - ┃ ┃ Input ┃ Adversar… ┃ ┃ ┃ ┃ - ┃ Sample ┃ Label ┃ Label ┃ Max Abs ┃ ┃ ┃ - ┃ Index ┃ (conf) ┃ (conf) ┃ Chg. ┃ Adversarial Input ┃ Success ┃ - ┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩ - │ 0 │ 5 │ 3 │ 2.1758 │ counterfit/targets/results/8f84add0/digits_mlp-95ce47f6.png │ [ True] │ - │ │ (0.999) │ (0.9289) │ │ │ │ - └───────────┴───────────┴───────────┴───────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴─────────┘ + ┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓ + ┃ ┃ ┃ Adversarial Label ┃ ┃ ┃ ┃ + ┃ Sample Index ┃ Input Label (conf) ┃ (conf) ┃ Max Abs Chg. ┃ Adversarial Input ┃ Success ┃ + ┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩ + │ 0 │ 5 (0.9990) │ 3 (0.9983) │ 1.0902 │ https://counterfit4s2tanqztopsc.blob.core.windows.net/counterfit/targets/results/92784548/digits_mlp-913d75ce… │ [ True] │ + └──────────────┴────────────────────┴────────────────────┴──────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────────┴─────────┘ ``` ## (Optional) Self Assessment 1. What are the model access requirements for `HopSkipJump`? 2. Find a set of _best_ parameters for `HopSkipJump` that provides excellent visual quality with few "Total Queries". -3. Try running with the parameters `set_params --sample_index 0 --max_eval 1250 --max_iter 60` and observe the adversarial class label? +3. Try running with the parameters `set_params --sample_index 0 --max_eval 500 --max_iter 10` and observe the adversarial class label? +[[Demo Home]](./README.MD) \ No newline at end of file diff --git a/static/counterfit_container_aci_terminal.png b/static/azure_aci_terminal.png similarity index 100% rename from static/counterfit_container_aci_terminal.png rename to static/azure_aci_terminal.png diff --git a/static/counterfit_arm_deployment.png b/static/azure_arm_deployment.png similarity index 100% rename from static/counterfit_arm_deployment.png rename to static/azure_arm_deployment.png diff --git a/static/counterfit_container_instance.png b/static/azure_container_instance.png similarity index 100% rename from static/counterfit_container_instance.png rename to static/azure_container_instance.png diff --git a/static/counterfit_azure_delete_rg.png b/static/azure_delete_rg.png similarity index 100% rename from static/counterfit_azure_delete_rg.png rename to static/azure_delete_rg.png diff --git a/static/counterfit_azure_account_portal.PNG b/static/azure_portal.png similarity index 100% rename from static/counterfit_azure_account_portal.PNG rename to static/azure_portal.png diff --git a/static/counterfit_resource_group.png b/static/azure_rg.png similarity index 100% rename from static/counterfit_resource_group.png rename to static/azure_rg.png diff --git a/static/counterfit_terminal_cmd.PNG b/static/counterfit_command_terminal.png similarity index 100% rename from static/counterfit_terminal_cmd.PNG rename to static/counterfit_command_terminal.png