microsoft
/
robustdg
зеркало из
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность

Updates to readme and notebooks; ready for merge with the master branch

This commit is contained in:
divyat09 2021-10-04 21:25:11 +00:00
Родитель 30a4aca36e
Коммит 1b5fd5daec
41 изменённых файлов: 377 добавлений и 3106 удалений
Показать статистику Скачать Patch файл Скачать Diff файл Показать все файлы Свернуть все файлы

18
README.rst
Просмотреть файл

@ -4,7 +4,7 @@ Toolkit for Building Robust ML models that generalize to unseen domains (RobustD
`Shruti Tople <https://www.microsoft.com/en-us/research/people/shtople/>`_,
`Amit Sharma <http://www.amitsharma.in>`_
`ICML 2020 Paper <https://arxiv.org/abs/1909.12732>`_ | `MatchDG paper <https://arxiv.org/abs/2006.07500>`_ | `Privacy & DG Connection paper <http://divy.at/privacy_dg.pdf>`_
` Privacy & Causal Learning (ICML 2020) <https://arxiv.org/abs/1909.12732>`_ | ` MatchDG: Causal View of DG (ICML 2021) <http://proceedings.mlr.press/v139/mahajan21b.html>`_ | `Privacy & DG Connection paper <http://divy.at/privacy_dg.pdf>`_
For machine learning models to be reliable, they need to generalize to data
beyond the train distribution. In addition, ML models should be robust to
@ -28,8 +28,7 @@ Let's first load the rotatedMNIST dataset in a suitable format for the resnet18
.. code:: shell
cd data/
python data_gen.py resnet18
python data/data_gen_mnist.py --dataset rot_mnist --model resnet18 --img_h 224 --img_w 224 --subset_size 2000
**Train and evaluate ML model**
@ -37,13 +36,14 @@ The following commands would train and evalute the MatchDG method on the Rotated
.. code:: shell
python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 100 --batch_size 256 --pos_metric cos
python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 1 --epochs 50 --batch_size 64 --pos_metric cos --match_func_aug_case 1
python train.py --dataset rot_mnist --method_name matchdg_erm --match_case -1 --penalty_ws 0.1 --epochs 25 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18
python test.py --dataset rot_mnist --method_name matchdg_erm --match_case -1 --penalty_ws 0.1 --epochs 25 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --test_metric acc
python train.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.0 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --epochs 25
python test.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --pos_metric cos --test_metric match_score
python test.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.0 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --epochs 25 --test_metric acc
python test.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 1 --pos_metric cos --test_metric match_score
Demo
@ -51,7 +51,7 @@ Demo
A quick introduction on how to use our repository can be accessed here in the `Getting Started notebook <https://github.com/microsoft/robustdg/blob/master/docs/notebooks/robustdg_getting_started.ipynb>`_.
If you are interested in reproducing results from the MatchDG paper, check out the `Reproducing results notebook <https://github.com/microsoft/robustdg/blob/master/docs/notebooks/reproducing_results_matchdg_paper.ipynb>`_.
If you are interested in reproducing results from the MatchDG paper, check out the `Reproducing results notebook <https://github.com/microsoft/robustdg/blob/master/docs/notebooks/reproduce_results.ipynb>`_.
Roadmap
-------

121
docs/notebooks/ChestXRay_Translate.ipynb
Просмотреть файл

@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -38,7 +38,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -53,7 +53,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -79,7 +79,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -149,7 +149,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -176,44 +176,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Lung Lesion doesn't exist. Adding nans instead.\n",
"Fracture doesn't exist. Adding nans instead.\n",
"Lung Opacity doesn't exist. Adding nans instead.\n",
"Enlarged Cardiomediastinum doesn't exist. Adding nans instead.\n",
"{'Pleural Other', 'Support Devices'} will be dropped\n",
"Infiltration doesn't exist. Adding nans instead.\n",
"Emphysema doesn't exist. Adding nans instead.\n",
"Fibrosis doesn't exist. Adding nans instead.\n",
"Pleural_Thickening doesn't exist. Adding nans instead.\n",
"Nodule doesn't exist. Adding nans instead.\n",
"Mass doesn't exist. Adding nans instead.\n",
"Hernia doesn't exist. Adding nans instead.\n",
"Atelectasis doesn't exist. Adding nans instead.\n",
"Consolidation doesn't exist. Adding nans instead.\n",
"Infiltration doesn't exist. Adding nans instead.\n",
"Pneumothorax doesn't exist. Adding nans instead.\n",
"Edema doesn't exist. Adding nans instead.\n",
"Emphysema doesn't exist. Adding nans instead.\n",
"Fibrosis doesn't exist. Adding nans instead.\n",
"Effusion doesn't exist. Adding nans instead.\n",
"Pleural_Thickening doesn't exist. Adding nans instead.\n",
"Cardiomegaly doesn't exist. Adding nans instead.\n",
"Nodule doesn't exist. Adding nans instead.\n",
"Mass doesn't exist. Adding nans instead.\n",
"Hernia doesn't exist. Adding nans instead.\n",
"Lung Lesion doesn't exist. Adding nans instead.\n",
"Fracture doesn't exist. Adding nans instead.\n",
"Enlarged Cardiomediastinum doesn't exist. Adding nans instead.\n"
]
}
],
"outputs": [],
"source": [
"d_nih = xrv.datasets.NIH_Dataset(imgpath=root_dir + \"/data/datasets/NIH/images_224/\",\n",
" views=[\"PA\",\"AP\"], unique_patients=False)\n",
@ -233,7 +198,7 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -362,28 +327,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(112120,)\n",
"torch.Size([800, 3, 224, 224]) torch.Size([800, 3, 224, 224]) torch.Size([800]) 400 400\n",
"torch.Size([200, 3, 224, 224]) torch.Size([200, 3, 224, 224]) torch.Size([200]) 100 100\n",
"torch.Size([400, 3, 224, 224]) torch.Size([400, 3, 224, 224]) torch.Size([400]) 200 200\n",
"(191010,)\n",
"torch.Size([800, 3, 224, 224]) torch.Size([800, 3, 224, 224]) torch.Size([800]) 400 400\n",
"torch.Size([200, 3, 224, 224]) torch.Size([200, 3, 224, 224]) torch.Size([200]) 100 100\n",
"torch.Size([400, 3, 224, 224]) torch.Size([400, 3, 224, 224]) torch.Size([400]) 200 200\n",
"(26684,)\n",
"torch.Size([800, 3, 224, 224]) torch.Size([800, 3, 224, 224]) torch.Size([800]) 400 400\n",
"torch.Size([200, 3, 224, 224]) torch.Size([200, 3, 224, 224]) torch.Size([200]) 100 100\n",
"torch.Size([400, 3, 224, 224]) torch.Size([400, 3, 224, 224]) torch.Size([400]) 200 200\n"
]
}
],
"outputs": [],
"source": [
"base_dir=root_dir + '/data/datasets/chestxray/'\n",
" \n",
@ -466,28 +412,9 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(112120,)\n",
"torch.Size([800, 3, 224, 224]) torch.Size([800, 3, 224, 224]) torch.Size([800]) 400 400\n",
"torch.Size([200, 3, 224, 224]) torch.Size([200, 3, 224, 224]) torch.Size([200]) 100 100\n",
"torch.Size([400, 3, 224, 224]) torch.Size([400, 3, 224, 224]) torch.Size([400]) 200 200\n",
"(191010,)\n",
"torch.Size([800, 3, 224, 224]) torch.Size([800, 3, 224, 224]) torch.Size([800]) 400 400\n",
"torch.Size([200, 3, 224, 224]) torch.Size([200, 3, 224, 224]) torch.Size([200]) 100 100\n",
"torch.Size([400, 3, 224, 224]) torch.Size([400, 3, 224, 224]) torch.Size([400]) 200 200\n",
"(26684,)\n",
"torch.Size([800, 3, 224, 224]) torch.Size([800, 3, 224, 224]) torch.Size([800]) 400 400\n",
"torch.Size([200, 3, 224, 224]) torch.Size([200, 3, 224, 224]) torch.Size([200]) 100 100\n",
"torch.Size([400, 3, 224, 224]) torch.Size([400, 3, 224, 224]) torch.Size([400]) 200 200\n"
]
}
],
"outputs": [],
"source": [
"base_dir=root_dir + '/data/datasets/chestxray/'\n",
" \n",
@ -565,29 +492,9 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"(112120,)\n",
"Error:\n",
"torch.Size([800, 3, 224, 224]) torch.Size([800, 3, 224, 224]) torch.Size([800]) 400 400\n",
"torch.Size([200, 3, 224, 224]) torch.Size([200, 3, 224, 224]) torch.Size([200]) 100 100\n",
"torch.Size([400, 3, 224, 224]) torch.Size([400, 3, 224, 224]) torch.Size([400]) 200 200\n",
"(191010,)\n",
"torch.Size([800, 3, 224, 224]) torch.Size([800, 3, 224, 224]) torch.Size([800]) 400 400\n",
"torch.Size([200, 3, 224, 224]) torch.Size([200, 3, 224, 224]) torch.Size([200]) 100 100\n",
"torch.Size([400, 3, 224, 224]) torch.Size([400, 3, 224, 224]) torch.Size([400]) 200 200\n",
"(26684,)\n",
"torch.Size([800, 3, 224, 224]) torch.Size([800, 3, 224, 224]) torch.Size([800]) 400 400\n",
"torch.Size([200, 3, 224, 224]) torch.Size([200, 3, 224, 224]) torch.Size([200]) 100 100\n",
"torch.Size([400, 3, 224, 224]) torch.Size([400, 3, 224, 224]) torch.Size([400]) 200 200\n"
]
}
],
"outputs": [],
"source": [
"base_dir=root_dir + '/data/datasets/chestxray/'\n",
" \n",

4
docs/notebooks/Spur_Rotated_MNIST.ipynb
Просмотреть файл

@ -157,9 +157,9 @@
],
"metadata": {
"kernelspec": {
"display_name": "main-envs",
"display_name": "matchdg-env",
"language": "python",
"name": "main-envs"
"name": "matchdg-env"
},
"language_info": {
"codemirror_mode": {

634
docs/notebooks/HParam_Plots.ipynb → docs/notebooks/beta/HParam_Plots.ipynb
Просмотреть файл

@ -804,640 +804,6 @@
"plt.savefig('results/slab_dataset.jpg')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"a= np.ones(2,3)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"torch.randint(0, 1, (5, 1))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import random\n",
"l=[[0, 1], [2, 3], [9, 8], [0, 2]]\n",
"random.shuffle(l)\n",
"print(l)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"x=np.array([1, 2, 3])\n",
"y=x"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"x[0]= 10\n",
"print(x, y)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import torch \n",
"import copy\n",
"a= torch.tensor([1, 2])\n",
"b= copy.deepcopy(a)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"a[0]=3\n",
"print(a, b)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"a=[1, 2, 3, 5, 6]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"random.choice(a)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"a=torch.tensor([7, 9, 3, 6, 8, 4, 9])\n",
"b, c= torch.sort(a)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(b)\n",
"print(c)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"torch.argmin(a)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"a[c[:3]]"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"(a ==9).nonzero()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"a=torch.rand(2,3,4)\n",
"b= a.flatten(start_dim=0, end_dim=1)\n",
"print(a.shape, b.shape)\n",
"\n",
"c= torch.stack(torch.split(b, a.shape[1]))\n",
"\n",
"print(a==c)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"a"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"b"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"c"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"n= np.array(range(1, 100))\n",
"c1= 181 \n",
"c2= -180\n",
"c3= -40\n",
"c4= -4\n",
"x= c1*(n**np.log2(5)) + c2*(n**2)+ c3*(np.log2(n))*(n**2) + c4*((np.log2(n))**2)*(n**2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"x"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"181*np.log2(5) - 448"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"5*109+64*9"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"n= np.array(range(1, 1000))\n",
"y= 181*np.log2(5)*(n**(np.log2(5)-1)) -400*n - 88*(n)*(np.log2(n)) - 8*n*(np.log2(n)**2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(y)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"181*np.log2(5)*(2**(np.log2(5)-1))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"np.log(n)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"n= np.array(range(1, 100))\n",
"z= 10*n**(np.log2(5)) - (n**2)*(np.log2(n))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"z"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"n=[1/100*np.array(range(1, 10))]\n",
"print(np.cos(n))\n",
"print(np.sin(n))\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"n= np.array(range(1, 10))\n",
"a=[]\n",
"for item in n:\n",
" a.append( np.math.factorial(item) )\n",
"a=np.array(a)\n",
"\n",
"a= np.log2(a)\n",
"b= 0.5*x*np.log2(n)\n",
"print(a >= b)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plt.scatter(a, b)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"print(a)\n",
"print(b)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"n= np.array(range(1, 100))\n",
"a= 2**(np.log2(n)**2)\n",
"b= n**2\n",
"print(a >= b)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"plt.plot(n, a, '--')\n",
"plt.plot(n, b, '.')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"n=10\n",
"x=np.array(range(1, n+1))\n",
"f=[]\n",
"for item in x:\n",
" f.append(item*(n+1-item))\n",
"f=np.array(f)\n",
"print(f >= n)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"k=222\n",
"n=5023"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"s=0\n",
"for i in range(k, n+1):\n",
" s+= i\n",
"b= 0.5*(n**2 + n - k*(k-1))\n",
"print(s, b)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import torch"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"f1='data/datasets/mnist/rot_mnist_lenet/train/seed_0_domain_15_data.pt'\n",
"f2='data/datasets/mnist_2/rot_mnist_lenet/train/seed_0_domain_15_data.pt'\n",
"\n",
"a= torch.load(f1)\n",
"b= torch.load(f2)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"torch.sum(torch.eq(a, b))/(1000*32*32)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [],
"source": [
"import torch\n",
"import torchvision\n",
"from torch import nn, optim\n",
" \n",
"from opacus.dp_model_inspector import DPModelInspector\n",
"from opacus.utils import module_modification"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"model= torchvision.models.resnet18(1)"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Parameter containing:\n",
"tensor([[-0.0185, -0.0705, -0.0518, ..., -0.0390, 0.1735, -0.0410],\n",
" [-0.0818, -0.0944, 0.0174, ..., 0.2028, -0.0248, 0.0372],\n",
" [-0.0332, -0.0566, -0.0242, ..., -0.0344, -0.0227, 0.0197],\n",
" ...,\n",
" [-0.0103, 0.0033, -0.0359, ..., -0.0279, -0.0115, 0.0128],\n",
" [-0.0359, -0.0353, -0.0296, ..., -0.0330, -0.0110, -0.0513],\n",
" [ 0.0021, -0.0248, -0.0829, ..., 0.0417, -0.0500, 0.0663]],\n",
" requires_grad=True)"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.fc.weight"
]
},
{
"cell_type": "code",
"execution_count": 31,
"metadata": {},
"outputs": [],
"source": [
"class GroupNorm(torch.nn.GroupNorm):\n",
" def __init__(self, num_channels, num_groups=32, **kwargs):\n",
" super().__init__(num_groups, num_channels, **kwargs)"
]
},
{
"cell_type": "code",
"execution_count": 32,
"metadata": {},
"outputs": [],
"source": [
"model= torchvision.models.resnet18(0, norm_layer=GroupNorm)"
]
},
{
"cell_type": "code",
"execution_count": 35,
"metadata": {},
"outputs": [],
"source": [
"opt= optim.SGD([ {'params': filter(lambda p: p.requires_grad, model.parameters()) }, \n",
" ], lr= 0.001, weight_decay= 0.0005, momentum= 0.9, nesterov=True )"
]
},
{
"cell_type": "code",
"execution_count": 41,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Linear(in_features=512, out_features=1000, bias=True)"
]
},
"execution_count": 41,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"model.fc"
]
},
{
"cell_type": "code",
"execution_count": 86,
"metadata": {},
"outputs": [],
"source": [
"import torch"
]
},
{
"cell_type": "code",
"execution_count": 87,
"metadata": {},
"outputs": [],
"source": [
"a=torch.rand(3)"
]
},
{
"cell_type": "code",
"execution_count": 88,
"metadata": {},
"outputs": [],
"source": [
"b=a\n",
"c=a.clone().detach()"
]
},
{
"cell_type": "code",
"execution_count": 89,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([0.7193, 0.8302, 0.5062])\n",
"tensor([0.7193, 0.8302, 0.5062])\n",
"tensor([0.7193, 0.8302, 0.5062])\n"
]
}
],
"source": [
"print(a)\n",
"print(b)\n",
"print(c)"
]
},
{
"cell_type": "code",
"execution_count": 90,
"metadata": {},
"outputs": [],
"source": [
"b[2]=-10\n",
"b[1]=-10"
]
},
{
"cell_type": "code",
"execution_count": 91,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([ 0.7193, -10.0000, -10.0000])\n",
"tensor([ 0.7193, -10.0000, -10.0000])\n",
"tensor([0.7193, 0.8302, 0.5062])\n"
]
}
],
"source": [
"print(a)\n",
"print(b)\n",
"print(c)"
]
},
{
"cell_type": "code",
"execution_count": 93,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor(2)"
]
},
"execution_count": 93,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"(b == -10).nonzero()[1,0]"
]
},
{
"cell_type": "code",
"execution_count": null,

157
docs/notebooks/adult_dataset.ipynb → docs/notebooks/beta/adult_dataset.ipynb
Просмотреть файл

@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -17,7 +17,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -340,7 +340,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -349,42 +349,9 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" age workclass education marital_status occupation race sex \\\n",
"0 39 Government Bachelors Single White-Collar White Male \n",
"1 50 Self-Employed Bachelors Married White-Collar White Male \n",
"2 38 Private HS-grad Divorced Blue-Collar White Male \n",
"3 53 Private School Married Blue-Collar Other Male \n",
"4 28 Private Bachelors Married Professional Other Female \n",
"\n",
" hours_per_week income \n",
"0 40 0 \n",
"1 13 0 \n",
"2 40 0 \n",
"3 40 0 \n",
"4 40 0 \n",
" age workclass education marital_status occupation race sex \\\n",
"0 39 Government Bachelors Single White-Collar White Male \n",
"1 50 Self-Employed Bachelors Married White-Collar White Male \n",
"2 38 Private HS-grad Divorced Blue-Collar White Male \n",
"3 53 Private School Married Blue-Collar Other Male \n",
"4 28 Private Bachelors Married Professional Other Female \n",
"\n",
" hours_per_week income \n",
"0 40 0 \n",
"1 13 0 \n",
"2 40 0 \n",
"3 40 0 \n",
"4 40 0 \n"
]
}
],
"outputs": [],
"source": [
"continuous_features=['age', 'hours_per_week']\n",
"outcome_name='income'\n",
@ -401,7 +368,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -416,49 +383,9 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['age',\n",
" 'hours_per_week',\n",
" 'workclass_Government',\n",
" 'workclass_Other/Unknown',\n",
" 'workclass_Private',\n",
" 'workclass_Self-Employed',\n",
" 'education_Assoc',\n",
" 'education_Bachelors',\n",
" 'education_Doctorate',\n",
" 'education_HS-grad',\n",
" 'education_Masters',\n",
" 'education_Prof-school',\n",
" 'education_School',\n",
" 'education_Some-college',\n",
" 'marital_status_Divorced',\n",
" 'marital_status_Married',\n",
" 'marital_status_Separated',\n",
" 'marital_status_Single',\n",
" 'marital_status_Widowed',\n",
" 'occupation_Blue-Collar',\n",
" 'occupation_Other/Unknown',\n",
" 'occupation_Professional',\n",
" 'occupation_Sales',\n",
" 'occupation_Service',\n",
" 'occupation_White-Collar',\n",
" 'race_Other',\n",
" 'race_White',\n",
" 'sex_Female',\n",
" 'sex_Male',\n",
" 'income']"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"outputs": [],
"source": [
"columns"
]
@ -601,19 +528,9 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Gender Stats: (21790, 30) (10771, 30)\n",
"Male: (15128, 30) (6662, 30)\n",
"Female: (9592, 30) (1179, 30)\n"
]
}
],
"outputs": [],
"source": [
"#train_data_vae= train_data_vae[ train_data_vae['income']==0 ]\n",
"#train_data_vae.drop('income', axis=1, inplace=True)\n",
@ -636,53 +553,9 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Domain: 0\n",
"Train Size for each group: 405 Val Size for each group: 45\n",
"Data: train Gender: male Income : low 122\n",
"Data: train Gender: male Income : high 283\n",
"Data: train Gender: female Income : low 284\n",
"Data: train Gender: female Income : high 121\n",
"torch.Size([810, 30]) torch.Size([810])\n",
"Data: val Gender: male Income : low 14\n",
"Data: val Gender: male Income : high 31\n",
"Data: val Gender: female Income : low 32\n",
"Data: val Gender: female Income : high 13\n",
"torch.Size([90, 30]) torch.Size([90])\n",
"\n",
"\n",
"Domain: 1\n",
"Train Size for each group: 405 Val Size for each group: 45\n",
"Data: train Gender: male Income : low 162\n",
"Data: train Gender: male Income : high 243\n",
"Data: train Gender: female Income : low 243\n",
"Data: train Gender: female Income : high 162\n",
"torch.Size([810, 30]) torch.Size([810])\n",
"Data: val Gender: male Income : low 18\n",
"Data: val Gender: male Income : high 27\n",
"Data: val Gender: female Income : low 27\n",
"Data: val Gender: female Income : high 18\n",
"torch.Size([90, 30]) torch.Size([90])\n",
"\n",
"\n",
"Domain: 2\n",
"Train Size for each group: 90 Val Size for each group: 10\n",
"Data: test Gender: male Income : low 81\n",
"Data: test Gender: male Income : high 9\n",
"Data: test Gender: female Income : low 9\n",
"Data: test Gender: female Income : high 81\n",
"torch.Size([180, 30]) torch.Size([180])\n",
"\n",
"\n"
]
}
],
"outputs": [],
"source": [
"#Two groups (Male, Female) equally represented in each domain\n",
"total_domains= 3 \n",
@ -1021,9 +894,9 @@
],
"metadata": {
"kernelspec": {
"display_name": "envs",
"display_name": "matchdg-env",
"language": "python",
"name": "envs"
"name": "matchdg-env"
},
"language_info": {
"codemirror_mode": {
@ -1035,7 +908,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
"version": "3.7.9"
}
},
"nbformat": 4,

204
docs/notebooks/mnist_results.ipynb → docs/notebooks/beta/mnist_results.ipynb
Просмотреть файл

@ -2645,210 +2645,6 @@
"# fig.savefig('a.jpg')"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import os\n",
"os.path.exists(save_path+'.pth')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"/home/t-dimaha/RobustDG/robustdg/results/rot_mnist/logit_hist/erm_0.jpg\">"
]
},
{
"cell_type": "code",
"execution_count": 33,
"metadata": {},
"outputs": [],
"source": [
"x= torch.rand((2,3))\n",
"y= torch.rand((2,1))"
]
},
{
"cell_type": "code",
"execution_count": 36,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([[0.4583, 0.4527, 0.3075, 0.8617],\n",
" [0.5896, 0.9299, 0.6985, 0.5716]])"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"torch.cat((x,y), dim=1)"
]
},
{
"cell_type": "code",
"execution_count": 44,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"torch.Size([6, 1])"
]
},
"execution_count": 44,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"torch.cat( (torch.zeros((3,1)), torch.ones((3,1))), dim=0).shape"
]
},
{
"cell_type": "code",
"execution_count": 47,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"tensor([1., 1., 0., 0., 0.])"
]
},
"execution_count": 47,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x=torch.tensor([1,2,3,4,5])\n",
"y=1.0*(x<3)\n",
"y"
]
},
{
"cell_type": "code",
"execution_count": 50,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"tensor([3]) tensor([4])\n"
]
}
],
"source": [
"x=torch.tensor([1,2,3])\n",
"y=torch.tensor([1,2,4])\n",
"print(x[x!=y], y[x!=y])"
]
},
{
"cell_type": "code",
"execution_count": 51,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1"
]
},
"execution_count": 51,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"1%10"
]
},
{
"cell_type": "code",
"execution_count": 52,
"metadata": {},
"outputs": [
{
"ename": "TypeError",
"evalue": "unsupported operand type(s) for +: 'int' and 'range'",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mTypeError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-52-75708452d08b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;36m1\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m10\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mTypeError\u001b[0m: unsupported operand type(s) for +: 'int' and 'range'"
]
}
],
"source": [
"1+range(1,10)"
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[101, 102, 103, 104, 105, 106, 107, 108, 109, 110]"
]
},
"execution_count": 53,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"arr= 100+ np.array(list(range(1, 11)))\n",
"arr.tolist()"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"2.0"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import numpy as np\n",
"x=[1,2,3]\n",
"np.mean(x)"
]
},
{
"cell_type": "code",
"execution_count": null,

39
docs/notebooks/helper_plots.ipynb
Просмотреть файл

@ -9,7 +9,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -34,11 +34,11 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data_obj= datasets.MNIST('../../data/rot_mnist',\n",
"data_obj= datasets.MNIST('../../data/datasets/mnist/',\n",
" train=True,\n",
" download=False,\n",
" transform=transforms.ToTensor()\n",
@ -55,7 +55,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -67,7 +67,7 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -80,7 +80,7 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
@ -89,13 +89,34 @@
" img.save('../../results/rot_mnist/images/' + str(angles[counter]) + '.jpg')\n",
" counter+=1"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "matchdg-env",
"language": "python",
"name": "python3"
"name": "matchdg-env"
},
"language_info": {
"codemirror_mode": {
@ -107,7 +128,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
"version": "3.7.9"
}
},
"nbformat": 4,

Двоичные данные
docs/notebooks/images/0.jpg
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 569 B

После

Ширина:  |  Высота:  |  Размер: 447 B

Двоичные данные
docs/notebooks/images/15.jpg
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 632 B

После

Ширина:  |  Высота:  |  Размер: 470 B

Двоичные данные
docs/notebooks/images/30.jpg
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 629 B

После

Ширина:  |  Высота:  |  Размер: 542 B

Двоичные данные
docs/notebooks/images/45.jpg
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 655 B

После

Ширина:  |  Высота:  |  Размер: 507 B

Двоичные данные
docs/notebooks/images/60.jpg
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 622 B

После

Ширина:  |  Высота:  |  Размер: 480 B

Двоичные данные
docs/notebooks/images/75.jpg
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 598 B

После

Ширина:  |  Высота:  |  Размер: 460 B

Двоичные данные
docs/notebooks/images/90.jpg
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 561 B

После

Ширина:  |  Высота:  |  Размер: 450 B

Двоичные данные
docs/notebooks/images/Merged.png
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом"

До

Ширина:  |  Высота:  |  Размер: 98 KiB

Двоичные данные
docs/notebooks/images/t_sne.png
Просмотреть файл

Двоичный файл не отображается.
Режим "рядом" Свайп Оверлей

До

Ширина:  |  Высота:  |  Размер: 71 KiB

После

Ширина:  |  Высота:  |  Размер: 94 KiB

286
docs/notebooks/privacy_plots.ipynb Normal file
Просмотреть файл

Различия файлов скрыты, потому что одна или несколько строк слишком длинны

0
docs/notebooks/beta_reproduce_results.ipynb → docs/notebooks/reproduce_results.ipynb
Просмотреть файл

359
docs/notebooks/reproducing_results_matchdg_paper.ipynb
Просмотреть файл

@ -1,359 +0,0 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Reproducing results\n",
"\n",
"\n",
"The following code reproduces results for Slab dataset, Rotated MNIST and Fashion-MNIST dataset, and PACS dataset corresponding to Tables 1, 2, 3, 4, 5, 6 in the main paper.\n",
"\n",
"### Slab Dataset\n",
"\n",
"We provide the exact commands for the slab dataset, follow the section 'Table 1' below to reproduce the results.\n",
"\n",
"### RotMNIST & FashionMNIST\n",
"\n",
"For convenience, we provide the exact commands for Rotated MNIST dataset with training domains set to [15, 30, 45, 60, 75] and the test domains set to [0, 90]. \n",
"\n",
"To obtain results for the FashionMNIST dataset, change the dataset parameter `--dataset` from `rot_mnist` to `fashion_mnist`.\n",
"\n",
"To obtain results for the different set of training domains in the paper, change the input to the parameter `--train_domains` with the list of training domains: `--train_domains [30, 45]` or `--train_domains [30, 45, 60]`\n",
"\n",
"### PACS\n",
"\n",
"For convenience, we provide the exact commands for ResNet-18 evaluation for the test domain Photo.\n",
"\n",
"To obtain results for ResNet-50, AlexNet, change the model_name parameter `--model_name` from `resnet18` to `resnet50`, `alexnet` and change the other hyperparameters accordingly (Details for hyperparameters in Section C.3, Supplementary materials)\n",
"\n",
"To obtain results on different domains than Photo, change the parameters `--train_domains`, `--test_domains` accordingly and change the other hyperparameters accordingly (Details for hyperparameters in Section C.3, Supplementary materials)\n",
"\n",
"## Chest X-ray\n",
"\n",
"Please perform the following step before running the code\n",
"\n",
" -Follow the steps in the Preprocess.ipynb notebook to download and process the Chest X-Ray datasets\n",
" -Then follow the steps in the ChestXRay_Translate.ipynb notebook to perform image translations\n",
"\n",
"### Note regarding hardware requirements\n",
"\n",
"The code requires a GPU device, also the batch size for MatchDG Phase 1 training might need to be adjusted according to the memory limits of the GPU device. In case of CUDA of out of memory issues, try with a smaller batch size."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Installing Libraries\n",
"\n",
"List of all the required packages are mentioned in the file 'requirements.txt'\n",
"\n",
"You may install them as follows: pip install -r requirements.txt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Table 1: Slab Dataset\n",
"\n",
"From the directory `slab`, run the following command\n",
"\n",
"bash final_res.sh"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This will save the results/logs for all the approaches inside the directory `slab/checkpoints/slab/` "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Table 2, 3: RotMNIST & Fashion MNIST\n",
"\n",
"## Prepare Data for Rot MNIST & Fashion MNIST\n",
"\n",
"From the directory `data`, run the following command"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"python data_gen.py resnet18"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Table 2\n",
"Now move back to the root directory.\n",
"\n",
"* ERM: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --epochs 25"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* ERM_RandomMatch:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --epochs 25"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* ERM_PerfectMatch:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --epochs 25"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* MatchDG:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 100 --batch_size 256 --pos_metric cos\n",
"\n",
"python train.py --dataset rot_mnist --method_name matchdg_erm --match_case -1 --penalty_ws 0.1 --epochs 25 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Table 3\n",
"\n",
"* ERM: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --test_metric match_score "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* MatchDG (Default):"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python test.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --pos_metric cos --test_metric match_score"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* MatchDG (PerfMatch):"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 1.0 --match_flag 1 --epochs 100 --batch_size 256 --pos_metric cos\n",
"\n",
"python test.py --dataset rot_mnist --method_name matchdg_ctr --match_case 1.0 --match_flag 1 --pos_metric cos --test_metric match_score "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Table 4, 5: PACS\n",
"\n",
"## Prepare Data for PACS\n",
"\n",
"Download the PACS dataset (https://drive.google.com/drive/folders/0B6x7gtvErXgfUU1WcGY5SzdwZVk) and place it in the directory '/data/datasets/pacs/' "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* RandMatch: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --train_domains art_painting cartoon sketch --test_domains photo --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --lr 0.001 --epochs 50 --model_name resnet18 --weight_decay 0.001 "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* MatchDG:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset pacs --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 100 --batch_size 64 --pos_metric cos --train_domains art_painting cartoon sketch --test_domains photo --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet50"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset pacs --method_name matchdg_erm --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --train_domains art_painting cartoon sketch --test_domains photo --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 50 --lr 0.0005 --batch_size 16 --weight_decay 0.001 --penalty_ws 0.1 --model_name resnet18"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* MDGHybrid:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset pacs --method_name hybrid --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --train_domains art_painting cartoon sketch --test_domains photo --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 50 --lr 0.0005 --batch_size 16 --weight_decay 0.001 --penalty_ws 0.1 --penalty_aug 0.1 --model_name resnet18"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Table 6: Chest X-Ray"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" -Follow the steps in the Preprocess.ipynb notebook to donwload and process the Chest X-Ray datasets\n",
" -Then follow the steps in the ChestXRay_Translate.ipynb notebook to perform image translations"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* ERM: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle_trans --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 0.0 --model_name densenet121"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* RandMatch: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle_trans --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 10.0 --model_name densenet121"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* MatchDG: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset chestxray --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 100 --batch_size 64 --pos_metric cos --train_domains nih_trans chex_trans --test_domains kaggle_trans --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset chestxray --method_name matchdg_erm --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --model_name densenet121 --train_domains nih_trans kaggle_trans --test_domains chex_trans --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --weight_decay 0.0005 --penalty_ws 50.0 --model_name densenet12"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"* MDGHybrid: "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"python train.py --dataset chestxray --method_name hybrid --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --model_name densenet121 --train_domains nih_trans kaggle_trans --test_domains chex_trans --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --weight_decay 0.0005 --penalty_ws 1.0 --penalty_aug 50.0 --model_name densenet12"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.9"
}
},
"nbformat": 4,
"nbformat_minor": 4
}

47
docs/notebooks/robustdg_getting_started.ipynb
Просмотреть файл

@ -50,11 +50,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"### Prepare Data for Rot MNIST & Fashion MNIST\n",
"### Prepare Data for Rot MNIST\n",
"\n",
"From the directory `data`, run the following command\n",
"Run the following command\n",
"\n",
"<code> python data_gen.py resnet18 </code>"
"<code> python data/data_gen_mnist.py --dataset rot_mnist --model resnet18 --img_h 224 --img_w 224 --subset_size 2000 </code>"
]
},
{
@ -69,7 +69,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"<code> python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --epochs 25 </code>"
"<code> python train.py --dataset rot_mnist --method_name erm_match --match_case 0.0 --penalty_ws 0.0 --epochs 25 </code>"
]
},
{
@ -78,7 +78,7 @@
"source": [
"### MatchDG: Domain generalization via causal matching\n",
"\n",
"The MatchDG model regularize the ERM training objective by matching data samples across domains that were generated from the same base object. More details are in the [Arxiv paper](https://arxiv.org/abs/2006.07500).\n",
"The MatchDG model regularize the ERM training objective by matching data samples across domains that were generated from the same base object. More details are in the [paper](https://arxiv.org/abs/2006.07500).\n",
"\n",
"Train the MatchDG model on Rotated MNIST by executing the following command\n",
"\n",
@ -91,7 +91,7 @@
"source": [
"#### Phase 1: Learning Match Function\n",
"\n",
"<code> python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 100 --batch_size 256 --pos_metric cos\n",
"<code> python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 1 --epochs 50 --batch_size 64 --pos_metric cos --match_func_aug_case 1\n",
"</code>"
]
},
@ -101,7 +101,7 @@
"source": [
"#### Phase 2: Learning Classifier regularised on the Match Function\n",
"\n",
"<code> python train.py --dataset rot_mnist --method_name matchdg_erm --match_case -1 --penalty_ws 0.1 --epochs 25 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 </code>"
"<code> python train.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.0 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --epochs 25 </code>"
]
},
{
@ -138,7 +138,7 @@
"source": [
"%%bash\n",
"cd ../..\n",
"python test.py --test_metric acc --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0"
"python test.py --test_metric acc --dataset rot_mnist --method_name erm_match --match_case 0.0 --penalty_ws 0.0"
]
},
{
@ -158,14 +158,7 @@
"source": [
"%%bash\n",
"cd ../..\n",
"python test.py --test_metric acc --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The results indicate that MatchDG (96.1) outperforms ERM (93.9) on OOD accuracy by approximately 2 percent "
"python test.py --test_metric acc --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.0 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18"
]
},
{
@ -192,7 +185,7 @@
"source": [
"%%bash\n",
"cd ../..\n",
"python test.py --test_metric t_sne --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --pos_metric cos"
"python test.py --test_metric t_sne --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 1 --pos_metric cos"
]
},
{
@ -212,7 +205,7 @@
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"\n",
"save_path= \"../../results/rot_mnist/matchdg_ctr/logit_match/train_['15', '30', '45', '60', '75']/Model_0.01_5_1_0_resnet18_label.json\"\n",
"save_path= \"../../results/rot_mnist/matchdg_ctr/logit_match/train_['15', '30', '45', '60', '75']/Model_0.0_5_1_0_resnet18_label.json\"\n",
"with open(save_path) as f:\n",
" data = json.load(f)\n",
"\n",
@ -260,7 +253,7 @@
"source": [
"%%bash\n",
"cd ../..\n",
"python test.py --test_metric mia --mia_logit 1 --mia_sample_size 2000 --batch_size 64 --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0"
"python test.py --test_metric mia --mia_logit 1 --mia_sample_size 2000 --batch_size 64 --dataset rot_mnist --method_name erm_match --match_case 0.0 --penalty_ws 0.0"
]
},
{
@ -278,22 +271,22 @@
"source": [
"%%bash\n",
"cd ../..\n",
"python test.py --test_metric mia --mia_logit 1 --mia_sample_size 2000 --batch_size 64 --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5"
"python test.py --test_metric mia --mia_logit 1 --mia_sample_size 2000 --batch_size 64 --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.0 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18"
]
},
{
"cell_type": "markdown",
"cell_type": "code",
"execution_count": null,
"metadata": {},
"source": [
"We evaluate ERM and MatchDG on MIA accuracy and observe a similar pattern to that of OOD accuracy; ERM gives test MIA accuracy of 57.2 percent, which gets imporved to 53.1 percent for MatchDG. MatchDG is able to improve the generalization, which makes it difficult for MIA to detemine samples derived from the train versus test distribution"
]
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "matchdg-env",
"language": "python",
"name": "python3"
"name": "matchdg-env"
},
"language_info": {
"codemirror_mode": {
@ -305,7 +298,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.7"
"version": "3.7.9"
}
},
"nbformat": 4,

857
docs/notebooks/usenix_results.ipynb
Просмотреть файл

Различия файлов скрыты, потому что одна или несколько строк слишком длинны

2
evaluation/t_sne.py
Просмотреть файл

@ -58,7 +58,7 @@ class TSNE(BaseEval):
domain_all=[]
for batch_idx, (x_e, y_e, d_e, idx_e) in enumerate(dataset):
for batch_idx, (x_e, y_e, d_e, idx_e, obj_e) in enumerate(dataset):
x_e= x_e.to(self.cuda)
y_e= torch.argmax(y_e, dim=1)
d_e= torch.argmax(d_e, dim=1)

29
misc_scripts/acc_pacs.sh
Просмотреть файл

@ -1,29 +0,0 @@
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains art_painting cartoon sketch --test_domains photo > results/pacs/acc/photo_erm.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo cartoon sketch --test_domains art_painting > results/pacs/acc/painting_erm.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo art_painting sketch --test_domains cartoon > results/pacs/acc/cartoon_erm.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo art_painting cartoon --test_domains sketch > results/pacs/acc/sketch_erm.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains art_painting cartoon sketch --test_domains photo > results/pacs/acc/photo_random.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0.5 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo cartoon sketch --test_domains art_painting > results/pacs/acc/painting_random.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo art_painting sketch --test_domains cartoon > results/pacs/acc/cartoon_random.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo art_painting cartoon --test_domains sketch > results/pacs/acc/sketch_random.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.1 --match_case -1 --train_domains art_painting cartoon sketch --test_domains photo > results/pacs/acc/photo_hybrid.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.01 --match_case -1 --train_domains photo cartoon sketch --test_domains art_painting > results/pacs/acc/painting_hybrid.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.01 --match_case -1 --train_domains photo art_painting sketch --test_domains cartoon > results/pacs/acc/cartoon_hybrid.txt
python test.py --test_metric acc --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.5 --match_case -1 --train_domains photo art_painting cartoon --test_domains sketch > results/pacs/acc/sketch_hybrid.txt

0
misc_scripts/untitled.txt → misc_scripts/adult.txt
Просмотреть файл

33
misc_scripts/adv_attack.sh
Просмотреть файл

@ -1,33 +0,0 @@
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 5
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 5
# python test.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 5
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 25
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 25
# python test.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 25
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 50
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 50
# python test.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 50
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 75
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 75
# python test.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 75
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 100
# python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 100
# python test.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --test_metric adv_attack --penalty_diff_ctr 0 --adv_eps 100
python3 test.py --test_metric adv_attack --dataset fashion_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --adv_eps 0.3 > perfect.txt
python3 test.py --test_metric adv_attack --dataset fashion_mnist --method_name erm_match --match_case 0.75 --penalty_ws 0.1 --adv_eps 0.3 > case_75.txt
python3 test.py --test_metric adv_attack --dataset fashion_mnist --method_name erm_match --match_case 0.5 --penalty_ws 0.1 --adv_eps 0.3 > case_50.txt
python3 test.py --test_metric adv_attack --dataset fashion_mnist --method_name erm_match --match_case 0.25 --penalty_ws 0.1 --adv_eps 0.3 > case_25.txt
python3 test.py --test_metric adv_attack --dataset fashion_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --adv_eps 0.3 > random.txt
python3 test.py --test_metric adv_attack --dataset fashion_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --adv_eps 0.3 > erm.txt
python test.py --test_metric adv_attack --dataset fashion_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --adv_eps 0.3 --n_runs 2 > match_dg.txt

72
misc_scripts/chex.sh
Просмотреть файл

@ -1,72 +0,0 @@
# Chex Evaluation
python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih_trans kaggle_trans --test_domains chex --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 0.0 --model_name densenet121 --n_runs 2 > erm_chex.txt
python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih_trans kaggle_trans --test_domains chex --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 50.0 --model_name densenet121 --n_runs 2 > rand_chex.txt
python train.py --dataset chestxray --method_name csd --match_case 0.01 --train_domains nih_trans kaggle_trans --test_domains chex --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 0.0 --model_name densenet121 --n_runs 2 --rep_dim 1024 > csd_chex.txt
python train.py --dataset chestxray --method_name irm --match_case 0.01 --train_domains nih_trans kaggle_trans --test_domains chex --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 50.0 --penalty_s 5 --model_name densenet121 --n_runs 2 > irm_chex.txt
# NIH Evaluation
python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains chex_trans kaggle_trans --test_domains nih --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 0.0 --model_name densenet121 --n_runs 2 > erm_nih.txt
python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains chex_trans kaggle_trans --test_domains nih --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 50.0 --model_name densenet121 --n_runs 2 > rand_nih.txt
python train.py --dataset chestxray --method_name csd --match_case 0.01 --train_domains chex_trans kaggle_trans --test_domains nih --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 0.0 --model_name densenet121 --n_runs 2 --rep_dim 1024 > csd_nih.txt
python train.py --dataset chestxray --method_name irm --match_case 0.01 --train_domains chex_trans kaggle_trans --test_domains nih --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 50.0 --penalty_s 5 --model_name densenet121 --n_runs 2 > irm_nih.txt
# Kaggle Evaluation
python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 0.0 --model_name densenet121 --n_runs 2 > erm_kaggle.txt
python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 50.0 --model_name densenet121 --n_runs 2 > rand_kaggle.txt
python train.py --dataset chestxray --method_name csd --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 0.0 --model_name densenet121 --n_runs 2 --rep_dim 1024 > csd_kaggle.txt
python train.py --dataset chestxray --method_name irm --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 50.0 --penalty_s 5 --model_name densenet121 --n_runs 2 > irm_kaggle.txt
# python train.py --dataset chestxray --method_name hybrid --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle_trans --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 0.0 --model_name densenet121 --n_runs 3 --penalty_aug 10.0 > perf_chex_trans.txt
# python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 32 --penalty_ws 0.0 --model_name densenet121 --n_runs 1 > erm_chex.txt
# python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 32 --penalty_ws 1.0 --model_name densenet121 --n_runs 1 > rand_chex.txt
# python train.py --dataset chestxray --method_name csd --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 32 --penalty_ws 0.0 --model_name densenet121 --n_runs 1 --rep_dim 1024 > csd_chex.txt
# python train.py --dataset chestxray --method_name irm --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 32 --penalty_ws 1.0 --penalty_s 5 --model_name densenet121 --n_runs 1 > irm_chex.txt
# python train.py --dataset chestxray --method_name hybrid --match_case 0.01 --train_domains nih_trans chex_trans --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 16 --penalty_ws 0.0 --model_name densenet121 --n_runs 1 --penalty_aug 10.0 > perf_chex.txt
# #Kaggle Evaluation
# python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih chex --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --penalty_ws 0.0 --model_name densenet121 > erm_.txt
# python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih chex --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --penalty_ws 1.0 --model_name densenet121 > rand_chex.txt
# python train.py --dataset chestxray --method_name csd --match_case 0.01 --train_domains nih chex --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --penalty_ws 0.0 --model_name densenet121 > csd_chex.txt
# python train.py --dataset chestxray --method_name irm --match_case 0.01 --train_domains nih chex --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --penalty_ws 1.0 --penalty_s 5 --model_name densenet121 > irm_chex.txt
# #NIH Evaluation
# python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains kaggle chex --test_domains nih --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --penalty_ws 0.0 --model_name densenet121 > erm_chex.txt
# python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains kaggle chex --test_domains nih --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --penalty_ws 1.0 --model_name densenet121 > rand_chex.txt
# python train.py --dataset chestxray --method_name csd --match_case 0.01 --train_domains kaggle chex --test_domains nih --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --penalty_ws 0.0 --model_name densenet121 > csd_chex.txt
# python train.py --dataset chestxray --method_name irm --match_case 0.01 --train_domains kaggle chex --test_domains nih --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --penalty_ws 1.0 --penalty_s 5 --model_name densenet121 > irm_chex.txt
#Alone
# python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains nih --test_domains nih --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 32 --penalty_ws 0.0 --model_name densenet121 --n_runs 1 > nih_alone.txt
# python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains chex --test_domains chex --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 32 --penalty_ws 0.0 --model_name densenet121 --n_runs 1 > chex_alone.txt
# python train.py --dataset chestxray --method_name erm_match --match_case 0.01 --train_domains kaggle --test_domains kaggle --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 40 --lr 0.001 --batch_size 32 --penalty_ws 0.0 --model_name densenet121 --n_runs 1 > kaggle_alone.txt

11
misc_scripts/domainbed.sh
Просмотреть файл

@ -1,11 +0,0 @@
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 1.0 --epochs 50 --model_name domain_bed_mnist --img_h 28 --img_w 28 --train_domains 15 30 45 60 75 --test_domains 0 > rand_1.txt
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.30 --penalty_ws 1.0 --epochs 50 --model_name domain_bed_mnist --img_h 28 --img_w 28 --train_domains 15 30 45 60 75 --test_domains 0 > mdg_1.txt
python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 1.0 --epochs 50 --model_name domain_bed_mnist --img_h 28 --img_w 28 --train_domains 15 30 45 60 75 --test_domains 0 > perf_1.txt
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --epochs 50 --model_name domain_bed_mnist --img_h 28 --img_w 28 --train_domains 15 30 45 60 75 --test_domains 0 > rand_0.1.txt
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.30 --penalty_ws 0.1 --epochs 50 --model_name domain_bed_mnist --img_h 28 --img_w 28 --train_domains 15 30 45 60 75 --test_domains 0 > mdg_0.1.txt
python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --epochs 50 --model_name domain_bed_mnist --img_h 28 --img_w 28 --train_domains 15 30 45 60 75 --test_domains 0 > perf_0.1.txt

16
misc_scripts/mnist_abl.sh
Просмотреть файл

@ -1,16 +0,0 @@
##Rotated MNIST
#Perfect MatchDG
python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 1.0 --match_flag 1 --epochs 50 --batch_size 64 --pos_metric cos
#Non Interative MatchDG
python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 0 --epochs 50 --batch_size 64 --pos_metric cos
##Fashion MNIST
#Perfect MatchDG
python train.py --dataset fashion_mnist --method_name matchdg_ctr --match_case 1.0 --match_flag 1 --epochs 50 --batch_size 64 --pos_metric cos
#Non Interative MatchDG
python train.py --dataset fashion_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 0 --epochs 50 --batch_size 64 --pos_metric cos

15
misc_scripts/pacs_ctr.sh
Просмотреть файл

@ -1,15 +0,0 @@
#Photo
python train.py --dataset pacs --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 15 --batch_size 128 --pos_metric cos --train_domains art_painting cartoon sketch --test_domains photo --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name $1
#Art Painting
python train.py --dataset pacs --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 15 --batch_size 128 --pos_metric cos --train_domains photo cartoon sketch --test_domains art_painting --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name $1
#Cartoon
python train.py --dataset pacs --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 15 --batch_size 128 --pos_metric cos --train_domains photo art_painting sketch --test_domains cartoon --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name $1
#Sketch
python train.py --dataset pacs --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 15 --batch_size 128 --pos_metric cos --train_domains photo art_painting cartoon --test_domains sketch --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name $1

36
misc_scripts/privacy.txt
Просмотреть файл

@ -1,36 +0,0 @@
#Entorpy Attack
python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 10 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle
#MIA Attack
python test.py --test_metric mia --mia_sample_size 1000 --mia_logit 1 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric mia --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 10 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric mia --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric mia --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric mia --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle
#Mean Rank
python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name erm_match --match_case 0.01 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle

104
misc_scripts/privacy_chest.sh
Просмотреть файл

@ -1,104 +0,0 @@
#Kaggle
python3 cxray_run.py kaggle acc train
python3 cxray_run.py kaggle acc test
python3 cxray_run.py kaggle privacy_loss_attack
python3 cxray_run.py kaggle privacy_entropy
python3 cxray_run.py kaggle match_score test
#ChexPert
python3 cxray_run.py chex acc train
python3 cxray_run.py chex acc test
python3 cxray_run.py chex privacy_loss_attack
python3 cxray_run.py chex privacy_entropy
python3 cxray_run.py chex match_score test
#NIH
python3 cxray_run.py nih acc train
python3 cxray_run.py nih acc test
python3 cxray_run.py nih privacy_loss_attack
python3 cxray_run.py nih privacy_entropy
python3 cxray_run.py nih match_score test
# #Accuracy Train
# python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case train --match_func_aug_case 1 --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_train/erm.txt
# python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case train --match_func_aug_case 1 --method_name erm_match --match_case 0.01 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_train/random.txt
# python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case train --match_func_aug_case 1 --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_train/csd.txt
# python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case train --match_func_aug_case 1 --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_train/irm.txt
# python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case train --match_func_aug_case 1 --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_train/matchdg.txt
# # #Accuracy Test
# # python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case test --match_func_aug_case 1 --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_test/erm.txt
# # python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case test --match_func_aug_case 1 --method_name erm_match --match_case 0.01 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_test/random.txt
# # python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case test --match_func_aug_case 1 --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_test/csd.txt
# # python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case test --match_func_aug_case 1 --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_test/irm.txt
# # python test.py --test_metric acc --batch_size 64 --dataset chestxray --acc_data_case test --match_func_aug_case 1 --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/acc_test/matchdg.txt
# # #Mean Rank
# # python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/match_score/erm.txt
# # python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name erm_match --match_case 0.01 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
# # python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
# # python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
# # python test.py --test_metric match_score --batch_size 64 --dataset chestxray --match_func_data_case test --match_func_aug_case 1 --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle
# # #MIA Attack
# # python test.py --test_metric mia --mia_sample_size 1000 --mia_logit 1 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy/erm.txt
# # python test.py --test_metric mia --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 10 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy/random.txt
# # python test.py --test_metric mia --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy/csd.txt
# # python test.py --test_metric mia --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy/irm.txt
# # python test.py --test_metric mia --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy/matchdg.txt
# # #Entorpy Attack
# # python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_entropy/erm.txt
# # python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 10 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_entropy/random.txt
# # python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_entropy/csd.txt
# # python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_entropy/irm.txt
# # python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_entropy/matchdg.txt
# # #Loss Attack
# # python test.py --test_metric privacy_loss_attack --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_loss/erm.txt
# # python test.py --test_metric privacy_loss_attack --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 10 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_loss/random.txt
# # python test.py --test_metric privacy_loss_attack --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_loss/csd.txt
# # python test.py --test_metric privacy_loss_attack --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_loss/irm.txt
# # python test.py --test_metric privacy_loss_attack --mia_sample_size 1000 --batch_size 64 --dataset chestxray --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/privacy_loss/matchdg.txt
#Attribute Attack
# python test.py --test_metric attribute_attack --mia_logit 1 --batch_size 64 --attribute_domain 1 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/attribute_attack/erm.txt
# python test.py --test_metric attribute_attack --mia_logit 1 --batch_size 64 --attribute_domain 1 --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/attribute_attack/random.txt
# python test.py --test_metric attribute_attack --mia_logit 1 --batch_size 64 --attribute_domain 1 --dataset chestxray --method_name csd --match_case 0.01 --penalty_ws 0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/attribute_attack/csd.txt
# python test.py --test_metric attribute_attack --mia_logit 1 --batch_size 64 --attribute_domain 1 --dataset chestxray --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/attribute_attack/irm.txt
# python test.py --test_metric attribute_attack --mia_logit 1 --batch_size 64 --attribute_domain 1 --dataset chestxray --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle > results/chestxray/attribute_attack/matchdg.txt

39
misc_scripts/privacy_mnist.sh
Просмотреть файл

@ -1,39 +0,0 @@
#Rot MNIST
# python3 mnist_run.py rot_mnist train_all acc train
# python3 mnist_run.py rot_mnist train_all acc test
# python3 mnist_run.py rot_mnist train_all privacy_loss_attack
# python3 mnist_run.py rot_mnist train_all privacy_entropy
# python3 mnist_run.py rot_mnist train_all match_score test
# python3 mnist_run.py rot_mnist train_abl_3 acc train
# python3 mnist_run.py rot_mnist train_abl_3 acc test
# python3 mnist_run.py rot_mnist train_abl_3 privacy_loss_attack
# python3 mnist_run.py rot_mnist train_abl_3 privacy_entropy
# python3 mnist_run.py rot_mnist train_abl_3 match_score test
# python3 mnist_run.py rot_mnist train_abl_2 acc train
# python3 mnist_run.py rot_mnist train_abl_2 acc test
# python3 mnist_run.py rot_mnist train_abl_2 privacy_loss_attack
# python3 mnist_run.py rot_mnist train_abl_2 privacy_entropy
# python3 mnist_run.py rot_mnist train_abl_2 match_score test
#Fashion MNIST
# python3 mnist_run.py fashion_mnist train_all acc train
# python3 mnist_run.py fashion_mnist train_all acc test
python3 mnist_run.py fashion_mnist train_all privacy_loss_attack
# python3 mnist_run.py fashion_mnist train_all privacy_entropy
# python3 mnist_run.py fashion_mnist train_all match_score test
# python3 mnist_run.py fashion_mnist train_abl_3 acc train
# python3 mnist_run.py fashion_mnist train_abl_3 acc test
python3 mnist_run.py fashion_mnist train_abl_3 privacy_loss_attack
# python3 mnist_run.py fashion_mnist train_abl_3 privacy_entropy
# python3 mnist_run.py fashion_mnist train_abl_3 match_score test
# python3 mnist_run.py fashion_mnist train_abl_2 acc train
# python3 mnist_run.py fashion_mnist train_abl_2 acc test
python3 mnist_run.py fashion_mnist train_abl_2 privacy_loss_attack
# python3 mnist_run.py fashion_mnist train_abl_2 privacy_entropy
# python3 mnist_run.py fashion_mnist train_abl_2 match_score test

24
misc_scripts/privacy_pacs.sh
Просмотреть файл

@ -1,24 +0,0 @@
# python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains art_painting cartoon sketch --test_domains photo > results/pacs/entropy/photo_erm.txt
# python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo cartoon sketch --test_domains art_painting > results/pacs/entropy/painting_erm.txt
# python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo art_painting sketch --test_domains cartoon > results/pacs/entropy/cartoon_erm.txt
# python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo art_painting cartoon --test_domains sketch > results/pacs/entropy/sketch_erm.txt
# python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains art_painting cartoon sketch --test_domains photo > results/pacs/entropy/photo_random.txt
# python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0.5 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo cartoon sketch --test_domains art_painting > results/pacs/entropy/painting_random.txt
# python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo art_painting sketch --test_domains cartoon > results/pacs/entropy/cartoon_random.txt
# python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains photo art_painting cartoon --test_domains sketch > results/pacs/entropy/sketch_random.txt
python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.1 --match_case -1 --train_domains art_painting cartoon sketch --test_domains photo > results/pacs/entropy/photo_hybrid.txt
python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.01 --match_case -1 --train_domains photo cartoon sketch --test_domains art_painting > results/pacs/entropy/painting_hybrid.txt
python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.01 --match_case -1 --train_domains photo art_painting sketch --test_domains cartoon > results/pacs/entropy/cartoon_hybrid.txt
python test.py --test_metric privacy_entropy --mia_sample_size 1000 --batch_size 64 --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.5 --match_case -1 --train_domains photo art_painting cartoon --test_domains sketch > results/pacs/entropy/sketch_hybrid.txt

104
misc_scripts/reproduce.txt
Просмотреть файл

@ -1,104 +0,0 @@
Table 1:
python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 1 --epochs 50 --batch_size 64 --pos_metric cos
python train.py --dataset rot_mnist --method_name matchdg_erm --match_case -1 --penalty_ws 0.1 --epochs 25 --ctr_match_case 0.0 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18
Table 2: (Match func evaluation on train domains)
python test.py --dataset rot_mnist --method_name erm_match --match_case 0.0 --penalty_ws 0.0 --test_metric match_score
python test.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 1 --pos_metric cos --test_metric match_score
Table 6: (LeNet)
#MatchDG
python train.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 1.0 --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --n_runs 3 --epochs 100 --lr 0.01 --model_name lenet --img_h 32 --img_w 32 --train_domains 15 30 45 60 75 --test_domains 0
#Perf
python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 1.0 --epochs 100 --model_name lenet --img_h 32 --img_w 32 --train_domains 0 15 30 45 60 --test_domains 75
Table 7: (DomainBed)
#MatchDG
python train.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 1.0 --match_case -1 --batch_size 64 --epochs 25 --model_name domain_bed_mnist --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --n_runs 3 --img_h 28 --img_w 28 --train_domains 15 30 45 60 75 --test_domains 0
#Perf
python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 1.0 --batch_size 64 --epochs 25 --model_name domain_bed_mnist --img_h 28 --img_w 28 --train_domains 15 30 45 60 75 --test_domains 0
Table 8: (Phase 2 Rankings)
#RandMatch
python test.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --test_metric match_score
#MatchDG
python test.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --n_runs 3 --test_metric match_score
#PerfMatch
python test.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --test_metric match_score
Chexpert: (Acc Evaluation)
#Matching Methods
python test.py --dataset chestxray --method_name erm_match --match_case 0.01 --penalty_ws 0 --test_metric acc --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains nih_trans
#CSD
python test.py --dataset chestxray --method_name csd --match_case 0.01 --penalty_ws 0 --test_metric acc --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
#IRM
python test.py --dataset chestxray --method_name irm --match_case 0.01 --penalty_s 5 --penalty_ws 10.0 --test_metric acc --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --train_domains nih_trans chex_trans --test_domains kaggle
python test.py --dataset chestxray --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --test_metric acc --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 0.0 --match_case 0.01 --train_domains nih_trans chex_trans --test_domains chex_tran
#MatchDG
python test.py --dataset chestxray --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --test_metric acc --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 50.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle
#Hybrid
python test.py --dataset chestxray --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name densenet121 --test_metric acc --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --penalty_ws 1.0 --match_case -1 --train_domains nih_trans chex_trans --test_domains kaggle_trans
PACS: (Acc Evaluation)
#Matching Methods
python test.py --dataset pacs --method_name erm_match --match_case 0.01 --penalty_ws 0 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --train_domains art_painting cartoon sketch --test_domains photo
#MatchDG
python test.py --dataset pacs --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.1 --match_case -1 --train_domains art_painting cartoon sketch --test_domains photo
python test.py --dataset pacs --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet50 --penalty_ws 0.1 --match_case -1 --train_domains art_painting cartoon sketch --test_domains photo
python test.py --dataset pacs --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.5 --match_case -1 --train_domains photo cartoon sketch --test_domains art_painting
python test.py --dataset pacs --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet50 --penalty_ws 0.1 --match_case -1 --train_domains photo cartoon sketch --test_domains art_painting
python test.py --dataset pacs --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 1.0 --match_case -1 --train_domains photo art_painting sketch --test_domains cartoon
python test.py --dataset pacs --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet50 --penalty_ws
1.0 --match_case -1 --train_domains photo art_painting sketch --test_domains cartoon
python test.py --dataset pacs --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.5 --match_case -1 --train_domains photo art_painting cartoon --test_domains sketch
python test.py --dataset pacs --method_name matchdg_erm --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet50 --penalty_ws
0.5 --match_case -1 --train_domains photo art_painting cartoon --test_domains sketch
#Hybrid
python test.py --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.1 --match_case -1 --train_domains art_painting cartoon sketch --test_domains photo
python test.py --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet50 --penalty_ws 0.1 --match_case -1 --train_domains art_painting cartoon sketch --test_domains photo
python test.py --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.01 --match_case -1 --train_domains photo cartoon sketch --test_domains art_painting
python test.py --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet50 --penalty_ws 0.01 --match_case -1 --train_domains photo cartoon sketch --test_domains art_painting
python test.py --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.01
--match_case -1 --train_domains photo art_painting sketch --test_domains cartoon
python test.py --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet50 --penalty_ws 0.01 --match_case -1 --train_domains photo art_painting sketch --test_domains cartoon
python test.py --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet18 --penalty_ws 0.5 --match_case -1 --train_domains photo art_painting cartoon --test_domains sketch
python test.py --dataset pacs --method_name hybrid --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet50 --test_metric acc --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name resnet50 --penalty_ws 0.01 --match_case -1 --train_domains photo art_painting cartoon --test_domains sketch

39
misc_scripts/reproduce_paper.sh
Просмотреть файл

@ -1,39 +0,0 @@
#RotMNIST
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --epochs 25
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --epochs 25
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.25 --penalty_ws 0.1 --epochs 25
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.50 --penalty_ws 0.1 --epochs 25
python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --epochs 25
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.75 --penalty_ws 0.1 --epochs 25
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --epochs 25
python train.py --dataset rot_mnist --method_name irm --match_case 0.01 --penalty_irm 1.0 --penalty_s 5 -- epochs 25
python train.py --dataset rot_mnist --method_name csd --match_case 0.01 --penalty_ws 0.0 -- epochs 25
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --epochs 25 --train_domains 30 45
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --epochs 25 --train_domains 30 45
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.25 --penalty_ws 0.1 --epochs 25 --train_domains 30 45
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.50 --penalty_ws 0.1 --epochs 25 --train_domains 30 45
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.75 --penalty_ws 0.1 --epochs 25 --train_domains 30 45
# python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --epochs 25 --train_domains 30 45
# python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 30 --batch_size 64 --pos_metric cos
# python train.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --n_runs 3 --epochs 25
# python train.py --dataset rot_mnist --method_name csd --match_case 0.01 --penalty_ws 0.0 -- epochs 25 --train_domains 30 45
# python train.py --dataset rot_mnist --method_name irm --match_case 0.01 --penalty_irm 1.0 --penalty_s 5 -- epochs 25 --train_domains 30 45
# python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 30 --batch_size 64 --pos_metric cos
# python train.py --dataset rot_mnist --method_name matchdg_erm --penalty_ws 0.1 --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --n_runs 3 --epochs 25
#FashionMNIST
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.0 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.25 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.50 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.75 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name csd --match_case 0.01 --penalty_ws 0.0 -- epochs 25
#ChestXRay
# python train.py --dataset chestxray --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 3 --batch_size 32 --pos_metric cos --train_domains nih_trans chex_trans --test_domains kaggle_trans --out_classes 2 --perfect_match 0 --img_c 3 --pre_trained 1 --model_name densenet121 --n_runs 2

16
misc_scripts/run_domainbed.sh
Просмотреть файл

@ -1,16 +0,0 @@
#ERM MatchDG
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.5 --penalty_ws 0.25 --model domain_bed_mnist --img_w 28 --img_h 28 --epochs 25 --train_domains 15 30 45 60 75 --test_domain 0
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.5 --penalty_ws 0.25 --model domain_bed_mnist --img_w 28 --img_h 28 --epochs 25 --train_domains 0 15 30 45 60 --test_domain 75
#ERM RandMatch
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.25 --model domain_bed_mnist --img_w 28 --img_h 28 --epochs 25 --train_domains 15 30 45 60 75 --test_domain 0
python train.py --dataset rot_mnist --method_name erm_match --match_case 0.01 --penalty_ws 0.25 --model domain_bed_mnist --img_w 28 --img_h 28 --epochs 25 --train_domains 0 15 30 45 60 --test_domain 75
#ERM PerfMatch
python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.25 --model domain_bed_mnist --img_w 28 --img_h 28 --epochs 25 --train_domains 15 30 45 60 75 --test_domain 0
python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.25 --model domain_bed_mnist --img_w 28 --img_h 28 --epochs 25 --train_domains 0 15 30 45 60 --test_domain 75

112
misc_scripts/slab-run.sh
Просмотреть файл

@ -1,112 +0,0 @@
# python train.py --dataset slab --model_name slab --method_name perf_match --batch_size 64 --lr 0.1 --penalty_ws 0.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3 > slab_temp/erm.txt
# python test.py --test_metric acc --dataset slab --model_name slab --method_name perf_match --penalty_ws 0.0 --out_classes 2 --train_domains 0.0 0.10 --n_runs 3 --acc_data_case train
# python test.py --test_metric slab_feat_eval --dataset slab --model_name slab --method_name perf_match --penalty_ws 0.0 --out_classes 2 --train_domains 0.0 0.10 --n_runs 3 --match_func_data_case train
# python test_slab.py --method_name perf_match --penalty_ws 0.0 --n_runs 3 --train_domains 0.0 0.10 --test_domain 0.3
# # python test.py --test_metric mia --dataset slab --model_name slab --method_name perf_match --penalty_ws 0.0 --mia_logit 1 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3 > slab_temp/erm-mi.txt
# # python test.py --test_metric privacy_entropy --dataset slab --model_name slab --method_name perf_match --penalty_ws 0.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3 > slab_temp/erm-entropy.txt
# # python test.py --test_metric privacy_loss_attack --dataset slab --model_name slab --method_name perf_match --penalty_ws 0.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3 > slab_temp/erm-loss.txt
# python train.py --dataset slab --model_name slab --method_name rand_match --batch_size 64 --lr 0.1 --penalty_ws 1.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 1.0 --n_runs 3 > slab_temp/rand.txt
# python test.py --test_metric acc --dataset slab --model_name slab --method_name rand_match --penalty_ws 1.0 --out_classes 2 --train_domains 0.0 0.10 --n_runs 3 --acc_data_case train
# python test_slab.py --method_name rand_match --penalty_ws 1.0 --n_runs 3 --train_domains 0.0 0.10 --test_domain 0.3
# # python test.py --test_metric mia --dataset slab --model_name slab --method_name rand_match --penalty_ws 10.0 --mia_logit 1 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3 > slab_temp/rand-mi.txt
# # python test.py --test_metric privacy_entropy --dataset slab --model_name slab --method_name rand_match --penalty_ws 10.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3 > slab_temp/rand-entropy.txt
# # python test.py --test_metric privacy_loss_attack --dataset slab --model_name slab --method_name rand_match --penalty_ws 10.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3 > slab_temp/rand-loss.txt
# python train.py --dataset slab --model_name slab --method_name perf_match --batch_size 64 --lr 0.1 --penalty_ws 1.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 1.0 --n_runs 3 > slab_temp/perf.txt
# python test.py --test_metric acc --dataset slab --model_name slab --method_name perf_match --penalty_ws 1.0 --out_classes 2 --train_domains 0.0 0.10 --n_runs 3 --acc_data_case train
# python test_slab.py --method_name perf_match --penalty_ws 1.0 --n_runs 3 --train_domains 0.0 0.10 --test_domain 0.3
# # python test.py --test_metric mia --dataset slab --model_name slab --method_name perf_match --penalty_ws 10.0 --mia_logit 1 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3 > slab_temp/perf-mi.txt
# # python test.py --test_metric privacy_entropy --dataset slab --model_name slab --method_name perf_match --penalty_ws 10.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3 > slab_temp/perf-entropy.txt
# # python test.py --test_metric privacy_loss_attack --dataset slab --model_name slab --method_name perf_match --penalty_ws 10.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.50 --n_runs 3
# #IRM
# python train.py --dataset slab --model_name slab --method_name irm_slab --batch_size 64 --lr 0.1 --penalty_irm 10.0 --penalty_s 2 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 1.0 --n_runs 3 > slab_temp/irm.txt
# #CSD
# python train.py --dataset slab --model_name slab --method_name csd_slab --batch_size 64 --lr 0.1 --penalty_ws 0.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 1.0 --n_runs 3 --rep_dim 100 > slab_temp/csd.txt
# ## Slab Spur
# python train.py --dataset slab_spur --model_name slab --method_name perf_match --batch_size 64 --lr 0.1 --penalty_ws 0.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_data_dim 3
# python train.py --dataset slab_spur --model_name slab --method_name perf_match --batch_size 64 --lr 0.1 --penalty_ws 50.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_data_dim 3
# python train.py --dataset slab_spur --model_name slab --method_name rand_match --batch_size 64 --lr 0.1 --penalty_ws 50.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 1.0 --n_runs 3 --slab_data_dim 3
# python train.py --dataset slab_spur --model_name slab --method_name irm_slab --batch_size 64 --lr 0.1 --penalty_irm 10.0 --penalty_s 2 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 1.0 --n_runs 3 --slab_data_dim 3
# python train.py --dataset slab_spur --model_name slab --method_name csd_slab --batch_size 64 --lr 0.1 --penalty_ws 0.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 1.0 --n_runs 3 --rep_dim 100 --slab_data_dim 3
# # Train Case
# python test.py --test_metric acc --dataset slab_spur --model_name slab --method_name perf_match --penalty_ws 50.0 --out_classes 2 --train_domains 0.0 0.10 --n_runs 3 --acc_data_case train --slab_data_dim 3
# # #Attribute Attack
# # python test.py --test_metric attribute_attack --mia_logit 1 --attribute_domain 0 --batch_size 64 --dataset slab_spur --model_name slab --method_name perf_match --penalty_ws 0.0 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_data_dim 3
# python test.py --test_metric attribute_attack --mia_logit 1 --attribute_domain 0 --batch_size 64 --dataset slab_spur --model_name slab --method_name perf_match --penalty_ws 0.0 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_data_dim 3
# python test.py --test_metric attribute_attack --mia_logit 1 --attribute_domain 0 --batch_size 64 --dataset slab_spur --model_name slab --method_name perf_match --penalty_ws 50.0 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_data_dim 3
# python3 slab-run.py train
# python3 slab-run.py test
# python3 slab-run.py train_plot
python test_slab.py --method_name perf_match --penalty_ws 0.0 --n_runs 3 --train_domains 0.0 0.10 --test_domain 0.90 --slab_noise 0.05 > slab_noise/erm_auc.txt
python test_slab.py --method_name perf_match --penalty_ws 1.0 --n_runs 3 --train_domains 0.0 0.10 --test_domain 0.90 --slab_noise 0.05 > slab_noise/perf_auc.txt
python test.py --test_metric privacy_entropy --dataset slab --model_name slab --method_name perf_match --penalty_ws 0.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_noise 0.05 > slab_noise/erm_entropy.txt
python test.py --test_metric privacy_entropy --dataset slab --model_name slab --method_name perf_match --penalty_ws 1.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_noise 0.05 > slab_noise/perf_entropy.txt
python test.py --test_metric privacy_loss_attack --dataset slab --model_name slab --method_name perf_match --penalty_ws 0.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_noise 0.05 > slab_noise/erm_loss.txt
python test.py --test_metric privacy_loss_attack --dataset slab --model_name slab --method_name perf_match --penalty_ws 1.0 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_noise 0.05 > slab_noise/perf_loss.txt
# python test.py --test_metric mia --dataset slab --model_name slab --method_name perf_match --penalty_ws 0.0 --mia_logit 1 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.15 --n_runs 3 --slab_noise 0.05 > slab_noise/erm_mi.txt
# python test.py --test_metric mia --dataset slab --model_name slab --method_name perf_match --penalty_ws 1.0 --mia_logit 1 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.15 --n_runs 3 --slab_noise 0.05 > slab_noise/perf_mi.txt
python train.py --dataset slab --model_name slab --method_name perf_match --batch_size 64 --lr 0.1 --penalty_ws 0.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_noise 0.20
python train.py --dataset slab --model_name slab --method_name mask_linear --batch_size 64 --lr 0.1 --penalty_ws 0.0 --epochs 30 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.90 --n_runs 3 --slab_noise 0.20
python test_slab.py --method_name perf_match --penalty_ws 0.0 --n_runs 3 --train_domains 0.0 0.10 --test_domain 0.30 --slab_noise 0.20
python test_slab.py --method_name mask_linear --penalty_ws 0.0 --n_runs 3 --train_domains 0.0 0.10 --test_domain 0.3 --slab_noise 0.20
python test.py --test_metric mia --dataset slab --model_name slab --method_name perf_match --penalty_ws 0.0 --mia_logit 1 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.30 --n_runs 3 --slab_noise 0.20
python test.py --test_metric mia --dataset slab --model_name slab --method_name mask_linear --penalty_ws 0.0 --mia_logit 1 --mia_sample_size 400 --out_classes 2 --train_domains 0.0 0.10 --test_domains 0.30 --n_runs 3 --slab_noise 0.20

3
misc_scripts/t3.sh
Просмотреть файл

@ -1,3 +0,0 @@
python3 cxray_run.py kaggle train
python3 cxray_run.py chex train
python3 cxray_run.py nih train

39
misc_scripts/temp.sh
Просмотреть файл

@ -1,39 +0,0 @@
# # python3 metric_eval.py fashion_mnist mia
# # python3 metric_eval.py rot_mnist mia
# # python train.py --dataset pacs --method_name hybrid --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --n_runs 3 --train_domains photo cartoon sketch --test_domains art_painting --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 50 --lr 0.001 --batch_size 16 --weight_decay 0.001 --penalty_ws 0.1 --penalty_aug 0.1 --model_name resnet18 --ctr_model_name resnet18
# #python train.py --dataset pacs --method_name hybrid --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --n_runs 3 --train_domains photo art_painting cartoon --test_domains sketch --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 50 --lr 0.01 --batch_size 16 --weight_decay 0.001 --penalty_ws 0.1 --penalty_aug 0.1 --model_name resnet18 --ctr_model_name resnet18
# #python train.py --dataset pacs --method_name hybrid --match_case -1 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --n_runs 3 --train_domains photo cartoon sketch --test_domains art_painting --out_classes 7 --perfect_match 0 --img_c 3 --pre_trained 1 --epochs 50 --lr 0.001 --batch_size 16 --weight_decay 0.001 --penalty_ws 0.1 --penalty_aug 0.1 --model_name resnet18 --ctr_model_name resnet18
# #python3 metric_eval.py fashion_mnist acc test
# #python3 metric_eval.py fashion_mnist match_score test
# #python3 metric_eval.py fashion_mnist mia
# python3 metric_eval.py fashion_mnist privacy_entropy
# #python3 metric_eval.py fashion_mnist attribute_attack
# #python3 metric_eval.py fashion_mnist privacy_loss_attack
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.0 --penalty_ws 0.0 --epochs 25
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.0 --penalty_ws 0.1 --epochs 25
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.25 --penalty_ws 0.1 --epochs 25
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.50 --penalty_ws 0.1 --epochs 25
# python train.py --dataset rot_mnist --method_name erm_match --match_case 0.75 --penalty_ws 0.1 --epochs 25
# python train.py --dataset rot_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.0 --penalty_ws 0.0 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.0 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.25 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.50 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 0.75 --penalty_ws 0.1 --epochs 25
# python train.py --dataset fashion_mnist --method_name erm_match --match_case 1.0 --penalty_ws 0.1 --epochs 25
#python train.py --dataset rot_mnist --method_name irm --match_case 0.0 --penalty_irm 1.0 --penalty_s 5 --epochs 25
#python train.py --dataset rot_mnist --method_name csd --match_case 0.0 --penalty_ws 0.0 --epochs 25 --rep_dim 512
python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.0 --match_flag 1 --epochs 50 --batch_size 64 --pos_metric cos
python train.py --dataset rot_mnist --method_name matchdg_erm --match_case -1 --penalty_ws 0.1 --epochs 25 --ctr_match_case 0.0 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18

19
misc_scripts/temp2.sh
Просмотреть файл

@ -1,19 +0,0 @@
# #MatchDG CTR Phase: RotMNIST
# python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 30 --batch_size 64 --pos_metric cos
# python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 1.0 --match_flag 1 --epochs 30 --batch_size 64 --pos_metric cos
# python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 0 --epochs 30 --batch_size 64 --pos_metric cos
# #MatchDG CTR Phase: FashionMNIST
# python train.py --dataset fashion_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 30 --batch_size 64 --pos_metric cos
# python train.py --dataset fashion_mnist --method_name matchdg_ctr --match_case 1.0 --match_flag 1 --epochs 30 --batch_size 64 --pos_metric cos
# python train.py --dataset fashion_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 0 --epochs 30 --batch_size 64 --pos_metric cos
python train.py --dataset rot_mnist --method_name matchdg_ctr --match_case 0.01 --match_flag 1 --epochs 100 --batch_size 128 --pos_metric cos
python train.py --dataset rot_mnist --method_name matchdg_erm --match_case -1 --penalty_ws 0.1 --epochs 25 --ctr_match_case 0.01 --ctr_match_flag 1 --ctr_match_interrupt 5 --ctr_model_name resnet18

44
misc_scripts/untitled1.txt
Просмотреть файл

@ -1,44 +0,0 @@
M2ExZ
In the domain generalization literature, a common objective is to learn representations independent of the domain after conditioning on the class label. We show that this objective is not sufficient: there exist counter-examples where a model fails to generalize to unseen domains even after satisfying class-conditional domain invariance. We formalize this observation through a structural causal model and show the importance of modeling within-class variations for generalization. Specifically, classes contain objects that characterize specific causal features, and domains can be interpreted as interventions on these objects that change non-causal features. We highlight an alternative condition: inputs across domains should have the same representation if they are derived from the same object. Based on this objective, we propose matching-based algorithms when base objects are observed (e.g., through data augmentation) and approximate the objective when objects are not observed (MatchDG). Our simple matching-based algorithms are competitive to prior work on out-of-domain accuracy for rotated MNIST, Fashion-MNIST, PACS, and Chest-Xray datasets. Our method MatchDG also recovers ground-truth object matches: on MNIST and Fashion-MNIST, top-10 matches from MatchDG have over 50% overlap with ground-truth matches.
Table 11:
88.2 + 98.6 + 97.7 + 97.5 + 97.0 + 85.6
91.0 + 99.7 + 99.6 + 99.4 + 99.7 + 93.1
93.0 + 99.5 + 99.9 + 99.4 + 99.7 + 93.3
96.5 + 99.1 + 99.2 + 98.6 + 98.6 + 94.9
Table 12:
95.4 + 98.2 + 97.9 + 98.5 + 98.1 + 94.3
95.9 + 98.4 + 98.6 + 98.9 + 98.7 + 95.1
Table 18:
95.38 + 77.68 + 78.98 + 74.75
95.37 + 78.16 + 78.83 + 75.13
95.93 + 79.77 + 80.03 + 77.11
96.15 + 81.71 + 80.75 + 78.79
95.57 + 79.09 + 79.37 + 77.60
96.53 + 81.32 + 80.70 + 79.72
96.67 + 82.80 + 81.61 + 81.05
97.89 + 82.16 + 81.68 + 80.45
97.94 + 85.61 + 82.12 + 78.76
98.36 + 86.74 + 82.32 + 82.66
Table 19:
85.29 + 64.23 + 66.61 + 59.25
85.42 + 65.54 + 68.41 + 59.46
85.41 + 66.21 + 68.47 + 59.56
85.67 + 66.89 + 68.89 + 60.39
86.04 + 67.35 + 69.71 + 64.66
86.52 + 67.99 + 69.92 + 65.64
87.03 + 67.97 + 71.06 + 67.19