Граф коммитов

558 Коммитов

Автор SHA1 Сообщение Дата
Peter Hessey 6f0cff82d6 ♻️ Style and fix mypy 2023-03-23 15:06:19 +00:00
Peter Hessey 722c0e0aa7 Merge branch 'main' into fperezgarcia/add-black 2023-03-23 14:32:48 +00:00
Peter Hessey c7c5b03a74 💄 Add black styling and configuration
MNT: Fix incorrect type annotations (#846)

ENH: Refactor Swin Transformer checkpointing (#843)

Refactor Swin Transformer checkpointing to enable patch embedding
checkpointing overriding without duplication the whole custom forward

MNT: Fix incorrect type annotations (#846)

ENH: Refactor Swin Transformer checkpointing (#843)

Refactor Swin Transformer checkpointing to enable patch embedding
checkpointing overriding without duplication the whole custom forward

Add multi-modal to root makefile

🐛 Fix Makefile updates

📝 Update dev documentation

🔧 Add pyproject.toml

👷 Update makefiles to use black config

📝 Add VS Code + black docs

🔧 Update settings.json for all repos
2023-03-23 14:32:03 +00:00
Ozan Oktay 82342cac33
ENH: Consume BioViL-T Pretrained Models in Inference Engines (#845)
- Implements helper functions to load BioViL-T text and image models.
- Updates the text and image inference engines to consume Biovil-T
models.
- Updates the grounding notebook with the new model and showing improved
results.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Fernando Pérez-García <fepegar@gmail.com>
Co-authored-by: Fernando Pérez-García <fperezgarcia@microsoft.com>
2023-03-22 17:37:11 +00:00
Kenza Bouzid 9e14c72f71
ENH: Add display name to montage script (#847) 2023-03-22 14:19:11 +00:00
Kenza Bouzid 3c54b305b4
ENH: Refactor Swin Transformer checkpointing (#843)
Refactor Swin Transformer checkpointing to enable patch embedding
checkpointing overriding without duplication the whole custom forward
2023-03-21 09:38:01 +00:00
Anton Schwaighofer 6f4065e8d1
MNT: Fix incorrect type annotations (#846) 2023-03-21 09:27:41 +00:00
Anton Schwaighofer 45235164a7
ENH: Add log files to AzureML runs to identify preemption (#841) 2023-03-21 08:59:45 +00:00
dependabot[bot] 6753ad0d10
Bump pypa/gh-action-pypi-publish from 1.5.0 to 1.8.1 (#844) 2023-03-20 22:49:38 +00:00
Ozan Oktay da13cdbb8c
ENH: Add BioViL-T image and text encoders (#830)
- Splits the content in `image/model/model.py` into separate files with
respect to their functionality, such as encoders and custom types.
- Creates a new encoder class that can handle multiple image inputs such
as current scan and previous scan. This way models can embed
multi-series data into a single vector representation.

<!--
## Guidelines

Please follow the guidelines for pull requests (PRs) in
[CONTRIBUTING](/CONTRIBUTING.md). Checklist:

- Ensure that your PR is small, and implements one change
- Give your PR title one of the prefixes listed in
[CONTRIBUTING](../docs/source/coding_guidelines.md#pull-request-titles)
to indicate what type of change that is
- Link the correct GitHub issue for tracking
- Add unit tests for all functions that you introduced or modified
- Run automatic code formatting / linting on all files ("Format
Document" Shift-Alt-F in VSCode)
- Ensure that documentation renders correctly in Sphinx by running `make
html` in the `docs` folder

## Change the default merge message

When completing your PR, you will be asked for a title and an optional
extended description. By default, the extended description will be a
concatenation of the individual
commit messages. Please DELETE/REPLACE that with a human readable
extended description for non-trivial PRs.
-->

---------

Co-authored-by: Stephanie Hyland <sthyland@microsoft.com>
Co-authored-by: Stephanie Hyland <corcra@users.noreply.github.com>
Co-authored-by: Fernando Pérez-García <fepegar@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-03-20 15:55:00 +00:00
Kenza Bouzid 97b9bae3d1
ENH: Enable activation checkpointing for Resnet and SwinTransformer (#842)
Add custom forward that checkpoints activations to trade compute for
memory

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-03-17 11:54:49 +00:00
Anton Schwaighofer 6ad33056dc
ENH: Allow variants of model parameters in the same class (#840)
Based on a commandline flag, models can now switch between different
hyperparameters
2023-03-16 14:25:53 +00:00
Anton Schwaighofer 201f5f0720
BUG: Misleading warnings when comparing metrics (#838) 2023-03-15 12:21:32 +00:00
Kenza Bouzid 273e84e6bb
ENH: Add non normalized confusion matrix to plot options (#791)
Add non normalized confusion matrix to plot options
2023-03-14 13:31:45 +00:00
Anton Schwaighofer fc5776875a
ENH: Improve handling of metric comparison for regression tests (#835)
- Regression metrics are now read and compared semantically rather than
as text files. Hence, formatting of the JSON files no longer matters.
- Fix for bug: if a scalar metric is read from AML before a list metric,
the dataframe conversion fails. Hence we can now mix scalar and list
metrics without problems.
2023-03-14 13:17:12 +00:00
Kenza Bouzid 14b43222de
BUG: Convert to tifffile if file doesn't exist or if it's empty (#836)
Fix issue of 0 bytes files
2023-03-13 16:10:50 +00:00
Kenza Bouzid 1a344600c2
ENH: Use Latest Monai GridPatch to filter both threshold and count (#834)
Temporary fix until next release of MONAI
Include fixes: 
https://github.com/Project-MONAI/MONAI/pull/6095
https://github.com/Project-MONAI/MONAI/pull/6055

Install dev version of MONAI pointing at commit
958aac78ab
2023-03-11 12:17:45 +00:00
Kenza Bouzid 1fa499b00d
MNT: Upgrade pytorch to 1.11 (#833)
Upgrade pytorch to 1.11
Add static graph flag
2023-03-10 11:01:59 +00:00
pre-commit-ci[bot] 219c35155e
ENH: Autoupdate hooks [pre-commit.ci] (#831)
Co-authored-by: Fernando Pérez-García <fepegar@gmail.com>
2023-03-09 17:00:30 +00:00
Kenza Bouzid cb97c59912
ENH: Add LoadMaskSubROId transform (#828)
LoadMaskSubROId loads a sub section from a slide based a foreground png
mask with categorical labels for each tissue section.

---------

Co-authored-by: Anton Schwaighofer <antonsc@microsoft.com>
2023-03-07 13:42:43 +00:00
Fernando Pérez-García 65bbf0e37a
FIX: Fix notebook CI when trying to merge from fork (#829)
Fixes #725.
2023-03-06 16:34:14 +00:00
Kenza Bouzid ea2ad93780
ENH: Enable flexible resolution unit for mock data generator (#827)
Add resolution_unit param to Mock slides generator
2023-03-06 15:10:38 +00:00
Peter Hessey ed8886d1a4
ENH: MONAI Auto3dSeg Updates (#819)
This PR contains changes necessary to enable running [MONAI's Auto3dSeg
tool](https://github.com/Project-MONAI/tutorials/tree/main/auto3dseg)
using hi-ml as a built-in feature for submitting to Azure:

- Allowing scripts of the form `python -m <module name>` to be run as an
AzureML job.
- Updating argument parsing to allow passing command line args
containing quotes.
- Replace logging with `print` statements to show job URL in other
packages.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-03-02 17:24:21 +00:00
Anton Schwaighofer 860da840fa
BUG: Doc build fails because of special characters in .md file (#826) 2023-03-02 16:40:00 +00:00
Kenza Bouzid 191d041b4e
ENH: WSI Conversion to tiff files (#820)
- Add ConvertWSIToTiffd transform 
- Add TiffConversionConfig for running on a slides dataset
- unit tests
2023-03-02 16:35:27 +00:00
Fernando Pérez-García 06acc07f12
DOC: Use symlinks to build docs instead of copying files (#824)
Fixes #823.

There was an issue with the Azure ML Python SDK v2 disliking symlinks in
the source dir, but that seems to have been resolved now so this PR
removes the copying operation and replaces the targets with symlinks.
2023-03-02 15:04:42 +00:00
Fernando Pérez-García 687fa69592
MNT: Increase verbosity while running notebook (#817)
Related to
- #725
2023-02-28 13:07:16 +00:00
Fernando Pérez-García 8274242e16
MNT: Bump multimodal version: 0.1.2 → 0.1.3 (#815) 2023-02-27 16:05:24 +00:00
Fernando Pérez-García 299163e5b9
MNT: Add make command to publish to PyPI (#816)
Running `make publish` from `/hi-ml-multimodal` will publish the package
to PyPI. It can be tested before by running `make publish_test`.
2023-02-27 14:00:26 +00:00
Fernando Pérez-García 81bd43f921
FIX: Use different image for multimodal notebook (#808)
Fixes #805.

Results with the new image:

![image](https://user-images.githubusercontent.com/12688084/220152485-418e0fb0-0e74-4e45-b399-8295b16ec8d7.png)

This image is very large, though :`(3950, 3300)`. Feel free to propose a
smaller one from the internet. Perhaps from Open-i.

<!--
## Guidelines

Please follow the guidelines for pull requests (PRs) in
[CONTRIBUTING](/CONTRIBUTING.md). Checklist:

- Ensure that your PR is small, and implements one change
- Give your PR title one of the prefixes listed in
[CONTRIBUTING](../docs/source/coding_guidelines.md#pull-request-titles)
to indicate what type of change that is
- Link the correct GitHub issue for tracking
- Add unit tests for all functions that you introduced or modified
- Run automatic code formatting / linting on all files ("Format
Document" Shift-Alt-F in VSCode)
- Ensure that documentation renders correctly in Sphinx by running `make
html` in the `docs` folder

## Change the default merge message

When completing your PR, you will be asked for a title and an optional
extended description. By default, the extended description will be a
concatenation of the individual
commit messages. Please DELETE/REPLACE that with a human readable
extended description for non-trivial PRs.
-->

---------

Co-authored-by: Stephanie Hyland <corcra@users.noreply.github.com>
2023-02-24 16:38:06 +00:00
Fernando Pérez-García dbc487dbe7
DOC: Replace arXiv links with ECCV links (#812)
Resolves #789.

<!--
## Guidelines

Please follow the guidelines for pull requests (PRs) in
[CONTRIBUTING](/CONTRIBUTING.md). Checklist:

- Ensure that your PR is small, and implements one change
- Give your PR title one of the prefixes listed in
[CONTRIBUTING](../docs/source/coding_guidelines.md#pull-request-titles)
to indicate what type of change that is
- Link the correct GitHub issue for tracking
- Add unit tests for all functions that you introduced or modified
- Run automatic code formatting / linting on all files ("Format
Document" Shift-Alt-F in VSCode)
- Ensure that documentation renders correctly in Sphinx by running `make
html` in the `docs` folder

## Change the default merge message

When completing your PR, you will be asked for a title and an optional
extended description. By default, the extended description will be a
concatenation of the individual
commit messages. Please DELETE/REPLACE that with a human readable
extended description for non-trivial PRs.
-->

---------

Co-authored-by: Shruthi42 <13177030+Shruthi42@users.noreply.github.com>
2023-02-24 15:30:51 +00:00
Anton Schwaighofer e9443bcdf9
ENH: Add ipykernel to environment (#814) 2023-02-23 16:59:17 +00:00
Anton Schwaighofer a4fe5e4353
MNT: Upgrade pytest (#813)
Upgrade to Pytest 7 and remove the manual setting of `sys.path` in
`conftest.py`
Upgrade mypy and flake8
2023-02-23 14:31:15 +00:00
Fernando Pérez-García 6a46779c5e
FIX: Set evaluation mode to compute number of dimensions (#809)
When guessing the number of output dimensions in the image encoder, the
model is in training mode and the batch norm stats get updated. This
shouldn't really happen and makes the checks in
https://github.com/microsoft/hi-ml/pull/807/files#r1112200449 fail. This
PR adds a line to ensure the encode is in `eval` mode before calculating
the number of output dims.
2023-02-22 14:45:13 +00:00
Harshita Sharma 5ce9e00fd5
ENH: Add background normalization transform (#803)
In this PR, background normalization transform is added and
corresponding tests are also added.
2023-02-22 12:28:02 +00:00
Kenza Bouzid efd08fd408
ENH: Add Swin Transformer Encoder and Projection layer (#810)
Add Swin transformer encoder and an intermediate projection layer
2023-02-21 20:13:04 +00:00
pre-commit-ci[bot] 7c7be2e499
ENH: Autoupdate hooks [pre-commit.ci] (#593)
<!--pre-commit.ci start-->
updates:
- [github.com/pre-commit/pre-commit-hooks: v4.3.0 →
v4.4.0](https://github.com/pre-commit/pre-commit-hooks/compare/v4.3.0...v4.4.0)
- [github.com/PyCQA/flake8: 5.0.2 →
6.0.0](https://github.com/PyCQA/flake8/compare/5.0.2...6.0.0)
- [github.com/pre-commit/mirrors-autopep8: v1.6.0 →
v2.0.1](https://github.com/pre-commit/mirrors-autopep8/compare/v1.6.0...v2.0.1)
<!--pre-commit.ci end-->

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Anton Schwaighofer <antonsc@microsoft.com>
Co-authored-by: Fernando Pérez-García <fperezgarcia@microsoft.com>
Co-authored-by: Fernando Pérez-García <fepegar@gmail.com>
2023-02-21 17:06:12 +00:00
Fernando Pérez-García 5b941d9a82
FIX: Copy hi-ml-multimodal README while building documentation (#811)
Fixes the issue described in 
- https://github.com/microsoft/hi-ml/pull/730#issuecomment-1419302573

<!--
## Guidelines

Please follow the guidelines for pull requests (PRs) in
[CONTRIBUTING](/CONTRIBUTING.md). Checklist:

- Ensure that your PR is small, and implements one change
- Give your PR title one of the prefixes listed in
[CONTRIBUTING](../docs/source/coding_guidelines.md#pull-request-titles)
to indicate what type of change that is
- Link the correct GitHub issue for tracking
- Add unit tests for all functions that you introduced or modified
- Run automatic code formatting / linting on all files ("Format
Document" Shift-Alt-F in VSCode)
- Ensure that documentation renders correctly in Sphinx by running `make
html` in the `docs` folder

## Change the default merge message

When completing your PR, you will be asked for a title and an optional
extended description. By default, the extended description will be a
concatenation of the individual
commit messages. Please DELETE/REPLACE that with a human readable
extended description for non-trivial PRs.
-->
2023-02-21 17:01:13 +00:00
Anton Schwaighofer d1dae07cda
ENH: Add instructions for HistoQC (#799)
Co-authored-by: Fernando Pérez-García <fepegar@gmail.com>
Co-authored-by: Harshita Sharma <t-hsharma@microsoft.com>
2023-02-14 14:53:34 +00:00
Anton Schwaighofer e149b19981
ENH: Add an argument to specify split for DSA upload (#800)
- Choose a split (test/extra_val/val) to upload
- Modify launch.json to use merged certificate file
2023-02-13 10:26:47 +00:00
Fernando Pérez-García 402d93c8c3
MNT: Run multimodal CI every morning (#790)
Enable multimodal CI schedule, every morning at 4 AM UTC.
2023-02-10 16:45:44 +00:00
Kenza Bouzid ebc4549ad0
ENH: Add workspace config path to generate checkpoint URL script (#798)
Enables to specify a different workspace config path to generate
checkpoint URLs
2023-02-10 16:26:52 +00:00
Anton Schwaighofer 1fb3a6d844
ENH: Add ability to choose a folder for upload to DSA (#796) 2023-02-09 20:45:47 +00:00
Anton Schwaighofer a055dfae9f
ENH: Add code to create montages from WSI (#787) 2023-02-09 19:50:07 +00:00
Peter Hessey 7b9a0d2453
BUG: Add flag for identity-based data access (#797)
Closes #795
2023-02-09 08:58:23 +00:00
Harshita Sharma ebc030e5f2
ENH: Stratify PR and ROC curves by metadata (#788)
In this PR:
- `stratify_plots_by` parameter is introduced in BaseMIL to specify
metadata column on which the output PR and ROC curves can be stratified
- Functionality is added to plot the output curves stratified by
metadata
- Corresponding tests are added

---------

Co-authored-by: Harshita Sharma <t-hsharma@microsoft.com>
2023-02-08 12:33:43 +00:00
Peter Hessey ee84421e79
BUG: Fix ordering of `local_datasets` (#794)
Closes #792
2023-02-08 09:12:50 +00:00
Peter Hessey 5155b5060f
ENH: Enable User Identity for accessing data (#775)
Closes #704. Includes the following changes:

* Allow users to access data using their own identity. This means that
we no longer need to store Account Keys / SAS tokens in the datastores
and avoids the information governance concerns there. Used by default,
if not available / not valid will revert to trying datastore identity
auth.
* Adds functionality for accessing input and output datasets in v2 jobs
via environment args rather than command line arguments.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-02-07 10:32:38 +00:00
Kenza Bouzid 33ef4cd28b
ENH: Enable resubmitting single cross validation runs manually (#781)
Add resubmit_crossval_child flag to be able to submit a single fold run.
This is useful in case one of the crossvalidation folds fails and needs
to be resubmitted manually.
2023-02-03 13:32:23 +00:00
Harshita Sharma ad72411cf4
ENH: Add ROC curve to plot options (#780)
ROC curve is added as a plot option. Tests are updated for ROC and PR
curves

---------

Co-authored-by: Harshita Sharma <t-hsharma@microsoft.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2023-02-03 13:20:01 +00:00