Граф коммитов

85 Коммитов

Автор SHA1 Сообщение Дата
Peter Hessey 69a5e8cd2c
DOC: Update environment documentation (#748)
* 📝 Update environment documentation

* 📝 Add quick setup, add review comment changes
2022-06-22 10:57:30 +00:00
Anton Schwaighofer 106c101ff7
DOC: Improve doc for checkpointing (#747) 2022-06-21 17:09:24 +01:00
Anton Schwaighofer ad9aac5413
DOC: Improve documentation for dataset creation (#723) 2022-06-06 16:12:10 +00:00
Peter Hessey 92d94799f2
ENH: Add Environment Locking (#735)
*  Add environment locking script

* 📝  Finish script, add documentation

* 🐛 Change Windows env file in workflow

* 📝 🐛 Add review changes + fixes

* 🚧 Temporarily alter tests and conda channels

* 🚧 Add logging

*  Fix TestSubmodule env file

* 🔥 Delete env test

* 🧑‍💻 Add warning to environment.yml

* 📝 ⚰️ Update based on review comments

* 📝 Add final changes
2022-06-01 10:05:54 +00:00
Anton Schwaighofer 14b968c282
DOC: Add a smoke-test model for AzureML setup (#708)
- Details how to test AzureML without uploading data.
- Also sneaked in an update of hi-ml to fffbb0f8
2022-04-13 14:05:09 +01:00
Fernando Pérez-García 6aab3b1f93
STYLE: Ensure line breaks at the end of all files (#717)
* Add line breaks at the end of all files

* Handle last line in VS Code settings
2022-04-04 15:08:43 +01:00
Fernando Pérez-García 2edbdc8d64
STYLE: Trim trailing whitespaces with pre-commit (#715) 2022-04-04 10:27:38 +01:00
Fernando Pérez-García 760cbe0305
ENH: Enforce prefixes for pull request titles (#713)
Enforce prefixes for pull request titles

Co-authored-by: Anton Schwaighofer <antonsc@microsoft.com>
2022-03-30 15:18:28 +01:00
Fernando Pérez-García 3eaf2f9053
Deprecate manual changelog (#707) 2022-03-29 13:43:50 +01:00
Fernando Pérez-García 45e7d5ff4d
Fix some Sphinx warnings (#699)
* Fix shell lexer name

* Update CHANGELOG

* Fix CHANGELOG

* Fix "html_static_path entry '_static' does not exist"

* Clean up preprocess script

* Fix link to InnerEye-DataQuality

* Use shutil.copy to copy files

* Remove extra info from CHANGELOG

* Fix broken link to LICENSE

* Fix lexer name for YAML

* Remove colons from headers

* Fix InnerEye module not being found
2022-03-22 09:46:15 +00:00
Anton Schwaighofer f0d2337d1b
Improve setup for running the HelloWorld model in AzureML (#693) 2022-03-15 14:07:28 +00:00
Javier 1606729c7a
Clean up legacy code (#671)
* Remove rnns

* Fix flake8

* Edit README

* Edit README

* Remove sequence

* Remove sequence

* Fix all

* Remove more

* Remove ignore

* Fix tests

* Undo config

* Fix config

* Revert pycharm

* Fix tests

* Undo outputlogger

* Fix flake8

* Fix ignore file

* Revert hi-ml

* Disable fail on alert
2022-03-09 14:53:12 +00:00
Anton Schwaighofer 0bc009e07d
Fixing SSL recovery, attempt 2 (#584)
Add linear head state and linear head optimizer to the checkpoints
2021-11-18 17:30:09 +00:00
maxilse 28d40c1f72
Update hello_world_model.md (#580) 2021-11-09 13:59:06 +00:00
Anton Schwaighofer bf4cb628c6
Switching batch time loading diagnostics to hi-ml (#577)
- Using the BatchTimeCallback from hi-ml
- Switches to trigger PL profiling
2021-11-03 15:48:10 +00:00
Jonathan Tripp 0e50ad58dd
Document Segmentation Model Evaluation (#544) 2021-10-22 11:51:01 +01:00
vale-salvatelli 521c004357
Generalize SSL functionality to work on other datasets (#555)
This PR contains some changes needed to make the SSLContainer compatible with new datasets and allow a user to run by simply creating a new augmentation config or defining a child class

* _get_transforms has been changed to accept new datasets without the need to touch the class
* get_cxr_ssl_transform has been changed to avoid the hidden channel expansion and make that optional. It has been also renamed to get_ssl_transform because it has nothing specific to cxr
* drop_last is now set as parameter of the InnerEyeVisionDataModule and the SSLContainer - that means it can be changed when initializing a new SSLContainer
* documentation about bringing your own SSL model has been updated
2021-09-15 10:48:29 +00:00
Anton Schwaighofer b35399fb84
Moving InnerEye's Azure code to hi-ml package (#548)
Moving InnerEye to use the new HI-ML package. 

See Issue 62 on the HI-ML package
2021-08-26 09:17:09 +01:00
vale-salvatelli c946c689aa
Environment and hello_world_model documentation updated (#546)
* Some additions and fix to the docs describing how to setup the environment. The new version should be a little more clear.
* A small change in the logging info when submitting azureml - the workspace name is now reporte
2021-07-27 09:28:17 +01:00
Shruthi42 9fcc08f6cd
Run inference using checkpoints from registered models (#509) 2021-07-15 14:31:15 +00:00
Tim Regan 3fc71b847e
Timregan/459 partial ground truth missed two things (#522)
* Fixing unit test

My approach was wrong, since it only tested the first time the mocked function was called.
I've swapped to Anton's suggestion by adding return values so that the mock can be called
multiple times in a test execution.

* Missing documentation
2021-07-06 11:18:53 +01:00
Alberto Santamaria-Pang d6d67bda10
Added ability to run segmentation inference module in the test data without or partial ground truth files. (#465)
Inference should become more flexible to work also on datasets where we do not have segmentations yet, or only some of the structures segmented. This will prove especially useful for partners like UCLH. Done by adding `allow_incomplete_labels` to the relevant methods.
2021-07-05 16:22:13 +00:00
Jonathan Tripp cab68ccc61
Split validation and test infer config (#502)
Split validation, test, ensemble inference flags
2021-07-05 16:25:49 +01:00
Tim Regan c2f0d668e8
Allow cross validation with 'bring your own' Lightning models (without ensemble building) (#483)
Allows cross validation with 'bring your own' Lightning models (without ensemble building).

Moved the code dealing with hyperdrive config retrieval from ModelConfigBase into it's base class WorkflowParams
so that it is available (and implemented) in LightningContainer

Removed the exception thrown in runner when trying to use cross validation with LightningContainer.

Added cross validation to HelloContainer (and especially its HelloDataModule) as exemplar code, and documentation to point to this change.

Three new unit tests:
1) That cross validation is handled correctly for LightningContainer in runner
2) That LightningContainer converts ScriptRunConfig into HyperDriveConfig
3) The cross validation functionality in HelloDataModule
2021-06-24 20:53:40 +01:00
Anton Schwaighofer be36e39206
Bug fix for regression test (#496)
- The regresssion test code in #492 missed out files in AzureML, because the wrong function was called in the main workflow.
- Cleaner folder structure for regression test results
2021-06-21 14:39:09 +01:00
Anton Schwaighofer 9749954923
Fix for stuck test set inference for LightningContainer models (#494)
Jobs got stuck if they used PyTorch Lightning built-in metrics objects, which were then trying to synchronize across GPUs. Resolved by shutting down torch.distributed
2021-06-17 17:34:26 +01:00
Jonathan Tripp 0f710dd452
Check bool parameter is either true or false (after lower) (#482)
Add test that a boolean command line parameter is one of: 'on', 't', 'true', 'y', 'yes', '1' (for true) or 'off', 'f', 'false', 'n', 'no', '0' (for false) after lower()
If a flag default is False, then allow --flag to set it to True (so the ==True is unnecessary)
If a flag default is True, then allow --no-flag to set it to False (so the ==False is unnecessary)
2021-06-10 17:11:46 +01:00
Anton Schwaighofer 5af8b01dc1
Update documentation for submodules (#481)
The current documentation uses the submodule setup at many places. However, we found that it can easily mislead users. Switching to suggesting plain forking.
2021-06-09 17:35:51 +01:00
Anton Schwaighofer 5c5687bf61
Extend documentation for fastMRI (#468) 2021-06-01 11:00:33 +01:00
melanibe 51274c8bdc
Update augmentation pipeline (#458)
* Refractor augmentations initial commit

* Test custom transforms and transform pipeline

* Make transforms independent of config

* Add placeholder to doc as reminder

* Update tests

* Update tests

* Add test for affine

* Add tests for color transform

* Add a todo list

* Adding new test and simplifying the transforms API further

* New tests for new custom transforms

* Remove deprecated file

* Update all transforms

* Add test for custom pipeline transform

* Fix the full pipeline test

* Rename

* SImplify feeding in transforms

* Updating CHANGELOG.md

* Updating CHANGELOG.md

* Flake8

* Simplify more

* Fixes

* Fixes

* Flake8

* Fix few things

* Fix few things

* Updating tests for new API

* Simplify image versus sample

* Update

* Update

* Update

* Update

* Error in test copy paste

* Style

* Add augmentation doc

* Add augmentation doc

* Doc strings

* Flake8

* Flake8

* Mypy

* Fix test and docstring

* Put back skipif windows

* flake8

* mypy

* Skip if OOM

* Skip if OOM

* Import was missing

* Separate transform pipeline from ScalarItem management.

* Update documentation to new API

* Accidental commit

* PR suggestion

* Make test for elastic human checkable

* Make test for elastic human checkable

* Update with p=0

* Update with p=0

* Add human readable test output for gamma transform

* Solve OOM, remove duplicate test input definitions

* Flake8

* Mypy

* Fix config

* Add check PR comment

* PR comment
2021-06-01 09:29:57 +01:00
Anton Schwaighofer 8bae42eb92
FastMRI dataset onboarding script and detailed examples (#444)
Add necessary tooling and examples for running fastMRI reconstruction models.
- Script to create and run an Azure Data Factory to download the raw data, and place them into a storage account
- Detailed examples to run the VarNet model from the fastMRI github repo
- Ability to work with fixed mounting points for datasets
2021-05-19 15:58:25 +00:00
melanibe 7b5b414d02
Add self-supervised learning capabilities to InnerEye (#440) 2021-05-07 13:27:38 +00:00
Javier 392724bdaa
Move models from one AML workspace to another (#441)
* Add move models

* Add move models

* Add test

* Fix flake8

* add space

* Fix issues with tests

* Fix issues mypy

* Add changelog
2021-04-28 10:25:07 +01:00
Javier f421234c3d
Add documentation and links for deployment components (#438)
* Add docs pointing to inference components

* Fix typo

* Add changelog

* Update deploy_on_aml.md
2021-04-20 08:40:21 +00:00
Anton Schwaighofer 0d479ba3d8
Enable Bring-your-own-Lightning-model (#417)
- Enable brining arbitrary PyTorch-Lightning models to the InnerEye toolbox
- Upgrade mypy and simplify the way we invoke it
2021-04-19 15:28:41 +00:00
Javier 917f8d0b30
Support Multilabel classification models with png files (#391)
- This PR adds support for multilabel classification models. Multiclass models (where labels are mutually exclusive) are not supported. Changes made:
  - `dataset.csv` can include multiple labels per sample
  - Loss functions changed to support multilabel classification tasks
  - Extra report for multilabel classification tasks
- Add support for png images

Co-authored-by: Shruthi42 <13177030+Shruthi42@users.noreply.github.com>
Co-authored-by: melanibe <32590828+melanibe@users.noreply.github.com>
Co-authored-by: Anton Schwaighofer <antonsc@microsoft.com>
2021-03-19 17:03:05 +01:00
Steve Haigh 6f475ffe4c
Clarified storage creation steps and fixed small typos (#422) 2021-03-18 20:59:50 +00:00
Anton Schwaighofer cc96b620c1
docu update (#399)
Closes #397
2021-02-15 16:37:29 +05:30
Anton Schwaighofer 7ef6ff9ecc
Improve documentation for dataset upload (#392) 2021-02-09 17:02:15 +05:30
Anton Schwaighofer b40d6d13c0
Enable multi-node training (#385)
- Enable training on multiple machines in AzureML
- Exclude private settings files from AzureML snapshot
2021-02-08 14:21:50 +00:00
Clemens Siebler 5d5e55e314
Documentation updates (#368)
Co-authored-by: Javier <jaalvare@microsoft.com>
Co-authored-by: Anton Schwaighofer <antonsc@microsoft.com>
2021-02-03 09:03:19 +00:00
Anton Schwaighofer 312afc3bf3
renaming master -> main (#384)
Updates to match the new default branch name "main".
2021-02-02 15:48:21 +00:00
Shruthi42 24585c7240
Remove sub-fold documentation (#369)
Remove old documentation about sub-fold cross validation, which is no longer supported.
2021-01-29 12:08:31 +05:30
Anton Schwaighofer bc90c65f0a
Migrate to Pytorch Lightning (#323)
This PR swaps out all of the previously hand-written training routines, and switches them to PyTorch Lightning.
2021-01-28 15:25:53 +00:00
Javier 0c424abdd6
Fix docs for HelloWorld (#360)
* Fix docs for HellowWorld

* Fix docs for HellowWorld

* Add more details
2021-01-15 13:11:23 +00:00
Shruthi42 787c6d8a80
Use mypy_runner with PyCharm (#358)
Update mypy_runner.py so that it can be used as a PyCharm external tool.
2021-01-13 10:49:06 +00:00
Jonathan Tripp 70f7861997
Add in unit tests and prototype for new config classes (#336)
* Renamed HeadAndNeckBase to HeadAndNeckPaper, and ProstateBase to ProstatePaper;
* Added new extensions of SegmentationModelBases HeadAndNeckBase and ProstateBase;
* Delete unused ProstatePaperBase.py;
* Added validation for SliceExclusionRule and SummedProbabilityRule.
2020-12-11 17:00:07 +00:00
Anton Schwaighofer 1a7b64f6c3
Remove more dead arguments (#333) 2020-12-04 11:31:37 +00:00
Anton Schwaighofer 014c74e34f
Fix ensemble checkpoint download (#326)
* test

* fix test

* download fix

* create separate model folder

* fixing tests

* making HD check better

* Tests

* inverted logic

* registering on parent run

* docu
2020-11-25 16:25:02 +01:00
Ivan Tarapov c434d31293
Broken formatting (#319)
Fixing broken formatting of a code snippet

Co-authored-by: Javier <jaalvare@microsoft.com>
2020-11-17 14:59:09 +00:00