Closes#696.
Changes the HelloWorld model to only use a single channel, as this is
the format that the [Inference
Service](https://github.com/microsoft/InnerEye-Inference/) expects for
its inputs.
As many tests require the 2-channel data, this PR also creates a new
class, `HelloWorld2Channel`, which inherits from `HelloWorld` but uses 2
channels and can be used for testing. This avoids the pain of having to
alter all unit and regression tests that relied on the 2-channel data
and model.
Both models can be trained on the same data, but now the `HelloWorld`
model can be run by the Inference Service.
This PR contains the hyperparameter updates for training and running
inference on InnerEye Lung segmentation model on a Standard_NC24rs_v3
Azure cluster (4 x V100).
It also includes all the documentation of the model will be released in
a tagged version in the same fashion as the [hippocampus
model](https://github.com/microsoft/InnerEye-DeepLearning/releases/tag/v0.5).
I will tag and release the model after this PR is merged to ensure that
the source code in the main branch is correct at the time of release
(docs currently include a link to v0.8 which doesn't yet exist).
* Update installation guide in WSL.md
- Removed long installation guide
- Included new shorter guide to install WSL
* Update WSL.md with given suggestions
- More clear language
* Update docs/WSL.md
* Update docs/WSL.md
* Update docs/WSL.md
* Trim trailing whitespace in WSL.md
- Trimmed trailing whitespace to pass the CI checks
Co-authored-by: Anton Schwaighofer <antonsc@microsoft.com>
* Fix shell lexer name
* Update CHANGELOG
* Fix CHANGELOG
* Fix "html_static_path entry '_static' does not exist"
* Clean up preprocess script
* Fix link to InnerEye-DataQuality
* Use shutil.copy to copy files
* Remove extra info from CHANGELOG
* Fix broken link to LICENSE
* Fix lexer name for YAML
* Remove colons from headers
* Fix InnerEye module not being found
This PR contains some changes needed to make the SSLContainer compatible with new datasets and allow a user to run by simply creating a new augmentation config or defining a child class
* _get_transforms has been changed to accept new datasets without the need to touch the class
* get_cxr_ssl_transform has been changed to avoid the hidden channel expansion and make that optional. It has been also renamed to get_ssl_transform because it has nothing specific to cxr
* drop_last is now set as parameter of the InnerEyeVisionDataModule and the SSLContainer - that means it can be changed when initializing a new SSLContainer
* documentation about bringing your own SSL model has been updated
* Some additions and fix to the docs describing how to setup the environment. The new version should be a little more clear.
* A small change in the logging info when submitting azureml - the workspace name is now reporte
* Fixing unit test
My approach was wrong, since it only tested the first time the mocked function was called.
I've swapped to Anton's suggestion by adding return values so that the mock can be called
multiple times in a test execution.
* Missing documentation
Inference should become more flexible to work also on datasets where we do not have segmentations yet, or only some of the structures segmented. This will prove especially useful for partners like UCLH. Done by adding `allow_incomplete_labels` to the relevant methods.
Allows cross validation with 'bring your own' Lightning models (without ensemble building).
Moved the code dealing with hyperdrive config retrieval from ModelConfigBase into it's base class WorkflowParams
so that it is available (and implemented) in LightningContainer
Removed the exception thrown in runner when trying to use cross validation with LightningContainer.
Added cross validation to HelloContainer (and especially its HelloDataModule) as exemplar code, and documentation to point to this change.
Three new unit tests:
1) That cross validation is handled correctly for LightningContainer in runner
2) That LightningContainer converts ScriptRunConfig into HyperDriveConfig
3) The cross validation functionality in HelloDataModule
- The regresssion test code in #492 missed out files in AzureML, because the wrong function was called in the main workflow.
- Cleaner folder structure for regression test results
Jobs got stuck if they used PyTorch Lightning built-in metrics objects, which were then trying to synchronize across GPUs. Resolved by shutting down torch.distributed
Add test that a boolean command line parameter is one of: 'on', 't', 'true', 'y', 'yes', '1' (for true) or 'off', 'f', 'false', 'n', 'no', '0' (for false) after lower()
If a flag default is False, then allow --flag to set it to True (so the ==True is unnecessary)
If a flag default is True, then allow --no-flag to set it to False (so the ==False is unnecessary)
The current documentation uses the submodule setup at many places. However, we found that it can easily mislead users. Switching to suggesting plain forking.
* Refractor augmentations initial commit
* Test custom transforms and transform pipeline
* Make transforms independent of config
* Add placeholder to doc as reminder
* Update tests
* Update tests
* Add test for affine
* Add tests for color transform
* Add a todo list
* Adding new test and simplifying the transforms API further
* New tests for new custom transforms
* Remove deprecated file
* Update all transforms
* Add test for custom pipeline transform
* Fix the full pipeline test
* Rename
* SImplify feeding in transforms
* Updating CHANGELOG.md
* Updating CHANGELOG.md
* Flake8
* Simplify more
* Fixes
* Fixes
* Flake8
* Fix few things
* Fix few things
* Updating tests for new API
* Simplify image versus sample
* Update
* Update
* Update
* Update
* Error in test copy paste
* Style
* Add augmentation doc
* Add augmentation doc
* Doc strings
* Flake8
* Flake8
* Mypy
* Fix test and docstring
* Put back skipif windows
* flake8
* mypy
* Skip if OOM
* Skip if OOM
* Import was missing
* Separate transform pipeline from ScalarItem management.
* Update documentation to new API
* Accidental commit
* PR suggestion
* Make test for elastic human checkable
* Make test for elastic human checkable
* Update with p=0
* Update with p=0
* Add human readable test output for gamma transform
* Solve OOM, remove duplicate test input definitions
* Flake8
* Mypy
* Fix config
* Add check PR comment
* PR comment
Peter Hessey
Dr Kenji Takeda
Anton Schwaighofer
arsenkhy
Melissa Bristow
Fernando Pérez-García
Javier
maxilse
Jonathan Tripp
vale-salvatelli
Shruthi42
Tim Regan
Alberto Santamaria-Pang
melanibe