Removed storage account keys from AML datastores. This PR handles the
consequences of that.
- Testing with an uploaded private wheel had to be removed because AML
SDK v1 will change a user delegation SAS token to a access-key SAS
token.
- Switch download of test datasets for cpath to mounting of datasets
- Skip some flaky distributed tests
- Use of `AzureCliCredential` rather than `DefaultAzureCredential`
When `hyperparam_args` is passed, the command is modified to add the
argument in the settings. For example, if the looks like
`{learning_rate: [1, 2, 3]}`, something like
`--learning_rate=${{inputs.learning_rate}}` will be added to the
command. But sometimes we wouldn't want the dashes, e.g., when using
Hydra (`learning_rate=${{inputs.learning_rate}}` or maybe
`+learning_rate=${{inputs.learning_rate}}`). This PR adds support to
specify the prefix for the argument (default: `"--"`). For Hydra, we
might want, e.g., `""` or `"+"`.
<!--
## Guidelines
Please follow the guidelines for pull requests (PRs) in
[CONTRIBUTING](/CONTRIBUTING.md). Checklist:
- Ensure that your PR is small, and implements one change
- Give your PR title one of the prefixes listed in
[CONTRIBUTING](../docs/source/coding_guidelines.md#pull-request-titles)
to indicate what type of change that is
- Link the correct GitHub issue for tracking
- Add unit tests for all functions that you introduced or modified
- Run automatic code formatting / linting on all files ("Format
Document" Shift-Alt-F in VSCode)
- Ensure that documentation renders correctly in Sphinx by running `make
html` in the `docs` folder
## Change the default merge message
When completing your PR, you will be asked for a title and an optional
extended description. By default, the extended description will be a
concatenation of the individual
commit messages. Please DELETE/REPLACE that with a human readable
extended description for non-trivial PRs.
-->
Remove the use of Service Principal authentication.
To make this work, I also had to add to modify the Service Principal. In
the "Clients & Secrets" section, add "Federated Credentials". Choose
"Scenario: GitHub Action". Fill in repo details, "Entity Type: Pull
Request". Name does not matter.
At present, only ServicePrincipal and InteractiveAuthentication are
supported for AzureML SDK v1. This PR adds AzureCLI as a third option.
This sets the stage for password-free authentication in pipelines
- Upgrade to Python 3.9 as a minimum
- Upgrade pillow because of security alerts. This required some API
changes and update of test result images
- The behaviour of the `LogCaptureFixture` changed, requiring changes
from `logging.info` to `logger.info`
- Downgraded `black` from 23 to 22 because of version constraints
- Changes in calls to `WorkflowParams` because of `mypy` issues that
were not detected until now
- `hi-ml-azure` now also has its own minimal Conda dev environment
svs has different metadata properties than Tiff and NDPI. This fix
handles the case when get_mpp from monai fails with a KeyError exception
missing "Tiff.XResolution" and "Tiff.YResolution"
TIFF conversion fails if input resolution is not in centimeters, but we
don't rely on that being the case. Hence, remove the error message and
just print an info line
Resolve issues with AML SDK v1/v2: Even in the v2 code path we are
creating v1 workspaces, leading to duplicate authentication requests.
To achieve that, a couple of other changes were necessary:
* Deprecate the use of the default datastore, which was read out of an
SDK v1 Workspace object even if SDK v2 was chosen
* No longer allowing SDK v2 when mounting datasets for local runs (v2
Datasets can't be mounted at all).
Also added more detailed logging for dataset creation, and a commandline
flag to control logging level.
Metadata columns are presently set via class variables, rather than in
the constructor. Hence, we can't have several datasets that use
different metadata columns, without them interfering with each other.
This caused odd test failures because test results now depend on the
order in which tests are executed.
Fix the order of hook call. on_run_extra_val_epoch needs to be run
before super().init_inference to account for any changes in datamodule
params before re-instantiation
This PR allows the user to determine whether to use ResNet weights
pre-trained on ImageNet or BioViL, or whether to have a randomly
initialised model: fixes#617 .
Closes#786
It should now be impossible to create data new data assets without also
specifying a datastore by its specific name, in both v1 and v2 pathways.
Default datastore interactions in all other code paths have been left
unchanged.
dependabot[bot]
Anton Schwaighofer
Fernando Pérez-García
Harshita Sharma
Kenza Bouzid
markpinnock
maxilse
Peter Hessey
Jubin Mathew
pre-commit-ci[bot]
Peter Hessey