Граф коммитов

143 Коммитов

Автор SHA1 Сообщение Дата
manipopopo c998634d40 Add `dim` to `pytorch_lightning.metrics.PSNR` (PL^5957)
* Add dim to PSNR

* Update CHANGELOG.md

* Update CHANGELOG.md

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Add reduction tests

* Recover warnings on reduction and add tests

* Add copyright texts

* Refactor PSNR

* Change warnings

* Update pytorch_lightning/metrics/functional/psnr.py

Change functional.psnr dim doc

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Change PSNR dim docs

* Apply suggestions from code review

* tests

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
(cherry picked from commit a3dd545b733f9b0cafbe725a4f2e57bb6bed2759)
2021-02-22 01:24:21 +01:00
Adrian Wälchli 4c2b7b24ec remove executable bit on source files (PL^5929)
* 644

(cherry picked from commit 39163e218a9407f69172cc79dca2ef019790a6cd)
2021-02-22 01:24:21 +01:00
Nicki Skafte 13c0db84b2 fix metric docs (PL^5880)
(cherry picked from commit 8b03a2dedd02f31c3ddc7f7d6271a43b79245969)
2021-02-22 01:24:21 +01:00
Jirka Borovec aee63969f0 mean relative error (PL^5706)
* empty

* sq

* obs

* int

* ts

* helpers

* chlog

* yapf

* avg

* dupl

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* fixes

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* fixes

* note

* warn

* 45

* link

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* yapf

* flake8

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
(cherry picked from commit 3eea2c3078e4d56949b34c298aca19f463542245)
2021-02-22 01:24:21 +01:00
Jirka Borovec 71e18a2961 fixing some compatibility with PT 1.8 (PL^5864)
* change default

* .

* p

* 0.21.2

* .

* fix

* .

(cherry picked from commit fdc1c66cdcfd07ea9a3a6e7a19d0467da1447944)
2021-02-22 01:24:21 +01:00
Jirka Borovec bfef808b77 Refactor simplify tests (PL^5861)
* add new

* restructure

* yapf

* move

* fix

(cherry picked from commit 004ddb04ee276c21ddf411d431cb8cf102c1675c)
2021-02-22 01:24:21 +01:00
Jirka Borovec a718ab3cca yapf tests metrics (PL^5845)
(cherry picked from commit cce1be3470a58aa46047d598ff00e9ad23f5f877)
2021-02-22 01:24:21 +01:00
Nicki Skafte 8bc17d8db8 Fix `num_classes` arg in F1 metric (PL^5663)
* fix f1 metric

* Apply suggestions from code review

* chlog

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
(cherry picked from commit ef8efc611c12209ebf158aab7f444756ba35927e)
2021-02-22 01:24:21 +01:00
Jirka Borovec 3dcd3cb037 flake8 + yapf
(cherry picked from commit 665919b38faa3c4daa490175c8a3d987f29f8d32)
2021-02-22 01:24:21 +01:00
manipopopo 635d9d8b7b Fix Metric.state_dict (PL^5614)
* Fix Metric.state_dict

* Update CHANGELOG.md

* Update CHANGELOG.md

* Detach tensors in a list if needed

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

(cherry picked from commit c3df5122e71d497982b6764bbe51c22a26aa4fef)
(cherry picked from commit 7b0c79a7920403cc543a4a79b6efa0cca0508434)
2021-02-22 01:24:21 +01:00
yuntai 2dd5749968 Update metrics to use Enum (PL^5689)
- Add DataType, AverageMethod and MDMCAverageMethod

(cherry picked from commit 0612034ea261375c1f6f1a2396aac9caeacc55a8)
2021-02-22 01:24:20 +01:00
Jirka Borovec 3367fac214 formatting 6/n: metrics (PL^5722)
* yapf metrics

* op

(cherry picked from commit 6dc63f3f50f407efaafece7e4dc61a29f434c412)
2021-02-22 01:24:20 +01:00
Nicki Skafte 855eea6ed2 AUC/AUROC class interface (PL^5479)
* base files

* auc done

* init files

* auc class interface

* fixing auc

* more fixes

* working auroc

* update auc

* add docs

* remove leftovers from merge

* suggestions

* fix f-string

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* add deprecated tests

* make logic clearer

* Update pytorch_lightning/metrics/classification/auroc.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* fix

* fix

* fix docs

* fix isort

* fix deprecated test

* fix tests

* fix tests

* fix isort

* Apply suggestions from code review

* add enum

* deprecate old impl

* update from suggestions

* chlog

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
(cherry picked from commit 86c96cdfd3a943046c2ad240d1b2e3f525e523b0)
2021-02-22 01:24:20 +01:00
Jirka Borovec 380938da1a move to Pages dir (PL^4869)
* folders

* common / advanced / extensions

* paths

* flake8

* isort

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
(cherry picked from commit b56c19af91a8c8a7e66bd4bd20b869aff5be3b92)
2021-02-22 01:24:20 +01:00
Jirka Borovec 7904df201b hotfix: skip unsupported metric compostions (PL^5664)
* skip PT unsupported compositions

* flake8

(cherry picked from commit 803bd25cce5998dc6cbf66befeb204aaf6819bd9)
2021-02-22 01:24:20 +01:00
Justus Schock ff01a0b9f2 Compositional metrics (PL^5464)
* implement compositional metrics

* implement composition functions for metrics

* test compositions

* docs

* pytest

* pep8

* fix argument resolution

* return all kwargs if filtering not possible

* fix typo

* implement hashing

* Update pytorch_lightning/metrics/compositional.py

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Update docs/source/metrics.rst

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* add representation

* Apply suggestions from code review

* Update docs/source/metrics.rst

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* chlog

* flake8

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
(cherry picked from commit a01f213bd7f4657c4444e2d8ca4244db768795cb)
2021-02-22 01:24:20 +01:00
Arnaud Gelas 9385e19d16 Fix isort failures in metrics (PL^5528)
Remove from skipped module in pyproject.toml and fix failures on:
- pytorch_lightning/metrics/*.py

(cherry picked from commit 135af5d0131c140f4522a33bb6ef5041281b4ff7)
2021-02-22 01:24:20 +01:00
Tadej Svetina 5cf4e45e6b Classification metrics overhaul: precision & recall (4/n) (PL^4842)
* Add stuff

* Change metrics documentation layout

* Add stuff

* Add stat scores

* Change testing utils

* Replace len(*.shape) with *.ndim

* More descriptive error message for input formatting

* Replace movedim with permute

* PEP 8 compliance

* WIP

* Add reduce_scores function

* Temporarily add back legacy class_reduce

* Division with float

* PEP 8 compliance

* Remove precision recall

* Replace movedim with permute

* Add back tests

* Add empty newlines

* Add precision recall back

* Add empty line

* Fix permute

* Fix some issues with old versions of PyTorch

* Style changes in error messages

* More error message style improvements

* Fix typo in docs

* Add more descriptive variable names in utils

* Change internal var names

* Revert unwanted changes

* Revert unwanted changes pt 2

* Update metrics interface

* Add top_k parameter

* Add back reduce function

* Add stuff

* PEP3

* Add depreciation

* PEP8

* Deprecate param

* PEP8

* Fix and simplify testing for older PT versions

* Update Changelog

* Remove redundant import

* Add tests to increase coverage

* Remove zero_division

* fix zero_division

* Add zero_div + edge case tests

* Reorder cls metric args

* Add back quotes for is_multiclass

* Add precision_recall and tests

* PEP8

* Fix docs

* Fix docs

* Update

* Change precision_recall output

* PEP8/isort

* Add method _get_final_stats

* Fix depr test

* Add comment to deprecation tests

* isort

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Add typing to test

* Add matc str to pytest.raises

Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
(cherry picked from commit 6e3a83752e54efe2cd2fc130e95e4ee10c66e6be)
2021-02-22 01:24:20 +01:00
Arnaud Gelas bb4ad8d407 Fix pre-commit isort failure on tests/metrics/*.py (PL^5424)
* Remove tests.metrics from skipped module in pyproject.toml

* Fix pre-commit isort failure on tests/metrics/*.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
(cherry picked from commit d4ed6240c46bd012c2b8c5d0ec3eecb90cd8f86f)
2021-02-22 01:24:20 +01:00
Nicki Skafte dd1e744efe Disable default reset after compute (PL^5409)
* reset

* self._cache -> cache (make cache local variable so it is not overwritten)

* pep8

* fix metric result integration

* rm print statements

* better comment

* changelog

* Update docs/source/metrics.rst

Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
(cherry picked from commit 2dc6e588697a40d7a3ee62a35433895734d05b71)
2021-02-22 01:24:20 +01:00
deng-cy 92c2e1923b Reformat iou [func] and add IoU class (PL^4704)
* added Iou

* Create iou.py

* Update iou.py

* Update iou.py

* Update CHANGELOG.md

* Update metrics.rst

* Update iou.py

* Update iou.py

* Update __init__.py

* Update iou.py

* Update iou.py

* Update classification.py

* Update classification.py

* Update classification.py

* Update __init__.py

* Update __init__.py

* Update iou.py

* Update classification.py

* Update metrics.rst

* Update CHANGELOG.md

* Update CHANGELOG.md

* add iou

* add test

* add test

* removed iou

* add iou

* add iou test

* add float

* reformat test_iou

* removed test_iou

* updated format

* updated format

* Update CHANGELOG.md

* updated format

* Update metrics.rst

* Apply suggestions from code review

merge suggestions

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* added equations

* reformat init

* change format

* change format

* deprecate iou and test for this

* fix changelog

* delete iou test in test_classification

* format change

* format change

* format

* format

* format

* delete white space

* delete white space

* fix tests

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* better deprecation

* fix docs

* Apply suggestions from code review

* fix todo

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
(cherry picked from commit 238040c9e5f2a9f03cecb171791fd8c4b88ca8b7)
2021-02-22 01:24:20 +01:00
Nicki Skafte bcb82d2a0c MetricCollection (PL^4318)
* docs + precision + recall + f_beta + refactor

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* rebase

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* fixes

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>

* added missing file

* docs

* docs

* extra import

* add metric collection

* add docs + integration with log_dict

* add test

* update

* update

* more test

* more test

* pep8

* fix doctest

* pep8

* add clone method

* add clone method

* merge-2

* changelog

* kwargs filtering and tests

* pep8

* fix test

* update docs

* Update docs/source/metrics.rst

Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>

* fix docs

* fix tests

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* fix docs

* fix doctest

* fix doctest

* fix doctest

* fix doctest

Co-authored-by: ananyahjha93 <ananya@pytorchlightning.ai>
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
(cherry picked from commit c371c57641947be0aa22ffa688d437bed6eeea82)
2021-02-22 01:24:20 +01:00
Pranjal Datta dfeaaf0713 added image-gradients (PL^4763) [1/2] (PL^5056)
* added iamge-gradients (PL^4763)

* fixed tests code format

* made recommended fixes

* removed explicit device flags

* tried to fix doctest failure

* pep8 and doctest fixes

* added to docs/metrics

* updated CHANGELOG

* added the noqa flag

* added suggested modification to changelog

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* recommended update to docstring

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* removed device from docstring

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* made recommended fixes

* Update CHANGELOG.md

* Apply suggestions from code review

* added 1-line docstrings

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
(cherry picked from commit b520321c9f89aaca67f63862089dded1de300c81)
2021-02-22 01:24:20 +01:00
Jirka Borovec ee03bce7f0 fix some minor typos in docs (PL^5369)
* fix docs typos

* Apply suggestions from code review

Co-authored-by: Wansoo Kim <rladhkstn8@gmail.com>

* flake8

Co-authored-by: Wansoo Kim <rladhkstn8@gmail.com>
(cherry picked from commit a047323cc796dcf5d4709399083bfc4b3b411de9)
2021-02-22 01:24:20 +01:00
Jirka Borovec 1cccdc73d9 drop duplicated metric helper (PL^5366)
* drop duplicated metric helper

* .

* fix tests

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
(cherry picked from commit 220dfaf7dcd12ad230ea2938d8d67a119a22e143)
2021-02-22 01:24:20 +01:00
Abhik Banerjee 261177efa2 Updated metrics/classification/precision_recall.py (PL^5348)
There was a typo in Documentation of Code of the ```compute()``` function of ```Recall``` metric at line 210. It said "Computes accuracy over state." which should have been "Computes recall over state."

(cherry picked from commit d568533b6ba0768ac7a6d028b3a25c5970a65e80)
(cherry picked from commit d2b7faf452890225dd4b7b62634bcbd8d0fdff96)
2021-02-22 01:24:20 +01:00
Tadej Svetina 20859b966e Fix metric state reset (PL^5273)
* Fix metric state reset

* Fix test

* Improve formatting

Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>
(cherry picked from commit 26488794e1af6e2fbe05bf21de52cf999252cbe7)
(cherry picked from commit c55f02a770b5fb5f6d59c48154256ef50f8fa6f1)
2021-02-22 01:24:20 +01:00
Jirka Borovec 21081669b0 drop deprecated fbeta metrics (PL^5322)
* drop deprecated fbeta metrics

* flake8

* imports

* chlog

(cherry picked from commit 89cc91060057d78a51682c9d6aebc580b194a44d)
2021-02-22 01:24:20 +01:00
Nicki Skafte 8736ee01eb R2Score (PL^5241)
* add r2metric

* change init

* add test

* add docs

* add math

* Apply suggestions from code review

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* changelog

* adjusted parameter

* add more test

* pep8

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* add warnings for adjusted score

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
(cherry picked from commit 4d44437074051a24366973e1ead758ff2f44c3b2)
2021-02-22 01:24:20 +01:00
Tadej Svetina 247e085d5f Classification metrics overhaul: stat scores (3/n) (PL^4839)
* Add stuff

* Change metrics documentation layout

* Add stuff

* Add stat scores

* Change testing utils

* Replace len(*.shape) with *.ndim

* More descriptive error message for input formatting

* Replace movedim with permute

* PEP 8 compliance

* WIP

* Add reduce_scores function

* Temporarily add back legacy class_reduce

* Division with float

* PEP 8 compliance

* Remove precision recall

* Replace movedim with permute

* Add back tests

* Add empty newlines

* Add empty line

* Fix permute

* Fix some issues with old versions of PyTorch

* Style changes in error messages

* More error message style improvements

* Fix typo in docs

* Add more descriptive variable names in utils

* Change internal var names

* Break down error checking for inputs into separate functions

* Remove the (N, ..., C) option in MD-MC

* Simplify select_topk

* Remove detach for inputs

* Fix typos

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update docs/source/metrics.rst

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Minor error message changes

* Update pytorch_lightning/metrics/utils.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Reuse case from validation in formatting

* Refactor code in _input_format_classification

* Small improvements

* PEP 8

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update docs/source/metrics.rst

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Alphabetical reordering of regression metrics

* Change default value of top_k and add error checking

* Extract basic validation into separate function

* Update to new top_k default

* Update desciption of parameters in input formatting

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Check that probabilities in preds sum to 1 (for MC)

* Fix coverage

* Split accuracy and hamming loss

* Remove old redundant accuracy

* Minor changes

* Fix imports

* Improve docstring descriptions

* Fix imports

* Fix edge case and simplify testing

* Fix docs

* PEP8

* Reorder imports

* Add top_k parameter

* Update changelog

* Update docstring

* Update docstring

* Reverse formatting changes for tests

* Change parameter order

* Remove formatting changes 2/2

* Remove formatting 3/3

* .

* Improve description of top_k parameter

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Remove unneeded assert

* Update pytorch_lightning/metrics/functional/accuracy.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Remove unneeded assert

* Explicit checking of parameter values

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Apply suggestions from code review

* Fix top_k checking

* PEP8

* Don't check dist_sync in test

* add back check_dist_sync_on_step

* Make sure half-precision inputs are transformed (#5013)

* Fix typo

* Rename hamming loss to hamming distance

* Fix tests for half precision

* Fix docs underline length

* Fix doc undeline length

* Replace mdmc_accuracy parameter with subset_accuracy

* Update changelog

* Fix unwanted accuracy change

* Enable top_k for ML prob inputs

* Test that default threshold is 0.5

* Fix typo

* Update top_k description in helpers

* updates

* Update styling and add back tests

* Remove excess spaces

* fix torch.where for old versions

* fix linting

* Update docstring

* Fix docstring

* Apply suggestions from code review (mostly docs)

* Default threshold to None, accept only (0,1)

* Change wrong threshold message

* Improve documentation and add tests

* Add back ddp tests

* Change stat reduce method and default

* Remove DDP tests and fix doctests

* Fix doctest

* Update changelog

* Refactoring

* Fix typo

* Refactor

* Increase coverage

* Fix linting

* Consistent use of backticks

* Fix too long line in docs

* Apply suggestions from code review

* Fix deprecation test

* Fix deprecation test

* Default threshold back to 0.5

* Minor documentation fixes

* Add types to tests

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
(cherry picked from commit 71556b5ddd91df86ab4135aab33c2d86eee5f286)
2021-02-22 01:24:20 +01:00
Prudhvi Rampey 527fa67a7b Implement partial auroc metric (PL^3790)
* Implement partial auroc metric

* Add pycodestyle changes

* Added tests for max_fpr

* changelog

* version for tests

* fix imports

* fix tests

* fix tests

* Added more thresholds in (0,1] to test max_fpr

* Removed deprecated 'reorder' param from auroc

* changelog

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* remove old structure

* Apply suggestions from code review

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* fix test error

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
(cherry picked from commit 36ea27be9d02ed4a702f0d5711808fb51bf13976)
2021-02-22 01:24:20 +01:00
Tadej Svetina 3ac52f16b5 Classification metrics overhaul: accuracy metrics (2/n) (PL^4838)
* Add stuff

* Change metrics documentation layout

* Add stuff

* Change testing utils

* Replace len(*.shape) with *.ndim

* More descriptive error message for input formatting

* Replace movedim with permute

* PEP 8 compliance

* Division with float

* Style changes in error messages

* More error message style improvements

* Fix typo in docs

* Add more descriptive variable names in utils

* Change internal var names

* Break down error checking for inputs into separate functions

* Remove the (N, ..., C) option in MD-MC

* Simplify select_topk

* Remove detach for inputs

* Fix typos

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update docs/source/metrics.rst

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Minor error message changes

* Update pytorch_lightning/metrics/utils.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Reuse case from validation in formatting

* Refactor code in _input_format_classification

* Small improvements

* PEP 8

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update docs/source/metrics.rst

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Alphabetical reordering of regression metrics

* Change default value of top_k and add error checking

* Extract basic validation into separate function

* Update to new top_k default

* Update desciption of parameters in input formatting

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Check that probabilities in preds sum to 1 (for MC)

* Fix coverage

* Split accuracy and hamming loss

* Remove old redundant accuracy

* Minor changes

* Fix imports

* Improve docstring descriptions

* Fix edge case and simplify testing

* Fix docs

* PEP8

* Reorder imports

* Update changelog

* Update docstring

* Update docstring

* Reverse formatting changes for tests

* Change parameter order

* Remove formatting changes 2/2

* Remove formatting 3/3

* .

* Improve description of top_k parameter

* Apply suggestions from code review

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Remove unneeded assert

* Update pytorch_lightning/metrics/functional/accuracy.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Remove unneeded assert

* Explicit checking of parameter values

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Apply suggestions from code review

* Fix top_k checking

* PEP8

* Don't check dist_sync in test

* add back check_dist_sync_on_step

* Make sure half-precision inputs are transformed (#5013)

* Fix typo

* Rename hamming loss to hamming distance

* Fix tests for half precision

* Fix docs underline length

* Fix doc undeline length

* Replace mdmc_accuracy parameter with subset_accuracy

* Update changelog

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Suggestions from code review

* Fix number in docs

* Update pytorch_lightning/metrics/classification/accuracy.py

* Replace topk by argsort in select_topk

* Fix changelog

* Add test for wrong params

* Add Google Colab badges (#5111)

* Add colab badges to notebook

Add colab badges to notebook to notebooks 4 & 5

* Add colab badges

Co-authored-by: chaton <thomas@grid.ai>

* Fix hanging metrics tests (#5134)

* Use torch.topk again as ddp hanging tests fixed in #5134

* Fix unwanted notebooks change

* Fix too long line in hamming_distance

* Apply suggestions from code review

* Apply suggestions from code review

* protect

* Update CHANGELOG.md

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Justus Schock <12886177+justusschock@users.noreply.github.com>
Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
Co-authored-by: Shachar Mirkin <shacharmirkin@gmail.com>
(cherry picked from commit 01365eb7b22ed4fc53bd95c77e44c643e0808484)
2021-02-22 01:24:20 +01:00
Jirka Borovec 59cb104296 fix/enable - check F401 (PL^5201)
* refactor - check F401

* missed

* fix

(cherry picked from commit 8003acbf68160c2f3357ed966464604a2d1510e9)
2021-02-22 01:24:20 +01:00
Jirka Borovec 07f2b802ad refactor - check E501 (PL^5200)
(cherry picked from commit 143714bd280aae92253609df77b5eac077debbe2)
2021-02-22 01:24:20 +01:00
Jirka Borovec 078fde2afa refactor - check F841 (PL^5202)
(cherry picked from commit 94d5b043ff701bd1bc88e36691ea1667ed894ea6)
2021-02-22 01:24:20 +01:00
Jirka Borovec 048a415b26 annotat unused vars (PL^5017)
* annotate all unused vars

* rank_zero_warn

* Apply suggestions from code review

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>

* f1 fixed

Co-authored-by: Carlos Mocholí <carlossmocholi@gmail.com>
(cherry picked from commit dfbad656c493bd392c59e7d563125006b95d225c)
2021-02-22 01:24:20 +01:00
Tadej Svetina b79e64dff0 Fix hanging metrics tests (PL^5134) 2021-02-22 01:24:20 +01:00
Roger Shieh 0f2f1fb9a0 Remove beta arg from F1 class and functional (PL^5076)
* remove beta from F1

* remove from functional

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2021-02-22 01:24:20 +01:00
Roger Shieh 0eef19942b Fix docs metrics formatting (PL^5077)
* fix functional f1 fbeta formatting

* Update f_beta.py

* remove line breaks

* Update f_beta.py

add line breaks and pad

* pad linea breaks with 2 spaces instead of tab
2021-02-22 01:24:20 +01:00
Jirka Borovec 829f5e8316 add back compatibility for deprecated metrics 2/n (PL^5068)
* add back compatibility for deprecated metrics

* fix

* imports

* imports
2021-02-22 01:24:20 +01:00
Jirka Borovec 75cd4df3ad add back compatibility for deprecated metrics 1/n (PL^5067)
* add back compatibility for metrics

* tests

* Add deprecated metric utility functions back to functional (#5062)

* add back *deprecated* metric utility functions to functional

* pep

* pep

* suggestions

* move

Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>

* more

* fix

* import

* docs

* tests

* fix

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
2021-02-22 01:24:20 +01:00
Jirka Borovec 0f173fa5ea drop duplicate metrics (PL^5014)
* drop duplicate metrics

* keep

* fix
2021-02-22 01:24:20 +01:00
Jirka Borovec e3befa7d8b drop deprecated reorder from AUC (PL^5004)
* drop deprecated reorder from AUC

* chlog

* fix

* fix

* simple

* fix

* fix

* fix

Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
2021-02-22 01:24:20 +01:00
Tadej Svetina 4b8d78b737 Classification metrics overhaul: input formatting standardization (1/n) (PL^4837)
* Add stuff

* Change metrics documentation layout

* Change testing utils

* Replace len(*.shape) with *.ndim

* More descriptive error message for input formatting

* Replace movedim with permute

* Style changes in error messages

* More error message style improvements

* Fix typo in docs

* Add more descriptive variable names in utils

* Change internal var names

* Break down error checking for inputs into separate functions

* Remove the (N, ..., C) option in MD-MC

* Simplify select_topk

* Remove detach for inputs

* Fix typos

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Update docs/source/metrics.rst

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Minor error message changes

* Update pytorch_lightning/metrics/utils.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* Reuse case from validation in formatting

* Refactor code in _input_format_classification

* Small improvements

* PEP 8

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update docs/source/metrics.rst

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Update pytorch_lightning/metrics/classification/utils.py

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Apply suggestions from code review

Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>

* Alphabetical reordering of regression metrics

* Change default value of top_k and add error checking

* Extract basic validation into separate function

* Update desciption of parameters in input formatting

* Apply suggestions from code review

Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>

* Check that probabilities in preds sum to 1 (for MC)

* Fix coverage

* Minor changes

* Fix edge case and simplify testing

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Rohit Gupta <rohitgr1998@gmail.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2021-02-22 01:24:20 +01:00
Nicki Skafte fadced267d Auto convert to contiguous format for all_gather (PL^4907)
* convert memory format

* changelog

* formatting

* suggestions

* retrigger tests

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: chaton <thomas@grid.ai>
2021-02-22 01:24:20 +01:00
Nicki Skafte dbd17ccc98 PrecisionRecallCurve, ROC and AveragePrecision class interface (PL^4549)
* initial changes

* remove old

* init files

* add average precision

* add precision_recall_curve

* add roc

* cleaning

* docs

* pep8

* docs

* pep8

* changelog

* examples prune duplicate roc

* format

* imports

* fix

* format

* flake8

* duplicate

* fix

* flake8

* docs

* docs

Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
2021-02-22 01:24:20 +01:00
Teddy Koker 57ac5569a3 Add formulas and references to metrics docs (PL^4823)
* precision

* precision

* recall

* f beta

* confusion matrix

* mse

* mae

* msle

* expalained variance

* psnr

* ssim

* text fp fn tp

* accuracy

* wiki -> sklearn for confusion metrix link

* confusion matrix logging note

Co-authored-by: Roger Shieh <sh.rog@protonmail.ch>
2021-02-22 01:24:20 +01:00
Adrian Wälchli abf61c0d58 Update test for logging a metric object and state reset (PL^4825)
* update test

* docstring

Co-authored-by: Ananya Harsh Jha <ananya@pytorchlightning.ai>
2021-02-22 01:24:20 +01:00
Nicki Skafte ccb9cee771 Unification of FBeta (PL^4656)
* implementation

* init files

* more stable reduction

* add tests

* docs

* remove old implementation

* pep8

* changelog

* Apply suggestions from code review

Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>

Co-authored-by: Nicki Skafte <nugginea@gmail.com>
Co-authored-by: Teddy Koker <teddy.koker@gmail.com>
Co-authored-by: Adrian Wälchli <aedu.waelchli@gmail.com>
2021-02-22 01:24:20 +01:00
Jeff Yang 44bb88f0fd Update SSIM (PL^4566)
* [metrics] Update SSIM

* [metrics] Update SSIM

* [metrics] Update SSIM

* [metrics] Update SSIM

* [metrics] update ssim

* dist_sync_on_step True

* [metrics] update ssim

* Update tests/metrics/regression/test_ssim.py

Co-authored-by: chaton <thomas@grid.ai>

* Update pytorch_lightning/metrics/functional/ssim.py

Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>

* ddp=True

* Update test_ssim.py

Co-authored-by: chaton <thomas@grid.ai>
Co-authored-by: Sean Naren <sean.narenthiran@gmail.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
2021-02-22 01:24:20 +01:00