* initial code, test failing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* increase requirement
* Apply suggestions from code review
* added docs+changelog and other review changes
* consistent shorted type hints
* fix tests
* remove arg
* minor merge fixes
* dep warning added to mean_rel_err
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Apply suggestions from code review
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* applied suggestions from code review
* update
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* added comments regarding test case fixes
* removed unused imports
* ,
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Refactor unnecessary `else` / `elif` when `if` block has a `return` statement
* Apply suggestions from code review
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: deepsource-autofix[bot] <62050782+deepsource-autofix[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* typing for is_differentiable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* update docs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix gather_all_tensors
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* pep
* shortcut for simple cases
* add better test and fix gather
* format
* add changelog
* address comments
* pep8
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: jirka <jirka.borovec@seznam.cz>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* AUROC: reduce allgather costs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* unused import
* A more intelligent implementation on the backend
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* pep8
* Update torchmetrics/metric.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* check that a thing is a list
* changelog
* ssim
* spearman
* pearson
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Added accuracy and all related arguments in Accuracy class
* Added code for subset_accuracy argument
* Set mdmc_average with default value and fixed some pep8 issues
* Added new tests for accuracy along with some minor modifications in to the accuracy class, created a new function in utilities for squeezing tensors
* Corrected wrong order of imports
* changelog
* Resolved mentioned issues
* fix isort
* fix cycle import
* remove unused import
* args
* Apply suggestions from code review
* Added test case for ValueError
* Apply suggestions from code review
* Added some more test cases
* Fixed a small issue
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* WIP: Binned PR-related metrics
* attempt to fix types
* switch to linspace to make old pytorch happy
* make flake happy
* clean up
* Add more testing, move test input generation to the approproate place
* bugfixes and more stable and thorough tests
* flake8
* Reuse python zip-based implementation as it can't be reproduced with torch.where/max
* address comments
* isort
* Add docs and doctests, make APIs same as non-binned versions
* pep8
* isort
* doctests likes longer title underlines :O
* use numpy's nan_to_num
* add atol to bleu tests to make them more stable
* atol=1e-2 for bleu
* add more docs
* pep8
* remove nlp test hack
* address comments
* pep8
* abc
* flake8
* remove typecheck
* chlog
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
* init transition to standard metric interface for IR metrics
* fixed typo in dtypes checks
* removed IGNORE_IDX, refactored tests using
* added pep8 compatibility
* fixed np.ndarray to np.array
* remove lambda functions
* fixed typos with numpy dtype
* fixed typo in doc example
* fixed typo in doc examples about new indexes position
* added paramter to class testing to divide kwargs as preds and targets. Fixed typo in doc format
* added typo in doc example
* added typo with new parameter frament_kwargs in MetricTester
* added typo in .cpu() conversion of non-tensor values
* improved test coverage
* improved test coverage
* added check on Tensor class to avoid calling .cpu() on non-tensor values
* implemented functional ndcg
* improved doc and changed default values for 'empty_target_action' argument
* Implemented Normalized Discounted Cumulative Gain
* refactored tests lists
* refactoring and fixed pep8 compatibility
* formatting
* simple
* agrs
* format
* _sk
* fixed typo in tests
* updated tests experiments from tuple to dict
* fixed typo in doc
* chlog
* Update torchmetrics/functional/retrieval/ndcg.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* moved dcg to global scope as _dcg
* Update torchmetrics/functional/retrieval/ndcg.py
* Update torchmetrics/functional/retrieval/ndcg.py
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* init transition to standard metric interface for IR metrics
* fixed typo in dtypes checks
* removed IGNORE_IDX, refactored tests using
* added pep8 compatibility
* fixed np.ndarray to np.array
* remove lambda functions
* fixed typos with numpy dtype
* fixed typo in doc example
* fixed typo in doc examples about new indexes position
* added paramter to class testing to divide kwargs as preds and targets. Fixed typo in doc format
* added typo in doc example
* added typo with new parameter frament_kwargs in MetricTester
* added typo in .cpu() conversion of non-tensor values
* improved test coverage
* improved test coverage
* added check on Tensor class to avoid calling .cpu() on non-tensor values
* improved doc and changed default values for 'empty_target_action' argument
* implemented fall-out
* refactored tests lists
* refactoring and fixed typos in tests
* formatting
* simple
* agrs
* format
* _sk
* fixed typo in tests
* updated tests experiments from tuple to dict
* fixed typo in doc
* Apply suggestions from code review
* chlog
* Update tests/retrieval/test_fallout.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Update torchmetrics/functional/retrieval/fall_out.py
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* added metric link to wikipedia
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* init transition to standard metric interface for IR metrics
* fixed typo in dtypes checks
* removed IGNORE_IDX, refactored tests using
* added pep8 compatibility
* fixed np.ndarray to np.array
* remove lambda functions
* fixed typos with numpy dtype
* fixed typo in doc example
* fixed typo in doc examples about new indexes position
* added paramter to class testing to divide kwargs as preds and targets. Fixed typo in doc format
* added typo in doc example
* added typo with new parameter frament_kwargs in MetricTester
* added typo in .cpu() conversion of non-tensor values
* improved test coverage
* improved test coverage
* added check on Tensor class to avoid calling .cpu() on non-tensor values
* improved doc and changed default values for 'empty_target_action' argument
* refactored tests lists
* formatting
* simple
* agrs
* format
* _sk
* fixed typo in tests
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
* fixed typo in comments
* implemented Mean Reciprocal Rank. Added checks on input dtypes also to Mean Average Precision.
* fixed flask8 and isort compatibility
* added tests for MRR and refactored tests for MAP
* fixed flask8 compatibility
* added docs entries. added other arguments dtype checks
* moved docs entries to specific section
* added tests with wrong input dtypes
* improved test code coverage
* improved test code coverage
* improved test code coverage
* improved test code coverage
* moved constants to global, refactores and reordered tests
* fixed compatibility with torch 1.4.0 and improved error messages in tests
* implemented improvements requested by @SkafteNicki
* format
* separated retrieval metric test files and removed useless loops
* fix tests
* update CHANGELOG, implemented pytest.raises to check exceptions in tests
* restricted allowed dtypes for IR metrics and updated doc
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* removed lightning seed_everything
* fixed code format
* fixed typo in doc
* changed in
* fixed typos in doc
* fixed typos in doc and test on windows
* fixed typos in doc
* updated seeding fn
* fixed typo in tests regarding randomness
Co-authored-by: Jirka Borovec <jirka.borovec@seznam.cz>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* init information retrieval metrics
* changed retrieval metrics names, expanded arguments and fixed typo
* added 'Retrieval' prefix to metrics and fixed conflict with already-present 'average_precision' file
* improved code formatting
* pep8 code compatibility
* features/implemented new Mean Average Precision metrics for Information Retrieval + doc
* fixed pep8 compatibility
* removed threshold parameter and fixed typo on types in RetrievalMAP and improved doc
* improved doc, put first class-specific args in RetrievalMetric and transformed RetrievalMetric in abstract class
* implemented tests for functional and class metric. fixed typo when input tensors are empty or when all targets are False
* fixed typos in doc and changed torch.true_divide to torch.div
* fixed typos pep8 compatibility
* fixed types in long division in ir_average_precision and example in mean_average_precision
* RetrievalMetric states are not lists and _metric method accepts predictions and targets for easier extension
* updated CHANGELOG file
* added '# noqa: F401' flag to not used imports
* added double space before '# noqa: F401' flag
* Update CHANGELOG.md
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* change get_mini_groups in get_group_indexes
* added checks on target inputs
* minor refactoring for code cleanness
* split tests over exception raising in separate function && refactored test code into multiple functions
* fixed pep8 compatibility
* implemented suggestions of @SkafteNicki
* fixed imports for isort and added types annontations to functions in test_map.py
* isort on test_map and fixed typing
* isort on retrieval and on __init__.py and utils.py in metrics package
* fixed typo in pytorch_lightning/metrics/__init__.py regarding code style
* fixed yapf compatibility
* fixed yapf compatibility
* fixed typo in doc
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Jirka
Pranjal Datta
deepsource-autofix[bot]
Bhadresh Savani
Jirka Borovec
Nicki Skafte
thomas chaton
Johannes Pitz
Justus Schock
Santiago Castro
Maxim Grechkin
Ihar
Arvind Muralie
Alan Du
hlin09
Maxim Grechkin
Luca Di Liello
Davide Fiocco
J. Sebastian Paez
victorjoos
Ethan Harris
Dipam Vasani
Frankie Robertson
prajakta0111
Akihiro Nitta
thomasgaudelet
Teddy Koker