* Start adding own BERTScore implementation
* Prepare the basic backbone for own BERTScore
* Make working BERTScore with all_layers=True + init bert_score in torchmetrics
* Fix cuda device placing
* Use IDF only if asked
* Add data collators
* Remove old parts of code
* Fix IDF rescaling and add new tests + fix type
* Add new tests for IDF
* Add explicit dtypes when used torch.cat to make this work with older PT versions
* Adjust code to work with the DDP plus some changes
* Add a support for the DDP mode and add the corresponding test
* Add a suport for verbose using tqdm loader
* Fix a bug with tokenizer and add hash_code
* Fix transformers import
* Add support for the user's own model
* Add the corresponding example
* Fix error raised by default tokenizer
* Add support for the rescale with baseline
* Clean some code + add some docstirngs
* Run bert-ddp tests only if torch.distributed.is_available()
* Use smaller model, 'albert-base-v2', for testing because of OOM issues
* Set join=False for mp_spawn in the ddp_test
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
* Added TextTester class
* Updating TextTester class with small naming changes
* Update TextTester with input processing, appropriate concatenations.
* Updated tests for BLEU Score based on TextTester.
* Updated WER metric to use the TextTester class.
* Updated tests for ROUGEScore based on TestTester.
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Remove dependency on jiwer for WER
* Add deprecation warning for concatenate_texts and add documnetation
* Use jiwer as reference in tests
* Apply suggestions from review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: SkafteNicki <skaftenicki@gmail.com>
* Make Rouge-N working
* Add RougeL score calculation
* Add some docs + enable using Porter stemmer
* Enable RougeLSum calculation
* Add a few references and clean some parts
* Clean some minor stuff
* Return decimal_places argument to ROUGEScore and prepare depreciation warning
* Use 0 in (x,y) instead of x == 0 or y == 0
* Enable _rouge_score_update method takes List of sentence_results
* Replace dangerous default dict() values
* Replace _RougeScore class with dict
* Update error messages
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Added docstrings for some functions
* Added docstrings for some more functions
* Fix flake8
* Fixed doctest fails
* Minor changes
* More minor changes
* Added some doctests
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
* Add docstrings for some functions in image, retrieval, text metrics
* Minor change
* Apply suggestions from code review
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* fix ignore_index
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix ignore_index in iou
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* changelog
Co-authored-by: CSautier <corentin.sautier@valeo.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Nicki Skafte <skaftenicki@gmail.com>
* Adding new metric ROUGE Metric for text
* Added tests for the ROUGE metric
* Updated docs and imports, added types
* Apply suggestions from code review
* Applied changes suggested in code review
* Updated text dependencies and CHANGELOG
* Fix typing issues
* Updated docs dependencies
* pkg
* pkg
* set jiwer
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Simplified the implementation for batches and added more tests.
* Updated docs requirements and removed unused imports.
* Fix typing, rigorously check rouge_keys, add tests for rouge_keys error, update doc strings
* Remove unused imports
* Apply suggestions from code review
* Fixed typing and added docstrings for update and compute method
* Changes based on review
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: Jirka <jirka.borovec@seznam.cz>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Test nDCG with negative relevance targets
* Fix: Check for non binary values for retrieval targets
- Use the scikit-learn implementation of nDCG
- Removed the test for non binary targets in test_ndcg.py and replaced the default parameters in the error test with a custom one that does not check for binary targets
- set the _input_retrieval_scores_non_binary_target low to -1 to reduce the test failure rate
* Fix removed unused imports in ndcg.py
* more stable tests
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: SkafteNicki <skaftenicki@gmail.com>
* Skipping empty classes in weighted auroc.py
added logic to omit empty classes with weighted AUROC, as they should have 0 weight anyway
* formatting
* Added test to test_auroc.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Removed comment, reformatted prediction matrix
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fmt
* debug -- fixing binary case
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Updated tests
Now checks for ValueError and UserWarning
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* updated changelog
* debug
* final bug
* Update auroc.py
* removed f-string
* Docstring; removed f-string
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Jirka Borovec <Borda@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
quancs
deepsource-autofix[bot]
Nicki Skafte
karthikrangasai
Jirka Borovec
Daniel Stancl
Yasser Souri
Nishant Prabhu
Justus Schock
kingyiusuen
Alexander Senov
Jirka
ananthsub
Gagan Bhatia
Pranjal Gulati
Raj Singh
Tony
Aki Nitta
Edward Williams
Vatsalya Chaubey
CSautier
Paul Grundmann
Artur Jaroszewicz