microsoft
/
huggingface-transformers
зеркало из https://github.com/microsoft/huggingface-transformers.git
1
0
Форкнуть 0
Код Задачи Пакеты Проекты Релизы Вики Активность
5 823 коммитов 244 Ветки 58 Теги 127 MiB
Граф коммитов

5823 Коммитов

Автор SHA1 Сообщение Дата
Sylvain Gugger 3213d3bfae
Question template (#8440) 
* Remove SO from question template

* Styling
 2020-11-10 10:07:56 -05:00
Stas Bekman 5d4972e608
[examples] better PL version check (#8429) 2020-11-10 09:33:23 -05:00
Shichao Sun ae1cb4ec22
[s2s/distill] hparams.tokenizer_name = hparams.teacher (#8382) 2020-11-10 09:32:01 -05:00
Lysandre aec51e5696 v3.5.0 documentation 2020-11-10 08:58:47 -05:00
Lysandre 818878dc88 Release: v3.5.0 2020-11-10 08:50:43 -05:00
Lysandre Debut 9cebee38ad
Model sharing rst (#8439) 
* Update RST

* Finer details

* Re-organize

* Style
 2020-11-10 08:35:11 -05:00
Julien Chaumond ad2303a401 Fix style 2020-11-10 14:28:30 +01:00
Julien Chaumond 55e8d0cea2 Update links from s3 to huggingface.co 2020-11-10 14:03:29 +01:00
Lysandre Debut 850afb422d
Patch token classification pipeline (#8364) 
* Patch token classification pipeline

* Some added tests for TokenClassificationArgumentHandler (#8366)

Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>
 2020-11-10 07:29:34 -05:00
Julien Chaumond 70f622fab4
Model versioning (#8324) 
* fix typo

* rm use_cdn & references, and implement new hf_bucket_url

* I'm pretty sure we don't need to `read` this file

* same here

* [BIG] file_utils.networking: do not gobble up errors anymore

* Fix CI 😇

* Apply suggestions from code review

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

* Tiny doc tweak

* Add doc + pass kwarg everywhere

* Add more tests and explain

cc @sshleifer let me know if better

Co-Authored-By: Sam Shleifer <sshleifer@gmail.com>

* Also implement revision in pipelines

In the case where we're passing a task name or a string model identifier

* Fix CI 😇

* Fix CI

* [hf_api] new methods + command line implem

* make style

* Final endpoints post-migration

* Fix post-migration

* Py3.6 compat

cc @stefan-it

Thank you @stas00

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-11-10 07:11:02 -05:00
Teven 4185b115d4
Changing XLNet default from not using memories to 512 context size following paper (#8417) 
* Move XLNet memory length FutureWarning

* isort

* style

* Changed default XLNet memory length
 2020-11-09 20:49:51 -05:00
Stas Bekman 190df58560
[github CI] add a multi-gpu job for all example tests (#8341) 
* add a multi-gpu job for all example tests

* run only ported tests

* rename

* explain why env is re-activated on each step

* mark all unported/checked tests with @require_torch_non_multigpu_but_fix_me

* style

* Apply suggestions from code review

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>

Co-authored-by: Sam Shleifer <sshleifer@gmail.com>
 2020-11-09 15:47:38 -05:00
Sylvain Gugger a39218b75b
Check all models are in an auto class (#8425) 2020-11-09 15:44:54 -05:00
Stas Bekman ef032ddd1e
[docs] [testing] gpu decorators table (#8422) 
* gpu decorators table

* whitespace

* Update docs/source/testing.rst

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>

* whitespace

Co-authored-by: Lysandre Debut <lysandre@huggingface.co>
 2020-11-09 14:27:42 -05:00
Sam Shleifer a8339b9ecc
Fix bart shape comment (#8423) 2020-11-09 13:25:33 -05:00
Sam Shleifer 46509d1c19
[docs] remove sshleifer from issue-template :( (#8418) 2020-11-09 12:51:38 -05:00
Patrick von Platen 9c83b96e62
[Tests] Add Common Test for Training + Fix a couple of bugs (#8415) 
* add training tests

* correct longformer

* fix docs

* fix some tests

* fix some more train tests

* remove ipdb

* fix multiple edge case model training

* fix funnel and prophetnet

* clean gpt models

* undo renaming of albert
 2020-11-09 18:24:41 +01:00
Sylvain Gugger 52040517b8
Deprecate old data/metrics functions (#8420) 2020-11-09 12:10:09 -05:00
Stas Bekman d4d1fbfc5a
[fsmt convert script] fairseq broke chkpt data - fixing that (#8377) 
* fairseq broke chkpt data - fixing that

* style

* support older bpecodes filenames - specifically "code" in iwslt14
 2020-11-09 11:57:42 -05:00
Sylvain Gugger 5c766ecb50 Fix typo 2020-11-09 11:50:51 -05:00
Sylvain Gugger 908a28894c
Add new token classification example (#8340) 
* Add new token classification example

* Remove txt file

* Add test

* With actual testing done

* Less warmup is better

* Update examples/token-classification/run_ner_new.py

Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>

* Address review comments

* Fix test

* Make Lysandre happy

* Last touches and rename

* Rename in tests

* Address review comments

* More run_ner -> run_ner_old

Co-authored-by: Thomas Wolf <thomwolf@users.noreply.github.com>
 2020-11-09 11:39:55 -05:00
Sylvain Gugger c7cb1aa26c
Bump tokenizers (#8419) 2020-11-09 11:32:10 -05:00
Stas Bekman 78d706f3ae
[fsmt tokenizer] support lowercase tokenizer (#8389) 
* support lowercase tokenizer

* fix arg pos
 2020-11-09 10:41:39 -05:00
Shashank Gupta 1e2acd0dcf
Bug fix for permutation language modelling (#8409) 2020-11-09 10:23:26 -05:00
Philip May bf8625e70b
add evaluate doc - trainer.evaluate returns 'epoch' from training (#8273) 
* add evaluate doc

* fix style with utils/style.doc

* Update src/transformers/trainer.py

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>

Co-authored-by: Sylvain Gugger <35901082+sgugger@users.noreply.github.com>
 2020-11-09 09:00:59 -05:00
Sam Shleifer ebde57acac
examples/docs: caveat that PL examples don't work on TPU (#8309) 2020-11-09 08:55:22 -05:00
Julien Plu 76e7a44dee
Fix some tooling for windows (#8359) 
* Fix some tooling for windows

* Fix conflict

* Trigger CI
 2020-11-09 13:50:38 +01:00
dartrevan 507dfb40c3
Update README.md (#8406) 2020-11-09 16:44:43 +08:00
smanjil 7247d0b4ea
updating tag for exbert viz (#8408) 2020-11-09 16:43:55 +08:00
Stas Bekman 4ab5617b0b
comet_ml temporary fix(#8410) 2020-11-09 16:36:06 +08:00
Sam Shleifer e6d9cdaafe
[s2s/distill] remove run_distiller.sh, fix xsum script (#8412) 2020-11-08 16:57:43 -05:00
Stas Bekman 66582492d3
[s2s test_finetune_trainer] failing multigpu test (#8400) 2020-11-08 16:45:40 -05:00
Stas Bekman f62755a600
[s2s examples test] fix data path (#8398) 2020-11-08 16:44:18 -05:00
Jonathan Chang 4a53e8e9e4
Fix DataCollatorForWholeWordMask again (#8397) 2020-11-08 09:53:01 -05:00
Manav Rathod 610730998f
fixed default labels for QA model (#8399) 2020-11-08 09:08:14 -05:00
Chengxi Guo 0b02489b2c
Add gpt2-medium-chinese model card (#8402) 
* Create README.md

* Update model_cards/mymusise/gpt2-medium-chinese/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-11-08 05:00:19 -05:00
Stas Bekman 187554366f
fix md table (#8395) 2020-11-08 04:25:14 -05:00
Jonathan Chang 77a257fc21
Fix DataCollatorForWholeWordMask (#8379) 
* Fix DataCollatorForWholeWordMask

* Replace all tensorize_batch in data_collator.py
 2020-11-07 12:51:56 -05:00
Stas Bekman 517eaf460b
[make] rewrite modified_py_files in python to be cross-platform (#8371) 
* rewrite modified_py_files in python to be cross-platform

* try a different way to test for variable not being ""

* improve comment
 2020-11-07 18:45:16 +01:00
Patrick von Platen 07708793f2
fix encoder outputs (#8368) 2020-11-06 21:03:25 +01:00
Yossi Synett bc0d26d1de
[All Seq2Seq model + CLM models that can be used with EncoderDecoder] Add cross-attention weights to outputs (#8071) 
* Output cross-attention with decoder attention output

* Update src/transformers/modeling_bert.py

* add cross-attention for t5 and bart as well

* fix tests

* correct typo in docs

* add sylvains and sams comments

* correct typo

Co-authored-by: Patrick von Platen <patrick.v.platen@gmail.com>
 2020-11-06 19:34:48 +01:00
hassoudi 30f2507a07
Update README.md (#8360) 
Fix websitr address
 2020-11-06 11:45:46 -05:00
Jonathan Chang 5807ba3fa9
Fix typo (#8351) 2020-11-06 11:19:41 -05:00
hassoudi 82146496b6
Update README.md (#8338) 
fixes
 2020-11-06 06:20:58 -05:00
ktrapeznikov 9e5c4d39ab
Create README.md (#8312) 
* Create README.md

* Update model_cards/ktrapeznikov/gpt2-medium-topic-news/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-11-06 06:19:59 -05:00
hasantanvir79 06ebc37967
Create README.md (#8255) 
* Create README.md

Initial commit

* Updated Read me

Updated

* Apply suggestions from code review

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-11-06 03:34:24 -05:00
Karthik Uppuluri 41cd031cf2
Create README.md (#8169) 2020-11-06 03:26:07 -05:00
Karthik Uppuluri f932ddeff5
Create README.md (#8170) 2020-11-06 03:25:52 -05:00
Karthik Uppuluri 08b92f78fa
Create README.md (#8168) 
* Create README.md

* Update README.md
 2020-11-06 03:25:33 -05:00
Karthik Uppuluri 77d62e78b0
Create README.md (#8167) 
* Create README.md

Telugu BERTU Readme file

* Update model_cards/kuppuluri/telugu_bertu/README.md

Co-authored-by: Julien Chaumond <chaumond@gmail.com>
 2020-11-06 03:24:31 -05:00