Daisy Deng
|
11551c5937
|
add minilm components
|
2020-04-10 21:51:05 +00:00 |
|
Daisy Deng
|
3dad522c5c
|
new
|
2020-03-20 20:00:07 +00:00 |
|
Daisy Deng
|
b27ae985d6
|
minor changes
|
2020-03-20 19:50:14 +00:00 |
|
Daisy Deng
|
219ab4b752
|
minor changes
|
2020-03-20 19:26:18 +00:00 |
|
Daisy Deng
|
b30a77c143
|
move iterabledistributedsampler and chunkdataloader out from extsum file
|
2020-03-20 19:21:01 +00:00 |
|
Daisy Deng
|
055645f590
|
fix move and parallelized model for DistributedDataParallel models
|
2020-03-20 19:18:29 +00:00 |
|
Daisy Deng
|
946ae06fb6
|
fixed dataset for abstractive sum
|
2020-03-20 19:14:27 +00:00 |
|
Daisy Deng
|
03948df1b0
|
model stays the same if there is no valid new gpu_ids
|
2020-03-19 20:13:32 +00:00 |
|
Daisy Deng
|
784c5ccac0
|
fixed test; slightly modiflied parallelize_model to capture errors and mismatched inputs
|
2020-03-19 18:36:17 +00:00 |
|
Daisy Deng
|
bad0cdf3cf
|
merge froms staging
|
2020-03-19 05:03:33 +00:00 |
|
Daisy Deng
|
1592ced9cc
|
verified test
|
2020-03-19 05:00:31 +00:00 |
|
Daisy Deng
|
fda15d37ba
|
notebook works for preprocessed data
|
2020-03-18 14:57:24 +00:00 |
|
Daisy Deng
|
868266117b
|
add example cells in the notebook to predict on the input text; sort the selected sentences as they appear in the input text and use the sorted version as summary
|
2020-03-18 06:30:03 +00:00 |
|
Daisy Deng
|
5b57bf4928
|
add DistributedSampler import
|
2020-03-17 21:55:31 +00:00 |
|
Daisy Deng
|
24c69d50ae
|
move clear cuda cache into the finetune and predict function
|
2020-03-17 20:56:51 +00:00 |
|
Daisy Deng
|
5ae6a5fae9
|
move max_pos_length into collate function
|
2020-03-16 03:50:39 +00:00 |
|
Daisy Deng
|
80883d520f
|
draft collocate
|
2020-03-13 14:45:08 +00:00 |
|
Daisy Deng
|
98536e099f
|
formatting
|
2020-03-13 04:11:05 +00:00 |
|
Daisy Deng
|
63f0d56539
|
formatting
|
2020-03-13 04:04:50 +00:00 |
|
Daisy Deng
|
eafb73e1c4
|
adjust the preprocessing for max_pos_length
|
2020-03-13 03:25:40 +00:00 |
|
Daisy Deng
|
7e36572b01
|
add copyright and license info for bertsum; polish the notebook; resolve review comments
|
2020-03-13 02:36:03 +00:00 |
|
Daisy Deng
|
8de8c9f501
|
remove bertsum git dependency; enable max_pos_length customization for extractive summarization
|
2020-03-12 04:22:24 +00:00 |
|
Daisy Deng
|
d05f359c55
|
add more in docstring for test argument
|
2020-03-11 20:48:45 +00:00 |
|
Daisy Deng
|
feeab6e23d
|
resolve review comments
|
2020-03-11 20:36:08 +00:00 |
|
Daisy Deng
|
b9c8a987d3
|
add documentation; add the original code source info
|
2020-03-10 16:11:57 +00:00 |
|
Daisy Deng
|
5b56f6e7d0
|
remove logging file; remove files that should not be included in this PR
|
2020-03-10 15:41:45 +00:00 |
|
Daisy Deng
|
2d4b13b681
|
formatting
|
2020-03-10 15:30:10 +00:00 |
|
Daisy Deng
|
b22f7f94e6
|
merge from staging, and made the necessary changes
|
2020-03-10 14:55:48 +00:00 |
|
Daisy Deng
|
d38bafe3bd
|
add docstring
|
2020-03-07 22:03:03 +00:00 |
hlums
|
bd27c25d81
|
Add all docstrings.
|
2020-03-06 22:07:26 +00:00 |
|
Daisy Deng
|
72786463fc
|
rename files
|
2020-03-06 21:08:27 +00:00 |
|
Daisy Deng
|
fd84b4b32f
|
Merge branch 'daden/presumm' of https://github.com/microsoft/nlp-recipes into daden/presumm
|
2020-03-06 15:56:44 +00:00 |
|
hlums
|
3a270fb2f7
|
Add some docstrings.
|
2020-03-06 05:19:00 +00:00 |
|
Daisy Deng
|
b527bdfcdb
|
enable multi-gpu inferencing for DataParallel and enable longer input length
|
2020-03-06 04:26:02 +00:00 |
|
hlums
|
804e9bc2df
|
Fix a few issues found in testing.
|
2020-03-05 00:06:07 +00:00 |
|
hlums
|
1d7796c35d
|
Improved data preprocessing functions.
|
2020-03-04 00:17:30 +00:00 |
|
Daisy Deng
|
0b42481d8b
|
enable multiple schedulers
|
2020-03-03 05:58:13 +00:00 |
|
Daisy Deng
|
9f24c6bde7
|
enable customizable maximum input length
|
2020-03-03 05:57:37 +00:00 |
|
hlums
|
bbc1287783
|
Minor code cleanup
|
2020-02-27 22:12:57 +00:00 |
|
hlums
|
520d0dd937
|
Merge branch 'hlu/unilm_abstractive_summarization' of https://github.com/Microsoft/nlp-recipes into hlu/unilm_abstractive_summarization
|
2020-02-26 19:56:43 +00:00 |
|
hlums
|
4734180eee
|
Add utf-8 encoding when reading files.
|
2020-02-26 19:55:49 +00:00 |
|
hlums
|
f6e4f1a282
|
Replace self.decode_tokenizer with self.tokenizer
|
2020-02-26 19:38:28 +00:00 |
|
Daisy Deng
|
cd8d8f00e9
|
add start token and end token for encoded target
|
2020-02-26 03:33:44 +00:00 |
|
hlums
|
c707614e6c
|
Merge branch 'hlu/unilm_abstractive_summarization' of https://github.com/Microsoft/NLP into hlu/unilm_abstractive_summarization
|
2020-02-25 20:10:10 +00:00 |
|
hlums
|
c723062385
|
Update model type checking.
|
2020-02-25 20:09:13 +00:00 |
|
Daisy Deng
|
c836249713
|
Merge branch 'daden/presumm' of https://github.com/microsoft/nlp-recipes into daden/presumm
|
2020-02-24 21:48:35 +00:00 |
|
Daisy Deng
|
0bfbba1000
|
deal with a list of schedulers
|
2020-02-24 21:46:09 +00:00 |
|
Daisy Deng
|
4b4b4d5396
|
add scheduler
|
2020-02-24 21:45:40 +00:00 |
|
Daisy Deng
|
10185d6f3c
|
Merge branch 'daden/presumm' of https://github.com/microsoft/nlp-recipes into daden/presumm
|
2020-02-24 21:31:54 +00:00 |
|
Daisy Deng
|
badbebb96d
|
only use 1 gpu for validation
|
2020-02-24 21:27:31 +00:00 |