* Create README.md

* Update README.md
This commit is contained in:
Antoine Louis 2020-09-16 16:16:01 +00:00 ΠΊΠΎΠΌΠΌΠΈΡ‚ ΠΏΡ€ΠΎΠΈΠ·Π²Ρ‘Π» GitHub
Π ΠΎΠ΄ΠΈΡ‚Π΅Π»ΡŒ 108c9aefcc
ΠšΠΎΠΌΠΌΠΈΡ‚ df165065c3
НС Π½Π°ΠΉΠ΄Π΅Π½ ΠΊΠ»ΡŽΡ‡, ΡΠΎΠΎΡ‚Π²Π΅Ρ‚ΡΡ‚Π²ΡƒΡŽΡ‰ΠΈΠΉ Π΄Π°Π½Π½ΠΎΠΉ подписи
Π˜Π΄Π΅Π½Ρ‚ΠΈΡ„ΠΈΠΊΠ°Ρ‚ΠΎΡ€ ΠΊΠ»ΡŽΡ‡Π° GPG: 4AEE18F83AFDEB23
1 ΠΈΠ·ΠΌΠ΅Π½Ρ‘Π½Π½Ρ‹Ρ… Ρ„Π°ΠΉΠ»ΠΎΠ²: 53 Π΄ΠΎΠ±Π°Π²Π»Π΅Π½ΠΈΠΉ ΠΈ 0 ΡƒΠ΄Π°Π»Π΅Π½ΠΈΠΉ

ΠŸΡ€ΠΎΡΠΌΠΎΡ‚Ρ€Π΅Ρ‚ΡŒ Ρ„Π°ΠΉΠ»

@ -0,0 +1,53 @@
---
language: "fr"
---
# BelGPT-2
**BelGPT-2** (*Belgian GPT-2* πŸ‡§πŸ‡ͺ) is a "small" GPT-2 model pre-trained on a very large and heterogeneous French corpus (around 60Gb). Please check [antoiloui/gpt2-french](https://github.com/antoiloui/gpt2-french) for more information about the pre-trained model, the data, the code to use the model and the code to pre-train it.
## Using BelGPT-2 for Text Generation in French
You can use BelGPT-2 with [πŸ€— transformers](https://github.com/huggingface/transformers) library as follows:
```python
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel
# Load pretrained model and tokenizer
model = GPT2LMHeadModel.from_pretrained("antoiloui/belgpt2")
tokenizer = GPT2Tokenizer.from_pretrained("antoiloui/belgpt2")
# Generate a sample of text
model.eval()
output = model.generate(
bos_token_id=random.randint(1,50000),
do_sample=True,
top_k=50,
max_length=100,
top_p=0.95,
num_return_sequences=1
)
# Decode it
decoded_output = []
for sample in output:
decoded_output.append(tokenizer.decode(sample, skip_special_tokens=True))
print(decoded_output)
```
## Data
Below is the list of all French copora used to pre-trained the model:
| Dataset | `$corpus_name` | Raw size | Cleaned size |
| :------| :--- | :---: | :---: |
| CommonCrawl | `common_crawl` | 200.2 GB | 40.4 GB |
| NewsCrawl | `news_crawl` | 10.4 GB | 9.8 GB |
| Wikipedia | `wiki` | 19.4 GB | 4.1 GB |
| Wikisource | `wikisource` | 4.6 GB | 2.3 GB |
| Project Gutenberg | `gutenberg` | 1.3 GB | 1.1 GB |
| EuroParl | `europarl` | 289.9 MB | 278.7 MB |
| NewsCommentary | `news_commentary` | 61.4 MB | 58.1 MB |
| **Total** | | **236.3 GB** | **57.9 GB** |