This commit is contained in:
Xu Tan (谭旭) 2023-05-21 18:22:51 +08:00 коммит произвёл GitHub
Родитель 1f139dabc3
Коммит 5f58fe806d
Не найден ключ, соответствующий данной подписи
Идентификатор ключа GPG: 4AEE18F83AFDEB23
1 изменённых файлов: 15 добавлений и 10 удалений

Просмотреть файл

@ -30,43 +30,47 @@ The current work in [Muzic](https://www.microsoft.com/en-us/research/project/ai-
+ Automatic Lyrics Transcription: [PDAugment](https://arxiv.org/abs/2109.07940)
+ Contrastive Language-Music Pre-training: [CLaMP](https://arxiv.org/abs/2304.11029)
* Music Generation
+ Song Writing: [SongMASS](https://arxiv.org/abs/2012.05168)
+ Lyric Generation: [DeepRapper](https://arxiv.org/abs/2107.01875)
+ Lyric-to-Melody Generation: [TeleMelody](https://arxiv.org/abs/2109.09617), [ReLyMe](https://arxiv.org/abs/2207.05688), [Re-creation of Creations (ROC)](https://arxiv.org/abs/2208.05697)
+ Melody Generation: [MeloForm](https://arxiv.org/abs/2208.14345), [Museformer](https://arxiv.org/abs/2210.10349)
+ Accompaniment Generation: [PopMAG](https://arxiv.org/abs/2008.07703)
+ Song Writing
+ Lyric-to-Melody and Melody-to-Lyric: [SongMASS](https://arxiv.org/abs/2012.05168)
+ Lyric Generation: [DeepRapper](https://arxiv.org/abs/2107.01875)
+ Lyric-to-Melody Generation: [TeleMelody](https://arxiv.org/abs/2109.09617), [ReLyMe](https://arxiv.org/abs/2207.05688), [Re-creation of Creations (ROC)](https://arxiv.org/abs/2208.05697)
+ Music Form/Structure Generation
+ Music Form Generation: [MeloForm](https://arxiv.org/abs/2208.14345)
+ Long/Short Structure Modeling: [Museformer](https://arxiv.org/abs/2210.10349)
+ Multi-Track Generation
+ Accompaniment Generation: [PopMAG](https://arxiv.org/abs/2008.07703)
+ Any Track Music Generation: [GETMusic](https://arxiv.org/abs/2305.10841)
+ Singing Voice Synthesis: [HiFiSinger](https://arxiv.org/abs/2009.01776)
You can find some music samples generated by our systems from this page: https://ai-muzic.github.io/.
<br>
For more speech related research, you can find from this page: https://github.com/microsoft/NeuralSpeech and https://speechresearch.github.io/.
For more speech related research, you can find from this page: https://speechresearch.github.io/ and https://github.com/microsoft/NeuralSpeech.
### We are hiring!
We are hiring both research FTEs and research interns on **AI music, speech, audio, language, and machine learning**. Please contact Xu Tan (xuta@microsoft.com) if you have interests.
### What is New!
+ We release [GETMusic](https://arxiv.org/abs/2305.10841), a versatile music copliot with a universal representation and diffusion framework to generate any music tracks.
+ We release the first model for cross-modal symbolic MIR: [CLaMP](https://arxiv.org/abs/2304.11029).
+ We release two new research work on music structure modeling: [MeloForm](https://arxiv.org/abs/2208.14345) and [Museformer](https://arxiv.org/abs/2210.10349).
+ We give a tutorial on [AI Music Composition](https://www.microsoft.com/en-us/research/uploads/prod/2021/10/Tutorial-on-AI-Music-Composition-@ACM-MM-2021.pdf) at [ACM Multimedia 2021](https://2021.acmmm.org/).
## Requirements
The operating system is Linux. We test on Ubuntu 16.04.6 LTS, CUDA 10, with Python 3.6.12. The requirements for running Muzic are listed in `requirements.txt`. To install the requirements, run:
```bash
pip install -r requirements.txt
```
We release the code of several research work: [MusicBERT](musicbert), [PDAugment](pdaugment), [DeepRapper](deeprapper), [SongMASS](songmass), [TeleMelody](telemelody), [ReLyMe](relyme), [Re-creation of Creations (ROC)](roc), [MeloForm](meloform), and [Museformer](museformer). You can find the README in the corresponding folder for detailed instructions on how to use.
We release the code of several research work: [MusicBERT](musicbert), [PDAugment](pdaugment), [CLaMP](clamp), [DeepRapper](deeprapper), [SongMASS](songmass), [TeleMelody](telemelody), [ReLyMe](relyme), [Re-creation of Creations (ROC)](roc), [MeloForm](meloform), [Museformer](museformer), and [GETMusic](getmusic). You can find the README in the corresponding folder for detailed instructions on how to use.
## Reference
If you find the Muzic project useful in your work, you can cite the following papers if there's a need:
If you find the Muzic project useful in your work, you can cite the papers as follows:
* [1] ***MusicBERT**: Symbolic Music Understanding with Large-Scale Pre-Training*, Mingliang Zeng, Xu Tan, Rui Wang, Zeqian Ju, Tao Qin, Tie-Yan Liu, **ACL 2021**.
* [2] ***PDAugment**: Data Augmentation by Pitch and Duration Adjustments for Automatic Lyrics Transcription*, Chen Zhang, Jiaxing Yu, Luchin Chang, Xu Tan, Jiawei Chen, Tao Qin, Kejun Zhang, **ISMIR 2022**.
@ -80,6 +84,7 @@ If you find the Muzic project useful in your work, you can cite the following pa
* [10] ***PopMAG**: Pop Music Accompaniment Generation*, Yi Ren, Jinzheng He, Xu Tan, Tao Qin, Zhou Zhao, Tie-Yan Liu, **ACM Multimedia 2020**.
* [11] ***HiFiSinger**: Towards High-Fidelity Neural Singing Voice Synthesis*, Jiawei Chen, Xu Tan, Jian Luan, Tao Qin, Tie-Yan Liu, arXiv 2020.
* [12] ***CLaMP**: Contrastive Language-Music Pre-training for Cross-Modal Symbolic Music Information Retrieval*, Shangda Wu, Dingyao Yu, Xu Tan, Maosong Sun, arXiv 2023.
* [13] ***GETMusic**: Generating Any Music Tracks with a Unified Representation and Diffusion Framework*, Ang Lv, Xu Tan, Peiling Lu, Wei Ye, Shikun Zhang, Jiang Bian, Rui Yan, arXiv 2023.
## Contributing