RoBERTa模型汇总#

下表汇总介绍了目前PaddleNLP支持的RoBERTa模型对应预训练权重。 关于模型的具体细节可以参考对应链接。

Pretrained Weight

Language

Details of the model

hfl/roberta-wwm-ext

Chinese

12-layer, 768-hidden, 12-heads, 102M parameters. Trained on English Text using Whole-Word-Masking with extended data.

hfl/roberta-wwm-ext-large

Chinese

24-layer, 1024-hidden, 16-heads, 325M parameters. Trained on English Text using Whole-Word-Masking with extended data.

hfl/rbt3

Chinese

3-layer, 768-hidden, 12-heads, 38M parameters.

hfl/rbtl3

Chinese

3-layer, 1024-hidden, 16-heads, 61M parameters.

hfl/rbt4

Chinese

4-layer, 768-hidden, 12-heads, 47M parameters.

hfl/rbt6

Chinese

6-layer, 768-hidden, 12-heads, 60M parameters.

deepset/roberta-base-squad2

English

12-layer, 768-hidden, 12-heads, 124M parameters. Trained on English text.

Please refer to: deepset/roberta-base-squad2

uer/roberta-base-chinese-extractive-qa

Chinese

12-layer, 768-hidden, 12-heads, 101M parameters. Trained on Chinese text.

Please refer to: uer/roberta-base-chinese-extractive-qa

uer/roberta-base-finetuned-chinanews-chinese

Chinese

12-layer, 768-hidden, 12-heads, 102M parameters. Trained on Chinese text.

Please refer to: uer/roberta-base-finetuned-chinanews-chinese

uer/roberta-base-finetuned-cluener2020-chinese

Chinese

12-layer, 768-hidden, 12-heads, 101M parameters. Trained on Chinese text.

Please refer to: uer/roberta-base-finetuned-cluener2020-chinese

roberta-base

English

Please refer to: roberta-base

cardiffnlp/twitter-roberta-base-sentiment

English

Please refer to: cardiffnlp/twitter-roberta-base-sentiment

roberta-large

English

Please refer to: roberta-large

distilroberta-base

English

Please refer to: distilroberta-base

cross-encoder/nli-distilroberta-base

English

Please refer to: cross-encoder/nli-distilroberta-base

siebert/sentiment-roberta-large-english

English

Please refer to: siebert/sentiment-roberta-large-english

j-hartmann/emotion-english-distilroberta-base

English

Please refer to: j-hartmann/emotion-english-distilroberta-base

roberta-base-openai-detector

English

Please refer to: roberta-base-openai-detector

huggingface/CodeBERTa-small-v1

English

Please refer to: huggingface/CodeBERTa-small-v1

mrm8488/distilroberta-finetuned-financial-news-sentiment-analysis

English

Please refer to: mrm8488/distilroberta-finetuned-financial-news-sentiment-analysis

cardiffnlp/twitter-roberta-base-emotion

English

Please refer to: cardiffnlp/twitter-roberta-base-emotion

seyonec/PubChem10M_SMILES_BPE_396_250

English

Please refer to: seyonec/PubChem10M_SMILES_BPE_396_250

textattack/roberta-base-SST-2

English

Please refer to: textattack/roberta-base-SST-2

sshleifer/tiny-distilroberta-base

English

Please refer to: sshleifer/tiny-distilroberta-base

thatdramebaazguy/roberta-base-squad

English

Please refer to: thatdramebaazguy/roberta-base-squad

ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli

English

Please refer to: ynie/roberta-large-snli_mnli_fever_anli_R1_R2_R3-nli

ufal/robeczech-base

Czech

Please refer to: ufal/robeczech-base

seyonec/PubChem10M_SMILES_BPE_450k

English

Please refer to: seyonec/PubChem10M_SMILES_BPE_450k

cardiffnlp/twitter-roberta-base

English

Please refer to: cardiffnlp/twitter-roberta-base

seyonec/PubChem10M_SMILES_BPE_50k

English

Please refer to: seyonec/PubChem10M_SMILES_BPE_50k

microsoft/codebert-base-mlm

English

Please refer to: microsoft/codebert-base-mlm

textattack/roberta-base-MNLI

English

Please refer to: textattack/roberta-base-MNLI

cardiffnlp/twitter-roberta-base-offensive

English

Please refer to: cardiffnlp/twitter-roberta-base-offensive

cross-encoder/stsb-roberta-large

English

Please refer to: cross-encoder/stsb-roberta-large

seyonec/ChemBERTa_zinc250k_v2_40k

English

Please refer to: seyonec/ChemBERTa_zinc250k_v2_40k

uklfr/gottbert-base

German

Please refer to: uklfr/gottbert-base

seyonec/ChemBERTa-zinc-base-v1

English

Please refer to: seyonec/ChemBERTa-zinc-base-v1

roberta-large-openai-detector

English

Please refer to: roberta-large-openai-detector

cross-encoder/quora-roberta-base

English

Please refer to: cross-encoder/quora-roberta-base

cross-encoder/stsb-roberta-base

English

Please refer to: cross-encoder/stsb-roberta-base

microsoft/graphcodebert-base

English

Please refer to: microsoft/graphcodebert-base

cardiffnlp/twitter-roberta-base-hate

English

Please refer to: cardiffnlp/twitter-roberta-base-hate

chkla/roberta-argument

English

Please refer to: chkla/roberta-argument

Salesforce/grappa_large_jnt

English

Please refer to: Salesforce/grappa_large_jnt

vinai/bertweet-large

English

Please refer to: vinai/bertweet-large

allenai/biomed_roberta_base

English

Please refer to: allenai/biomed_roberta_base

facebook/muppet-roberta-base

English

Please refer to: facebook/muppet-roberta-base

Rakib/roberta-base-on-cuad

English

Please refer to: Rakib/roberta-base-on-cuad

cross-encoder/stsb-distilroberta-base

English

Please refer to: cross-encoder/stsb-distilroberta-base

nyu-mll/roberta-base-1B-1

English

Please refer to: nyu-mll/roberta-base-1B-1

nyu-mll/roberta-med-small-1M-1

English

Please refer to: nyu-mll/roberta-med-small-1M-1

SkolkovoInstitute/roberta_toxicity_classifier

English

Please refer to: SkolkovoInstitute/roberta_toxicity_classifier

facebook/muppet-roberta-large

English

Please refer to: facebook/muppet-roberta-large

lassl/roberta-ko-small

Korean

Please refer to: lassl/roberta-ko-small

huggingface/CodeBERTa-language-id

English

Please refer to: huggingface/CodeBERTa-language-id

textattack/roberta-base-imdb

English

Please refer to: textattack/roberta-base-imdb

macedonizer/mk-roberta-base

Macedonian

Please refer to: macedonizer/mk-roberta-base

cross-encoder/nli-MiniLM2-L6-H768

English

Please refer to: cross-encoder/nli-MiniLM2-L6-H768

textattack/roberta-base-QNLI

English

Please refer to: textattack/roberta-base-QNLI

deepset/roberta-base-squad2-covid

English

Please refer to: deepset/roberta-base-squad2-covid

textattack/roberta-base-MRPC

English

Please refer to: textattack/roberta-base-MRPC

bhadresh-savani/roberta-base-emotion

English

Please refer to: bhadresh-savani/roberta-base-emotion

aychang/roberta-base-imdb

English

Please refer to: aychang/roberta-base-imdb

cross-encoder/quora-distilroberta-base

English

Please refer to: cross-encoder/quora-distilroberta-base

csarron/roberta-base-squad-v1

English

Please refer to: csarron/roberta-base-squad-v1

seyonec/ChemBERTA_PubChem1M_shard00_155k

English

Please refer to: seyonec/ChemBERTA_PubChem1M_shard00_155k

mental/mental-roberta-base

English

Please refer to: mental/mental-roberta-base

textattack/roberta-base-CoLA

English

Please refer to: textattack/roberta-base-CoLA

navteca/quora-roberta-base

English

Please refer to: navteca/quora-roberta-base

cardiffnlp/twitter-roberta-base-emoji

English

Please refer to: cardiffnlp/twitter-roberta-base-emoji

benjamin/roberta-base-wechsel-german

Multilingual

Please refer to: benjamin/roberta-base-wechsel-german

textattack/roberta-base-ag-news

English

Please refer to: textattack/roberta-base-ag-news

johngiorgi/declutr-base

English

Please refer to: johngiorgi/declutr-base

salesken/query_wellformedness_score

English

Please refer to: salesken/query_wellformedness_score

blinoff/roberta-base-russian-v0

Russian

Please refer to: blinoff/roberta-base-russian-v0

allenai/reviews_roberta_base

English

Please refer to: allenai/reviews_roberta_base

ruiqi-zhong/roberta-base-meta-tuning-test

English

Please refer to: ruiqi-zhong/roberta-base-meta-tuning-test

mrm8488/distilroberta-finetuned-tweets-hate-speech

English

Please refer to: mrm8488/distilroberta-finetuned-tweets-hate-speech

cointegrated/roberta-large-cola-krishna2020

English

Please refer to: cointegrated/roberta-large-cola-krishna2020

deepset/roberta-base-squad2-distilled

English

Please refer to: deepset/roberta-base-squad2-distilled

tli8hf/unqover-roberta-base-squad

English

Please refer to: tli8hf/unqover-roberta-base-squad

cross-encoder/nli-roberta-base

English

Please refer to: cross-encoder/nli-roberta-base

nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large

English

Please refer to: nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large

seyonec/BPE_SELFIES_PubChem_shard00_160k

English

Please refer to: seyonec/BPE_SELFIES_PubChem_shard00_160k

CLTL/MedRoBERTa.nl

Dutch

Please refer to: CLTL/MedRoBERTa.nl

HooshvareLab/roberta-fa-zwnj-base

Persian

Please refer to: HooshvareLab/roberta-fa-zwnj-base

nyu-mll/roberta-base-100M-1

English

Please refer to: nyu-mll/roberta-base-100M-1

deepset/tinyroberta-squad2

English

Please refer to: deepset/tinyroberta-squad2

youscan/ukr-roberta-base

Ukrainian

Please refer to: youscan/ukr-roberta-base

navteca/roberta-base-squad2

English

Please refer to: navteca/roberta-base-squad2

bertin-project/bertin-roberta-base-spanish

Spanish

Please refer to: bertin-project/bertin-roberta-base-spanish

shiyue/roberta-large-tac08

English

Please refer to: shiyue/roberta-large-tac08

softcatala/julibert

Catalan

Please refer to: softcatala/julibert

elozano/tweet_sentiment_eval

English

Please refer to: elozano/tweet_sentiment_eval

cahya/roberta-base-indonesian-1.5G

Indonesian

Please refer to: cahya/roberta-base-indonesian-1.5G

elozano/tweet_emotion_eval

English

Please refer to: elozano/tweet_emotion_eval

navteca/roberta-large-squad2

English

Please refer to: navteca/roberta-large-squad2

elozano/tweet_offensive_eval

English

Please refer to: elozano/tweet_offensive_eval

ynie/roberta-large_conv_contradiction_detector_v0

English

Please refer to: ynie/roberta-large_conv_contradiction_detector_v0