GPT模型汇总#

下表汇总介绍了目前PaddleNLP支持的GPT模型对应预训练权重。 关于模型的具体细节可以参考对应链接。

Pretrained Weight

Language

Details of the model

gpt-cpm-large-cn

Chinese

32-layer, 2560-hidden, 32-heads, 2.6B parameters. Trained on Chinese text.

gpt-cpm-small-cn-distill

Chinese

12-layer, 768-hidden, 12-heads, 109M parameters. The model distilled from the GPT model gpt-cpm-large-cn

gpt2-en

English

12-layer, 768-hidden, 12-heads, 117M parameters. Trained on English text.

gpt2-medium-en

English

24-layer, 1024-hidden, 16-heads, 345M parameters. Trained on English text.

gpt2-large-en

English

36-layer, 1280-hidden, 20-heads, 774M parameters. Trained on English text.

gpt2-xl-en

English

48-layer, 1600-hidden, 25-heads, 1558M parameters. Trained on English text.

microsoft/DialoGPT-medium

English

24-layer, 1024-hidden, 16-heads, 354M parameters. Trained on English text.

Please refer to: microsoft/DialoGPT-medium

uer/gpt2-chinese-poem

Chinese

12-layer, 768-hidden, 12-heads, 103M parameters. Trained on Chinese poetry corpus.

Please refer to: uer/gpt2-chinese-poem

distilgpt2

English

Please refer to: distilgpt2

w11wo/javanese-gpt2-small-imdb

Javanese

Please refer to: w11wo/javanese-gpt2-small-imdb

remotejob/tweetsDISTILGPT2fi_v4

English

Please refer to: remotejob/tweetsDISTILGPT2fi_v4

TrLOX/gpt2-tdk

English

Please refer to: TrLOX/gpt2-tdk

huggingtweets/slime_machine

English

Please refer to: huggingtweets/slime_machine

microsoft/DialoGPT-small

English

Please refer to: microsoft/DialoGPT-small

sberbank-ai/rugpt3large_based_on_gpt2

Russian

Please refer to: sberbank-ai/rugpt3large_based_on_gpt2

sshleifer/tiny-gpt2

English

Please refer to: sshleifer/tiny-gpt2

microsoft/DialoGPT-large

English

Please refer to: microsoft/DialoGPT-large

sberbank-ai/rugpt3small_based_on_gpt2

Russian

Please refer to: sberbank-ai/rugpt3small_based_on_gpt2

uw-hai/polyjuice

English

Please refer to: uw-hai/polyjuice

NYTK/text-generation-poem-petofi-gpt2-small-hungarian

Hungarian

Please refer to: NYTK/text-generation-poem-petofi-gpt2-small-hungarian

microsoft/DialogRPT-human-vs-rand

English

Please refer to: microsoft/DialogRPT-human-vs-rand

hf-internal-testing/tiny-random-gpt2

English

Please refer to: hf-internal-testing/tiny-random-gpt2

Grossmend/rudialogpt3_medium_based_on_gpt2

Russian

Please refer to: Grossmend/rudialogpt3_medium_based_on_gpt2

pranavpsv/genre-story-generator-v2

English

Please refer to: pranavpsv/genre-story-generator-v2

microsoft/DialogRPT-updown

English

Please refer to: microsoft/DialogRPT-updown

microsoft/DialogRPT-human-vs-machine

English

Please refer to: microsoft/DialogRPT-human-vs-machine

pierreguillou/gpt2-small-portuguese

Portuguese

Please refer to: pierreguillou/gpt2-small-portuguese

mrm8488/GPT-2-finetuned-covid-bio-medrxiv

English

Please refer to: mrm8488/GPT-2-finetuned-covid-bio-medrxiv

anonymous-german-nlp/german-gpt2

German

Please refer to: anonymous-german-nlp/german-gpt2

microsoft/CodeGPT-small-py

English

Please refer to: microsoft/CodeGPT-small-py

antoiloui/belgpt2

French

Please refer to: antoiloui/belgpt2

benjamin/gerpt2

German

Please refer to: benjamin/gerpt2

asi/gpt-fr-cased-small

French

Please refer to: asi/gpt-fr-cased-small

microsoft/CodeGPT-small-java-adaptedGPT2

English

Please refer to: microsoft/CodeGPT-small-java-adaptedGPT2

GroNLP/gpt2-small-dutch

Dutch

Please refer to: GroNLP/gpt2-small-dutch

lvwerra/gpt2-imdb

English

Please refer to: lvwerra/gpt2-imdb

DeepESP/gpt2-spanish

Spanish

Please refer to: DeepESP/gpt2-spanish

microsoft/CodeGPT-small-py-adaptedGPT2

English

Please refer to: microsoft/CodeGPT-small-py-adaptedGPT2

microsoft/DialogRPT-width

English

Please refer to: microsoft/DialogRPT-width

dbddv01/gpt2-french-small

French

Please refer to: dbddv01/gpt2-french-small

GroNLP/gpt2-small-italian

Italian

Please refer to: GroNLP/gpt2-small-italian

flax-community/gpt2-medium-persian

Persian

Please refer to: flax-community/gpt2-medium-persian

microsoft/DialogRPT-depth

English

Please refer to: microsoft/DialogRPT-depth

Nokia/nlgp-natural

English

Please refer to: Nokia/nlgp-natural

macedonizer/hr-gpt2

English

Please refer to: macedonizer/hr-gpt2

mrm8488/GPT-2-finetuned-common_gen

English

Please refer to: mrm8488/GPT-2-finetuned-common_gen

pranavpsv/gpt2-genre-story-generator

English

Please refer to: pranavpsv/gpt2-genre-story-generator

rbhushan/distilgpt2-finetuned-wikitext2

English

Please refer to: rbhushan/distilgpt2-finetuned-wikitext2

readerbench/RoGPT2-large

Romanian

Please refer to: readerbench/RoGPT2-large

flax-community/gpt2-small-indonesian

Indonesian

Please refer to: flax-community/gpt2-small-indonesian

HooshvareLab/gpt2-fa

Persian

Please refer to: HooshvareLab/gpt2-fa

cahya/gpt2-small-indonesian-522M

Indonesian

Please refer to: cahya/gpt2-small-indonesian-522M

DingleyMaillotUrgell/homer-bot

English

Please refer to: DingleyMaillotUrgell/homer-bot

datificate/gpt2-small-spanish

Spanish

Please refer to: datificate/gpt2-small-spanish

ericzhou/tsundere_v1

English

Please refer to: ericzhou/tsundere_v1

huggingtweets/wwm_shakespeare

English

Please refer to: huggingtweets/wwm_shakespeare

SIC98/GPT2-python-code-generator

English

Please refer to: SIC98/GPT2-python-code-generator

GroNLP/gpt2-small-italian-embeddings

Italian

Please refer to: GroNLP/gpt2-small-italian-embeddings

huggingtweets/hel_ql-shahdashrf_-sinnerslayerr-witheredstrings

English

Please refer to: huggingtweets/hel_ql-shahdashrf_-sinnerslayerr-witheredstrings

salesken/grammar_correction

English

Please refer to: salesken/grammar_correction

flax-community/gpt2-medium-indonesian

Indonesian

Please refer to: flax-community/gpt2-medium-indonesian

gorkemgoknar/gpt2-small-turkish

Turkish

Please refer to: gorkemgoknar/gpt2-small-turkish

deepparag/DumBot

English

Please refer to: deepparag/DumBot

jcblaise/gpt2-tagalog

Tagalog

Please refer to: jcblaise/gpt2-tagalog

BigSalmon/InformalToFormalLincoln21

English

Please refer to: BigSalmon/InformalToFormalLincoln21

LorenzoDeMattei/GePpeTto

English

Please refer to: LorenzoDeMattei/GePpeTto

macedonizer/sr-gpt2

English

Please refer to: macedonizer/sr-gpt2

indonesian-nlp/gpt2

English

Please refer to: indonesian-nlp/gpt2

ceostroff/harry-potter-gpt2-fanfiction

English

Please refer to: ceostroff/harry-potter-gpt2-fanfiction

akhooli/gpt2-small-arabic-poetry

Arabic

Please refer to: akhooli/gpt2-small-arabic-poetry

asi/gpt-fr-cased-base

French

Please refer to: asi/gpt-fr-cased-base

congcongwang/gpt2_medium_fine_tuned_coder

English

Please refer to: congcongwang/gpt2_medium_fine_tuned_coder

cambridgeltl/simctg_wikitext103

English

Please refer to: cambridgeltl/simctg_wikitext103