Reformer模型汇总

Reformer模型汇总#

下表汇总介绍了目前PaddleNLP支持的Reformer模型对应预训练权重。 关于模型的具体细节可以参考对应链接。

Pretrained Weight

Language

Details of the model

reformer-enwik8

English

12-layer, 1024-hidden, 8-heads, 148M parameters.

reformer-crime-and-punishment

English

6-layer, 256-hidden, 2-heads, 3M parameters.