SqueezeBert模型汇总#

下表汇总介绍了目前PaddleNLP支持的SqueezeBert模型对应预训练权重。 关于模型的具体细节可以参考对应链接。

Pretrained Weight

Language

Details of the model

squeezebert-uncased

English

12-layer, 768-hidden, 12-heads, 51M parameters. SqueezeBert Uncased model.

squeezebert-mnli

English

12-layer, 768-hidden, 12-heads, 51M parameters. SqueezeBert Mnli model.

squeezebert-mnli-headless

English

12-layer, 768-hidden, 12-heads, 51M parameters. SqueezeBert Mnli Headless model.