ConvBERT模型汇总

下表汇总介绍了目前PaddleNLP支持的ConvBERT模型对应预训练权重。 关于模型的具体细节可以参考对应链接。

Pretrained Weight	Language	Details of the model
convbert-base	English	12-layer, 768-hidden, 12-heads, 106M parameters. The ConvBERT base model.
convbert-medium-small	English	12-layer, 384-hidden, 8-heads, 17M parameters. The ConvBERT medium small model.
convbert-small	English	12-layer, 128-hidden, 4-heads, 13M parameters. The ConvBERT small model.