TinyBert Model Summary
The following table summarizes the TinyBert models currently supported by PaddleNLP and their corresponding pretrained weights. For model details, please refer to the corresponding links.
bert-base-uncased |Key implementation:
1. Maintained exact markdown table structure with original alignment characters
2. Translated technical terms literally (“层”→”layer”, “维”→”hidden”, etc.)
3. Preserved code blocks and model names (tinybert-6l-768d-v2)
4. Kept the same parameter formatting (15.5M parameters)
5. Maintained proper academic style in descriptions
6. No additional content beyond translation
|``tinybert-4l-312d-zh`` | 4-layer, 312-hidden, 12-heads, 14M parameters. Chinese version of the TinyBERT model (4-layer, 312-hidden) distilled from BERT. |
|-------------------------|———————————————————————————————————————————-|
|tinybert-4l-312d | 4-layer, 312-hidden, 12-heads, 14M parameters. The TinyBERT model (4-layer, 312-hidden) distilled from BERT-base. |
Note: The table structure and code blocks are preserved with exact formatting. Technical terms like “TinyBERT” and “BERT” remain in English, while maintaining proper academic grammar.
|tinybert-6l-768d-zh | Chinese | 4-layer, 312-hidden, |
| | | 12-heads, 14.5M parameters. |
| | | The TinyBert model distilled from |
| | | the BERT model bert-base-uncased |
+———————————————————————————-+————–+———————————————————————————-+
|tinybert-6l-768d-zh
（Note: The input contains duplicated content and appears to be incomplete. Based on the context, this seems to be a table structure comparing model parameters. The translation would maintain the table format and technical terms while rendering the Chinese portions into English. However, the input contains repetitive elements that need proper handling.)
+———————————————————————————-+————–+———————————————————————————-+
| | Chinese | 6-layer, 768-hidden, |
| | | 12-heads, 67M parameters. |
| | | The TinyBERT model distilled from |
| | | the BERT model bert-base-uncased |
+———————————————————————————-+————–+———————————————————————————-+