roformerv2
- modeling
- tokenizer
RoFormerv2Tokenizer
RoFormerv2Tokenizer.vocab_size
RoFormerv2Tokenizer.convert_tokens_to_string()
RoFormerv2Tokenizer.num_special_tokens_to_add()
RoFormerv2Tokenizer.build_inputs_with_special_tokens()
RoFormerv2Tokenizer.build_offset_mapping_with_special_tokens()
RoFormerv2Tokenizer.create_token_type_ids_from_sequences()
RoFormerv2Tokenizer.get_special_tokens_mask()
RoFormerv2Tokenizer.get_vocab()