unified_transformer#
- convert
- modeling
- tokenizer
UnifiedTransformerTokenizer
UnifiedTransformerTokenizer.vocab_size
UnifiedTransformerTokenizer.get_vocab()
UnifiedTransformerTokenizer.convert_tokens_to_string()
UnifiedTransformerTokenizer.convert_ids_to_string()
UnifiedTransformerTokenizer.num_special_tokens_to_add()
UnifiedTransformerTokenizer.build_inputs_with_special_tokens()
UnifiedTransformerTokenizer.build_offset_mapping_with_special_tokens()
UnifiedTransformerTokenizer.create_token_type_ids_from_sequences()
UnifiedTransformerTokenizer.get_special_tokens_mask()
UnifiedTransformerTokenizer.save_resources()
UnifiedTransformerTokenizer.load_vocabulary()
UnifiedTransformerTokenizer.dialogue_encode()