prophetnet
- modeling
- tokenizer
load_vocab()
ProphetNetTokenizer
ProphetNetTokenizer.vocab_size
ProphetNetTokenizer.get_vocab()
ProphetNetTokenizer.tokenize()
ProphetNetTokenizer.convert_tokens_to_ids()
ProphetNetTokenizer.convert_ids_to_tokens()
ProphetNetTokenizer.convert_tokens_to_string()
ProphetNetTokenizer.get_special_tokens_mask()
ProphetNetTokenizer.create_token_type_ids_from_sequences()
ProphetNetTokenizer.build_inputs_with_special_tokens()
ProphetNetTokenizer.save_vocabulary()