tokenizer

Tokenization classes for LayoutLMv2 model.

class LayoutLMv2Tokenizer(vocab_file, do_lower_case=True, unk_token='[UNK]', sep_token='[SEP]', pad_token='[PAD]', cls_token='[CLS]', mask_token='[MASK]', **kwargs)[source]

Bases: paddlenlp.transformers.bert.tokenizer.BertTokenizer

The usage of LayoutLMv2Tokenizer is the same as BertTokenizer. For more information regarding those methods, please refer to this superclass.