paddlenlp#
- paddlenlp.data
- paddlenlp.dataaug
- paddlenlp.datasets
- paddlenlp.embeddings
- paddlenlp.experimental
- paddlenlp.layers
- paddlenlp.losses
- paddlenlp.metrics
- paddlenlp.ops
- paddlenlp.seq2vec
- paddlenlp.taskflow
- models
- code_generation
- dependency_parsing
- dialogue
- information_extraction
- knowledge_mining
- lexical_analysis
- named_entity_recognition
- poetry_generation
- pos_tagging
- question_answering
- sentiment_analysis
- task
- taskflow
- text_to_image
- text_correction
- text_generation
- text_similarity
- utils
download_file()download_check()add_docstrings()cut_chinese_sent()TermTreeNodeTermTreelevenstein_distance()BurkhardKellerNodeBurkhardKellerTreeTriedTreeCustomizationSchemaTreeget_id_and_prob()WordTagRelationExtractorDataCollatorGPDataCollatorForErnieCtmDocSpanExampleFeatureFeature.bio_seq_labelsFeature.bioes_seq_labelsFeature.boxesFeature.doc_labelsFeature.doc_span_indexFeature.doc_tokensFeature.example_indexFeature.im_base64Feature.image_roisFeature.key_labelsFeature.keysFeature.model_typeFeature.overlapsFeature.position_idsFeature.qas_idFeature.se_seq_labelsFeature.segment_idsFeature.seq_labelsFeature.textFeature.text_symbol_idsFeature.text_type_idsFeature.token_idsFeature.token_is_max_contextFeature.token_to_orig_mapFeature.tokensFeature.unique_id
ComposeProcessReaderpad_batch_data()get_final_text()find_bio_pos()calEuclidean()longestCommonSequence()
- word_segmentation
- paddlenlp.trainer
- utils
- argparser
- integrations
- trainer_base
- trainer_callback
TrainerStateTrainerControlTrainerCallbackTrainerCallback.on_init_end()TrainerCallback.on_train_begin()TrainerCallback.on_train_end()TrainerCallback.on_epoch_begin()TrainerCallback.on_epoch_end()TrainerCallback.on_step_begin()TrainerCallback.on_substep_end()TrainerCallback.on_step_end()TrainerCallback.on_evaluate()TrainerCallback.on_save()TrainerCallback.on_log()TrainerCallback.on_prediction_step()
CallbackHandlerCallbackHandler.on_init_end()CallbackHandler.on_train_begin()CallbackHandler.on_train_end()CallbackHandler.on_epoch_begin()CallbackHandler.on_epoch_end()CallbackHandler.on_step_begin()CallbackHandler.on_substep_end()CallbackHandler.on_step_end()CallbackHandler.on_evaluate()CallbackHandler.on_save()CallbackHandler.on_log()CallbackHandler.on_prediction_step()
DefaultFlowCallbackProgressCallbackPrinterCallbackEarlyStoppingCallback
- trainer_utils
- training_args
default_logdir()TrainingArgumentsTrainingArguments.train_batch_sizeTrainingArguments.eval_batch_sizeTrainingArguments.current_deviceTrainingArguments.world_sizeTrainingArguments.process_indexTrainingArguments.logical_process_indexTrainingArguments.local_process_indexTrainingArguments.should_logTrainingArguments.should_saveTrainingArguments.should_save_model_stateTrainingArguments.main_process_first()TrainingArguments.get_warmup_steps()TrainingArguments.to_dict()TrainingArguments.to_json_string()TrainingArguments.to_sanitized_dict()TrainingArguments.print_config()
- paddlenlp.transformers
- albert
- artist
- auto
- modeling
AutoBackboneAutoModelAutoModelForPretrainingAutoModelForSequenceClassificationAutoModelForTokenClassificationAutoModelForQuestionAnsweringAutoModelForMultipleChoiceAutoModelForMaskedLMAutoModelForCausalLMAutoModelForCausalLMPipeAutoEncoderAutoDecoderAutoGeneratorAutoDiscriminatorAutoModelForConditionalGeneration
- tokenizer
- modeling
- bart
- bert
- bert_japanese
- bigbird
- blenderbot
- blenderbot_small
- chinesebert
- codegen
- convbert
- modeling
ConvBertModelConvBertForMaskedLMConvBertPretrainedModelConvBertForTotalPretrainingConvBertDiscriminatorConvBertGeneratorConvBertClassificationHeadConvBertForSequenceClassificationConvBertForTokenClassificationConvBertPretrainingCriterionConvBertForQuestionAnsweringConvBertForMultipleChoiceConvBertForPretraining
- tokenizer
- modeling
- ctrl
- dallebart
- distilbert
- electra
- modeling
ElectraModelElectraPretrainedModelElectraForTotalPretrainingElectraDiscriminatorElectraGeneratorElectraClassificationHeadElectraForSequenceClassificationElectraForTokenClassificationElectraPretrainingCriterionElectraForMultipleChoiceElectraForQuestionAnsweringElectraForMaskedLMElectraForPretrainingErnieHealthForTotalPretrainingErnieHealthPretrainingCriterionErnieHealthDiscriminator
- tokenizer
- modeling
- ernie
- ernie_ctm
- ernie_doc
- ernie_gen
- ernie_gram
- ernie_m
- fnet
- funnel
- gau_alpha
- gpt
- layoutlm
- layoutlmv2
- layoutxlm
- luke
- mbart
- megatronbert
- mobilebert
- mpnet
- nezha
- opt
- ppminilm
- prophetnet
- reformer
- rembert
- roberta
- roformer
- roformerv2
- semantic_search
- skep
- squeezebert
- t5
- tinybert
- transformer
- unified_transformer
- unimo
- xlm
- xlnet
- attention_utils
- convert_slow_tokenizer
- distill_utils
- export
- generation_utils
- model_outputs
tuple_output()convert_encoder_output()ModelOutputBaseModelOutputBaseModelOutputWithNoAttentionBaseModelOutputWithPoolingBaseModelOutputWithPastBaseModelOutputWithPastAndCrossAttentionsBaseModelOutputWithPoolingAndCrossAttentionsSequenceClassifierOutputTokenClassifierOutputQuestionAnsweringModelOutputMultipleChoiceModelOutputMaskedLMOutputCausalLMOutputWithPastCausalLMOutputWithCrossAttentionsSeq2SeqModelOutputSeq2SeqLMOutputSeq2SeqQuestionAnsweringModelOutputSeq2SeqSequenceClassifierOutputSequenceClassifierOutputWithPastBackboneOutputBaseModelOutputWithPoolingAndNoAttentionImageClassifierOutputWithNoAttentionDepthEstimatorOutputSemanticSegmenterOutputSeq2SeqSpectrogramOutput
- model_utils
PretrainedModelPretrainedModel.init_weights()PretrainedModel.from_config()PretrainedModel.base_modelPretrainedModel.model_name_listPretrainedModel.can_generate()PretrainedModel.recompute_enable()PretrainedModel.recompute_disable()PretrainedModel.get_memory_footprint()PretrainedModel.get_input_embeddings()PretrainedModel.set_input_embeddings()PretrainedModel.get_output_embeddings()PretrainedModel.tie_weights()PretrainedModel.resize_position_embeddings()PretrainedModel.constructed_from_pretrained_config()PretrainedModel.save_model_config()PretrainedModel.save_to_hf_hub()PretrainedModel.save_to_aistudio()PretrainedModel.resize_token_embeddings()PretrainedModel.from_pretrained()PretrainedModel.save_pretrained()
register_base_model()
- optimization
- sentencepiece_model_pb2
- tokenizer_utils
PretrainedTokenizerPretrainedTokenizer.vocab_sizePretrainedTokenizer.get_added_vocab()PretrainedTokenizer.prepare_for_tokenization()PretrainedTokenizer.tokenize()PretrainedTokenizer.convert_tokens_to_string()PretrainedTokenizer.load_vocabulary()PretrainedTokenizer.save_vocabulary()PretrainedTokenizer.get_special_tokens_mask()PretrainedTokenizer.num_special_tokens_to_add()PretrainedTokenizer.get_offset_mapping()PretrainedTokenizer.decode_token()
BPETokenizertokenize_chinese_chars()is_chinese_char()normalize_chars()tokenize_special_chars()convert_to_unicode()
- tokenizer_utils_base
AddedTokenFastEncodingExplicitEnumPaddingStrategyTensorTypeto_py_obj()TruncationStrategyCharSpanTokenSpanBatchEncodingBatchEncoding.n_sequencesBatchEncoding.is_fastBatchEncoding.keys()BatchEncoding.values()BatchEncoding.items()BatchEncoding.encodingsBatchEncoding.tokens()BatchEncoding.sequence_ids()BatchEncoding.words()BatchEncoding.word_ids()BatchEncoding.token_to_sequence()BatchEncoding.token_to_word()BatchEncoding.word_to_tokens()BatchEncoding.token_to_chars()BatchEncoding.char_to_token()BatchEncoding.word_to_chars()BatchEncoding.char_to_word()BatchEncoding.convert_to_tensors()
SpecialTokensMixinSpecialTokensMixin.sanitize_special_tokens()SpecialTokensMixin.add_special_tokens()SpecialTokensMixin.add_tokens()SpecialTokensMixin.bos_tokenSpecialTokensMixin.eos_tokenSpecialTokensMixin.unk_tokenSpecialTokensMixin.sep_tokenSpecialTokensMixin.pad_tokenSpecialTokensMixin.cls_tokenSpecialTokensMixin.mask_tokenSpecialTokensMixin.additional_special_tokensSpecialTokensMixin.pad_token_type_idSpecialTokensMixin.bos_token_idSpecialTokensMixin.eos_token_idSpecialTokensMixin.unk_token_idSpecialTokensMixin.sep_token_idSpecialTokensMixin.pad_token_idSpecialTokensMixin.cls_token_idSpecialTokensMixin.mask_token_idSpecialTokensMixin.additional_special_tokens_idsSpecialTokensMixin.special_tokens_mapSpecialTokensMixin.special_tokens_map_extendedSpecialTokensMixin.all_special_tokensSpecialTokensMixin.all_special_tokens_extendedSpecialTokensMixin.all_special_ids
PretrainedTokenizerBasePretrainedTokenizerBase.max_len_single_sentencePretrainedTokenizerBase.max_len_sentences_pairPretrainedTokenizerBase.get_vocab()PretrainedTokenizerBase.from_pretrained()PretrainedTokenizerBase.save_pretrained()PretrainedTokenizerBase.save_resources()PretrainedTokenizerBase.save_to_hf_hub()PretrainedTokenizerBase.save_to_aistudio()PretrainedTokenizerBase.tokenize()PretrainedTokenizerBase.__call__()PretrainedTokenizerBase.encode()PretrainedTokenizerBase.encode_plus()PretrainedTokenizerBase.batch_encode()PretrainedTokenizerBase.pad()PretrainedTokenizerBase.create_token_type_ids_from_sequences()PretrainedTokenizerBase.build_inputs_with_special_tokens()PretrainedTokenizerBase.build_offset_mapping_with_special_tokens()PretrainedTokenizerBase.prepare_for_model()PretrainedTokenizerBase.truncate_sequences()PretrainedTokenizerBase.convert_tokens_to_string()PretrainedTokenizerBase.batch_decode()PretrainedTokenizerBase.decode()PretrainedTokenizerBase.get_special_tokens_mask()PretrainedTokenizerBase.clean_up_tokenization()
- tokenizer_utils_fast
- utils
convert_ndarray_dtype()get_scale_by_dtype()fn_args_to_dict()adapt_stale_fwd_patch()InitTrackerMetaparam_in_func()resolve_cache_dir()find_transformer_model_type()find_transformer_model_class_by_name()convert_file_size_to_int()cached_file()get_checkpoint_shard_files()ContextManagersdtype_byte_size()CaptureStd
- paddlenlp.utils