modeling#

class DebertaModel(config: DebertaConfig)[源代码]#

基类:DebertaPreTrainedModel

get_input_embeddings()[源代码]#

get input embedding of model

返回:

embedding of model

返回类型:

nn.Embedding

set_input_embeddings(value)[源代码]#

set new input embedding for model

参数:

value (Embedding) -- the new embedding of model

抛出:

NotImplementedError -- Model has not implement set_input_embeddings method

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, inputs_embeds=None, output_attentions=None, output_hidden_states=None, return_dict=None)[源代码]#

Defines the computation performed at every call. Should be overridden by all subclasses.

参数:
  • *inputs (tuple) -- unpacked tuple arguments

  • **kwargs (dict) -- unpacked dict arguments

class DebertaForSequenceClassification(config)[源代码]#

基类:DebertaPreTrainedModel

get_input_embeddings()[源代码]#

get input embedding of model

返回:

embedding of model

返回类型:

nn.Embedding

set_input_embeddings(new_embeddings)[源代码]#

set new input embedding for model

参数:

value (Embedding) -- the new embedding of model

抛出:

NotImplementedError -- Model has not implement set_input_embeddings method

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, output_attentions=None, output_hidden_states=None, return_dict=None)[源代码]#

Defines the computation performed at every call. Should be overridden by all subclasses.

参数:
  • *inputs (tuple) -- unpacked tuple arguments

  • **kwargs (dict) -- unpacked dict arguments

class DebertaForQuestionAnswering(config)[源代码]#

基类:DebertaPreTrainedModel

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, inputs_embeds=None, start_positions=None, end_positions=None, output_attentions=None, output_hidden_states=None, return_dict=None)[源代码]#

Defines the computation performed at every call. Should be overridden by all subclasses.

参数:
  • *inputs (tuple) -- unpacked tuple arguments

  • **kwargs (dict) -- unpacked dict arguments

class DebertaForTokenClassification(config)[源代码]#

基类:DebertaPreTrainedModel

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, output_attentions=None, output_hidden_states=None, return_dict=None)[源代码]#

Defines the computation performed at every call. Should be overridden by all subclasses.

参数:
  • *inputs (tuple) -- unpacked tuple arguments

  • **kwargs (dict) -- unpacked dict arguments

class DebertaPreTrainedModel(*args, **kwargs)[源代码]#

基类:PretrainedModel

An abstract class for pretrained BERT models. It provides BERT related model_config_file, resource_files_names, pretrained_resource_files_map, pretrained_init_configuration, base_model_prefix for downloading and loading pretrained models. See PretrainedModel for more details.

config_class#

DebertaConfig 的别名

init_weights(layer)[源代码]#

Initialization hook

base_model_class#

DebertaModel 的别名

class DebertaForMultipleChoice(config: DebertaConfig)[源代码]#

基类:DebertaPreTrainedModel

Deberta Model with a linear layer on top of the hidden-states output layer, designed for multiple choice tasks like RocStories/SWAG tasks.

参数:
  • bert (DebertaModel) -- An instance of DebertaModel.

  • num_choices (int, optional) -- The number of choices. Defaults to 2.

  • dropout (float, optional) -- The dropout probability for output of Bert. If None, use the same value as hidden_dropout_prob of DebertaModel instance bert. Defaults to None.

forward(input_ids=None, token_type_ids=None, position_ids=None, attention_mask=None, inputs_embeds=None, labels=None, output_hidden_states=None, output_attentions=None, return_dict=None)[源代码]#

The DebertaForMultipleChoice forward method, overrides the __call__() special method.

参数:
  • input_ids (Tensor) -- See DebertaModel and shape as [batch_size, num_choice, sequence_length].

  • token_type_ids (Tensor, optional) -- See DebertaModel and shape as [batch_size, num_choice, sequence_length].

  • position_ids (Tensor, optional) -- See DebertaModel and shape as [batch_size, num_choice, sequence_length].

  • attention_mask (list, optional) -- See DebertaModel and shape as [batch_size, num_choice, sequence_length].

  • inputs_embeds (list, optional) -- See DebertaModel and shape as [batch_size, num_choice, sequence_length].

  • labels (Tensor of shape (batch_size, ), optional) -- Labels for computing the multiple choice classification loss. Indices should be in [0, ..., num_choices-1] where num_choices is the size of the second dimension of the input tensors. (See input_ids above)

  • output_hidden_states (bool, optional) -- Whether to return the hidden states of all layers. Defaults to False.

  • output_attentions (bool, optional) -- Whether to return the attentions tensors of all attention layers. Defaults to False.

  • return_dict (bool, optional) -- Whether to return a MultipleChoiceModelOutput object. If False, the output will be a tuple of tensors. Defaults to False.

返回:

An instance of MultipleChoiceModelOutput if return_dict=True. Otherwise it returns a tuple of tensors corresponding to ordered and not None (depending on the input arguments) fields of MultipleChoiceModelOutput.

示例

import paddle
from paddlenlp.transformers import BertForMultipleChoice, BertTokenizer
from paddlenlp.data import Pad, Dict

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForMultipleChoice.from_pretrained('bert-base-uncased', num_choices=2)

data = [
    {
        "question": "how do you turn on an ipad screen?",
        "answer1": "press the volume button.",
        "answer2": "press the lock button.",
        "label": 1,
    },
    {
        "question": "how do you indent something?",
        "answer1": "leave a space before starting the writing",
        "answer2": "press the spacebar",
        "label": 0,
    },
]

text = []
text_pair = []
for d in data:
    text.append(d["question"])
    text_pair.append(d["answer1"])
    text.append(d["question"])
    text_pair.append(d["answer2"])

inputs = tokenizer(text, text_pair)
batchify_fn = lambda samples, fn=Dict(
    {
        "input_ids": Pad(axis=0, pad_val=tokenizer.pad_token_id),  # input_ids
        "token_type_ids": Pad(
            axis=0, pad_val=tokenizer.pad_token_type_id
        ),  # token_type_ids
    }
): fn(samples)
inputs = batchify_fn(inputs)

reshaped_logits = model(
    input_ids=paddle.to_tensor(inputs[0], dtype="int64"),
    token_type_ids=paddle.to_tensor(inputs[1], dtype="int64"),
)
print(reshaped_logits.shape)
# [2, 2]