modeling

class DebertaModel(config: DebertaConfig)

Bases: DebertaPreTrainedModel

get_input_embeddings()

get input embedding of model

Returns:

embedding of model

Return type:

nn.Embedding

set_input_embeddings(value)

set new input embedding for model

Parameters:

value (Embedding) – the new embedding of model

Raises:

NotImplementedError – Model has not implement set_input_embeddings method

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, inputs_embeds=None, output_attentions=None, output_hidden_states=None, return_dict=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters:

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class DebertaForSequenceClassification(config)

Bases: DebertaPreTrainedModel

get_input_embeddings()

get input embedding of model

Returns:

embedding of model

Return type:

nn.Embedding

set_input_embeddings(new_embeddings)

set new input embedding for model

Parameters:

value (Embedding) – the new embedding of model

Raises:

NotImplementedError – Model has not implement set_input_embeddings method

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, output_attentions=None, output_hidden_states=None, return_dict=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters:

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class DebertaForQuestionAnswering(config)

Bases: DebertaPreTrainedModel

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, inputs_embeds=None, start_positions=None, end_positions=None, output_attentions=None, output_hidden_states=None, return_dict=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters:

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class DebertaForTokenClassification(config)

Bases: DebertaPreTrainedModel

forward(input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, output_attentions=None, output_hidden_states=None, return_dict=None)

Defines the computation performed at every call. Should be overridden by all subclasses.

Parameters:

• *inputs (tuple) – unpacked tuple arguments

• **kwargs (dict) – unpacked dict arguments

class DebertaPreTrainedModel(*args, **kwargs)

Bases: PretrainedModel

An abstract class for pretrained BERT models. It provides BERT related model_config_file, resource_files_names, pretrained_resource_files_map, pretrained_init_configuration, base_model_prefix for downloading and loading pretrained models. See PretrainedModel for more details.

config_class

alias of DebertaConfig

init_weights(layer)

Initialization hook

base_model_class

alias of DebertaModel

class DebertaForMultipleChoice(config: DebertaConfig)

Bases: DebertaPreTrainedModel

Deberta Model with a linear layer on top of the hidden-states output layer, designed for multiple choice tasks like RocStories/SWAG tasks.

Parameters:

• bert (DebertaModel) – An instance of DebertaModel.

• num_choices (int, optional) – The number of choices. Defaults to 2.

• dropout (float, optional) – The dropout probability for output of Bert. If None, use the same value as hidden_dropout_prob of DebertaModel instance bert. Defaults to None.

forward(input_ids=None, token_type_ids=None, position_ids=None, attention_mask=None, inputs_embeds=None, labels=None, output_hidden_states=None, output_attentions=None, return_dict=None)

The DebertaForMultipleChoice forward method, overrides the __call__() special method.

Parameters:

• input_ids (Tensor) – See DebertaModel and shape as [batch_size, num_choice, sequence_length].

• token_type_ids (Tensor, optional) – See DebertaModel and shape as [batch_size, num_choice, sequence_length].

• position_ids (Tensor, optional) – See DebertaModel and shape as [batch_size, num_choice, sequence_length].

• attention_mask (list, optional) – See DebertaModel and shape as [batch_size, num_choice, sequence_length].

• inputs_embeds (list, optional) – See DebertaModel and shape as [batch_size, num_choice, sequence_length].

• labels (Tensor of shape (batch_size, ), optional) – Labels for computing the multiple choice classification loss. Indices should be in [0, ..., num_choices-1] where num_choices is the size of the second dimension of the input tensors. (See input_ids above)

• output_hidden_states (bool, optional) – Whether to return the hidden states of all layers. Defaults to False.

• output_attentions (bool, optional) – Whether to return the attentions tensors of all attention layers. Defaults to False.

• return_dict (bool, optional) – Whether to return a MultipleChoiceModelOutput object. If False, the output will be a tuple of tensors. Defaults to False.

Returns:

An instance of MultipleChoiceModelOutput if return_dict=True. Otherwise it returns a tuple of tensors corresponding to ordered and not None (depending on the input arguments) fields of MultipleChoiceModelOutput.

Example

import paddle
from paddlenlp.transformers import BertForMultipleChoice, BertTokenizer
from paddlenlp.data import Pad, Dict

tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForMultipleChoice.from_pretrained('bert-base-uncased', num_choices=2)

data = [
    {
        "question": "how do you turn on an ipad screen?",
        "answer1": "press the volume button.",
        "answer2": "press the lock button.",
        "label": 1,
    },
    {
        "question": "how do you indent something?",
        "answer1": "leave a space before starting the writing",
        "answer2": "press the spacebar",
        "label": 0,
    },
]

text = []
text_pair = []
for d in data:
    text.append(d["question"])
    text_pair.append(d["answer1"])
    text.append(d["question"])
    text_pair.append(d["answer2"])

inputs = tokenizer(text, text_pair)
batchify_fn = lambda samples, fn=Dict(
    {
        "input_ids": Pad(axis=0, pad_val=tokenizer.pad_token_id),  # input_ids
        "token_type_ids": Pad(
            axis=0, pad_val=tokenizer.pad_token_type_id
        ),  # token_type_ids
    }
): fn(samples)
inputs = batchify_fn(inputs)

reshaped_logits = model(
    input_ids=paddle.to_tensor(inputs[0], dtype="int64"),
    token_type_ids=paddle.to_tensor(inputs[1], dtype="int64"),
)
print(reshaped_logits.shape)
# [2, 2]