modeling

class BlenderbotModel(config: paddlenlp.transformers.blenderbot.configuration.BlenderbotConfig)[source]

Bases: paddlenlp.transformers.blenderbot.modeling.BlenderbotPretrainedModel

Construct a bare Blenderbot Model.

This model inherits from PretrainedModel. Check the superclass documentation for the generic methods and the library implements for all its model.

This model is also a Paddle paddle.nn.Layer subclass. Use it as a regular Paddle Layer and refer to the Paddle documentation for all matter related to general usage and behavior.

forward(input_ids=None, attention_mask=None, decoder_input_ids=None, decoder_attention_mask=None, encoder_output=None, use_cache=False, cache=None, **kwargs)[source]
Parameters

  • input_ids (Tensor) – Indices of input sequence tokens in the vocabulary. They are numerical representations of tokens that build the input sequence. It’s data type should be int64 and has a shape of [batch_size, sequence_length].

  • attention_mask (Tensor, optional) –

    Mask to indicate whether to perform attention on each input token or not. The values should be either 0 or 1. The attention scores will be set to -infinity for any positions in the mask that are 0, and will be unchanged for positions that are 1.

    • 1 for tokens that are not masked,

    • 0 for tokens that are masked.

    It’s data type should be float32 and has a shape of [batch_size, sequence_length]. Defaults to None.

  • decoder_input_ids (Tensor, optional) – If not provided, decoder_input_ids will be automatically generated based on decoder_start_token_id and input_ids.

  • decoder_attention_mask (Tensor, optional) – If not provided, the default decoder_attention_mask will be a tensor with upper triangular part being -np.inf. the shape will be (decoder_length, decoder_length)

  • encoder_output (Tensor, optional) – The output of encoder. If not provided, a encoder_output will be generated from BlenderbotEncoder. Defaults to None.

  • use_cache (bool, optional) – Indicates whether to use cache to speed up decoding. Defaults to False

  • cache (list, optional) – It is a list, and each element in the list is a tuple( (incremental_cache, static_cache) ). See paddle.nn.TransformerDecoder.gen_cache for more details. It is only used for inference and should be None for training. Default None.

Returns

If use_cache=False, the return will be the last hidden state of decoder with shape of [batch_size, seq_lens, hidden_size]. seq_lens corresponds to the length of input sequence. Otherwise, the return will be a tuple of (decoder_output, cache). Please refer to class paddle.nn.TransformerDecoder for more information regarding cache.

Return type

Tensor|tuple

Example

import paddle
from paddlenlp.transformers import BlenderbotTokenizer, BlenderbotModel

# "blenderbot-400M-distill" is the pretrained weight of BlenderbotForConditionalGeneration,
# Therefore some weight of additional layers in BlenderbotForConditionalGeneration
# might not be loaded and used regarding the following sample code.
pretrained_model_name = "blenderbot-400M-distill"
tokenizer = BlenderbotTokenizer.from_pretrained(pretrained_model_name)
model = BlenderbotModel.from_pretrained(pretrained_model_name)

sample_text = "My friends are cool but they eat too many carbs."
inputs = tokenizer(sample_text, return_attention_mask=True, return_token_type_ids=False)
inputs = {k:paddle.to_tensor([v]) for (k, v) in inputs.items()}
decoder_output = model(**inputs)
get_input_embeddings()[source]

get input embedding of model

Returns

embedding of model

Return type

nn.Embedding

set_input_embeddings(value)[source]

set new input embedding for model

Parameters

value (Embedding) – the new embedding of model

Raises

NotImplementedError – Model has not implement set_input_embeddings method

get_encoder()[source]

This method is required for model with encoder-decoder architecture.

class BlenderbotPretrainedModel(*args, **kwargs)[source]

Bases: paddlenlp.transformers.model_utils.PretrainedModel

An abstract class for pretrained Blenderbot models. It provides Blenderbot related model_config_file, resource_files_names, pretrained_resource_files_map, pretrained_init_configuration, base_model_prefix for downloading and loading pretrained models. Refer to PretrainedModel for more details.

config_class

alias of paddlenlp.transformers.blenderbot.configuration.BlenderbotConfig

init_weights(layer)[source]

Initialization hook

base_model_class

alias of paddlenlp.transformers.blenderbot.modeling.BlenderbotModel

class BlenderbotEncoder(config: paddlenlp.transformers.blenderbot.configuration.BlenderbotConfig, embed_tokens=None)[source]

Bases: paddlenlp.transformers.blenderbot.modeling.BlenderbotPretrainedModel

The encoder of Blenderbot Model. Please refer to PretrainedModel or BlenderbotModel for more information regarding methods and arguments.

forward(input_ids, attention_mask=None)[source]
Returns

The last hidden states at the last layer of the encoder. It’s data type should be float and has a shape of (batch_size, seq_lens, hidden_size). seq_lens corresponds to the length of input sequence.

Return type

Tensor

class BlenderbotDecoder(config: paddlenlp.transformers.blenderbot.configuration.BlenderbotConfig, embed_tokens=None)[source]

Bases: paddlenlp.transformers.blenderbot.modeling.BlenderbotPretrainedModel

The decoder of Blenderbot Model. Please refer to PretrainedModel and BlenderbotModel for more information regarding methods and arguments.

forward(decoder_input_ids=None, decoder_attention_mask=None, encoder_output=None, memory_mask=None, use_cache=False, cache=None)[source]

Please refer to BlenderbotModel for more information regarding the arguments.

class BlenderbotForConditionalGeneration(config: paddlenlp.transformers.blenderbot.configuration.BlenderbotConfig)[source]

Bases: paddlenlp.transformers.blenderbot.modeling.BlenderbotPretrainedModel

forward(input_ids=None, attention_mask=None, decoder_input_ids=None, decoder_attention_mask=None, encoder_output=None, use_cache=False, cache=None, **kwargs)[source]

Please refer to BlenderbotModel for more information regarding arguments. :returns:

If use_cache=False, the return will be a tensor with shape of

[batch_size, seq_lens, hidden_size]. Otherwise, the return will be a tuple of (decoder_output, cache).

Return type

Tensor|tuple

Example

import paddle from paddlenlp.transformers import BlenderbotTokenizer, BlenderbotForConditionalGeneration

pretrained_model_name = “blenderbot-400M-distill” tokenizer = BlenderbotTokenizer.from_pretrained(pretrained_model_name) model = BlenderbotForConditionalGeneration.from_pretrained(pretrained_model_name)

sample_text = “My friends are cool but they eat too many carbs.” inputs = tokenizer(sample_text, return_attention_mask=True, return_token_type_ids=False) inputs = {k: paddle.to_tensor([v]) for (k, v) in inputs.items()}

# Generate response using beam search result_ids, scores = model.generate(input_ids=inputs[‘input_ids’],

max_length=60, min_length=20, decode_strategy=’beam_search’, num_beams=10, length_penalty=0.65)

for sequence_ids in result_ids.numpy().tolist():

print(“User: “, sample_text) print(“bot: “, tokenizer.convert_ids_to_string(sequence_ids)) # “bot: That’s unfortunate. Are they trying to lose weight?”

prepare_inputs_for_generation(decoder_input_ids, attention_mask=None, encoder_output=None, use_cache=True, cache=None, **kwargs)[source]

Prepare inputs for decoder to generate sentences. :returns: A dictionary containing necessary inputs for generating next token. :rtype: dict

get_encoder()[source]

This method is required for model with encoder-decoder architecture.

class BlenderbotForCausalLM(config: paddlenlp.transformers.blenderbot.configuration.BlenderbotConfig)[source]

Bases: paddlenlp.transformers.blenderbot.modeling.BlenderbotPretrainedModel

Constructs BLenderbot For Causal Language Model. This model is equivalent to the blenderbot decoder without cross-attention.

forward(input_ids=None, attention_mask=None, use_cache=False, cache=None, **kwargs)[source]
Parameters

  • input_ids (Tensor) – Indices of input sequence tokens in the vocabulary. They are numerical representations of tokens that build the input sequence. It’s data type should be int64 and has a shape of [batch_size, sequence_length].

  • attention_mask (Tensor, optional) –

    Mask to indicate whether to perform attention on each input token or not. The values should be either 0 or 1. The attention scores will be set to -infinity for any positions in the mask that are 0, and will be unchanged for positions that are 1.

    • 1 for tokens that are not masked,

    • 0 for tokens that are masked.

    It’s data type should be float32 and has a shape of [batch_size, sequence_length]. Defaults to None.

  • use_cache (bool, optional) – Indicates whether to use cache to speed up decoding. Defaults to False

  • cache (list, optional) – It is a list, and each element in the list is a tuple( (incremental_cache, static_cache) ). See paddle.nn.TransformerDecoder.gen_cache for more details. It is only used for inference and should be None for training. Default None.

Returns

If use_cache=False, the return will be a tensor with shape of

[batch_size, seq_lens, hidden_size]. Otherwise, the return will be a tuple of (lm_logits, cache).

Return type

Tensor|tuple

Example

import paddle from paddlenlp.transformers import BlenderbotTokenizer, BlenderbotForCausalLM use_cache = False text = “My friends are cool but they eat too many carbs.” model_name = “blenderbot-400M-distill” tokenizer = BlenderbotTokenizer.from_pretrained(model_name) model = BlenderbotForCausalLM.from_pretrained(model_name) model.eval() inputs = tokenizer(text) inputs = {k: paddle.to_tensor([v]) for (k, v) in inputs.items()}

with paddle.no_grad():

outputs = model(**inputs, use_cache=use_cache) # outputs is a tuple of (lm_logits, cache) if use_cache=True.

prepare_inputs_for_generation(input_ids, attention_mask=None, use_cache=True, cache=None, **kwargs)[source]

Prepare inputs for decoder to generate sentences. :returns: A dictionary containing necessary inputs for generating next token. :rtype: dict