modeling#
Modeling classes for FNet model.
- class FNetPretrainedModel(*args, **kwargs)[源代码]#
-
An abstract class for pretrained FNet models. It provides FNet related
model_config_file
,pretrained_init_configuration
,resource_files_names
,pretrained_resource_files_map
,base_model_prefix
for downloading and loading pretrained models. SeePretrainedModel
for more details.- config_class#
FNetConfig
的别名
- class FNetModel(config: FNetConfig)[源代码]#
-
The model can behave as an encoder, following the architecture described in FNet: Mixing Tokens with Fourier Transforms by James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, Santiago Ontanon.
- set_input_embeddings(value)[源代码]#
set new input embedding for model
- 参数:
value (Embedding) -- the new embedding of model
- 抛出:
NotImplementedError -- Model has not implement
set_input_embeddings
method
- forward(input_ids=None, token_type_ids=None, position_ids=None, inputs_embeds=None, output_hidden_states=None, return_dict=None)[源代码]#
The FNetModel forward method.
- 参数:
input_ids (Tensor) -- Indices of input sequence tokens in the vocabulary. They are numerical representations of tokens that build the input sequence. Its data type should be
int64
and it has a shape of [batch_size, sequence_length].token_type_ids (Tensor, optional) --
Segment token indices to indicate different portions of the inputs. Selected in the range
[0, type_vocab_size - 1]
. Iftype_vocab_size
is 2, which means the inputs have two portions. Indices can either be 0 or 1:0 corresponds to a sentence A token,
1 corresponds to a sentence B token.
Its data type should be
int64
and it has a shape of [batch_size, sequence_length]. Defaults toNone
, which means we don't add segment embeddings.position_ids (Tensor, optional) -- Indices of positions of each input sequence tokens in the position embeddings. Selected in the range
[0, max_position_embeddings - 1]
. Shape as(batch_size, num_tokens)
and dtype as int64. Defaults toNone
.inputs_embeds -- If you want to control how to convert
inputs_ids
indices into associated vectors, you can pass an embedded representation directly instead of passinginputs_ids
.
- 返回:
Returns tuple (
sequence_output
,pooled_output
,encoder_outputs[1:]
) or a dict with last_hidden_state`,pooled_output
,all_hidden_states
, fields.With the fields:
sequence_output
(Tensor):Sequence of hidden-states at the last layer of the model. It's data type should be float32 and has a shape of [
batch_size, sequence_length, hidden_size
].
pooled_output
(Tensor):The output of first token (
[CLS]
) in sequence. We "pool" the model by simply taking the hidden state corresponding to the first token. Its data type should be float32 and has a shape of [batch_size, hidden_size].
last_hidden_state
(Tensor):The output of the last encoder layer, it is also the
sequence_output
. It's data type should be float32 and has a shape of [batch_size, sequence_length, hidden_size].
all_hidden_states
(Tensor):Hidden_states of all layers in the Transformer encoder. The length of
all_hidden_states
isnum_hidden_layers + 1
. For all element in the tuple, its data type should be float32 and its shape is [batch_size, sequence_length, hidden_size
].
- 返回类型:
tuple or Dict
示例
import paddle from paddlenlp.transformers.fnet.modeling import FNetModel from paddlenlp.transformers.fnet.tokenizer import FNetTokenizer tokenizer = FNetTokenizer.from_pretrained('fnet-base') model = FNetModel.from_pretrained('fnet-base') inputs = tokenizer("Welcome to use PaddlePaddle and PaddleNLP!") inputs = {k:paddle.to_tensor([v]) for (k, v) in inputs.items()} output = model(**inputs)
- class FNetForSequenceClassification(config: FNetConfig, num_classes=2)[源代码]#
-
FNet Model with a linear layer on top of the output layer, designed for sequence classification/regression tasks like GLUE tasks.
- 参数:
fnet (
FNetModel
) -- An instance of FNetModel.num_classes (int, optional) -- The number of classes. Defaults to
2
.
- forward(input_ids=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, output_hidden_states=None, return_dict=None)[源代码]#
The FNetForSequenceClassification forward method.
- 参数:
input_ids (Tensor) -- Indices of input sequence tokens in the vocabulary. They are numerical representations of tokens that build the input sequence. Its data type should be
int64
and it has a shape of [batch_size, sequence_length].token_type_ids (Tensor, optional) --
Segment token indices to indicate different portions of the inputs. Selected in the range
[0, type_vocab_size - 1]
. Iftype_vocab_size
is 2, which means the inputs have two portions. Indices can either be 0 or 1:0 corresponds to a sentence A token,
1 corresponds to a sentence B token.
Its data type should be
int64
and it has a shape of [batch_size, sequence_length]. Defaults toNone
, which means we don't add segment embeddings.position_ids (Tensor, optional) -- Indices of positions of each input sequence tokens in the position embeddings. Selected in the range
[0, max_position_embeddings - 1]
. Shape as(batch_size, num_tokens)
and dtype as int64. Defaults toNone
.inputs_embeds -- If you want to control how to convert
inputs_ids
indices into associated vectors, you can pass an embedded representation directly instead of passinginputs_ids
.
- 返回:
Returns tensor
logits
, or a dict withlogits
,hidden_states
,attentions
fields.With the fields:
logits
(Tensor):A tensor of the input text classification logits. Shape as
[batch_size, num_classes]
and dtype as float32.
hidden_states
(Tensor):Hidden_states of all layers in the Transformer encoder. The length of
hidden_states
isnum_hidden_layers + 1
. For all element in the tuple, its data type should be float32 and its shape is [batch_size, sequence_length, hidden_size
].
- 返回类型:
Tensor or Dict
示例
import paddle from paddlenlp.transformers.fnet.modeling import FNetForSequenceClassification from paddlenlp.transformers.fnet.tokenizer import FNetTokenizer tokenizer = FNetTokenizer.from_pretrained('fnet-base') model = FNetModel.from_pretrained('fnet-base') inputs = tokenizer("Welcome to use PaddlePaddle and PaddleNLP!") inputs = {k:paddle.to_tensor([v]) for (k, v) in inputs.items()} output = model(**inputs)
- class FNetForPreTraining(config: FNetConfig)[源代码]#
-
FNet Model with two heads on top as done during the pretraining: a
masked language modeling
head and anext sentence prediction (classification)
head.- get_output_embeddings()[源代码]#
To be overwrited for models with output embeddings
- 返回:
the otuput embedding of model
- 返回类型:
Optional[Embedding]
- forward(input_ids=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, next_sentence_label=None, output_hidden_states=None, return_dict=None)[源代码]#
The FNetForPretraining forward method.
- 参数:
input_ids (Tensor) -- See
FNetModel
.token_type_ids (Tensor, optional) -- See
FNetModel
.position_ids (Tensor, optional) -- See
FNetModel
.labels (LongTensor of shape (batch_size, sequence_length), optional) -- Labels for computing the masked language modeling loss.
inputs_embeds (Tensor, optional) -- See
FNetModel
.next_sentence_labels (Tensor) -- The labels of the next sentence prediction task, the dimensionality of
next_sentence_labels
is equal toseq_relation_labels
. Its data type should be int64 and its shape is [batch_size, 1]output_hidden_states (bool, optional) -- See
FNetModel
.return_dict (bool, optional) -- See
FNetModel
.
- 返回:
Returns tuple (
prediction_scores
,seq_relationship_score
) or a dict withprediction_logits
,seq_relationship_logits
,hidden_states
fields.- 返回类型:
tuple or Dict
- class FNetForMaskedLM(config: FNetConfig)[源代码]#
-
FNet Model with a
masked language modeling
head on top.- get_output_embeddings()[源代码]#
To be overwrited for models with output embeddings
- 返回:
the otuput embedding of model
- 返回类型:
Optional[Embedding]
- forward(input_ids=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, next_sentence_label=None, output_hidden_states=None, return_dict=None)[源代码]#
The FNetForMaskedLM forward method.
- 参数:
input_ids (Tensor) -- See
FNetModel
.token_type_ids (Tensor, optional) -- See
FNetModel
.position_ids (Tensor, optional) -- See
FNetModel
.inputs_embeds (Tensor, optional) -- See
FNetModel
.labels (Tensor, optional) -- See
FNetForPreTraining
.next_sentence_label (Tensor, optional) -- See
FNetForPreTraining
.output_hidden_states (Tensor, optional) -- See
FNetModel
.return_dict (bool, optional) -- See
FNetModel
.
- 返回:
Returns tensor
prediction_scores
or a dict withprediction_logits
,hidden_states
fields.With the fields:
prediction_scores
(Tensor):The scores of masked token prediction. Its data type should be float32. and its shape is [batch_size, sequence_length, vocab_size].
hidden_states
(Tensor):Hidden_states of all layers in the Transformer encoder. The length of
hidden_states
isnum_hidden_layers + 1
. For all element in the tuple, its data type should be float32 and its shape is [batch_size, sequence_length, hidden_size
].
- 返回类型:
Tensor or Dict
- class FNetForNextSentencePrediction(config: FNetConfig)[源代码]#
-
FNet Model with a
next sentence prediction
head on top.- get_output_embeddings()[源代码]#
To be overwrited for models with output embeddings
- 返回:
the otuput embedding of model
- 返回类型:
Optional[Embedding]
- forward(input_ids=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, next_sentence_label=None, output_hidden_states=None, return_dict=None)[源代码]#
Defines the computation performed at every call. Should be overridden by all subclasses.
- 参数:
*inputs (tuple) -- unpacked tuple arguments
**kwargs (dict) -- unpacked dict arguments
- class FNetForMultipleChoice(config: FNetConfig)[源代码]#
-
FNet Model with a linear layer on top of the hidden-states output layer, designed for multiple choice tasks like SWAG tasks .
- 参数:
fnet (
FNetModel
) -- An instance of FNetModel.
- forward(input_ids=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, output_hidden_states=None, return_dict=None)[源代码]#
Defines the computation performed at every call. Should be overridden by all subclasses.
- 参数:
*inputs (tuple) -- unpacked tuple arguments
**kwargs (dict) -- unpacked dict arguments
- class FNetForTokenClassification(config: FNetConfig, num_classes=2)[源代码]#
-
FNet Model with a linear layer on top of the hidden-states output layer, designed for token classification tasks like NER tasks.
- 参数:
fnet (
FNetModel
) -- An instance of FNetModel.num_classes (int, optional) -- The number of classes. Defaults to
2
.
- forward(input_ids=None, token_type_ids=None, position_ids=None, inputs_embeds=None, labels=None, output_hidden_states=None, return_dict=None)[源代码]#
Defines the computation performed at every call. Should be overridden by all subclasses.
- 参数:
*inputs (tuple) -- unpacked tuple arguments
**kwargs (dict) -- unpacked dict arguments
- class FNetForQuestionAnswering(config: FNetConfig, num_labels)[源代码]#
-
FNet Model with a linear layer on top of the hidden-states output to compute
span_start_logits
andspan_end_logits
, designed for question-answering tasks like SQuAD.- 参数:
fnet (
FNetModel
) -- An instance of FNetModel.num_labels (int) -- The number of labels.
- forward(input_ids=None, token_type_ids=None, position_ids=None, inputs_embeds=None, start_positions=None, end_positions=None, output_hidden_states=None, return_dict=None)[源代码]#
Defines the computation performed at every call. Should be overridden by all subclasses.
- 参数:
*inputs (tuple) -- unpacked tuple arguments
**kwargs (dict) -- unpacked dict arguments