glue¶
-
class
AccuracyAndF1
(topk=(1), pos_label=1, name='acc_and_f1', *args, **kwargs)[源代码]¶ 基类:
paddle.metric.metrics.Metric
This class encapsulates Accuracy, Precision, Recall and F1 metric logic, and
accumulate
function returns accuracy, precision, recall and f1. The overview of all metrics could be seen at the document of paddle.metric for details.- 参数
topk (int or tuple(int), optional) -- Number of top elements to look at for computing accuracy. Defaults to (1,).
pos_label (int, optional) -- The positive label for calculating precision and recall. Defaults to 1.
name (str, optional) -- String name of the metric instance. Defaults to 'acc_and_f1'.
示例
import paddle from paddlenlp.metrics import AccuracyAndF1 x = paddle.to_tensor([[0.1, 0.9], [0.5, 0.5], [0.6, 0.4], [0.7, 0.3]]) y = paddle.to_tensor([[1], [0], [1], [1]]) m = AccuracyAndF1() correct = m.compute(x, y) m.update(correct) res = m.accumulate() print(res) # (0.5, 0.5, 0.3333333333333333, 0.4, 0.45)
-
compute
(pred, label, *args)[源代码]¶ Accepts network's output and the labels, and calculates the top-k (maximum value in topk) indices for accuracy.
- 参数
pred (Tensor) -- Predicted tensor, and its dtype is float32 or float64, and has a shape of [batch_size, num_classes].
label (Tensor) -- The ground truth tensor, and its dtype is is int64, and has a shape of [batch_size, 1] or [batch_size, num_classes] in one hot representation.
- 返回
Correct mask, each element indicates whether the prediction equals to the label. Its' a tensor with a data type of float32 and has a shape of [batch_size, topk].
- 返回类型
Tensor
-
update
(correct, *args)[源代码]¶ Updates the metrics states (accuracy, precision and recall), in order to calculate accumulated accuracy, precision and recall of all instances.
- 参数
correct (Tensor) -- Correct mask for calculating accuracy, and it's a tensor with shape [batch_size, topk] and has a dtype of float32.
-
accumulate
()[源代码]¶ Calculates and returns the accumulated metric.
- 返回
The accumulated metric. A tuple of shape (acc, precision, recall, f1, average_of_acc_and_f1)
With the fields:
acc
(numpy.float64):The accumulated accuracy.
precision
(numpy.float64):The accumulated precision.
recall
(numpy.float64):The accumulated recall.
f1
(numpy.float64):The accumulated f1.
average_of_acc_and_f1
(numpy.float64):The average of accumulated accuracy and f1.
- 返回类型
tuple
-
class
Mcc
(name='mcc', *args, **kwargs)[源代码]¶ 基类:
paddle.metric.metrics.Metric
This class calculates Matthews correlation coefficient .
- 参数
name (str, optional) -- String name of the metric instance. Defaults to 'mcc'.
示例
import paddle from paddlenlp.metrics import Mcc x = paddle.to_tensor([[-0.1, 0.12], [-0.23, 0.23], [-0.32, 0.21], [-0.13, 0.23]]) y = paddle.to_tensor([[1], [0], [1], [1]]) m = Mcc() (preds, label) = m.compute(x, y) m.update((preds, label)) res = m.accumulate() print(res) # (0.0,)
-
compute
(pred, label, *args)[源代码]¶ Processes the pred tensor, and returns the indices of the maximum of each sample.
- 参数
pred (Tensor) -- The predicted value is a Tensor with dtype float32 or float64. Shape is [batch_size, 1].
label (Tensor) -- The ground truth value is Tensor with dtype int64, and its shape is [batch_size, 1].
- 返回
A tuple of preds and label. Each shape is [batch_size, 1], with dtype float32 or float64.
- 返回类型
tuple
-
update
(preds_and_labels)[源代码]¶ Calculates states, i.e. the number of true positive, false positive, true negative and false negative samples.
- 参数
preds_and_labels (tuple[Tensor]) -- Tuple of predicted value and the ground truth label, with dtype float32 or float64. Each shape is [batch_size, 1].
-
class
PearsonAndSpearman
(name='pearson_and_spearman', *args, **kwargs)[源代码]¶ 基类:
paddle.metric.metrics.Metric
The class calculates Pearson correlation coefficient and Spearman's rank correlation coefficient .
- 参数
name (str, optional) -- String name of the metric instance. Defaults to 'pearson_and_spearman'.
示例
import paddle from paddlenlp.metrics import PearsonAndSpearman x = paddle.to_tensor([[0.1], [1.0], [2.4], [0.9]]) y = paddle.to_tensor([[0.0], [1.0], [2.9], [1.0]]) m = PearsonAndSpearman() m.update((x, y)) res = m.accumulate() print(res) # (0.9985229081857804, 1.0, 0.9992614540928901)
-
update
(preds_and_labels)[源代码]¶ Ensures the type of preds and labels is numpy.ndarray and reshapes them into [-1, 1].
- 参数
preds_and_labels (tuple[Tensor] or list[Tensor]) -- Tuple or list of predicted value and the ground truth label. Its data type should be float32 or float64 and its shape is [batch_size, d0, ..., dN].
-
accumulate
()[源代码]¶ Calculates and returns the accumulated metric.
- 返回
Returns the accumulated metric, a tuple of (pearson, spearman, the_average_of_pearson_and_spearman).
With the fields:
pearson
(numpy.float64):The accumulated pearson.
spearman
(numpy.float64):The accumulated spearman.
the_average_of_pearson_and_spearman
(numpy.float64):The average of accumulated pearson and spearman correlation coefficient.
- 返回类型
tuple
-
class
MultiLabelsMetric
(num_labels, name='multi_labels_metric')[源代码]¶ 基类:
paddle.metric.metrics.Metric
This class encapsulates Accuracy, Precision, Recall and F1 metric logic in multi-labels setting (also the binary setting). Some codes are taken and modified from sklearn.metrics .
- 参数
num_labels (int) -- The total number of labels which is usually the number of classes
name (str, optional) -- String name of the metric instance. Defaults to 'multi_labels_metric'.
示例
import paddle from paddlenlp.metrics import MultiLabelsMetric x = paddle.to_tensor([[0.1, 0.2, 0.9], [0.5, 0.8, 0.5], [0.6, 1.5, 0.4], [2.8, 0.7, 0.3]]) y = paddle.to_tensor([[2], [1], [2], [1]]) m = MultiLabelsMetric(num_labels=3) args = m.compute(x, y) m.update(args) result1 = m.accumulate(average=None) # (array([0.0, 0.5, 1.0]), array([0.0, 0.5, 0.5]), array([0.0, 0.5, 0.66666667])) result2 = m.accumulate(average='binary', pos_label=0) # (0.0, 0.0, 0.0) result3 = m.accumulate(average='binary', pos_label=1) # (0.5, 0.5, 0.5) result4 = m.accumulate(average='binary', pos_label=2) # (1.0, 0.5, 0.6666666666666666) result5 = m.accumulate(average='micro') # (0.5, 0.5, 0.5) result6 = m.accumulate(average='macro') # (0.5, 0.3333333333333333, 0.38888888888888884) result7 = m.accumulate(average='weighted') # (0.75, 0.5, 0.5833333333333333)
- Note: When zero_division is encountered (details as followed), the corresponding metrics will be set to 0.0
precision is zero_division if there are no positive predictions recall is zero_division if there are no positive labels fscore is zero_division if all labels AND predictions are negative
-
update
(args)[源代码]¶ Updates the metrics states (accuracy, precision and recall), in order to calculate accumulated accuracy, precision and recall of all instances.
- 参数
args (tuple of Tensor) -- the tuple returned from
compute
function
-
accumulate
(average=None, pos_label=1)[源代码]¶ Calculates and returns the accumulated metric.
- 参数
average (str in {‘binary’, ‘micro’, ‘macro’, ’weighted’} or None, optional) --
to None. If None (Defaults) --
scores for each class are returned. (the) --
Otherwise --
determines the type of averaging performed on the data (this) --
binary (-) -- Only report results for the class specified by pos_label.
micro (-) -- Calculate metrics globally by counting the total true positives, false negatives and false positives.
macro (-) -- Calculate metrics for each label, and find their unweighted mean. This does not take label imbalance into account.
weighted (-) -- Calculate metrics for each label, and find their average weighted by support (the number of true instances for each label). This alters
macro
to account for label imbalance; it can result in an F-score that is not between precision and recall.pos_label (int, optional) -- The positive label for calculating precision and recall in binary settings. Noted: Only when
average='binary'
, this arguments will be used. Otherwise, it will be ignored. Defaults to 1.
- 返回
- The accumulated metric. A tuple of shape (precision, recall, f1)
With the fields:
precision
(numpy.float64 or numpy.ndarray if average=None):The accumulated precision.
recall
(numpy.float64 or numpy.ndarray if average=None):The accumulated recall.
f1
(numpy.float64 or numpy.ndarray if average=None):The accumulated f1.
- 返回类型
tuple