rouge¶

class RougeL(trans_func=None, vocab=None, gamma=1.2, name='rouge-l', *args, **kwargs)[source]¶

Bases: paddle.metric.metrics.Metric

Rouge-L is Recall-Oriented Understudy for Gisting Evaluation based on Longest Common Subsequence (LCS). Longest common subsequence problem takes into account sentence level structure similarity naturally and identifies longest co-occurring in sequence n-grams automatically.

\[ \begin{align}\begin{aligned}R_{LCS} & = \frac{LCS(C,S)}{len(S)}\\P_{LCS} & = \frac{LCS(C,S)}{len(C)}\\F_{LCS} & = \frac{(1 + \gamma^2)R_{LCS}P_{LCS}}{R_{LCS}} + \gamma^2{R_{LCS}}\end{aligned}\end{align} \]

where C is the candidate sentence, and S is the reference sentence.

Parameters

trans_func (callable, optional) – trans_func transforms the network output to string to calculate.
vocab (dict|paddlenlp.data.vocab, optional) – Vocab for target language. If trans_func is None and RougeL is used as paddle.metric.Metric instance, default_trans_func will be performed and vocab must be provided.
gamma (float) – A hyperparameter to decide the weight of recall. Defaults to 1.2.
name (str, optional) – Name of paddle.metric.Metric instance. Defaults to “rouge-l”.

Examples

from paddlenlp.metrics import RougeL
rougel = RougeL()
cand = ["The","cat","The","cat","on","the","mat"]
ref_list = [["The","cat","is","on","the","mat"], ["There","is","a","cat","on","the","mat"]]
rougel.add_inst(cand, ref_list)
print(rougel.score()) # 0.7800511508951408

lcs(string, sub)[source]¶

Calculate the length of longest common subsequence of string and sub.

Parameters

string (str) – The string to be calculated, usually longer the sub string.
sub (str) – The sub string to be calculated.

Returns

Returns the length of the longest common subsequence of string and sub.

Return type

float

add_inst(cand, ref_list)[source]¶

Update the states based on the a pair of candidate and references.

Parameters

cand (str) – The candidate sentence generated by model.
ref_list (list) – List of ground truth sentences.

update(output, label, seq_mask=None)[source]¶

Update states for metric

Inputs of update is the outputs of Metric.compute, if compute is not defined, the inputs of update will be flatten arguments of output of mode and label from data: update(output1, output2, ..., label1, label2,...)

see Metric.compute

accumulate()[source]¶: Calculate the final rouge-l metric.

reset()[source]¶: Reset states and result

name()[source]¶: Returns metric name

class RougeLForDuReader(alpha=1.0, beta=1.0, gamma=1.2)[source]¶

Bases: paddlenlp.metrics.rouge.RougeL

Rouge-L metric with bonus for DuReader contest.

Please refer to `DuReader Homepage<https://ai.baidu.com//broad/subordinate?dataset=dureader>`_ for more details.

Parameters

alpha (float, optional) – Weight of YesNo dataset when adding bonus for DuReader contest. Defaults to 1.0.
beta (float, optional) – Weight of Entity dataset when adding bonus for DuReader contest. Defaults to 1.0.

add_inst(cand, ref_list, yn_label=None, yn_ref=None, entity_ref=None)[source]¶

Update the states based on the a pair of candidate and references.

Parameters

cand (str) – The candidate sentence generated by model.
ref_list (list) – List of ground truth sentences.