tools

static_params_to_dygraph(model, static_tensor_dict)

Simple tool for convert static paramters to dygraph paramters dict.

NOTE The model must both support static graph and dygraph mode.

Parameters

• model (nn.Layer) – the model of a neural network.

• static_tensor_dict (string) – path of which locate the saved paramters in static mode. Usualy load by paddle.static.load_program_state.

Returns

a state dict the same as the dygraph mode.

Return type

[tensor dict]

dygraph_params_to_static(model, dygraph_tensor_dict, topo=None)

Simple tool for convert dygraph paramters to static paramters dict.

NOTE The model must both support static graph and dygraph mode.

Parameters

• model (nn.Layer) – the model of a neural network.

• dygraph_tensor_dict (string) – path of which locate the saved paramters in static mode.

Returns

a state dict the same as the dygraph mode.

Return type

[tensor dict]

class TimeCostAverage

Bases: object

Simple tool for calcluating time average cost in the process of training and inferencing.

reset()

Reset the recoder state, and reset the cnt to zero.

record(usetime)

Recoding the time cost in current step and accumulating the cnt.

get_average()

Returning the average time cost after the start of training.

get_env_device()

Return the device name of running enviroment.

compare_version(version, pair_version)

Parameters

• version (str) – The first version string needed to be compared. The format of version string should be as follow : “xxx.yyy.zzz”.

• pair_version (str) – The second version string needed to be compared. The format of version string should be as follow : “xxx.yyy.zzz”.

Returns

The result of comparasion. 1 means version > pair_version; 0 means

version = pair_version; -1 means version < pair_version.

Return type

int

Examples

>>> compare_version("2.2.1", "2.2.0")
>>> 1
>>> compare_version("2.2.0", "2.2.0")
>>> 0
>>> compare_version("2.2.0-rc0", "2.2.0")
>>> -1
>>> compare_version("2.3.0-rc0", "2.2.0")
>>> 1

get_bool_ids_greater_than(probs, limit=0.5, return_prob=False)

Get idx of the last dimension in probability arrays, which is greater than a limitation.

Parameters

• probs (List[List[float]]) – The input probability arrays.

• limit (float) – The limitation for probability.

• return_prob (bool) – Whether to return the probability

Returns

The index of the last dimension meet the conditions.

Return type

List[List[int]]

get_span(start_ids, end_ids, with_prob=False)

Get span set from position start and end list.

Parameters

• start_ids (List[int]/List[tuple]) – The start index list.

• end_ids (List[int]/List[tuple]) – The end index list.

• with_prob (bool) – If True, each element for start_ids and end_ids is a tuple aslike: (index, probability).

Returns

The span set without overlapping, every id can only be used once .

Return type

set

class DataConverter(label_studio_file, negative_ratio=5, prompt_prefix='情感倾向', options=['正向', '负向'], separator='##', layout_analysis=False, expand_to_a4_size=True, schema_lang='ch', ocr_lang='en', anno_type='text')

Bases: object

DataConverter to convert data export from annotation platform

convert_cls_examples(raw_examples)

Convert labeled data for classification task.

convert_ext_examples(raw_examples, is_train=True)

Convert labeled data for extraction task.