helper

paddle_pad_and_concatenate(tensor1, tensor2, padding_index=-100)

Concatenates tensor1 and tensor2 on first axis, applying padding on the second if necessary.

nested_concat(tensors, new_tensors, padding_index=-100)

Concat the new_tensors to tensors on the first dim and pad them on the second if needed. Works for tensors or nested list/tuples of tensors.

nested_detach(tensors)

Detach tensors (even if it’s a nested list/tuple of tensors).

nested_numpify(tensors)

Numpify tensors (even if it’s a nested list/tuple of tensors).

nested_truncate(tensors, limit)

Truncate tensors at limit (even if it’s a nested list/tuple of tensors).