千言：面向事实一致性的生成评测比赛基线

比赛简介

自然语言生成旨在让机器能够像人一样使用自然语言进行表达和交互，它是人工智能领域重要的前沿课题，近年来受到学术界和工业界广泛关注。

随着神经网络生成模型特别是预训练语言模型的迅速发展，机器生成文本的可读性和流畅性不断提升。然而，自动生成的文本中依然经常出现不符合原文或背景的错误事实描述，这种生成的事实一致性问题是自然语言生成进行落地应用的主要障碍之一，并逐渐受到研究学者的关注。鉴于当前国内外关于事实一致性的生成评测比赛十分匮乏，为了促进自然语言生成的技术发展和实际应用，我们计划组织面向事实一致性的生成评测比赛。

在此比赛中，我们将提供三个对事实一致性有较高要求的生成任务，包括文案生成、摘要生成和问题生成。同时，在系统评价中，我们将结合文本流畅性和事实一致性两项指标综合评估参赛生成系统的水平。通过这样的任务设定和评价方式，此评测将有助于研究者和开发者更多关注自然语言生成的事实一致性难题，并为大家提供学术交流平台，从而进一步提升自然语言生成的研究水平，推动相关技术的应用发展。

本比赛得到中国中文信息学会自然语言生成专业委员会（筹）支持，将在2021年11月7日首届中国自然语言生成大会（CCNLG-2021）召开评测研讨会，并在大会上对获奖团队颁奖。

本示例以比赛中的问题生成任务为例，展示如何使用PaddleNLP和UNIMO-text快速实现比赛基线。

安装说明

• PaddlePaddle 安装

  本项目依赖于 PaddlePaddle 2.0 及以上版本，请参考 安装指南 进行安装

• PaddleNLP 安装

  pip install --upgrade paddlenlp -i https://pypi.org/simple
  

• 环境依赖

  Python的版本要求 3.6+

AI Studio平台默认安装了Paddle和PaddleNLP，并定期更新版本。 如需手动更新Paddle，可参考飞桨安装说明，安装相应环境下最新版飞桨框架。

使用如下命令确保安装最新版PaddleNLP：

!pip install --upgrade paddlenlp -i https://pypi.org/simple

Requirement already up-to-date: paddlenlp in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (2.1.1)
Requirement already satisfied, skipping upgrade: multiprocess in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.70.11.1)
Requirement already satisfied, skipping upgrade: colorlog in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (4.1.0)
Requirement already satisfied, skipping upgrade: paddlefsl==1.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (1.0.0)
Requirement already satisfied, skipping upgrade: h5py in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (2.9.0)
Requirement already satisfied, skipping upgrade: colorama in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.4.4)
Requirement already satisfied, skipping upgrade: jieba in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (0.42.1)
Requirement already satisfied, skipping upgrade: seqeval in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlenlp) (1.2.2)
Requirement already satisfied, skipping upgrade: dill>=0.3.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from multiprocess->paddlenlp) (0.3.3)
Requirement already satisfied, skipping upgrade: pillow==8.2.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlefsl==1.0.0->paddlenlp) (8.2.0)
Collecting numpy~=1.19.2 (from paddlefsl==1.0.0->paddlenlp)
[?25l  Downloading https://files.pythonhosted.org/packages/08/d6/a6aaa29fea945bc6c61d11f6e0697b325ff7446de5ffd62c2fa02f627048/numpy-1.19.5-cp37-cp37m-manylinux2010_x86_64.whl (14.8MB)
     |████████████████████████████████| 14.8MB 5.0MB/s eta 0:00:01
[?25hRequirement already satisfied, skipping upgrade: tqdm~=4.27.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlefsl==1.0.0->paddlenlp) (4.27.0)
Requirement already satisfied, skipping upgrade: requests~=2.24.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from paddlefsl==1.0.0->paddlenlp) (2.24.0)
Requirement already satisfied, skipping upgrade: six in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from h5py->paddlenlp) (1.15.0)
Requirement already satisfied, skipping upgrade: scikit-learn>=0.21.3 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from seqeval->paddlenlp) (0.24.2)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests~=2.24.0->paddlefsl==1.0.0->paddlenlp) (2.8)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests~=2.24.0->paddlefsl==1.0.0->paddlenlp) (2019.9.11)
Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests~=2.24.0->paddlefsl==1.0.0->paddlenlp) (1.25.6)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from requests~=2.24.0->paddlefsl==1.0.0->paddlenlp) (3.0.4)
Requirement already satisfied, skipping upgrade: threadpoolctl>=2.0.0 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (2.1.0)
Requirement already satisfied, skipping upgrade: scipy>=0.19.1 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (1.6.3)
Requirement already satisfied, skipping upgrade: joblib>=0.11 in /opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages (from scikit-learn>=0.21.3->seqeval->paddlenlp) (0.14.1)
Installing collected packages: numpy
  Found existing installation: numpy 1.20.3
    Uninstalling numpy-1.20.3:
      Successfully uninstalled numpy-1.20.3
Successfully installed numpy-1.19.5

示例流程

与大多数NLP任务相同，本次生成任务的示例展示分为以下四步：

首先我们从数据准备开始。

数据准备

数据准备流程如下：

1. 加载PaddleNLP内置数据集

使用PaddleNLP提供的load_datasetAPI，即可一键完成数据集加载。

from paddlenlp.datasets import load_dataset

train_ds, dev_ds = load_dataset('dureader_qg', splits=('train', 'dev'))

for idx in range(2):
    print(train_ds[idx]['source'])
    print(train_ds[idx]['title'])
    print(train_ds[idx]['target'])
    print()

第35集雪见缓缓张开眼睛，景天又惊又喜之际，长卿和紫萱的仙船驶至，见众人无恙，也十分高兴。众人登船，用尽合力把自身的真气和水分输给她。雪见终于醒过来了，但却一脸木然，全无反应。众人向常胤求助，却发现人世界竟没有雪见的身世纪录。长卿询问清微的身世，清微语带双关说一切上了天界便有答案。长卿驾驶仙船，众人决定立马动身，往天界而去。众人来到一荒山，长卿指出，魔界和天界相连。由魔界进入通过神魔之井，便可登天。众人至魔界入口，仿若一黑色的蝙蝠洞，但始终无法进入。后来花楹发现只要有翅膀便能飞入。于是景天等人打下许多乌鸦，模仿重楼的翅膀，制作数对翅膀状巨物。刚佩戴在身，便被吸入洞口。众人摔落在地，抬头发现魔界守卫。景天和众魔套交情，自称和魔尊重楼相熟，众魔不理，打了起来。
第35集
仙剑奇侠传3第几集上天界

选择燃气热水器时，一定要关注这几个问题：1、出水稳定性要好，不能出现忽热忽冷的现象2、快速到达设定的需求水温3、操作要智能、方便4、安全性要好，要装有安全报警装置 市场上燃气热水器品牌众多，购买时还需多加对比和仔细鉴别。方太今年主打的磁化恒温热水器在使用体验方面做了全面升级：9秒速热，可快速进入洗浴模式；水温持久稳定，不会出现忽热忽冷的现象，并通过水量伺服技术将出水温度精确控制在±0.5℃，可满足家里宝贝敏感肌肤洗护需求；配备CO和CH4双气体报警装置更安全（市场上一般多为CO单气体报警）。另外，这款热水器还有智能WIFI互联功能，只需下载个手机APP即可用手机远程操作热水器，实现精准调节水温，满足家人多样化的洗浴需求。当然方太的磁化恒温系列主要的是增加磁化功能，可以有效吸附水中的铁锈、铁屑等微小杂质，防止细菌滋生，使沐浴水质更洁净，长期使用磁化水沐浴更利于身体健康。
方太
燃气热水器哪个牌子好

2. 加载 paddlenlp.transformers.UNIMOTokenizer用于数据处理

文本数据在输入unimo-text预训练模型之前，需要经过数据处理转化为Feature。这一过程通常包括分词，token to id，add special token等步骤。

PaddleNLP对于各种预训练模型已经内置了相应的tokenizer，指定想要使用的模型名字即可加载对应的tokenizer。

可以通过调用tokenizer中的方法简单的完成上述数据处理。

import paddlenlp

# 设置模型名称
MODEL_NAME = 'unimo-text-1.0'
tokenizer = paddlenlp.transformers.UNIMOTokenizer.from_pretrained(MODEL_NAME)

[2021-10-23 09:45:26,651] [    INFO] - Already cached /home/aistudio/.paddlenlp/models/unimo-text-1.0/unimo-text-1.0-vocab.txt

3. 调用map()方法批量处理数据

由于我们传入了lazy=False，所以我们使用load_dataset()自定义的数据集是MapDataset对象。

MapDataset是paddle.io.Dataset的功能增强版本。其内置的map()方法适合用来进行批量数据集处理。

map()方法传入的是一个用于数据处理的function。正好可以与tokenizer相配合。

以下是本示例中的用法：

from functools import partial

def convert_example(example,
                    tokenizer,
                    max_seq_len=512,
                    max_target_len=128,
                    max_title_len=256,
                    mode='train'):
    """Convert all examples into necessary features."""
    source = example['source']
    title = None
    if 'title' in example.keys():
        title = example['title']

    if mode != 'test':
        tokenized_example = tokenizer.gen_encode(
            source,
            title=title,
            target=example['target'],
            max_seq_len=max_seq_len,
            max_target_len=max_target_len,
            max_title_len=max_title_len,
            return_position_ids=True,
            return_length=True)
        target_start = tokenized_example['input_ids'].index(
            tokenizer.cls_token_id, 1)
        target_end = tokenized_example['seq_len']
        # Use to gather the logits corresponding to the labels during training
        tokenized_example['masked_positions'] = list(
            range(target_start, target_end - 1))
        tokenized_example['labels'] = tokenized_example['input_ids'][
            target_start + 1:target_end]

        return tokenized_example
    else:
        tokenized_example = tokenizer.gen_encode(
            source,
            title=title,
            max_seq_len=max_seq_len,
            max_title_len=max_title_len,
            add_start_token_for_decoding=True,
            return_position_ids=True)

        if 'target' in example and example['target']:
            tokenized_example['target'] = example['target']
        return tokenized_example

train_trans_func = partial(
        convert_example,
        tokenizer=tokenizer,
        max_seq_len=360,
        max_target_len=30,
        max_title_len=30,
        mode='train')

train_ds = train_ds.map(train_trans_func, lazy=False)

dev_trans_func = partial(
        convert_example,
        tokenizer=tokenizer,
        max_seq_len=360,
        max_target_len=30,
        max_title_len=30,
        mode='test')
        
dev_ds = dev_ds.map(dev_trans_func, lazy=False)

for idx in range(2):
    print(train_ds[idx]['input_ids'])
    print(train_ds[idx]['token_type_ids'])
    print(train_ds[idx]['position_ids'])
    print(train_ds[idx]['masked_positions'])
    print(train_ds[idx]['labels'])
    print()

[1, 131, 1773, 271, 2, 131, 1773, 271, 1097, 373, 1427, 1427, 501, 88, 662, 1906, 4, 561, 125, 311, 1168, 311, 692, 46, 430, 4, 84, 2073, 14, 1264, 3967, 5, 1034, 1020, 1829, 268, 4, 373, 539, 8, 154, 5210, 4, 105, 167, 59, 69, 685, 12043, 539, 8, 883, 1020, 4, 29, 720, 95, 90, 427, 67, 262, 5, 384, 266, 14, 101, 59, 789, 416, 237, 12043, 1097, 373, 616, 37, 1519, 93, 61, 15, 4, 255, 535, 7, 1529, 619, 187, 4, 62, 154, 451, 149, 12043, 539, 8, 253, 223, 3679, 323, 523, 4, 535, 34, 87, 8, 203, 280, 1186, 340, 9, 1097, 373, 5, 262, 203, 623, 704, 12043, 84, 2073, 1137, 358, 334, 702, 5, 262, 203, 4, 334, 702, 405, 360, 653, 129, 178, 7, 568, 28, 15, 125, 280, 518, 9, 1179, 487, 12043, 84, 2073, 1621, 1829, 1034, 1020, 4, 539, 8, 448, 91, 202, 466, 70, 262, 4, 638, 125, 280, 83, 299, 12043, 539, 8, 61, 45, 7, 1537, 176, 4, 84, 2073, 288, 39, 4, 889, 280, 14, 125, 280, 156, 538, 12043, 190, 889, 280, 71, 109, 124, 93, 292, 889, 46, 1248, 4, 518, 48, 883, 125, 12043, 539, 8, 268, 889, 280, 109, 270, 4, 1586, 845, 7, 669, 199, 5, 3964, 3740, 1084, 4, 255, 440, 616, 154, 72, 71, 109, 12043, 49, 61, 283, 3591, 34, 87, 297, 41, 9, 1993, 2602, 518, 52, 706, 109, 12043, 37, 10, 561, 125, 43, 8, 445, 86, 576, 65, 1448, 2969, 4, 469, 1586, 118, 776, 5, 1993, 2602, 4, 108, 25, 179, 51, 1993, 2602, 498, 1052, 122, 12043, 1082, 1994, 1616, 11, 262, 4, 518, 171, 813, 109, 1084, 270, 12043, 539, 8, 3006, 580, 11, 31, 4, 2473, 306, 34, 87, 889, 280, 846, 573, 12043, 561, 125, 14, 539, 889, 810, 276, 182, 4, 67, 351, 14, 889, 1182, 118, 776, 156, 952, 4, 539, 889, 16, 38, 4, 445, 15, 200, 61, 12043, 2, 1, 1034, 1189, 734, 2003, 241, 284, 131, 553, 271, 28, 125, 280, 3]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]
[337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 348, 349]
[1034, 1189, 734, 2003, 241, 284, 131, 553, 271, 28, 125, 280, 3]

[1, 58, 512, 2, 352, 790, 1404, 266, 506, 101, 361, 36, 4, 7, 91, 41, 129, 490, 47, 553, 27, 358, 281, 74, 208, 6, 39, 101, 862, 91, 92, 41, 170, 4, 16, 52, 39, 87, 1745, 506, 1745, 888, 5, 87, 528, 249, 6, 532, 537, 45, 302, 94, 91, 5, 413, 323, 101, 565, 284, 6, 868, 25, 41, 826, 52, 6, 58, 518, 397, 6, 204, 62, 92, 41, 170, 4, 41, 371, 9, 204, 62, 337, 1023, 371, 521, 99, 191, 28, 1404, 266, 506, 101, 361, 100, 664, 539, 65, 4, 817, 1042, 36, 201, 413, 65, 120, 51, 277, 14, 2081, 541, 1190, 348, 12043, 58, 512, 508, 17, 57, 445, 5, 1512, 73, 1664, 565, 506, 101, 361, 11, 175, 29, 82, 412, 58, 76, 388, 15, 62, 76, 658, 222, 74, 701, 1866, 537, 506, 4, 48, 532, 537, 71, 109, 1123, 1600, 469, 220, 12048, 101, 565, 303, 876, 862, 91, 4, 16, 32, 39, 87, 1745, 506, 1745, 888, 5, 87, 528, 4, 145, 124, 93, 101, 150, 3466, 231, 164, 133, 174, 39, 101, 565, 130, 326, 524, 586, 108, 11, 17963, 42, 17963, 4, 48, 596, 581, 50, 155, 707, 1358, 1443, 345, 1455, 1411, 1123, 455, 413, 323, 12048, 483, 366, 4850, 14, 6215, 9488, 653, 266, 82, 337, 1023, 371, 521, 263, 204, 62, 78, 99, 191, 28, 7, 689, 65, 13, 4850, 269, 266, 82, 337, 1023, 77, 12043, 770, 137, 4, 47, 699, 506, 101, 361, 201, 9, 826, 52, 4177, 756, 387, 369, 52, 4, 297, 413, 86, 763, 27, 247, 98, 3887, 444, 48, 29, 247, 98, 629, 163, 868, 25, 506, 101, 361, 4, 79, 87, 326, 378, 290, 377, 101, 565, 4, 596, 581, 50, 8, 65, 314, 73, 5, 1123, 1600, 413, 323, 12043, 153, 187, 58, 512, 5, 1512, 73, 1664, 565, 135, 517, 57, 41, 5, 10, 385, 120, 1512, 73, 369, 52, 4, 48, 22, 9, 344, 813, 912, 101, 12, 5, 754, 2337, 6, 754, 2880, 43, 702, 96, 792, 2, 1, 1404, 266, 506, 101, 361, 1256, 27, 664, 85, 170, 3]
[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, 266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, 279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289, 290, 291, 292, 293, 294, 295, 296, 297, 298, 299, 300, 301, 302, 303, 304, 305, 306, 307, 308, 309, 310, 311, 312, 313, 314, 315, 316, 317, 318, 319, 320, 321, 322, 323, 324, 325, 326, 327, 328, 329, 330, 331, 332, 333, 334, 335, 336, 337, 338, 339, 340, 341, 342, 343, 344, 345, 346, 347, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
[348, 349, 350, 351, 352, 353, 354, 355, 356, 357, 358]
[1404, 266, 506, 101, 361, 1256, 27, 664, 85, 170, 3]

从以上结果可以看出，数据集中的example已经被转换成了模型可以接收的feature，包括input_ids、token_type_ids、position_ids和labels等。
其中：

• input_ids: 表示输入文本的token ID。
• token_type_ids: 用于区分source和target。
• position_ids: 表示输入token的位置。
• masked_positions: 表示target的位置。
• labels: target部分的token ID.

更多有关数据处理的内容，请参考数据处理。

4. Batchify和数据读入

使用paddle.io.BatchSampler和paddlenlp.data中提供的方法把数据组成batch。

然后使用paddle.io.DataLoader接口多线程异步加载数据。

batchify_fn详解：

本示例中我们将batchify方法写在了gen_utils.py中。

import paddle
from gen_utils import batchify_fn

batch_size = 16

# 初始化BatchSampler
train_batch_sampler = paddle.io.BatchSampler(
    train_ds, batch_size=batch_size, shuffle=True)

dev_batch_sampler = paddle.io.BatchSampler(
    dev_ds, batch_size=batch_size, shuffle=False)

train_collate_fn = partial(batchify_fn, pad_val=tokenizer.pad_token_id, mode='train')
dev_collate_fn = partial(batchify_fn, pad_val=tokenizer.pad_token_id, mode='test')

# 初始化DataLoader
train_data_loader = paddle.io.DataLoader(train_ds, batch_sampler=train_batch_sampler, 
                collate_fn=train_collate_fn, return_list=True)

dev_data_loader = paddle.io.DataLoader(dev_ds, batch_sampler=dev_batch_sampler, 
                collate_fn=dev_collate_fn, return_list=True)

更多PaddleNLP内置的batchify相关API，请参考collate。

到这里数据集准备就全部完成了，下一步我们需要组网并设计loss function。

模型结构

使用PaddleNLP一键加载预训练模型

在本次比赛的三个文本生成任务中，我们基于本基线使用的模型是UNIMO-text,是基于ERNIE-UNIMO框架在文本数据上预训练得到模型。

PaddleNLP已经内置了unimo-text-1.0，使用PaddleNLP API即可一键调用。

from paddlenlp.transformers import UNIMOLMHeadModel

model = UNIMOLMHeadModel.from_pretrained(MODEL_NAME)

[2021-10-23 09:45:57,084] [    INFO] - Downloading https://paddlenlp.bj.bcebos.com/models/transformers/unimo/unimo-text-1.0.pdparams and saved to /home/aistudio/.paddlenlp/models/unimo-text-1.0
[2021-10-23 09:45:57,087] [    INFO] - Downloading unimo-text-1.0.pdparams from https://paddlenlp.bj.bcebos.com/models/transformers/unimo/unimo-text-1.0.pdparams
100%|██████████| 444205/444205 [00:09<00:00, 44556.93it/s]

设计loss function

本示例中的问题生成任务是一个典型的seq2seq任务，所以我们使用的loss应该是多分类任务常用的CrossEntropy loss。Paddle中也内置了该loss，可以一键调用。之后在训练代码中可以看到。

选择网络结构后，我们需要设置Fine-Tune优化策略。

设置Fine-Tune优化策略

适用于ERNIE/BERT这类Transformer模型的学习率为warmup的动态学习率。

图3：动态学习率示意图

# 训练过程中的最大学习率
learning_rate = 5e-5 

# 训练轮次
epochs = 6

# 学习率预热比例
warmup_proportion = 0.02

# 权重衰减系数，类似模型正则项策略，避免模型过拟合
weight_decay = 0.01

max_grad_norm = 1.0

num_training_steps = len(train_data_loader) * epochs

# 学习率衰减策略
lr_scheduler = paddlenlp.transformers.LinearDecayWithWarmup(learning_rate, num_training_steps,
                                    warmup_proportion)

decay_params = [
    p.name for n, p in model.named_parameters()
    if not any(nd in n for nd in ["bias", "norm"])
]

# 定义优化器
optimizer = paddle.optimizer.AdamW(
    learning_rate=lr_scheduler,
    parameters=model.parameters(),
    weight_decay=weight_decay,
    beta1=0.9,
    beta2=0.98,
    epsilon=1e-6,
    apply_decay_param_fun=lambda x: x in decay_params,
    grad_clip=paddle.nn.ClipGradByGlobalNorm(max_grad_norm))

现在万事俱备，我们可以开始训练模型。

模型训练与评估

模型训练的过程通常有以下步骤：

1. 从dataloader中取出一个batch data
2. 将batch data喂给model，做前向计算
3. 将前向计算结果传给损失函数，计算loss。
4. loss反向回传，更新梯度。重复以上步骤。

每训练100个steps后，程序会自动调用gen_utils.py中的evaluation()方法计算任务的BLEU score。

import paddle.nn.functional as F
from gen_utils import evaluation, save_ckpt
import time

step = 0
total_time = 0.0
for epoch in range(epochs):
    print('\nEpoch %d/%d' % (epoch + 1, epochs))
    batch_start_time = time.time()
    for inputs in train_data_loader:
        step += 1
        labels = inputs[-1]
        logits = model(*inputs[:-1])
        labels = paddle.nn.functional.one_hot(
            labels, num_classes=logits.shape[-1])
        labels = paddle.nn.functional.label_smooth(labels)
        loss = F.cross_entropy(logits, labels, soft_label=True)

        loss.backward()
        optimizer.step()
        lr_scheduler.step()
        optimizer.clear_grad()

        total_time += (time.time() - batch_start_time)
        if step % 10 == 0:
            ppl = paddle.exp(loss)
            print(
                'step %d - loss: %.4f - ppl: %.4f - lr: %.7f - %.3fs/step'
                % (step, loss, ppl, optimizer.get_lr(),
                    total_time / 100))
            total_time = 0.0

        if step % 100 == 0 or step >= num_training_steps:
            save_ckpt(model, tokenizer, './checkpoints', step)
            print('Saved step {} model.\n'.format(step))
            model_eval = model._layers if isinstance(
                model, paddle.DataParallel) else model
            evaluation(model_eval, dev_data_loader, tokenizer)

        batch_start_time = time.time()

print('\nTraining completed.')

Epoch 1/6
step 10 - loss: 12.7858 - ppl: 357112.6875 - lr: 0.0000046 - 0.039s/step
step 20 - loss: 9.1871 - ppl: 9770.0098 - lr: 0.0000093 - 0.036s/step
step 30 - loss: 7.2957 - ppl: 1473.9421 - lr: 0.0000139 - 0.036s/step
step 40 - loss: 6.0092 - ppl: 407.1726 - lr: 0.0000185 - 0.036s/step
step 50 - loss: 5.8009 - ppl: 330.5934 - lr: 0.0000231 - 0.036s/step
step 60 - loss: 5.3549 - ppl: 211.6459 - lr: 0.0000278 - 0.036s/step
step 70 - loss: 4.7274 - ppl: 113.0066 - lr: 0.0000324 - 0.036s/step
step 80 - loss: 5.3654 - ppl: 213.8846 - lr: 0.0000370 - 0.036s/step
step 90 - loss: 4.6764 - ppl: 107.3812 - lr: 0.0000417 - 0.036s/step
step 100 - loss: 4.0520 - ppl: 57.5105 - lr: 0.0000463 - 0.037s/step
Saved step 100 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.10903393447280288
step 110 - loss: 3.9474 - ppl: 51.8024 - lr: 0.0000500 - 0.036s/step
step 120 - loss: 4.3713 - ppl: 79.1455 - lr: 0.0000499 - 0.036s/step
step 130 - loss: 3.9407 - ppl: 51.4565 - lr: 0.0000498 - 0.037s/step
step 140 - loss: 3.5864 - ppl: 36.1048 - lr: 0.0000497 - 0.036s/step
step 150 - loss: 3.4473 - ppl: 31.4143 - lr: 0.0000496 - 0.036s/step
step 160 - loss: 3.5283 - ppl: 34.0664 - lr: 0.0000495 - 0.037s/step
step 170 - loss: 3.7139 - ppl: 41.0127 - lr: 0.0000494 - 0.036s/step
step 180 - loss: 3.7959 - ppl: 44.5170 - lr: 0.0000493 - 0.037s/step
step 190 - loss: 3.5926 - ppl: 36.3276 - lr: 0.0000492 - 0.037s/step
step 200 - loss: 3.6892 - ppl: 40.0145 - lr: 0.0000491 - 0.036s/step
Saved step 200 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.2797706458843031
step 210 - loss: 3.2174 - ppl: 24.9633 - lr: 0.0000490 - 0.036s/step
step 220 - loss: 3.1577 - ppl: 23.5160 - lr: 0.0000490 - 0.036s/step
step 230 - loss: 3.3734 - ppl: 29.1769 - lr: 0.0000489 - 0.037s/step
step 240 - loss: 3.2700 - ppl: 26.3114 - lr: 0.0000488 - 0.037s/step
step 250 - loss: 3.3709 - ppl: 29.1055 - lr: 0.0000487 - 0.036s/step
step 260 - loss: 3.5547 - ppl: 34.9768 - lr: 0.0000486 - 0.037s/step
step 270 - loss: 3.0423 - ppl: 20.9527 - lr: 0.0000485 - 0.037s/step
step 280 - loss: 2.9760 - ppl: 19.6101 - lr: 0.0000484 - 0.037s/step
step 290 - loss: 3.2797 - ppl: 26.5675 - lr: 0.0000483 - 0.037s/step
step 300 - loss: 2.6343 - ppl: 13.9332 - lr: 0.0000482 - 0.037s/step
Saved step 300 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.29703924697075107
step 310 - loss: 3.1453 - ppl: 23.2262 - lr: 0.0000481 - 0.036s/step
step 320 - loss: 3.3483 - ppl: 28.4555 - lr: 0.0000480 - 0.037s/step
step 330 - loss: 3.4337 - ppl: 30.9923 - lr: 0.0000479 - 0.037s/step
step 340 - loss: 3.3462 - ppl: 28.3959 - lr: 0.0000478 - 0.037s/step
step 350 - loss: 3.1732 - ppl: 23.8835 - lr: 0.0000477 - 0.037s/step
step 360 - loss: 2.8486 - ppl: 17.2640 - lr: 0.0000476 - 0.037s/step
step 370 - loss: 3.1872 - ppl: 24.2203 - lr: 0.0000475 - 0.037s/step
step 380 - loss: 3.3895 - ppl: 29.6515 - lr: 0.0000475 - 0.036s/step
step 390 - loss: 3.0387 - ppl: 20.8782 - lr: 0.0000474 - 0.036s/step
step 400 - loss: 3.1261 - ppl: 22.7858 - lr: 0.0000473 - 0.036s/step
Saved step 400 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.31525123652710446
step 410 - loss: 3.5418 - ppl: 34.5285 - lr: 0.0000472 - 0.036s/step
step 420 - loss: 3.1074 - ppl: 22.3635 - lr: 0.0000471 - 0.036s/step
step 430 - loss: 3.1056 - ppl: 22.3216 - lr: 0.0000470 - 0.036s/step
step 440 - loss: 3.0530 - ppl: 21.1796 - lr: 0.0000469 - 0.036s/step
step 450 - loss: 3.2937 - ppl: 26.9416 - lr: 0.0000468 - 0.036s/step
step 460 - loss: 2.9580 - ppl: 19.2603 - lr: 0.0000467 - 0.036s/step
step 470 - loss: 3.1693 - ppl: 23.7918 - lr: 0.0000466 - 0.036s/step
step 480 - loss: 3.0417 - ppl: 20.9417 - lr: 0.0000465 - 0.036s/step
step 490 - loss: 2.9991 - ppl: 20.0679 - lr: 0.0000464 - 0.037s/step
step 500 - loss: 3.4030 - ppl: 30.0551 - lr: 0.0000463 - 0.037s/step
Saved step 500 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.33143947532072143
step 510 - loss: 3.0930 - ppl: 22.0426 - lr: 0.0000462 - 0.036s/step
step 520 - loss: 2.5499 - ppl: 12.8052 - lr: 0.0000461 - 0.036s/step
step 530 - loss: 2.6142 - ppl: 13.6569 - lr: 0.0000460 - 0.036s/step
step 540 - loss: 2.6469 - ppl: 14.1103 - lr: 0.0000460 - 0.036s/step
step 550 - loss: 2.6818 - ppl: 14.6115 - lr: 0.0000459 - 0.037s/step
step 560 - loss: 3.3470 - ppl: 28.4165 - lr: 0.0000458 - 0.036s/step
step 570 - loss: 3.1077 - ppl: 22.3698 - lr: 0.0000457 - 0.037s/step
step 580 - loss: 2.8071 - ppl: 16.5618 - lr: 0.0000456 - 0.037s/step
step 590 - loss: 3.1872 - ppl: 24.2198 - lr: 0.0000455 - 0.036s/step
step 600 - loss: 2.6443 - ppl: 14.0734 - lr: 0.0000454 - 0.037s/step
Saved step 600 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3329463794961518
step 610 - loss: 3.4308 - ppl: 30.9015 - lr: 0.0000453 - 0.036s/step
step 620 - loss: 2.7005 - ppl: 14.8879 - lr: 0.0000452 - 0.036s/step
step 630 - loss: 2.9503 - ppl: 19.1108 - lr: 0.0000451 - 0.037s/step
step 640 - loss: 3.4411 - ppl: 31.2218 - lr: 0.0000450 - 0.036s/step
step 650 - loss: 3.0281 - ppl: 20.6586 - lr: 0.0000449 - 0.037s/step
step 660 - loss: 3.3157 - ppl: 27.5407 - lr: 0.0000448 - 0.036s/step
step 670 - loss: 2.9843 - ppl: 19.7726 - lr: 0.0000447 - 0.037s/step
step 680 - loss: 2.9049 - ppl: 18.2636 - lr: 0.0000446 - 0.037s/step
step 690 - loss: 3.1769 - ppl: 23.9732 - lr: 0.0000446 - 0.036s/step
step 700 - loss: 3.1583 - ppl: 23.5299 - lr: 0.0000445 - 0.037s/step
Saved step 700 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3522047809390708
step 710 - loss: 2.9169 - ppl: 18.4843 - lr: 0.0000444 - 0.036s/step
step 720 - loss: 3.4773 - ppl: 32.3719 - lr: 0.0000443 - 0.036s/step
step 730 - loss: 3.0892 - ppl: 21.9599 - lr: 0.0000442 - 0.036s/step
step 740 - loss: 3.0610 - ppl: 21.3491 - lr: 0.0000441 - 0.036s/step
step 750 - loss: 3.0579 - ppl: 21.2839 - lr: 0.0000440 - 0.036s/step
step 760 - loss: 2.7247 - ppl: 15.2516 - lr: 0.0000439 - 0.036s/step
step 770 - loss: 2.9204 - ppl: 18.5479 - lr: 0.0000438 - 0.036s/step
step 780 - loss: 3.0387 - ppl: 20.8789 - lr: 0.0000437 - 0.036s/step
step 790 - loss: 3.1677 - ppl: 23.7537 - lr: 0.0000436 - 0.036s/step
step 800 - loss: 2.6275 - ppl: 13.8394 - lr: 0.0000435 - 0.036s/step
Saved step 800 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.34164585816947296
step 810 - loss: 2.2354 - ppl: 9.3504 - lr: 0.0000434 - 0.036s/step
step 820 - loss: 3.5113 - ppl: 33.4926 - lr: 0.0000433 - 0.036s/step
step 830 - loss: 3.0235 - ppl: 20.5625 - lr: 0.0000432 - 0.036s/step
step 840 - loss: 2.6856 - ppl: 14.6669 - lr: 0.0000431 - 0.036s/step
step 850 - loss: 2.7245 - ppl: 15.2488 - lr: 0.0000431 - 0.036s/step
step 860 - loss: 2.7352 - ppl: 15.4122 - lr: 0.0000430 - 0.036s/step
step 870 - loss: 2.9736 - ppl: 19.5613 - lr: 0.0000429 - 0.037s/step
step 880 - loss: 3.0008 - ppl: 20.1013 - lr: 0.0000428 - 0.036s/step
step 890 - loss: 3.0778 - ppl: 21.7112 - lr: 0.0000427 - 0.036s/step
step 900 - loss: 2.8358 - ppl: 17.0446 - lr: 0.0000426 - 0.037s/step
Saved step 900 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3451996548162081

Epoch 2/6
step 910 - loss: 2.9591 - ppl: 19.2811 - lr: 0.0000425 - 0.035s/step
step 920 - loss: 2.9661 - ppl: 19.4158 - lr: 0.0000424 - 0.036s/step
step 930 - loss: 2.3252 - ppl: 10.2290 - lr: 0.0000423 - 0.036s/step
step 940 - loss: 2.7328 - ppl: 15.3754 - lr: 0.0000422 - 0.036s/step
step 950 - loss: 2.8285 - ppl: 16.9204 - lr: 0.0000421 - 0.036s/step
step 960 - loss: 2.6771 - ppl: 14.5425 - lr: 0.0000420 - 0.036s/step
step 970 - loss: 2.7520 - ppl: 15.6747 - lr: 0.0000419 - 0.036s/step
step 980 - loss: 2.6941 - ppl: 14.7929 - lr: 0.0000418 - 0.036s/step
step 990 - loss: 2.7595 - ppl: 15.7921 - lr: 0.0000417 - 0.036s/step
step 1000 - loss: 2.8390 - ppl: 17.0987 - lr: 0.0000416 - 0.036s/step
Saved step 1000 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3434393919324781
step 1010 - loss: 2.5956 - ppl: 13.4052 - lr: 0.0000416 - 0.037s/step
step 1020 - loss: 2.7560 - ppl: 15.7371 - lr: 0.0000415 - 0.036s/step
step 1030 - loss: 2.6757 - ppl: 14.5219 - lr: 0.0000414 - 0.036s/step
step 1040 - loss: 2.7803 - ppl: 16.1238 - lr: 0.0000413 - 0.036s/step
step 1050 - loss: 2.6717 - ppl: 14.4650 - lr: 0.0000412 - 0.036s/step
step 1060 - loss: 2.4910 - ppl: 12.0730 - lr: 0.0000411 - 0.037s/step
step 1070 - loss: 2.7729 - ppl: 16.0044 - lr: 0.0000410 - 0.036s/step
step 1080 - loss: 2.7114 - ppl: 15.0509 - lr: 0.0000409 - 0.037s/step
step 1090 - loss: 2.7026 - ppl: 14.9182 - lr: 0.0000408 - 0.037s/step
step 1100 - loss: 2.6854 - ppl: 14.6636 - lr: 0.0000407 - 0.036s/step
Saved step 1100 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3560799252870019
step 1110 - loss: 2.8307 - ppl: 16.9568 - lr: 0.0000406 - 0.036s/step
step 1120 - loss: 2.6309 - ppl: 13.8865 - lr: 0.0000405 - 0.036s/step
step 1130 - loss: 2.8233 - ppl: 16.8320 - lr: 0.0000404 - 0.036s/step
step 1140 - loss: 3.0231 - ppl: 20.5554 - lr: 0.0000403 - 0.036s/step
step 1150 - loss: 2.5476 - ppl: 12.7769 - lr: 0.0000402 - 0.036s/step
step 1160 - loss: 2.5039 - ppl: 12.2305 - lr: 0.0000401 - 0.037s/step
step 1170 - loss: 2.6913 - ppl: 14.7506 - lr: 0.0000401 - 0.036s/step
step 1180 - loss: 2.8845 - ppl: 17.8940 - lr: 0.0000400 - 0.036s/step
step 1190 - loss: 2.7891 - ppl: 16.2669 - lr: 0.0000399 - 0.036s/step
step 1200 - loss: 2.7661 - ppl: 15.8959 - lr: 0.0000398 - 0.036s/step
Saved step 1200 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3541472124617531
step 1210 - loss: 2.5566 - ppl: 12.8920 - lr: 0.0000397 - 0.037s/step
step 1220 - loss: 2.5174 - ppl: 12.3958 - lr: 0.0000396 - 0.037s/step
step 1230 - loss: 2.9576 - ppl: 19.2520 - lr: 0.0000395 - 0.037s/step
step 1240 - loss: 2.6588 - ppl: 14.2788 - lr: 0.0000394 - 0.037s/step
step 1250 - loss: 2.7020 - ppl: 14.9099 - lr: 0.0000393 - 0.037s/step
step 1260 - loss: 2.5633 - ppl: 12.9781 - lr: 0.0000392 - 0.037s/step
step 1270 - loss: 2.3900 - ppl: 10.9139 - lr: 0.0000391 - 0.037s/step
step 1280 - loss: 2.7610 - ppl: 15.8161 - lr: 0.0000390 - 0.036s/step
step 1290 - loss: 2.5673 - ppl: 13.0305 - lr: 0.0000389 - 0.037s/step
step 1300 - loss: 2.5631 - ppl: 12.9766 - lr: 0.0000388 - 0.037s/step
Saved step 1300 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3669669536384197
step 1310 - loss: 2.2246 - ppl: 9.2494 - lr: 0.0000387 - 0.037s/step
step 1320 - loss: 2.7285 - ppl: 15.3100 - lr: 0.0000387 - 0.037s/step
step 1330 - loss: 2.4106 - ppl: 11.1404 - lr: 0.0000386 - 0.037s/step
step 1340 - loss: 2.7551 - ppl: 15.7225 - lr: 0.0000385 - 0.036s/step
step 1350 - loss: 2.8719 - ppl: 17.6697 - lr: 0.0000384 - 0.036s/step
step 1360 - loss: 2.5673 - ppl: 13.0306 - lr: 0.0000383 - 0.036s/step
step 1370 - loss: 2.4229 - ppl: 11.2784 - lr: 0.0000382 - 0.037s/step
step 1380 - loss: 2.6113 - ppl: 13.6171 - lr: 0.0000381 - 0.036s/step
step 1390 - loss: 2.7364 - ppl: 15.4317 - lr: 0.0000380 - 0.036s/step
step 1400 - loss: 2.9766 - ppl: 19.6212 - lr: 0.0000379 - 0.037s/step
Saved step 1400 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.37127651521081784
step 1410 - loss: 2.9148 - ppl: 18.4446 - lr: 0.0000378 - 0.036s/step
step 1420 - loss: 2.4830 - ppl: 11.9773 - lr: 0.0000377 - 0.036s/step
step 1430 - loss: 2.6804 - ppl: 14.5910 - lr: 0.0000376 - 0.036s/step
step 1440 - loss: 2.5549 - ppl: 12.8706 - lr: 0.0000375 - 0.036s/step
step 1450 - loss: 2.4551 - ppl: 11.6473 - lr: 0.0000374 - 0.036s/step
step 1460 - loss: 2.7822 - ppl: 16.1546 - lr: 0.0000373 - 0.037s/step
step 1470 - loss: 2.5540 - ppl: 12.8583 - lr: 0.0000372 - 0.036s/step
step 1480 - loss: 2.5386 - ppl: 12.6614 - lr: 0.0000372 - 0.036s/step
step 1490 - loss: 2.3709 - ppl: 10.7073 - lr: 0.0000371 - 0.037s/step
step 1500 - loss: 2.5368 - ppl: 12.6395 - lr: 0.0000370 - 0.036s/step
Saved step 1500 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.37815449607312623
step 1510 - loss: 2.5198 - ppl: 12.4256 - lr: 0.0000369 - 0.038s/step
step 1520 - loss: 2.3159 - ppl: 10.1336 - lr: 0.0000368 - 0.037s/step
step 1530 - loss: 2.4976 - ppl: 12.1528 - lr: 0.0000367 - 0.037s/step
step 1540 - loss: 2.4564 - ppl: 11.6631 - lr: 0.0000366 - 0.037s/step
step 1550 - loss: 2.9612 - ppl: 19.3204 - lr: 0.0000365 - 0.037s/step
step 1560 - loss: 2.9010 - ppl: 18.1915 - lr: 0.0000364 - 0.037s/step
step 1570 - loss: 2.7340 - ppl: 15.3937 - lr: 0.0000363 - 0.037s/step
step 1580 - loss: 2.5176 - ppl: 12.3990 - lr: 0.0000362 - 0.037s/step
step 1590 - loss: 2.4378 - ppl: 11.4480 - lr: 0.0000361 - 0.037s/step
step 1600 - loss: 2.7711 - ppl: 15.9763 - lr: 0.0000360 - 0.037s/step
Saved step 1600 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3783972703813113
step 1610 - loss: 2.5230 - ppl: 12.4655 - lr: 0.0000359 - 0.037s/step
step 1620 - loss: 2.7631 - ppl: 15.8496 - lr: 0.0000358 - 0.037s/step
step 1630 - loss: 2.8825 - ppl: 17.8584 - lr: 0.0000357 - 0.037s/step
step 1640 - loss: 2.6621 - ppl: 14.3261 - lr: 0.0000357 - 0.037s/step
step 1650 - loss: 2.5582 - ppl: 12.9123 - lr: 0.0000356 - 0.038s/step
step 1660 - loss: 2.8928 - ppl: 18.0436 - lr: 0.0000355 - 0.037s/step
step 1670 - loss: 2.8574 - ppl: 17.4160 - lr: 0.0000354 - 0.037s/step
step 1680 - loss: 2.4173 - ppl: 11.2150 - lr: 0.0000353 - 0.037s/step
step 1690 - loss: 2.2631 - ppl: 9.6125 - lr: 0.0000352 - 0.037s/step
step 1700 - loss: 2.7484 - ppl: 15.6170 - lr: 0.0000351 - 0.037s/step
Saved step 1700 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.38341942928025413
step 1710 - loss: 2.8887 - ppl: 17.9707 - lr: 0.0000350 - 0.037s/step
step 1720 - loss: 2.6021 - ppl: 13.4925 - lr: 0.0000349 - 0.037s/step
step 1730 - loss: 3.0668 - ppl: 21.4727 - lr: 0.0000348 - 0.037s/step
step 1740 - loss: 3.0824 - ppl: 21.8116 - lr: 0.0000347 - 0.037s/step
step 1750 - loss: 2.7579 - ppl: 15.7659 - lr: 0.0000346 - 0.037s/step
step 1760 - loss: 2.3157 - ppl: 10.1322 - lr: 0.0000345 - 0.037s/step
step 1770 - loss: 2.5510 - ppl: 12.8200 - lr: 0.0000344 - 0.038s/step
step 1780 - loss: 2.4753 - ppl: 11.8857 - lr: 0.0000343 - 0.037s/step
step 1790 - loss: 2.3660 - ppl: 10.6545 - lr: 0.0000343 - 0.037s/step
step 1800 - loss: 2.2695 - ppl: 9.6745 - lr: 0.0000342 - 0.037s/step
Saved step 1800 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3791853890436889
step 1810 - loss: 2.3898 - ppl: 10.9116 - lr: 0.0000341 - 0.037s/step

Epoch 3/6
step 1820 - loss: 2.5415 - ppl: 12.6991 - lr: 0.0000340 - 0.036s/step
step 1830 - loss: 2.0309 - ppl: 7.6206 - lr: 0.0000339 - 0.037s/step
step 1840 - loss: 2.4498 - ppl: 11.5860 - lr: 0.0000338 - 0.037s/step
step 1850 - loss: 2.3366 - ppl: 10.3463 - lr: 0.0000337 - 0.037s/step
step 1860 - loss: 2.3200 - ppl: 10.1759 - lr: 0.0000336 - 0.037s/step
step 1870 - loss: 2.3267 - ppl: 10.2437 - lr: 0.0000335 - 0.037s/step
step 1880 - loss: 2.6154 - ppl: 13.6724 - lr: 0.0000334 - 0.038s/step
step 1890 - loss: 2.2624 - ppl: 9.6058 - lr: 0.0000333 - 0.037s/step
step 1900 - loss: 2.3735 - ppl: 10.7351 - lr: 0.0000332 - 0.037s/step
Saved step 1900 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3736918893723728
step 1910 - loss: 2.3255 - ppl: 10.2314 - lr: 0.0000331 - 0.037s/step
step 1920 - loss: 2.5070 - ppl: 12.2679 - lr: 0.0000330 - 0.037s/step
step 1930 - loss: 2.3195 - ppl: 10.1704 - lr: 0.0000329 - 0.037s/step
step 1940 - loss: 2.6797 - ppl: 14.5808 - lr: 0.0000328 - 0.037s/step
step 1950 - loss: 2.3691 - ppl: 10.6881 - lr: 0.0000328 - 0.036s/step
step 1960 - loss: 2.1678 - ppl: 8.7392 - lr: 0.0000327 - 0.037s/step
step 1970 - loss: 2.5114 - ppl: 12.3224 - lr: 0.0000326 - 0.037s/step
step 1980 - loss: 2.7571 - ppl: 15.7542 - lr: 0.0000325 - 0.037s/step
step 1990 - loss: 2.4835 - ppl: 11.9830 - lr: 0.0000324 - 0.037s/step
step 2000 - loss: 2.2833 - ppl: 9.8092 - lr: 0.0000323 - 0.037s/step
Saved step 2000 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.3779025879461599
step 2010 - loss: 2.5694 - ppl: 13.0584 - lr: 0.0000322 - 0.037s/step
step 2020 - loss: 2.6586 - ppl: 14.2768 - lr: 0.0000321 - 0.037s/step
step 2030 - loss: 2.4564 - ppl: 11.6622 - lr: 0.0000320 - 0.037s/step
step 2040 - loss: 2.1576 - ppl: 8.6507 - lr: 0.0000319 - 0.037s/step
step 2050 - loss: 2.5024 - ppl: 12.2118 - lr: 0.0000318 - 0.037s/step
step 2060 - loss: 2.3578 - ppl: 10.5680 - lr: 0.0000317 - 0.037s/step
step 2070 - loss: 2.5245 - ppl: 12.4841 - lr: 0.0000316 - 0.037s/step
step 2080 - loss: 2.6190 - ppl: 13.7216 - lr: 0.0000315 - 0.037s/step
step 2090 - loss: 2.5256 - ppl: 12.4984 - lr: 0.0000314 - 0.038s/step
step 2100 - loss: 2.5341 - ppl: 12.6046 - lr: 0.0000313 - 0.037s/step
Saved step 2100 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.37272013400106907
step 2110 - loss: 2.2579 - ppl: 9.5634 - lr: 0.0000313 - 0.037s/step
step 2120 - loss: 2.6708 - ppl: 14.4519 - lr: 0.0000312 - 0.037s/step
step 2130 - loss: 2.4001 - ppl: 11.0243 - lr: 0.0000311 - 0.037s/step
step 2140 - loss: 2.5737 - ppl: 13.1143 - lr: 0.0000310 - 0.037s/step
step 2150 - loss: 2.1106 - ppl: 8.2530 - lr: 0.0000309 - 0.037s/step
step 2160 - loss: 2.7373 - ppl: 15.4453 - lr: 0.0000308 - 0.037s/step
step 2170 - loss: 2.4718 - ppl: 11.8432 - lr: 0.0000307 - 0.038s/step
step 2180 - loss: 2.1996 - ppl: 9.0213 - lr: 0.0000306 - 0.037s/step
step 2190 - loss: 2.3058 - ppl: 10.0322 - lr: 0.0000305 - 0.037s/step
step 2200 - loss: 2.7691 - ppl: 15.9443 - lr: 0.0000304 - 0.037s/step
Saved step 2200 model.


Eval begin...

Save inference result into: ./output.txt

***************
The auto evaluation result is:
BLEU-4: 0.386648892057624
step 2210 - loss: 2.5017 - ppl: 12.2036 - lr: 0.0000303 - 0.036s/step
step 2220 - loss: 2.6945 - ppl: 14.7978 - lr: 0.0000302 - 0.036s/step
step 2230 - loss: 2.3235 - ppl: 10.2110 - lr: 0.0000301 - 0.037s/step
step 2240 - loss: 2.4091 - ppl: 11.1238 - lr: 0.0000300 - 0.038s/step
step 2250 - loss: 2.3000 - ppl: 9.9744 - lr: 0.0000299 - 0.037s/step
step 2260 - loss: 2.3532 - ppl: 10.5195 - lr: 0.0000299 - 0.037s/step
step 2270 - loss: 2.6025 - ppl: 13.4968 - lr: 0.0000298 - 0.037s/step
step 2280 - loss: 2.4017 - ppl: 11.0420 - lr: 0.0000297 - 0.036s/step

到这里，整个问题生成任务就完成了，脚本输出的output.txt是可以直接提交的结果。关于比赛的其他任务，可以在PaddleNLP中的unimo-text找到详细的基线代码

以上内容实现基于PaddleNLP，⭐⭐⭐STAR⭐⭐⭐收藏起来，跟进最新功能吧~

GitHub地址：https://github.com/PaddlePaddle/PaddleNLP

更多使用方法可参考PaddleNLP教程

• 使用seq2vec模块进行句子情感分类
• 使用预训练模型ERNIE优化情感分析
• 使用BiGRU-CRF模型完成快递单信息抽取
• 使用预训练模型ERNIE优化快递单信息抽取
• 使用Seq2Seq模型完成自动对联
• 使用预训练模型ERNIE-GEN自动写诗
• 使用TCN网络完成新冠疫情病例数预测
• 自定义数据集实现文本多分类任务
• 项目合集

加入交流群，一起学习吧

现在就加入PaddleNLP的QQ技术交流群，一起交流NLP技术吧！