{"version":"2026-10-19.observations.2","reading":{"id":"reading-transfer","title":"代码训练会让数学和语言一起变好吗？","deck":"训练配比、相同后续训练和程序提示是三种不同干预，结论要各自保留。","editorialAt":"2026-10-15","sourceIds":["D-PETTY","D-DSM","D-COCO"],"sections":[{"title":"把“迁移”说成具体改变","paragraphs":["迁移是改训练或表示方法后，另一个任务的表现改变。它可以是正向，也可能牺牲别的任务；不能由两个强模型共同得高分推出训练因果关系。"],"sourceIds":[]},{"title":"先区分三种改变","paragraphs":["训练配比：改变学习材料中代码与语言的比例，检查其他任务有没有受益或受损。","继续训练：比较不同前期训练，再接上相同数学训练，检查这种起点差异的影响。","提示接口：保留同一个模型，把任务改写成代码格式，检查产物是否更好。三种证据不能合并成一个通用增益。"],"sourceIds":[]},{"title":"训练配比：收益和代价同时看","paragraphs":["Petty等的小规模实验发现：部分结构化输出与算术任务受益，纯语言或知识任务也可能受损；算术表现并不总随代码比例单调提高。","核对条件：374M（3.74亿参数）模型。competitive固定总训练token量，代码增加时会挤占语言；additive固定语言token量，再额外添加代码。前者换了材料，后者还增加了总训练量，应分别阅读。"],"sourceIds":["D-PETTY"]},{"title":"继续训练：在同一后续阶段比较","paragraphs":["人工核对训练的一种办法，是让不同起点接相同的后续训练。DeepSeekMath表6在1.3B（13亿参数）模型上比较400B代码或通用预训练，再接相同150B数学训练。","GSM8K小学数学文字题的公布成绩分别为21.9%与19.1%；本站相减为2.8个百分点。这是该模型与训练条件的配对差，不是置信区间，也不是前沿系统普遍收益。"],"sourceIds":["D-DSM"]},{"title":"提示接口：模型不换，任务表示改变","paragraphs":["CoCoGen在同一DAVINCI检查点上，用代码格式和文字格式处理PROSCRIPT事件关系预测；正式表5的F1分别为49.30与43.06，本站相减为6.24个F1点。","F1同时考虑找出的关系是否正确、应有关系是否找全，不等于整题答案准确率。这里变的是提示和输出接口，不能当作代码预训练的因果收益。"],"sourceIds":["D-COCO"]},{"title":"对工作工具的实际启发","paragraphs":["看到训练迁移线索时，优先核对自己的产物：是公式/结构化字段、文字解释还是代码补丁？按任务分别验收，并保留反例。没有实际材料和验证，不能因此给数学、金融或语言工作增加统一可委派百分比。"],"sourceIds":[]},{"title":"核对术语","paragraphs":["参数是模型可学习的数值；token是模型处理文本的片段，不等于汉字数。M表示百万，B表示十亿；模型参数量和训练token量是两种不同数量。","消融是有控制地改变一个设定，观察表现变化。它只能支持被控制的条件下的解释。"],"sourceIds":[]}],"actions":["分清训练配比、继续训练、提示接口","保留正向与负向结果的任务范围","把机制线索带回F01/M01/C03的小范围验收"],"seriesIds":["epoch-code","aime-answers"],"taskIds":["C03","F01","M01"],"frontierIds":["F03","F04","F07","F08"],"evidenceStatus":"active","actualCoverage":null},"sources":[{"id":"D-PETTY","title":"代码预训练影响任务表现","url":"https://arxiv.org/html/2409.04556v2","version":"arXiv v2","locator":"§3/4训练设置；§5结果；Tables3/5/6/7","scope":"374M模型，控制代码/自然语言配比；competitive总token固定、additive语言token固定","license":"CC BY 4.0；只写解释与表格事实","limit":"小模型与特定任务；加代码不统一改善所有语言任务；以正文定义为准，abs摘要标签有倒置","publishedAt":"2025-02-25","readAt":"2026-10-15","collectedAt":null,"status":"active","nextReview":"2027-01-13","methods":[],"family":"D-PETTY"},{"id":"D-DSM","title":"DeepSeekMath的匹配阶段消融","url":"https://arxiv.org/html/2402.03300v3","version":"arXiv v3","locator":"§5.1.1 Table6，同一Math150B后续训练","scope":"1.3B消融：400B代码或通用预训练，接相同150B数学训练","license":"公开阅读；不复制论文全文","limit":"单规模与指定训练；表中算术差不是置信区间或前沿系统通用增益","publishedAt":"2024-04-27","readAt":"2026-10-15","collectedAt":null,"status":"active","nextReview":"2027-01-13","methods":[],"family":"D-DSM"},{"id":"D-COCO","title":"CoCoGen代码形式的提示接口","url":"https://aclanthology.org/2022.emnlp-main.90/","version":"EMNLP2022正式论文","locator":"Table5，印刷p1391；PROSCRIPT edge prediction","scope":"同检查点用代码结构或文本输出比较","license":"ACL CC BY 4.0；只写方法与有限事实","limit":"F1分数不是正确率；提示格式比较不能识别代码预训练的因果效果","publishedAt":null,"readAt":"2026-10-15","collectedAt":null,"status":"active","nextReview":"2027-01-13","methods":[],"family":"D-COCO"}],"actualCoverage":null}