{"version":"2026-10-19.boundary.1","asOf":"2026-10-19","month":"2026-10","domain":{"id":"computing","title":"计算机 · 软件工程","summary":"从写小程序、修缺陷到维护和整仓迁移，最明显的变化是交接范围扩大；真实收益因任务和验收标准而异。","frontierId":"F08","sourceIds":["BC03","BC06","BC08","BC10"],"terms":[{"term":"仓库 / issue / PR","definition":"仓库是项目代码集合；issue是待解决的问题单；PR是送给人审查的代码变更，draft PR是其草稿。"},{"term":"HTTP server","definition":"接收网络请求并返回响应的小程序；此处实验任务规定了规格和测试。"},{"term":"BM25 / oracle","definition":"BM25按关键词找文件；oracle用参考修复指定相关文件，代表理想检索条件。"},{"term":"Agent框架 / token","definition":"框架组织模型、工具与运行步骤；token是模型处理的文字片段，13k约为1.3万片段。"},{"term":"技术栈 / 反例","definition":"技术栈指语言、框架等组成；反例是可执行输入，显示新旧程序行为不一致。"}]},"focusTask":"C03","caseStudy":{"id":"C03","explanationId":"maintenance-outcomes","title":"熟悉项目维护：先看完整交付，再看生成速度","lead":"补全新代码和改熟悉仓库，是两种任务。AI生成得快，不代表你连同核对和返工也更快。","fit":"相近软件工作实验支持试点设计，不能给这个仓库承诺收益。先限定一个可回滚问题。","editorialAt":"2026-10-19","sourceIds":["BC06","BC07","FS071","E18-J2026","BC13","BC12","R-HARNESS2"],"steps":[{"title":"定义问题","actor":"人决定","action":"选择一个可回滚修复，列出原行为与不能改变的行为。","output":"任务说明与验收用例","gate":"问题范围、权限与回归门槛先明确","judgment":"C03-step-1","editorialReason":"本步骤是本站为交付和接收提出的组织安排；没有本地逐步成功率。","evidenceRelations":[{"sourceId":"BC13","role":"context","reason":"重复解决会影响PR接收；提醒先核对问题状态，不证明本项目效果","locator":"§3.2 R1"}]},{"title":"定位并草拟改动","actor":"AI草拟","action":"在给定文件与日志中提出原因和最小补丁；不自行扩展改造。","output":"定位依据与待审补丁","gate":"每处改动能回到问题和依据","judgment":"C03-step-2","editorialReason":"本步骤是本站为交付和接收提出的组织安排；没有本地逐步成功率。","evidenceRelations":[{"sourceId":"R-HARNESS2","role":"context","reason":"相邻方法启示，未验证本站案例；原有现场研究及其条件保留。","locator":"§2.1–2.3；§3.1；§4"}]},{"title":"检查行为","actor":"程序执行","action":"运行现有测试和针对问题的回归检查，记录失败而非只留成功输出。","output":"检查记录与全部尝试","gate":"修复目标通过、原有行为未退化","judgment":"C03-step-3","editorialReason":"本步骤是本站为交付和接收提出的组织安排；没有本地逐步成功率。","evidenceRelations":[{"sourceId":"BC12","role":"condition","reason":"评测修订说明独立验证和环境会改变判断；用于设计回归条件","locator":"9月22日更新说明"}]},{"title":"审查与接收","actor":"人核验","action":"核对遗漏行为、错误路径与回滚，再决定是否合并。","output":"审查决定与人时记录","gate":"审查、返工、等待和接管都有记录","judgment":"C03-step-4","editorialReason":"本步骤是本站为交付和接收提出的组织安排；没有本地逐步成功率。","evidenceRelations":[{"sourceId":"BC13","role":"context","reason":"接收还受协作与优先级影响，不能仅看生成","locator":"§3.2；表1"}]}],"example":{"title":"一个限定维护任务的交付格式","body":"【人工构造，未执行AI任务】\n问题：输入为空列表时应返回空列表，不应崩溃。\n保持：非空输入的原输出、错误输入的原错误语义。\n产物：待审补丁 + 空/非空/错误输入回归记录 + 未覆盖行为。\n审查决定：____；回滚入口：____。\n提示/查找人时：____；审查：____；返工：____。","note":"这是一份格式示例，没有代码改动、真实仓库或完成成绩。"},"failures":[{"trigger":"补丁通过新增用例，却改变了错误路径","why":"局部通过没有覆盖全部原行为","action":"补错误路径回归；无法核对时回滚并由人接管"},{"trigger":"生成很快，但审查和返工更久","why":"只看生成时长漏掉了人参与成本","action":"记录全部人时，按同质量比较；允许净收益为负"},{"trigger":"额外检查与返工抵消生成收益","why":"局部结果不足以证明完整交付合格","action":"保留完整人时及原始依据，停止扩大委派并由人复核"}],"trial":{"scope":"一个熟悉仓库中可回滚的维护问题；不要直接扩大到整仓改造","baseline":"保存原工作量/质量门槛下的人独立流程；一次任务不能估计通用收益","acceptance":["目标问题和保持行为分别通过","保留全部尝试、审查/返工/接管记录","只在同工作量同质量下比较人时"],"timeFields":["活动人时：准备与查找","活动人时：提示与交互","活动人时：人工审查","活动人时：返工","活动人时：接管与维护","另记：机器运行、等待及端到端历时；重叠活动不重复累计；若等待占用本人，注明计量约定"],"stop":"行为未能核对、范围扩大或人参与成本持续上升时，暂停委派并接管"},"comparisons":[{"title":"熟悉开源仓库","sourceId":"BC06","relationship":"相近任务","task":"真实维护问题","people":"16开发者、246任务","setup":"主要Cursor+Claude3.5/3.7，2025早期","outcome":"耗时增加19%，95%区间+2%至+39%","metric":"任务耗时","limitation":"小而专门人群；不是所有开发者"},{"title":"三项企业开发实验","sourceId":"FS071","relationship":"相近任务","task":"企业日常开发任务","people":"4,867开发者","setup":"随机提供代码补全工具访问","outcome":"完成任务数+26.08%；标准误10.3%（SE，表示估计的不确定性）","metric":"完成任务数","limitation":"不是省时26.08%；三研究异质，质量/利润未由本摘要测得"},{"title":"METR后续","sourceId":"BC07","relationship":"受限后续","task":"后续维护任务","people":"10回访/47新招募","setup":"选择、报酬、并行计时改变","outcome":"耗时−18%/−4%；两个区间跨零","metric":"任务耗时，弱证据","limitation":"研究者认为无法可靠估计当前收益"},{"title":"咨询任务的越界反例","sourceId":"E18-J2026","relationship":"背景参考","task":"18项范围内任务及一项范围外管理任务","people":"758知识工作者","setup":"无AI/GPT4/GPT4+说明","outcome":"范围内更快；范围外正确解概率低约19个百分点","metric":"不同任务的时间/正确解指标","limitation":"不是软件维护直接结果；不与上三行平均"},{"title":"修复PR的接收过程","sourceId":"BC13","relationship":"相邻真实协作观察","task":"8106个AI参与修复PR；人工分析326个关闭未合并样本","people":"五类代理所在开源仓库，非随机配对试验","setup":"依据采集时merged/closed/open状态；闭合原因人工编码","outcome":"65.0%已合并；另有26.1%关闭未合并、8.9%开放","metric":"PR状态分布，不是修复正确率或净人时","limitation":"重复解决、优先级和审阅流程也影响未合并；没有测本团队收益"},{"title":"运行方法与完整成本","sourceId":"R-HARNESS2","relationship":"adjacent：方法相邻，不是本任务完成率","task":"冻结模型下的运行系统改进","people":"作者实验；本站没有参与者或任务复跑","setup":"须分别看执行模型、编辑器、预算与验收","outcome":"提供过程改进的研究方法，未测本站收益","metric":"本页不摘录分数；不与现有现场研究合并","limitation":"相邻方法启示，未验证本站案例；原有现场研究及其条件保留。"}],"receiptFields":["任务/版本/资料与环境","已授权范围与工具配置","实际输入/产物路径","全部尝试与逐项验收","最终接收/拒绝与接管","基线/准备/核验/返工人时","未知与未验证项"],"exampleStatus":"synthetic_not_ai_executed","actualCoverage":null,"evidenceStatus":"active","sourceRelations":[{"sourceId":"BC06","role":"support","reason":"支持研究本身所述范围；对本工作组织和收益仍需独立验证。"},{"sourceId":"BC07","role":"support","reason":"支持研究本身所述范围；对本工作组织和收益仍需独立验证。"},{"sourceId":"FS071","role":"support","reason":"支持研究本身所述范围；对本工作组织和收益仍需独立验证。"},{"sourceId":"E18-J2026","role":"support","reason":"支持研究本身所述范围；对本工作组织和收益仍需独立验证。"},{"sourceId":"BC13","role":"context","reason":"支持研究本身所述范围；对本工作组织和收益仍需独立验证。"},{"sourceId":"BC12","role":"context","reason":"支持研究本身所述范围；对本工作组织和收益仍需独立验证。"},{"sourceId":"R-HARNESS2","role":"context","reason":"相邻方法启示，未验证本站案例；原有现场研究及其条件保留。"}]},"caseOptions":[{"id":"C03","title":"熟悉项目维护：先看完整交付，再看生成速度","lead":"补全新代码和改熟悉仓库，是两种任务。AI生成得快，不代表你连同核对和返工也更快。"}],"tasks":[{"id":"C03","domainId":"computing","title":"完成熟悉项目中的维护任务","input":"长期维护者选定的缺陷、功能或重构","output":"符合团队标准的可审阅PR","human":"记录准备、提示、核对、返工与接管的活动人时；机器运行、等待与历时另记，重叠活动不重复累计","acceptance":["质量相当再比较耗时","全部尝试/未完成记录","区分并行等待与人时"],"relation":"真实工作RCT，但仅代表参加研究的仓库、人群和当时工具","workTaskId":null,"actualCoverage":null}],"events":[{"id":"C03-2025","taskId":"C03","availableAt":"2025-07-10","eventDate":"2025-02至2025-06","dateBasis":"按该条支持材料的公开日期进入时间轴；不是实验统一发生日","title":"熟悉大仓库的开发者反而更慢","kind":"caution","mode":"assist","result":"16名开发者完成246个自己项目的真实任务，允许AI时耗时增加19%（区间+2%至+39%）。","aiRole":"辅助编码、查找与迭代；在这批真实维护任务中，净耗时增加19%。","config":"2025年2–6月，主要Cursor Pro+Claude3.5/3.7；熟悉仓库、真实PR质量要求","metric":"相近质量下的任务耗时变化","meaning":"评审与返工可能吃掉编码收益；是否采用需记录全程人时。","limit":"不是所有软件工作的普遍结论；不能与2022 HTTP server实验相减。","sourceIds":["BC06"],"comparison":"任务/配置或指标不同，不计算统一增长率","supersedes":null,"actualCoverage":null,"measurementWindow":{"start":"2025-02","end":"2025-06","precision":"month","label":"2025年2–6月；仅月粒度，不表示每天都在观测"}},{"id":"C03-2026","taskId":"C03","availableAt":"2026-02-24","eventDate":"2025-08起的后续研究","dateBasis":"按该条支持材料的公开日期进入时间轴；不是实验统一发生日","title":"后续看似加速，研究方认为测量仍不可靠","kind":"caution","mode":"assist","result":"后续57名开发者/800多任务；旧参与者点估计耗时−18%，新参与者−4%，两个区间均跨过零。","aiRole":"后续工具继续辅助维护；因样本选择和并行计时，当前净收益大小未可靠确定。","config":"2025年8月起；参与选择、报酬变化与并行计时影响结果","metric":"后续耗时点估计；仅弱证据","meaning":"工具和工作方式有变化，真实净人时趋势仍需新的研究设计。","limit":"研究方明确认为该数据不能可靠衡量当前增益大小；不画收益增长线。","sourceIds":["BC07"],"comparison":"任务/配置或指标不同，不计算统一增长率","supersedes":"C03-2025","actualCoverage":null,"measurementWindow":null},{"id":"C03-field-comparison","taskId":"C03","availableAt":"2026-02-27","eventDate":null,"dateBasis":"按所引用固定版本的公开日进入；不是首次发现或运行日","title":"不同开发情境出现任务数收益","kind":"field","mode":"assist","result":"三项企业现场实验合计4,867名开发者；随机提供代码补全工具的合并估计为完成任务数增加26.08%（标准误10.3%，表示估计的不确定性）。","aiRole":"代码补全辅助日常开发；这批研究记录任务数，未提供本任务完整净人时。","config":"Microsoft、Accenture及匿名大型企业；工具访问随机分配，三项结果有噪声和差异","metric":"完成任务数变化，不能改写为耗时下降","meaning":"与熟悉仓库的耗时实验并列，提示选择本团队任务并测审查返工。","limit":"相近开发情境，不是C03同任务重测；质量、利润与全流程人时不能由摘要推出。","sourceIds":["FS071"],"comparison":"只解释这一来源/任务/配置；不与其他题集连成增长曲线","supersedes":null,"actualCoverage":null,"measurementWindow":null}],"sources":[{"id":"BC03","title":"SWE-bench首稿","url":"https://arxiv.org/html/2310.06770v1","publishedAt":"2023-10-10","kind":"研究基准","locator":"§2任务；§4/表4–5 BM25与oracle条件","limit":"1.96%是Claude2+BM25结果，oracle条件4.8%；不选择性把前者说成最高能力","family":"BC03","readAt":"2026-10-14","readingScope":"本轮直接阅读所列原文位置；未复跑实验","dateBasis":"页面标示发布日期或固定预印本v1页首日期","status":"active"},{"id":"BC06","title":"METR早期2025开发效率RCT","url":"https://metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/","publishedAt":"2025-07-10","kind":"独立工作实验","locator":"Methodology；Core Result；FAQ","limit":"16位熟悉所维护仓库的开发者/246任务；不代表所有程序员","family":"metr-productivity","readAt":"2026-10-14","readingScope":"本轮直接阅读所列原文位置；未复跑实验","dateBasis":"页面标示发布日期或固定预印本v1页首日期","status":"active"},{"id":"BC07","title":"METR后续研究设计修正","url":"https://metr.org/blog/2026-02-24-uplift-update/","publishedAt":"2026-02-24","kind":"方法修正","locator":"首段及Wider adoption/Details of the productivity study","limit":"选择、支付和并行计时问题导致结果仅弱证据；不能据此计算可靠趋势","family":"metr-productivity","readAt":"2026-10-14","readingScope":"本轮直接阅读所列原文位置；未复跑实验","dateBasis":"页面标示发布日期或固定预印本v1页首日期","status":"active"},{"id":"BC08","title":"SWE-bench Verified测量审计","url":"https://openai.com/index/why-we-no-longer-evaluate-swe-bench-verified/","publishedAt":"2026-02-23","kind":"方法修正","locator":"Background；Too narrow and too wide tests；Contamination","limit":"59.4%针对138道困难审计题，不是全部500题；发布方审计","family":"swe-verified","readAt":"2026-10-14","readingScope":"本轮直接阅读所列原文位置；未复跑实验","dateBasis":"页面标示发布日期或固定预印本v1页首日期","status":"active"},{"id":"BC10","title":"SWE Refactor Bench首稿","url":"https://arxiv.org/html/2608.23564v1","publishedAt":"2026-08-24","kind":"研究基准","locator":"§2.2任务预算；§3.1指标；§3.2/表2","limit":"28/520为8模型26配置混合运行结果，不是单一模型通用通过率","family":"BC10","readAt":"2026-10-14","readingScope":"本轮直接阅读所列原文位置；未复跑实验","dateBasis":"页面标示发布日期或固定预印本v1页首日期","status":"active"},{"id":"FS071","title":"开发者三项现场实验（正式期刊版本）","url":"https://pubsonline.informs.org/doi/10.1287/mnsc.2025.00535","publishedAt":"2026-02-27","kind":"现场随机实验论文","locator":"Abstract；Article Information / Published Online；Microsoft Research对应记录","limit":"复用现站FS071同一研究；4,867开发者，完成任务数指标，不是耗时/质量；正式版本日不是首次披露日","family":"developer-three-field-rcts","readAt":"2026-10-15","readingScope":"本轮直接核对所列原文位置/版本；未复跑外部实验","dateBasis":"本次引用期刊Published Online；机构记录已有2025年6月公开摘要","status":"active"},{"id":"E18-J2026","title":"咨询工作的锯齿边界（正式期刊版本）","url":"https://pubsonline.informs.org/doi/10.1287/orsc.2025.21838","publishedAt":"2026-03-11","kind":"知识工作随机实验论文","locator":"Published Online；Abstract；§4.2 Figure 5/Table 7；§6 Discussion（19个百分点）","limit":"与旧E18同一研究家族；咨询任务仅作背景，不是软件维护或财报直接证据；正式发表不是首次公开","family":"bcg-jagged-frontier","readAt":"2026-10-15","readingScope":"本轮直接核对所列原文位置/版本；未复跑外部实验","dateBasis":"本次引用期刊Published Online；已有较早工作论文，不称2026新能力","status":"active"},{"id":"BC12","title":"SWE-Bench Pro V2协议修订","url":"https://labs.scale.com/leaderboard/swe_bench_pro_public_v2?tab=full","publishedAt":"2026-09-22","kind":"评测发布方修订说明","locator":"Update September 22, 2026，任务筛选与隔离重新评分","limit":"发布方自述修订，本轮未独立复跑；V1/V2分母与协议不同","family":"swe-bench-pro-v2","readAt":"2026-10-16","readingScope":"本轮直接阅读所列原文和固定版本；未复跑外部实验","dateBasis":"页面明确标修订日期，不视为原始SWE-Bench Pro首次公开日","status":"active"},{"id":"BC13","title":"AI参与修复PR的接收与未合并原因（v1）","url":"https://arxiv.org/html/2602.00164v1","publishedAt":"2026-01-29","kind":"软件工程观察研究","locator":"§2.1–2.3；表1；§3.2 R1/R2例子","limit":"样本非随机分配代理；65%分母含开放PR；未合并不全是技术失败","family":"aidev-pr-integration","readAt":"2026-10-16","readingScope":"本轮直接阅读所列原文和固定版本；未复跑外部实验","dateBasis":"固定v1 arXiv摘要页标2026-01-29提交","status":"active"},{"id":"R-HARNESS2","title":"Harness²：用执行对照改进运行方法","url":"https://arxiv.org/html/2610.14224v1","publishedAt":"2026-10-09","kind":"论文","locator":"§2.1–2.3；§3.1；§4","limit":"作者研究，本站未复现；行为对照不能验证正确性，多次运行和编辑器成本需计入。","family":"R-HARNESS2","readAt":"2026-10-19","readingScope":"本轮实际阅读指定方法段落，未复跑","dateBasis":"arXiv固定v1公开日期","status":"active"}],"scope":"全模块16个代表任务切片；当前聚焦1个C03；不是完整职业清单或任务覆盖比例。","actualCoverage":null,"knowledgeRule":"主轴按原始材料公开日期；实验/上线时点另列。历史空白仅表示本次未收录证据；后来的质疑不偷偷改写早期结果。","method":{"coverage":"覆盖的含义：在指定输入、环境和验收下，材料支持AI参与哪些动作；未知比例保持null。","speed":"速度的含义：证据节点之间的日历间隔、流程交接变化；指标/样本/预算不一致时，不计算能力增长率或外推一年三年。","sourcePolicy":"综述用于发现任务和原始文献；每条公开主张引用原始论文/官方记录。转载及同一文献家族不累计成独立验证。","selection":"目的性选择三个领域的代表任务，不是穷尽检索或无偏职业采样；基础时间轴于2026-10-14核对，本次精选补充与证据截止2026-10-15，固定论文优先，不把动态榜排名倒填历史。"}}