{"version":"1.0","generated_at":"2026-09-28T07:54:35.313913","id":916,"slug":"bytedance-study-finds-that-asking-lmms-questions-beats-making-it-transcribe-text-for-long-document-training","title":"趋势解读：ByteDance study finds that asking LMMs questions beats，评估 LLM Agent 表现","summary":"字节跳动与港科大联合研究发现，使用问答对训练多模态模型能显著提升长文档理解能力，而字符识别任务反而损害性能。基于此训练的MMProLong模型在预算有限的情况下超越了多个大型开源模型。","abstract":"趋势解读：ByteDance study finds that asking LMMs questions beats，评估 LLM Agent 表现 字节跳动与港科大联合研究发现，使用问答对训练多模态模型能显著提升长文档理解能力，而字符识别任务反而损害性能。基于此训练的MMProLong模型在预算有限的情况下超越了多个大型开源模型。 字符识别训练损害多模态模型长文本性能 问答对训练显著提升长文档理解能力 长短混合训练并非必要，长问答数据即可保持短任务能力 信息提取任务比推理计算更关键 低成本训练（12.8万tokens）超越大模型 多模态AI模型本应处理越来越长的文档，但如何训练它们这样做通常仍是商业机密。一项新研究表明，字符识别作为训练任务实际上损害了性能，而问答对的效果要好得多。 这是什么信号： 字节跳动Seed团队与港科大的研究揭示了一个反直觉的发现：在训练多模态长文档理解模型时，使用字符识别任务会拉低性能，而基于问答对的训练方法能带来显著提升。这意味着当前主流模型训练中“认字”优先的思路可能需要重新审视。 为什么重要： 随着多模态模型需要处理长篇PDF、视频和智能体记忆，长上下文能力成为关键竞争点。然而各AI实验室对训练数据的配方秘而不宣。该研究开源了MMProLong模型（基于Qwen2.5-VL），仅用12.8万tokens的训练预算就打败了InternVL3-38B和Gemma3-27B等更大模型，为行业提供了一个低成本、高回报的训练范式。 对谁有价值： AI研究者可直接复用其方法优化长文档理解模型；企业AI团队可减少训练成本并提升产品体验（如文档问答、视频分…","access_level":"member","access_label":"免费阅读","access_mode":"full","is_preview":false,"canonical_url":"https://opc.beizhux.com/content/916/bytedance-study-finds-that-asking-lmms-questions-beats-making-it-transcribe-text-for-long-document-training","html_url":"https://opc.beizhux.com/content/916/bytedance-study-finds-that-asking-lmms-questions-beats-making-it-transcribe-text-for-long-document-training","json_url":"https://opc.beizhux.com/content/916/bytedance-study-finds-that-asking-lmms-questions-beats-making-it-transcribe-text-for-long-document-training.json","published_at":"2026-05-24T21:28:45","updated_at":"2026-09-28T07:17:39","category":{"slug":"ai-skills","name":"AI技能杠杆"},"source":{"site":"the-decoder.com","author":"Jonathan Kemper","url":"https://the-decoder.com/bytedance-study-finds-that-asking-lmms-questions-beats-making-it-transcribe-text-for-long-document-training/"},"tags":["AI","LLM训练","The Decoder","多模态AI","字节跳动","研究","自动化","长文档理解","问答对"],"topics":[{"slug":"agent-workflow","name":"Agent工作流","url":"https://opc.beizhux.com/topics/agent-workflow","reason":"这篇内容命中「Agent、工作流」等主题信号。"},{"slug":"ai-tools","name":"AI工具","url":"https://opc.beizhux.com/topics/ai-tools","reason":"这篇内容命中「自动化、模型」等主题信号。"},{"slug":"ai-super-individual","name":"AI超级个体","url":"https://opc.beizhux.com/topics/ai-super-individual","reason":"这篇内容命中「技能」等主题信号。"}],"keywords":["AI技能杠杆","Agent工作流","AI工具","AI超级个体","工具","自动化","模型","Cursor","Claude","Agent","智能体","工作流"],"questions":[{"question":"趋势解读：ByteDance study finds that asking LMMs questions beats，评估 LLM Agent 表现主要讲什么？","answer":"字节跳动与港科大联合研究发现，使用问答对训练多模态模型能显著提升长文档理解能力，而字符识别任务反而损害性能。基于此训练的MMProLong模型在预算有限的情况下超越了多个大型开源模型。"},{"question":"这篇文章最值得关注的要点是什么？","answer":"字节跳动与港科大联合研究发现，使用问答对训练多模态模型能显著提升长文档理解能力，而字符识别任务反而损害性能。基于此训练的MMProLong模型在预算有限的情况下超越了多个大型开源模型。；字符识别训练损害多模态模型长文本性能；问答对训练显著提升长文档理解能力；长短混合训练并非必要，长问答数据即可保持短任务能力"},{"question":"这篇文章和哪些AI专题相关？","answer":"它适合放在Agent工作流、AI工具、AI超级个体专题里阅读。 关联原因：这篇内容命中「Agent、工作流」等主题信号。；这篇内容命中「自动化、模型」等主题信号。；这篇内容命中「技能」等主题信号。"},{"question":"阅读这篇文章建议先理解哪些关键词？","answer":"建议先理解AI工具、工具、自动化、模型、Cursor这些关键词，再结合正文判断工具、机会或风险是否值得进入自己的工作流。"}],"terms":[{"slug":"ai-tools-term","name":"AI工具","definition":"在AI觉醒星球里，「AI工具」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/ai-tools-term","path":"/glossary/ai-tools-term","json_url":"https://opc.beizhux.com/glossary/ai-tools-term.json"},{"slug":"tools","name":"工具","definition":"在AI觉醒星球里，「工具」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/tools","path":"/glossary/tools","json_url":"https://opc.beizhux.com/glossary/tools.json"},{"slug":"automation","name":"自动化","definition":"在AI觉醒星球里，「自动化」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/automation","path":"/glossary/automation","json_url":"https://opc.beizhux.com/glossary/automation.json"},{"slug":"ai-model","name":"模型","definition":"在AI觉醒星球里，「模型」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/ai-model","path":"/glossary/ai-model","json_url":"https://opc.beizhux.com/glossary/ai-model.json"},{"slug":"cursor","name":"Cursor","definition":"在AI觉醒星球里，「Cursor」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/cursor","path":"/glossary/cursor","json_url":"https://opc.beizhux.com/glossary/cursor.json"},{"slug":"claude","name":"Claude","definition":"在AI觉醒星球里，「Claude」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/claude","path":"/glossary/claude","json_url":"https://opc.beizhux.com/glossary/claude.json"},{"slug":"agent","name":"Agent","definition":"在AI觉醒星球里，「Agent」属于「Agent工作流」方向。聚合Agent、AI编程、Claude Code、Cursor、工作流自动化等内容，帮助你把智能体能力落到真实任务链路。 把智能体当成任务系统，而不是聊天窗口。","topic_slug":"agent-workflow","topic_name":"Agent工作流","topic_title":"Agent工作流：智能体、AI编程与自动化协作","topic_url":"https://opc.beizhux.com/topics/agent-workflow","topic_path":"/topics/agent-workflow","url":"https://opc.beizhux.com/glossary/agent","path":"/glossary/agent","json_url":"https://opc.beizhux.com/glossary/agent.json"},{"slug":"ai-agent","name":"智能体","definition":"在AI觉醒星球里，「智能体」属于「Agent工作流」方向。聚合Agent、AI编程、Claude Code、Cursor、工作流自动化等内容，帮助你把智能体能力落到真实任务链路。 把智能体当成任务系统，而不是聊天窗口。","topic_slug":"agent-workflow","topic_name":"Agent工作流","topic_title":"Agent工作流：智能体、AI编程与自动化协作","topic_url":"https://opc.beizhux.com/topics/agent-workflow","topic_path":"/topics/agent-workflow","url":"https://opc.beizhux.com/glossary/ai-agent","path":"/glossary/ai-agent","json_url":"https://opc.beizhux.com/glossary/ai-agent.json"}],"preview_hidden_sections":[],"related_items":[{"id":3094,"title":"研究发现：AI 访问让人们几乎完全不愿说“我不知道”","url":"https://opc.beizhux.com/content/3094/ai-access-makes-people-almost-entirely-unwilling-to-say-i-don-t-know-study-finds","summary":"一项涵盖 3,132 名参与者的五项实验发现，只要 AI 建议可用，人们就几乎不再愿意说“我不知道”，即使 AI 答案大多错误。参与者对 AI 答案更自信，但正确率显著下降；财务激励能减少求助 AI，却无法显著改变这种判断放弃效应。","access_level":"member"},{"id":3081,"title":"Arena：GPT-6 Sol （Max） 以 +7.7% 净改进重塑 Agent Arena Pareto 前沿","url":"https://opc.beizhux.com/content/3081/arena-gpt-6-sol-max-7-7-agent-arena-pareto","summary":"Arena 宣布 OpenAI 的 GPT-6 Sol（Max）进入 Agent Arena，基于 4K+ 真实智能体会话取得 +7.7% 净改进，排名第 6，中位成本 $0.75/task。榜单方给出成本与得分对比，便于评估其性价比位置。","access_level":"member"},{"id":3105,"title":"Meta 的 Muse agent 为每位用户提供运行 Ubuntu Linux 的完整云电脑","url":"https://opc.beizhux.com/content/3105/meta-s-muse-agent-gives-every-user-a-full-cloud-computer-running-ubuntu-linux","summary":"Meta 的 Muse 为每位用户提供免费完整的 Ubuntu Linux 云电脑，可安装软件、编写编译代码和浏览网页；采用 Muse Secure VM、Runtime Cell 和 Sentinel 隔离敏感操作，首周获超 50 万用户并登顶 App Store。","access_level":"member"}],"usage_policy":{"summarizable":true,"preferred_url":"https://opc.beizhux.com/content/916/bytedance-study-finds-that-asking-lmms-questions-beats-making-it-transcribe-text-for-long-document-training","private_fields_excluded":true}}