{"version":"1.0","generated_at":"2026-10-14T13:04:48.956220","id":1666,"slug":"cloudflare-replaces-its-blanket-ai-bot-block-with-granular-controls-for-search-training-and-agent-crawlers","title":"Cloudflare 将全面封禁 AI 机器人改为按搜索、训练和代理爬虫的精细控制","summary":"Cloudflare 为所有客户提供新工具，让网站所有者按搜索、训练和代理三类分别控制 AI 爬虫访问权限；从 2026 年 9 月 15 日起，带广告的页面将默认阻止训练和代理机器人，同时企业版新增 BotBase 数据库，验证标签不再足以自动放行。","abstract":"Cloudflare 将全面封禁 AI 机器人改为按搜索、训练和代理爬虫的精细控制 Cloudflare 为所有客户提供新工具，让网站所有者按搜索、训练和代理三类分别控制 AI 爬虫访问权限；从 2026 年 9 月 15 日起，带广告的页面将默认阻止训练和代理机器人，同时企业版新增 BotBase 数据库，验证标签不再足以自动放行。 Cloudflare 为所有客户提供新工具，让网站所有者按搜索、训练和代理三类分别控制 AI 爬虫访问权限；从 2026 年 9 月 15 日起，带广告的页面将默认阻止训练和代理机器人，同时企业版新增 BotBase 数据库，验证标签不再足以自动放行。 原贴提到：Cloudflare gives all customers new tools to control how AI bots access t 来源：the-decoder.com Cloudflare 为所有客户提供控制 AI 机器人访问其网站的新工具。自 2024 年 7 月起，Cloudflare 客户只需单击一次即可阻止所有 AI 爬虫。现在，网站所有者不再使用全面封禁，而是可以在三个类别之间进行选择：搜索（搜索引擎索引）、训练（AI 模型训练数据收集）、代理（代表用户行动的机器人，如 ChatGPT）。 这是什么信号？ Cloudflare 从“一刀切”式禁止 AI 爬虫转向按用途精细化管理，表明 AI 爬虫流量已成为网站运营的核心变量。这既是技术策略调整，也是商业博弈——通过分级控制，Cloudflare 既维护了普通网站的利益（防止被无偿抓取训练），又为付费企业客户提供…","access_level":"member","access_label":"免费阅读","access_mode":"full","is_preview":false,"canonical_url":"https://opc.beizhux.com/content/1666/cloudflare-replaces-its-blanket-ai-bot-block-with-granular-controls-for-search-training-and-agent-crawlers","html_url":"https://opc.beizhux.com/content/1666/cloudflare-replaces-its-blanket-ai-bot-block-with-granular-controls-for-search-training-and-agent-crawlers","json_url":"https://opc.beizhux.com/content/1666/cloudflare-replaces-its-blanket-ai-bot-block-with-granular-controls-for-search-training-and-agent-crawlers.json","published_at":"2026-07-07T02:54:45","updated_at":"2026-10-14T11:57:18","category":{"slug":"ai-business","name":"AI小生意项目库"},"source":{"site":"the-decoder.com","author":"Matthias Bastian","url":"https://the-decoder.com/cloudflare-replaces-its-blanket-ai-bot-block-with-granular-controls-for-search-training-and-agent-crawlers/"},"tags":["AI","AI机器人","AI爬虫","AI训练","AI训练数据","BotBase","Cloudflare","The Decoder","代理","内容保护","内容控制","广告"],"topics":[{"slug":"ai-side-business","name":"AI副业","url":"https://opc.beizhux.com/topics/ai-side-business","reason":"这篇内容命中「项目、小生意、变现」等主题信号。"},{"slug":"ai-tools","name":"AI工具","url":"https://opc.beizhux.com/topics/ai-tools","reason":"这篇内容命中「工具、自动化」等主题信号。"},{"slug":"ai-content-growth","name":"AI内容增长","url":"https://opc.beizhux.com/topics/ai-content-growth","reason":"这篇内容命中「流量、内容」等主题信号。"}],"keywords":["AI小生意项目库","AI副业","AI工具","AI内容增长","工具","自动化","模型","Cursor","Claude","Agent","工作流","副业"],"questions":[{"question":"Cloudflare 将全面封禁 AI 机器人改为按搜索、训练和代理爬虫的精细控制主要讲什么？","answer":"Cloudflare 为所有客户提供新工具，让网站所有者按搜索、训练和代理三类分别控制 AI 爬虫访问权限；从 2026 年 9 月 15 日起，带广告的页面将默认阻止训练和代理机器人，同时企业版新增 BotBase 数据库，验证标签不再足以自动放行。"},{"question":"这篇文章最值得关注的要点是什么？","answer":"Cloudflare 为所有客户提供新工具，让网站所有者按搜索、训练和代理三类分别控制 AI 爬虫访问权限；从 2026 年 9 月 15 日起，带广告的页面将默认阻止训练和代理机器人，同时企业版新增 BotBase 数据库，验证标签不再…；原贴提到：Cloudflare gives all customers new tools to control how AI bots access t；来源：the-decoder.com"},{"question":"这篇文章和哪些AI专题相关？","answer":"它适合放在AI副业、AI工具、AI内容增长专题里阅读。 关联原因：这篇内容命中「项目、小生意、变现」等主题信号。；这篇内容命中「工具、自动化」等主题信号。；这篇内容命中「流量、内容」等主题信号。"},{"question":"阅读这篇文章建议先理解哪些关键词？","answer":"建议先理解AI工具、工具、自动化、模型、Cursor这些关键词，再结合正文判断工具、机会或风险是否值得进入自己的工作流。"}],"terms":[{"slug":"ai-tools-term","name":"AI工具","definition":"在AI觉醒星球里，「AI工具」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/ai-tools-term","path":"/glossary/ai-tools-term","json_url":"https://opc.beizhux.com/glossary/ai-tools-term.json"},{"slug":"tools","name":"工具","definition":"在AI觉醒星球里，「工具」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/tools","path":"/glossary/tools","json_url":"https://opc.beizhux.com/glossary/tools.json"},{"slug":"automation","name":"自动化","definition":"在AI觉醒星球里，「自动化」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/automation","path":"/glossary/automation","json_url":"https://opc.beizhux.com/glossary/automation.json"},{"slug":"ai-model","name":"模型","definition":"在AI觉醒星球里，「模型」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/ai-model","path":"/glossary/ai-model","json_url":"https://opc.beizhux.com/glossary/ai-model.json"},{"slug":"cursor","name":"Cursor","definition":"在AI觉醒星球里，「Cursor」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/cursor","path":"/glossary/cursor","json_url":"https://opc.beizhux.com/glossary/cursor.json"},{"slug":"claude","name":"Claude","definition":"在AI觉醒星球里，「Claude」属于「AI工具」方向。围绕AI工具、模型能力、自动化流程和真实使用场景做整理，优先关注能提升效率、降低成本和创造新交付的工具。 不只看工具热度，更看它能不能进入真实流程。","topic_slug":"ai-tools","topic_name":"AI工具","topic_title":"AI工具：模型、插件、自动化与实战场景","topic_url":"https://opc.beizhux.com/topics/ai-tools","topic_path":"/topics/ai-tools","url":"https://opc.beizhux.com/glossary/claude","path":"/glossary/claude","json_url":"https://opc.beizhux.com/glossary/claude.json"},{"slug":"agent","name":"Agent","definition":"在AI觉醒星球里，「Agent」属于「Agent工作流」方向。聚合Agent、AI编程、Claude Code、Cursor、工作流自动化等内容，帮助你把智能体能力落到真实任务链路。 把智能体当成任务系统，而不是聊天窗口。","topic_slug":"agent-workflow","topic_name":"Agent工作流","topic_title":"Agent工作流：智能体、AI编程与自动化协作","topic_url":"https://opc.beizhux.com/topics/agent-workflow","topic_path":"/topics/agent-workflow","url":"https://opc.beizhux.com/glossary/agent","path":"/glossary/agent","json_url":"https://opc.beizhux.com/glossary/agent.json"},{"slug":"workflow","name":"工作流","definition":"在AI觉醒星球里，「工作流」属于「Agent工作流」方向。聚合Agent、AI编程、Claude Code、Cursor、工作流自动化等内容，帮助你把智能体能力落到真实任务链路。 把智能体当成任务系统，而不是聊天窗口。","topic_slug":"agent-workflow","topic_name":"Agent工作流","topic_title":"Agent工作流：智能体、AI编程与自动化协作","topic_url":"https://opc.beizhux.com/topics/agent-workflow","topic_path":"/topics/agent-workflow","url":"https://opc.beizhux.com/glossary/workflow","path":"/glossary/workflow","json_url":"https://opc.beizhux.com/glossary/workflow.json"}],"preview_hidden_sections":[],"related_items":[{"id":3398,"title":"Character.ai 破产文件显示 AI 商业模式难以为继","url":"https://opc.beizhux.com/content/3398/character-ai-ai","summary":"Character.ai 进入破产程序，法院文件披露其算力支出与收入严重失衡：2024 年月度算力支出 1380 万美元、2025 年为 1000 万美元，订阅收入峰值仅每月 280 万美元，含广告全部收入峰值每月 530 万美元；2026 年 1 至 8 月算力支出 5610 万美元，8 月时每月仍亏损约 800-…","access_level":"member"},{"id":3399,"title":"OpenAI 办公室发现窃密录音装置，涉事员工被捕","url":"https://opc.beizhux.com/content/3399/openai-20","summary":"OpenAI 在办公室发现隐蔽录音装置，利用自家模型分析确认系一名员工几天前安放。公司已解雇该员工并移交 FBI，FBI 迅速逮捕相关人员。OpenAI 称暂无迹象显示模型权重、源代码或客户数据被不当访问或外泄，并将此案视为 AI 协助应对复杂安全威胁的实例。","access_level":"member"},{"id":3402,"title":"XBOW 团队借助 OpenAI GPT-5.6 Sol 获 Google Chrome 25 万美元全链漏洞赏金","url":"https://opc.beizhux.com/content/3402/xbow-openai-gpt-5-6-sol-google-chrome-25","summary":"XBOW 团队使用 OpenAI GPT-5.6 Sol 辅助，6 天内以 4 万美元算力构建 Chrome 全链漏洞，获 Google 25 万美元赏金；这是 Google 第二次发放该类赏金、2026 年新规则下首次，影响 Windows 版 Chrome。","access_level":"member"}],"usage_policy":{"summarizable":true,"preferred_url":"https://opc.beizhux.com/content/1666/cloudflare-replaces-its-blanket-ai-bot-block-with-granular-controls-for-search-training-and-agent-crawlers","private_fields_excluded":true}}