敏捷深思：AI 如何帮助人类定义灰色地带

Otilia Stretcu, Research Scientist, and Leijie Wang, Student Researcher, Google Research Agile Deliberation addresses the challenges of building AI image classifiers for subjective categories by guiding users through an interactive deliberation workflow, from scoping sub-categories to resolving edge cases, using an LLM-driven optimization loop to automatically build an accurate, aligned classifier. Machine learning models learn to recognize the world around us from examples labeled by humans. Underlying this process is an assumption that the person doing the labeling has a clear and stable definition of the category in mind before they begin. This assumption holds well when training a model on objective categories, like a stop sign or a golden retriever, where boundaries are clear and universally recognized. In practice, however, real-world applications often require vision systems to navigate the subjective "gray areas" of human experience. For example, consider training a vision classifier to identify "healthy food", curate "inspiring artwork”, or assist in content moderation. In such domains, category boundaries may not be universal or static, since they can shift with cultural context, personal philosophies, or evolving operational policies. Furthermore, people rarely start with a complete specification for subjective concepts, even in their own minds. Instead, they begin with an intuitive idea of a concept and only discover the concept’s true boundaries, exceptions, and nuances as they encounter real-world examples. Traditional training workflows don’t account for this concept evolution , forcing users to provide rigid labels to an AI system even when their own mental model is still shifting. To bridge this gap, we introduce Agile Deliberation , a framework that treats the creation of subjective AI classifiers as an iterative conversation. Rather than requiring humans to act as static data labelers, Agile Deliberation turns the AI system into a collaborative partner that helps users "deliberate" and sharpen their own definitions. By resolving human ambiguity first, the framework systematically builds more effective, robust, and aligned AI systems. To support the wider research community, we have also open-sourced the underlying code and algorithms on GitHub . A key differentiator of this work is its focus on the live user experience. Because standard, static AI benchmarks can’t capture subjective concepts, we bypassed simulated testing and evaluated Agile Deliberation across 18 live, 1.5-hour sessions with human participants. This allowed us to directly observe and measure how a person's intent dynamically evolves when paired with an intelligent system. We found that Agile Deliberation drastically reduced user fatigue and anxiety by replacing manual prompt engineering with direct feedback on borderline images, which resulted in more consistent, higher-performing AI models that outperform standard baselines. Agile Deliberation splits the complex task of defining a subjective concept into two structured stages: concept scoping and concept iteration . Together, these two phases transform a fuzzy idea into a highly precise instructional blueprint for an AI model. The Agile Deliberation workflow. The framework decomposes a broad target concept into in-scope (i.e., positive examples that the classifier should recognize and flag as matching the user's intent) and out-of-scope (i.e., explicit negative boundaries or "false alarms" that the user wants the classifier not to flag) definitions, retrieves borderline images to expose edge cases, and guides human feedback into an automatic prompt refinement loop to generate ready-to-use image classifiers. When confronted with a broad topic like "Healthy Food," human intuition naturally struggles to list every rule upfront. In the concept scoping stage, the system guides the user to break down their overarching concept into a structured hierarchy of sub-concepts and explicit exclus