Learning Outcomes
Slide Idea
This slide articulates three interrelated learning objectives for working with generative AI systems. By the end of instruction, students should be able to translate ambiguous creative intent into structured specifications, identify three categories of failure modes that arise when systems encounter ambiguity or operate beyond their capabilities or ethical boundaries, and document at least one revision decision with explicit justification showing the reasoning that informed the change.
Key Concepts & Definitions
Constrained Specification
A constrained specification is a formalized description of a design or system that defines both what the artifact should accomplish (functional requirements) and the boundaries within which it must operate (constraints). In design thinking methodology, constraints typically encompass feasibility (technical capability), viability (economic sustainability), and desirability (user needs). Specifications reduce ambiguity by translating open-ended creative intent into parameters that guide implementation and evaluation. The transformation from creative intent to specification requires identifying essential features, establishing success criteria, and acknowledging limitations.
Source: Brown, T. (2008). Design thinking. Harvard Business Review, 86(6), 84-92.
Ambiguity (as Failure Mode)
In generative AI systems, ambiguity refers to unclear, vague, or underspecified inputs that prevent a system from determining user intent or generating appropriate outputs. Ambiguous prompts lack necessary context, contain contradictory instructions, or use language susceptible to multiple interpretations. This constitutes a failure mode because the system cannot reliably map input to meaningful output—the request itself is malformed relative to the system's input requirements. Ambiguity differs from other failure modes in that it originates in the communication interface rather than in the system's technical or ethical limitations.
Source: Raji, I. D., et al. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 33-44.
Model Limitation (as Failure Mode)
Model limitation refers to situations where a generative AI system lacks the capability—due to training data gaps, architectural constraints, or computational boundaries—to perform a requested task. These are inherent technical boundaries that exist regardless of how well the request is formulated. Model limitations manifest as inability to process certain input types, generate outputs requiring knowledge beyond training data, or perform reasoning tasks that exceed the system's architectural capacity. Unlike ambiguity, model limitations represent the frontier of what the system can technically achieve.
Source: Bender, E. M., et al. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610-623.
Ethical Risk (as Failure Mode)
Ethical risk in AI systems encompasses potential harms to individuals, groups, or society that may result from system deployment or use, including discrimination, privacy violations, misinformation amplification, or violations of fundamental rights. This failure mode differs from technical limitations in that the system may be capable of generating the requested output, but doing so would produce harm that outweighs potential benefits. Ethical risks require assessment of consequences, stakeholder impacts, and alignment with values that extend beyond functional requirements.
Source: Gebru, T., et al. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86-92.
Design Rationale Documentation
Design rationale documentation is the practice of recording not only what decisions were made during a design or development process, but also why those decisions were chosen, what alternatives were considered, and what criteria informed the selection. This documentation creates a historical record that supports traceability, facilitates iteration, enables team alignment, and makes implicit reasoning explicit. In educational contexts, requiring documented rationale helps students develop metacognitive awareness of their decision-making processes and learn to justify choices with evidence rather than preference.
Source: Cross, N. (2011). Design thinking: Understanding how designers think and work. Berg Publishers.
Revision Decision
A revision decision is a deliberate choice to modify, refine, or fundamentally change an aspect of work based on evaluation against criteria, discovery of new information, or changed understanding of requirements. Unlike initial design decisions, revisions occur in response to feedback, testing, or reflection and represent iterative improvement. Documenting revision decisions requires articulating what changed, what prompted the change, what evidence supported the decision, and what consequences or trade-offs the change introduces.
Source: Schön, D. A. (1983). The reflective practitioner: How professionals think in action. Basic Books.
Why This Matters for Students' Work
These three learning outcomes establish a progression from input formulation through failure diagnosis to iterative improvement—a cycle fundamental to working with any complex system, generative or otherwise.
The ability to translate creative intent into constrained specifications addresses a persistent challenge in creative and technical work: bridging the gap between vision and implementation. Students often begin projects with expansive ideas but struggle to operationalize them. Learning to identify essential requirements, acknowledge constraints, and articulate success criteria enables movement from aspiration to actionable plan. This skill transfers across disciplines—specifying a film production requires different technical knowledge than specifying a software application, but both demand the same analytical process of defining scope, constraints, and success criteria.
Identifying failure modes develops diagnostic thinking. When generative systems produce unsatisfactory results, students must determine whether the problem originated in their specification (ambiguity), exceeded the system's capabilities (model limitation), or raised concerns about consequences (ethical risk). This diagnostic capacity prevents common pitfalls: iterating endlessly on poorly-formed requests, attempting tasks systems cannot perform, or pursuing outputs that may cause harm. Beyond generative AI contexts, this pattern of failure mode analysis appears throughout systems engineering, design thinking, and creative production. Students who can categorize failures can address root causes rather than treating symptoms.
Documenting revision decisions builds the metacognitive practice of making reasoning visible. Professional creative and technical work involves continuous revision, but the value of iteration depends on learning from each cycle. Students who document what changed and why create an externalized record of their thinking that can be reviewed, shared, and refined. This practice supports both individual learning—making implicit decision criteria explicit—and collaborative work, where team members must understand the logic behind choices. In professional contexts, design rationale documentation enables teams to reconstruct reasoning months or years later, avoid revisiting rejected alternatives, and maintain consistency across projects.
Together, these outcomes scaffold a complete cycle: specify clearly, diagnose failures accurately, and learn from revisions systematically. This cycle applies whether students are producing films, designing interfaces, writing code, or conducting research. The specific technologies and methods change; the underlying process of intentional, reflective, iterative work remains constant.
How This Shows Up in Practice (Non-Tool-Specific)
Translating Creative Intent into Constrained Specification
In film pre-production, a director's creative vision ("a tense confrontation scene") must be translated into constrained specifications before production can begin: location requirements (interior/exterior, lighting conditions, acoustic properties), cast needs (number of actors, character relationships), equipment specifications (camera angles, lens choices, audio capture), and time constraints (scene duration, shooting schedule). Each constraint narrows possibilities while enabling concrete planning.
In writing, an assignment to "write persuasively about climate change" becomes specified through constraints: target audience (policy makers vs. general public), word count (800 words), required sources (minimum five peer-reviewed), tone (formal academic vs. accessible journalism), and argumentative structure (problem-solution vs. comparative analysis). These constraints do not eliminate creativity—they channel it productively.
In software development, a request for "an intuitive user interface" translates into specifications addressing accessibility standards (WCAG compliance level), platform constraints (responsive design breakpoints), interaction patterns (touch vs. cursor, gesture vocabulary), visual hierarchy principles, and performance requirements (maximum load time, minimum supported devices). The specification makes "intuitive" measurable and actionable.
Identifying Three Failure Modes
Ambiguity: A cinematographer asked to create "dramatic lighting" might reasonably interpret this as high-contrast chiaroscuro, Rembrandt lighting, or stylized color gels. The failure is not in the cinematographer's capability—it is in the underspecified request. Clarification requires specifying mood, visual references, narrative function, and technical parameters.
Model Limitation: A novice film editor cannot produce effects that require advanced compositing skills they have not yet developed. The creative vision may be clear, but it exceeds current technical capability. Addressing this limitation requires either skill development, collaboration with specialists, or revision of creative goals to match available capabilities.
Ethical Risk: A documentary filmmaker gathering footage must consider whether capturing certain scenes might endanger subjects, violate privacy expectations, or perpetuate harmful stereotypes. The equipment can technically record the content, but ethical considerations may prevent its capture or use. Responsible practice requires assessing consequences beyond technical feasibility.
Documenting One Revision Decision with Clear Justification
In design iteration, a product team might document: "Revised navigation from hamburger menu to tab bar based on usability testing. Three of five participants failed to locate key features within menu structure. Tab bar increases surface area of primary actions, aligns with platform conventions users expect, and improves task completion rates. Trade-off: reduced screen space for content. Justification: task success prioritizes content density for this application type."
In research writing, documentation might read: "Revised thesis statement from descriptive to argumentative framing. The initial version stated what happened; revision argues why it matters and what readers should conclude. Change prompted by peer feedback indicating unclear purpose. New framing provides clearer organizational logic and establishes stakes. Evidence from sources now supports claims rather than illustrating facts."
In production planning, revision documentation: "Shifted shoot schedule from chronological to location-based sequencing. Initial plan minimized actor confusion by filming in story order. Revision consolidates all scenes per location to reduce company moves. Prompted by budget analysis showing transport costs exceeding 15% of total. Mitigation: detailed script breakdown for actors, additional rehearsal time. Justification: $8,000 savings enables two additional shoot days."
Common Misunderstandings
"Constraints limit creativity"
This misunderstanding inverts the relationship between constraints and creative problem-solving. Constraints do not eliminate creativity—they define the problem space within which creative solutions emerge. Unlimited possibility often produces paralysis; bounded problems invite inventive responses. The confusion arises from conflating constraints (which focus creative energy) with restrictions (which may prevent optimal solutions). Constraints are intrinsic to any real-world problem; restrictions may be arbitrary or counterproductive. Learning to work productively within constraints differs from accepting unnecessary limitations.
"Failure modes are isolated problems with discrete solutions"
Students sometimes treat the three failure modes as separate diagnostic categories requiring distinct fixes. In practice, these modes frequently interact and compound each other. An ambiguous specification may mask model limitations—the system fails not solely because the request was unclear, but because no achievable specification exists within current capabilities. Ethical risks often emerge from combinations: ambiguous objectives pursued through capable-but-misapplied systems. Effective diagnosis requires considering how failure modes interact rather than selecting a single category.
"Revision decisions correct mistakes"
This framing positions revision as remediation of error rather than iterative refinement of understanding. Many revision decisions do not fix mistakes—they respond to new information, changed requirements, feedback from testing, or evolved understanding of the problem. A documented revision decision may record a successful pivot rather than error correction. The misunderstanding leads students to view revision as evidence of initial failure rather than as intrinsic to design and creative processes. Professional practice involves continuous revision as understanding deepens; this is expertise, not incompetence.
"Documentation of rationale is bureaucratic overhead"
Students sometimes perceive documentation requirements as administrative burden separate from the actual work of creation or problem-solving. This misses the cognitive function of documentation: the act of articulating reasoning clarifies thinking, reveals gaps in logic, and creates external memory that supports learning. Documentation serves the documenter as much as future readers. The misunderstanding treats documentation as output for evaluation rather than as tool for thought. Professionals document because it improves decision quality, not solely for compliance or communication purposes.
Scholarly Foundations
Cross, N. (2011). Design thinking: Understanding how designers think and work. Berg Publishers.
Cross examines how expert designers navigate problem-solving under constraints, emphasizing that design problems are characteristically ill-defined and require iterative cycles of specification, generation, and evaluation. Particularly relevant for understanding how creative intent translates into constrained specifications through a process of problem framing and reframing.
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020). Closing the AI accountability gap: Defining an end-to-end framework for internal algorithmic auditing. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 33-44.
Introduces the SMACTR framework for algorithmic auditing throughout the AI development lifecycle, emphasizing identification of failure modes including model limitations, data quality issues, and ethical risks. Provides structured methodology for diagnosing system failures beyond technical performance metrics.
Collins, A., Brown, J. S., & Holum, A. (1991). Cognitive apprenticeship: Making thinking visible. American Educator, 15(3), 6-11, 38-46.
Describes methods for teaching complex cognitive skills by making expert thinking processes explicit through modeling, coaching, scaffolding, and reflection. Central to understanding why documentation of revision decisions supports learning: it externalizes reasoning that would otherwise remain implicit.
Schön, D. A. (1983). The reflective practitioner: How professionals think in action. Basic Books.
Foundational text on reflection-in-action and reflection-on-action, examining how professionals make decisions in situations of uncertainty and complexity. Essential for understanding revision decisions as sites of learning rather than error correction, and for framing documentation as a tool for developing reflective practice.
Brown, T. (2008). Design thinking. Harvard Business Review, 86(6), 84-92.**
Articulates the design thinking framework emphasizing the intersection of desirability, feasibility, and viability as the space where innovation occurs. Directly relevant for understanding how constraints in specifications balance user needs, technical capabilities, and practical limitations.
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021). On the dangers of stochastic parrots: Can language models be too big? Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, 610-623.
Examines inherent limitations of large language models including environmental costs, inscribed biases, inability to understand meaning, and risks of deployment in high-stakes contexts. Critical for understanding model limitations as a failure mode category distinct from ambiguous inputs.
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé III, H., & Crawford, K. (2021). Datasheets for datasets. Communications of the ACM, 64(12), 86-92.**
Proposes systematic documentation of datasets to surface ethical risks, embedded biases, appropriate use cases, and limitations. Relevant for understanding ethical risk as a failure mode requiring proactive identification and mitigation rather than reactive response to harm.
Tang, A., & Lau, M. F. (2014). Software architecture review by association. Journal of Systems and Software, 88, 87-101.**
Examines design decision documentation in software engineering contexts, analyzing what information must be captured to support traceability, maintenance, and knowledge transfer. Provides empirical grounding for claims about the value of documenting rationale beyond recording outcomes.
Boundaries of the Claim
This slide presents learning outcomes for a specific instructional session; it does not claim these are the only skills required for working with generative AI systems or that mastery follows from a single lesson. The three failure modes represent categories useful for diagnostic purposes, not an exhaustive taxonomy of all possible system failures—additional failure mode categories exist in technical and systems engineering literature.
The slide does not specify which generative AI systems, modalities, or task domains the learning applies to. The outcomes are framed at a level of abstraction intended to transfer across different systems and contexts, but students will need domain-specific knowledge to apply these frameworks effectively in their particular disciplines.
The requirement to document "one" revision decision establishes a minimum threshold for demonstrating the practice, not a claim that single documentation is sufficient for developing expertise. Iteration and revision documentation are ongoing practices; this outcome establishes initial capability.
The slide does not address how students will learn these skills (instructional methods), how they will be assessed (evaluation criteria), or what prerequisite knowledge is assumed. These are pedagogical implementation questions beyond the scope of the learning outcomes themselves.
The framework does not claim that all failures fall neatly into one of three categories or that diagnosis is always straightforward. Real-world systems exhibit complex, interacting failures that may span multiple categories. The three-mode framework provides conceptual scaffolding for diagnosis, not a deterministic classification system.
Reflection / Reasoning Check
1. Consider a creative project where initial attempts were unsatisfying. Looking back, which of these best describes the central difficulty: (a) unclear objectives, (b) objectives that exceeded available skills or resources, or (c) concerns about whether pursuing the initial idea was appropriate or responsible? How might explicitly identifying which type of difficulty was primary have changed the approach to the problem?
This question asks students to apply the failure mode framework retrospectively to their own experience, assessing whether ambiguity, capability limitations, or ethical concerns were primary obstacles. It develops metacognitive awareness by prompting students to categorize their own struggles, and tests whether they can distinguish between these conceptually distinct types of problems. An effective response would identify the failure mode type, provide specific evidence from the experience supporting that classification, and articulate how recognizing the failure mode earlier might have suggested different problem-solving strategies.
2. Imagine being midway through a project and realizing a significant change to the original plan is necessary. What information would need to be recorded about this revision decision for it to be useful six months later? What would make such documentation actually valuable rather than merely descriptive?
This question probes understanding of design rationale documentation by asking students to reason about what makes documentation functionally useful versus superficially compliant. It tests whether students recognize that valuable documentation captures reasoning, alternatives, trade-offs, and criteria—not just outcomes. An effective response would distinguish between documenting what changed and documenting why it changed, would identify specific information types (evidence considered, alternatives rejected, criteria applied, consequences anticipated), and would articulate how such documentation enables future learning or decision-making rather than simply creating a record.