Unit 3: Assumption mapping and risk reduction for AI products

Learning objectives

Identify and categorize assumptions specific to AI products. Prioritize assumptions by risk and impact. Design lightweight experiments to test the riskiest assumptions before committing resources.

Video script

Reading material

Testing assumptions cheaply

The goal of assumption testing is to increase confidence without building the full feature. Here are approaches for each assumption category.

Technical assumptions. Build a spike or proof-of-concept. Can the model actually perform the task at acceptable quality? Test with real data, not just benchmarks. A one-week technical spike can save months of misguided development.

User behavior assumptions. Run a prototype test with 5-8 real users. You don't need a working model; you can simulate the AI output manually (a "Wizard of Oz" test) and observe whether users trust and act on the output. If they don't trust a human-generated version, they won't trust an AI-generated one either.

Value assumptions. Calculate the time savings and quality improvement the feature would deliver and validate with users. "This would save you approximately 30 minutes per day. What would you do with that time?" If users can't articulate how they'd use the saved time, the value might be theoretical.

Data assumptions. Audit your data before you start building. How much do you have? Is it labeled? Is it representative of the use cases you're targeting? Data quality issues are the most common cause of AI feature failure, and they're the most preventable if you check early.

Market assumptions. Competitive analysis and customer demand research. Talk to 10 customers and ask about their interest in the capability. Check what competitors are doing or announcing.

The assumption map template

Create a simple grid. Rows are your assumptions, grouped by category. Columns are: assumption statement, criticality (1-5), confidence (1-5), test approach, timeline, and result. Fill in the first four columns before you build. Fill in the last column as you test.

Review the map weekly with your product trio. Update confidence scores as you learn. If a critical assumption fails a test, that's not a setback. It's a gift. You just saved your team from building something that wouldn't work.

Practical exercise

Exercise: Build an assumption map

Choose an AI feature idea (from a previous exercise or create a new one). List at least 8 assumptions across the five categories. Rate each for criticality and confidence.

Identify the top 3 assumptions that need testing (high criticality, low confidence). For each, design a lightweight test: what would you do, how long would it take, and what result would increase your confidence?

Write up your assumption map as a one-page document.