PRDs for AI products: specifying when the model is the feature
Acceptance criteria don't survive non-determinism. This is the template we use at Collybrix to write PRDs when the model is the feature.
Translated automatically from Spanish.

"The summary has to be correct." That was the first line of acceptance criteria I wrote for a feature with a model inside. I read it twice and felt embarrassed. Not because it was wrong, but because nobody could verify it. Correct according to whom? Always? Nine times out of ten? What happens on the tenth?
It was my first project with a model inside, when everything was still very green. Models have improved enormously since then, and today they'd probably even interpret that instruction well. But "well" isn't what a project needs. After a lot of back and forth, I understood that specifying the product and the expected outcome is key, and not just for the model. Also for the people who are going to use it.
The classic PRD was written for software that does the same thing every time you give it the same thing. You pass it an input, it gives you back an output, and the acceptance criterion is a yes or a no. With a model, that breaks down. You give it the same input twice and it returns two different things, both reasonable, and a third that has nothing to do with either. And the problem isn't the model. The problem is that the document we use to decide what to build assumes a world that no longer exists.
This post is the template we ended up using at Collybrix when the model isn't an implementation detail but the feature itself. And why I arrived at it by reading people who thought about it much more carefully than I did.
Why the usual PRD doesn't hold up
There are three assumptions in the traditional PRD that AI breaks.
The first is that behavior can be specified before building. Hamel Husain and Shreya Shankar, who are probably the two people who've taught the most about evals in the last two years, say it bluntly: people aren't good at specifying all their requirements for an AI up front. The real criteria emerge when someone who knows the domain looks at actual responses and says "this is wrong, and it's wrong for this reason." Shankar has an entire paper about what she calls criteria drift: the criteria you evaluate against change as you see more outputs. It's not a lack of rigor. It's just how it works.
The second is that acceptance is binary. In normal software, the test passes or it doesn't. With a model, the question stops being "does it work?" and becomes "how often does it work, how does it fail when it fails, and how much does that failure cost us?" These are three different questions, and the classic PRD has no room for any of them.
The third is that the launch is the end. Aishwarya Naresh Reganti and Kiriti Badam explain this very well in Lenny's Newsletter: AI products need a different lifecycle, one where the system earns autonomy gradually. They call it CC/CD, continuous calibration / continuous development, and the underlying idea is that every increase in autonomy is a loss of human control. You don't launch a feature; you launch a low-power version, watch how it behaves, and only then give it more.
"Evals are the new PRD." Almost.
There's a strong current of thought saying the PRD gets replaced by evals. Braintrust published exactly that title this year: functional requirements become test scenarios, acceptance criteria become rubrics, launch review becomes an automated gate in CI. There's a lot of truth to it. A "responses must be useful" is a specification that can't be verified, and an eval can be.
But I'll stick with "almost." A set of evals tells you what is good. It doesn't tell you why you're building this, for whom, how much error that user can tolerate, or what happens in the product when the model gets it wrong. Those are product decisions, and if they're not written down somewhere, they end up being made by whoever writes the prompt on a Friday afternoon.
So we didn't throw out the PRD. We changed its shape. The evals live inside it, as the heart of the document, but surrounded by the decisions that give them meaning.
The template
There are ten sections. Some are the usual ones under a different name; others didn't exist before.
1. The problem, without the word AI
One or two sentences describing the job the user wants to do, without mentioning models, agents, or anything like that. If you can't write it without naming the technology, you're probably in love with the solution and not the problem.
2. Why a model (and what's the alternative)
What this product would do without AI: a rule, a form, a human. That alternative is your baseline. If the model doesn't clearly beat it, there's no feature. And if it does beat it, now you know what to measure against.
3. The behavior contract
What the system has to do, what it must never do, and what a good response looks like. This is where concrete examples go: Aakash Gupta recommends between 15 and 25, split into bad, good, and excellent. I agree. A well-chosen example specifies more than three paragraphs of prose, and it's the only thing the model, the team, and the client understand the same way.
4. The reference set
Between 20 and 100 real inputs to start, which is the range Reganti and Badam propose. Real means real: with typos, ambiguous questions, requests that have nothing to do with the feature. The neat inputs the team writes don't teach anyone anything. This set is the seed of the evals and grows every time production shows you a case you hadn't imagined.
5. Measurable success criteria
One per dimension that matters: task quality, consistency, tone, latency, cost. Anthropic's guide on success criteria sums it up well with a counterexample that stuck with me: "the model should classify sentiment well" is not a criterion; "F1 of at least 0.85 on a set of 10,000 posts, 5% better than the current baseline" is. Specific, measurable, achievable, and relevant.
And a detail I learned from Husain and Shankar: when the criterion is qualitative, judge it as pass/fail, not on a 1-to-5 scale. Scales invite wishy-washiness ("a 3, let's say") and nobody agrees on what separates a 3 from a 4. The nuance isn't lost: it goes in the written critique that explains why it passed or didn't.
6. The error budget
This is the section that saves the most arguments. Not all errors cost the same, and the PRD has to say which ones are acceptable and which aren't. A summary omitting a minor detail might be fine one time out of ten. Making up a figure that wasn't in the document, never. Writing it this way forces you to separate the error rate from the severity of the error, which is exactly what the "it has to be correct" criterion was conflating.
A made-up but very typical example: an assistant that summarizes a sales team's meetings and logs next steps in the CRM. The error budget might look like this:
| Error type | Severity | Tolerance |
|---|---|---|
| Omits a secondary topic from the meeting | Low | Up to 1 in 10 summaries |
| Attributes a statement to the wrong person | Medium | Up to 1 in 50, and if unsure, flags it |
| Invents a commitment, date, or price nobody said | Critical | None. Blocks launch |
Look at the last row. It's not a percentage: it's a design rule. If a commitment doesn't appear in the transcript, it can't appear in the summary, and that changes the whole product. It forces every next step to cite the moment in the meeting it comes from, and gives the eval something concrete to verify. No salesperson is going to call a client to confirm a discount the assistant made up, because the product doesn't allow it by design.
And notice also what the table doesn't say: it doesn't say "the summary has to be correct." It says what kind of incorrect we can live with, how much, and which kind never.
7. Autonomy and human handoff
What level of power this version has and how it gets taken away. Reganti and Badam's example is a support system: first it only classifies tickets and a human can reassign them; then it drafts responses that a human reviews; only at the end does it resolve a limited set of cases on its own, always with a handoff to a person. Each version earns the next one with evidence.
This section also covers the "I don't know." What the system does when it's not sure, how it says so, and who it hands off to. I learned this taking agents to production: the user doesn't forgive the error, they forgive the surprise.
8. Cost and latency per interaction
Not per request: per complete interaction, which in a conversation includes more and more context. With a maximum number for each. This section is also the one that forces you to ask whether a smaller, cheaper model passes the same evals, a question almost nobody asks until the bill arrives.
9. Launch and monitoring criteria
Which evals have to pass, and at what threshold, for the version to ship. How often they run in production. And what drop triggers an alert or a rollback, because the model you're using today isn't the one you'll have in three months: the provider updates it, changes it, or retires it, and your product changes with it even if you haven't touched a single line.
10. What we're going to learn by looking at traces
The last section is an honest list of what we don't know. Questions that can only be answered by reading what happens in production, with a date for the first review. Husain repeats this in everything he writes: the typical mistake is writing the rubric before having looked at a single data point. The rubric is a result of error analysis, not its starting point. This section puts in writing that the PRD is going to change, and who's going to change it.
The template, to copy
# PRD: <name>
## 1. Problem (without mentioning AI)
## 2. Why a model
- Non-AI alternative (baseline):
- What the model needs to improve relative to the baseline:
## 3. Behavior contract
- Must:
- Must never:
- Examples (15–25): bad / good / excellent
## 4. Reference set
- 20–100 real inputs, including the ugly ones. Source and owner:
## 5. Success criteria (one per dimension, pass/fail)
| Dimension | Criterion | How it's measured | Threshold |
## 6. Error budget
| Error type | Severity | Tolerance |
## 7. Autonomy
- Level of this version:
- How a human intervenes:
- What the system does when it's not sure:
## 8. Cost and latency per interaction (maximums)
## 9. Launch and monitoring
- Evals that block launch and thresholds:
- Frequency in production:
- What drop triggers alert / rollback:
## 10. Open questions
- What we're going to learn from the traces, and when we'll review it:What changes for whoever writes the PRD
The most uncomfortable part of all this isn't the template. It's that whoever writes the PRD has to read outputs. A lot of them. By hand. Not a dashboard: the responses, one by one, saying this one yes, this one no, and why.
It's the least glamorous work in the process and it's where almost all the value is. Husain and Shankar insist it should be done by someone who understands the user, not just the system, because the failures that matter are the product's failures, not the model's. A technically correct summary that gives the user the information in the wrong order won't be caught by any generic "hallucination" eval. It's caught by someone who knows what the user wanted it for.
For me, that changed the question. Before, the PRD answered "what are we building?" Now it answers "how are we going to know it's good, and what do we do when it isn't?" It's a harder question, and it's the right one.
Since we started working this way at Collybrix, projects reach clients in much better shape. We stopped "trying it out to see what comes up" and started narrowing down, in writing and before the first prompt, what we expect from the model in terms of product goals.
The savings are enormous: in time, in rewritten code, in back-and-forth with the client, and in headaches. But what matters most isn't measured in hours. It's trust. A client who knows in advance what the model is going to do, how much it can get wrong, and what happens when it does, doesn't get blindsided. And being blindsided, as I said, is the one thing that doesn't get forgiven.
Sources
- Aishwarya Naresh Reganti and Kiriti Badam, Why your AI product needs a different development lifecycle, Lenny's Newsletter (August 2025).
- Hamel Husain and Shreya Shankar, Building eval systems that improve your AI product, Lenny's Newsletter (September 2025).
- Hamel Husain, Automating error analysis (June 2026).
- Shreya Shankar et al., Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences (2024).
- Ameya Bhatawdekar, Evals are the new PRD, Braintrust (March 2026).
- Anthropic, Define success criteria and build evaluations.
- Aakash Gupta, AI product PRD template.