The AI ConferenceLast week · my advice to the room
Strip the guardrails as the models earn it.
SF Tech WeekToday’s prompt
How to build the guardrails.
Full circle
You can’t strip what you never built.
And you shouldn’t strip what you haven’t tested.
labs.google.com/pomelli
1Business DNA
Recorded in Pomelli · opulize.com
The spectrum
From Do It With Me to Do It For Me.
“Please just take care of all of this for me.”
Do It With MeYou drive. You spell out what you want.
Do It For MeYou hand it over. It may even suggest what’s next.
The spectrum
Not everything should be Do It For Me.
Stay in it · Do It With Me
Your users enjoy doing it
They’re learning and growing
AI should amplify them. Not replace them.
Hand it over · Do It For Me
It’s tedious
It’s clearly defined
Nobody wanted to do it anyway
Set it and forget it.
Not clearly defined yet?Keep them in the loop→Fill in what’s missing→Build up context→Fewer gaps over time
Do It With Me
Do It For Me
The catch
The magic and the risk come from the same place.
Do It With Me
Do It For Me
Day oneOver time, as the models earn it
The spectrum, today
Same check-ins. A much bigger job.
A few months agoS
Today · the same check-insXL
Do It With Me
Do It For Me
Pomelli
One product. More than one point on the spectrum.
Pomelli Agent
Chat, upload docs and photos, build your Business DNA together.
Together
Photoshoot
A product photo in. An on-brand shoot out.
One click
Do It With Me
Do It For Me
01
Why agents break.
Why agents break
Five ways agentic workflows break in production.
01
No guaranteed output
02
Errors compound
03
Users you didn’t plan for
04
Costs get away from you
05
Open-ended gets trickier
Do It With Me
Do It For Me
Why agents break · 1 of 5
You don’t get a guaranteed output.
Is it complete?Fonts, colors, values, imagery. All there?
Is it good?Does it actually feel like this business?
It never arrives.Try, check, try again.
Not completeThe wrong shape.You asked for fields. You got a paragraph.
Not goodComplete, but mediocre.Every field filled in. Still thin.
If you don’t define success, the agent will. And your users might disagree.
Why agents break · 2 of 5
Errors compound.
95%
right at every step
One agentThe hand-offs are still there.
The hand-offs don’t go away. You just can’t see them anymore.
Why agents break · 3 of 5
Your users will use it in ways you didn’t predict.
Expected
A product photo
Their storefront
A screenshot of their website
Six products, one photo
Out of distribution
A normal app might throw an error. An agent improvises.
Why agents break · 4 of 5
Costs get away from you.
?steps it decides to take
?tools it decides to call
?how long it decides to think
Cost of one good result
Cheaper model8 tries
Stronger model1 try
Not price per token. Price per successful task.
Why agents break · 5 of 5
The more open-ended, the harder it is to make reliable.
Photoshoot · narrow
Product photo in. Photoshoot out.
Websites · wide open
“Build me a website, and keep improving it.”
Every extra degree of freedom is another place to wander.
Answer each one for your product, and you have the first spec for your guardrails.
02
Build.
Build
Your first guardrails are product decisions.
01Scope
What it does. And what it doesn’t.
Photoshoot: a product photo and a set of templates
Narrow on purpose, before a single model call
02The interface
Where do I see what happened?
Where can I step in? How do I undo?
Keep editing: let them finish the last 10%
03The system
Contracts and exits
Checks between steps
A contained blast radius
Clever prompting can get you a demo. Product-grade engineeringis what gets you to launch.
Contracts and exits
Every step gets a contract. Every agent gets an exit.
The contract
The exits
Loop cap
Time budget
Cost budget
FallbackRetry. Go simpler. Or ask the user.
An agent with no exit never tells you it’s stuck. It just keeps going. And you keep paying for it.
Check between steps
Check every black box before it feeds the next one.
Example · Business DNA from a websiteHard rule · is it complete?Did we get colors, fonts and a logo?Semantic · is it good?Do the values we extracted feel like something this type of business would believe in?
5checks+5judge calls of latencylowrisk of a silent miss
Photoshoot · under the hood
One click on top. A lot of decisions underneath.
1 · Your photo
→
2 · Templates we suggest
→
3 · One clickThe full shoot ↓
On IceFloral WhimsyFlatlayModel Try OnIn UseHalloween
In Use→Your Business DNA→
SunglassesOn a facePushed up on a head
Makeup brandIn a purseApplying lipstick
60+options in the interface
Before you add a setting, ask if your context already knows the answer.
Blast radius
Contain the blast radius.
01
Only what it needs.In Pomelli: your Business DNA, and likely the context of the feature you’re using.
02
Sandbox where it operates.Creations stay in Pomelli until a person shares them.
03
Reads: low risk. Writes: human in the loop.The test: can the user undo it? Posting to Instagram or Google Ads, not easily.
If it only lives in the prompt, it’s not a guardrail. It’s wishful thinking.
03
Validate.
Validate
How do you know it’s ready? Validation has two jobs.
Job 01 · the expected
Quality, in distribution
Does it do what you expect? And is it good?
Your businessSponsored
✓
Your business Handmade frames, for every face and every occasion.
Job 02 · the unexpected
Misuse, out of distribution
Does the damage stay inside the blast radius?Skip it, and an agent invents a sale the business never offered.
Your businessSponsored
FLASH SALE50% OFF EVERYTHING
Your business Biggest sale ever. Every frame, half price.
Two jobs, two tools
Hill climb the expected. Red team the unexpected.
Red team goals · sandboxed, no script
01Make it write something off-brand.
02Make a claim the business never made.
03Put someone else’s logo in the shot.
You’ll be amazed, and probably a little horrified, by how creative people truly are.
The loop
Break it. Bucket it. Guard it. Re-check it.
The point fix · one symptom
Block the phrase “50% off”
The bucket · the root cause
The agent invents offers the business never made.
Fix the root cause, and you also fix the failures you haven’t seen yet.
04
Strip.
Strip
What we’re stripping now.
Pomelli Agentdoing more→
Websitesbeyond templates→
Check-insrethought→
How you know it’s earnedBetter model. Simpler prompt. Looser guardrail.→Re-run evals and red team→Failure stays gone? Strip it.
When the models get better, revisit your guardrails. Some of them have done their job.
Build the guardrails. Validate them.
Then strip them as the models earn it.
Thank you.Jaclyn Konzelmann Product Director · Google Labs · @jacalulu
1 / 31
← → slides · O grid · N speaker notes
Enter to go · Esc to close
00:00N notes · T timer (starts on first press) · P window · F full screen
✓
Expected
