What Is Non-Determinism, and When Should You Use It?

When all you have is a hammer, everything looks like a nail.

Now that we have AI agents, it’s tempting to have them reason through every step of a task. Check a status? Have the agent interpret a record. Calculate a total? Have the agent read the numbers and tell you what they add up to. Decide whether a customer said yes? Have the agent judge the reply.

The agent can handle all three tasks. The question is how you ask it to do each one. It can check a field, use a calculator, or interpret a conversation. Those choices matter when bots start doing work for you and interacting with your customers. Two concepts help explain why: determinism and non-determinism.

A deterministic system produces the same output from the same inputs under the same conditions. A non-deterministic system can produce different results. Language models are generally non-deterministic, so repeating a prompt can produce a different response. Google’s machine learning glossary

That sounds abstract. Let’s make it practical. Suppose you’re building a bot with this instruction: “When the customer says ‘yes’, move to the next step.”

The user responds, “Yes.”, with a capital Y and period. The app checks whether their response matches all lower case “yes”, so the algorithm misses. You decide to account for capitalization and punctuation.

The next person writes “sure.” Then “okay.” Then “sounds good to me.” Damn! Missed again. You add more rules. Those rules are deterministic. You have explicitly defined which responses will match and what happens when it does.

Then someone writes: “I don’t see why we wouldn’t.” Damn Damn. Got me again.

This is where a language model becomes useful. You can ask it to interpret whether the person agreed, using the conversation as context. You no longer have to anticipate every possible phrase yourself. That flexibility is powerful. It also introduces a different kind of decision into your workflow.

There’s a distinction worth making here: interpreting language, inference, and non-determinism are related in these systems, but they describe different things.

Inference is the process of running a trained model on new input to produce an output. It doesn’t specifically mean understanding someone’s intent, and it doesn’t inherently require non-determinism. For our bot, inference is how we run the model. Interpreting the customer’s response is the task we give it. Non-determinism describes the possibility that its answer may vary between runs.

Keeping those ideas separate helps us ask better questions: where does the ability to interpret justify the cost and uncertainty and what did the customer actually approve?

Consider a few more responses:

  • “Sure, if you can deliver by Friday.”
  • “Okay, but send me the revised price first.”
  • “k”
  • “Yah, sometimes…”

It would take a lot of guess work and lists of hardcoded matches to catch all the ways a person could say yes. So we decide to use an agent for non-deterministic results. Even then, these deserve different treatment. A positive tone can may contain a condition, or no approval at all. “Sometimes” is especially revealing. It does not clearly lay out what approval requires to get to the next step. Even “k” can mean different things depending on what came before it. The agent can interpret the words and the context, but it cannot decide your standard for approval on its own every time. That standard is partly a matter of judgment and policy, and different people may draw the line in different places. Human or bot.

If your bot’s instruction is “look for anything that sounds like positive movement,” you may have built a system that turns interest into permission. The work is in telling the agent what counts as approval for this task, and what to do when the answer is unclear.

For a bot suggesting another article, a loose interpretation might be acceptable. For a bot placing an order, you may want explicit confirmation of the item, price, and delivery terms.

The model’s ability to interpret language doesn’t decide how much ambiguity your business should accept. You have to decide that.

This amazing ability to allow for a great deal of flexibility and variation has changed our world of work. That doesn’t mean every problem requires it now. Model calls consume massive compute, and asking an agent to interpret more information can add cost and wait time. Sometimes that expense buys something valuable: understanding an unusual request, connecting ideas across documents, or drafting a useful response. Other times, the answer is already available as a field, a formula, or a rule. I wrote more on how you can balance agentic power with deterministic costs here.

Let’s use a common email bot helper that many are using to increase productibity today. Suppose you ask your agent to find the messages that need your attention (archiving or deleting all the slop). It could read your entire inbox every time you ask. That gives it plenty of context, but it also takes time and uses inference on messages a simple filter could handle.

Instead, ask your agent to create a filter for the obvious parts of that task. It can easily screen out information from site you like but only when it doesn’t meat pattern you’ve seen. Those filters will (deterministically) trim the data your agent needs to interpret down – sometimes to zero. Then it can read what’s left. The agent still does the work. You’ve asked it to spend its reasoning on the messages that need reasoning. Then you can even add a routine to have it create filters based on probably patterns it’s deleted without reading. Each run decreases costs.

Using the customer confirmation example from before, I’d redesign the handoff like this: Have the model interpret the reply as approval, rejection, conditional approval, or unclear. You can even include definitive reply patterns to increase speed and save a cent or two each run.

  • Ask you dev agent to create a deterministic list of positive exact match cases that ignore periods and captialization.
  • Provide explicit non-deterministic rules to understand what each category should permits.
  • Ask the user for clarification when a condition or ambiguity affects the next action.

The filter trims the use of the model, model helps interpret the conversation. The rules establish what the system may do with that interpretation. That’s the design decision that deserves more attention as we put agents to work. Thinking, along with the agent, while developing or setting up our personal bots saved the world time, energy, air quality, speed, cost, and more.

Every request for interpretation should earn its place. What ambiguity is the agent resolving? What flexibility are we gaining? What happens when its interpretation is wrong? How can I trim the list of variations with certainty first? An agent can build and run the whole workflow, including creating the filters, lookups, and calculations that let it focus its judgment where it counts.

Knowing when to remove from you agent with your agent, implement with your rules your agent can reason with, and when to have it check back in to be sure, is part of using all the tools on your toolbelt based on what works best.

Cheap Code Doesn’t Change What a Product Is

“If AI can rebuild your product in a weekend, why would anyone pay for it?”

I hear this question just about every single day on X, Instagram, Reddit and more. It sounds like a question about the future of software. But underneath it is an old assumption: that what people buy when they buy software is the code.

If that were true, cheaper code would make the whole business fall apart.

But think about what you’re actually buying.

When you pay for a scheduling tool, you aren’t looking for access to a calendar interface. You want to stop emailing people about when they’re free. You want the invitation to arrive, the time zones to work, and the meeting to show up where it belongs.

You want something taken off your plate.

The code is part of how that happens. It isn’t the whole transaction.

You could make the brownies, too

I wrote about this in Convenience Is the Product.

Brownie mix is a simple proposition. Someone measured the ingredients, worked out the proportions, and put the instructions on the box.

You could do those things yourself. Buying the box doesn’t mean you couldn’t.

It means you’d rather spend your attention somewhere else.

Software has a similar bargain. Someone has decided how a process should work, which details matter, and what you shouldn’t have to think about.

AI changes how much effort it takes to make your own version. That matters. Some things you used to buy may become things you make.

But the ability to make something doesn’t automatically create the desire to be responsible for it.

A weekend is only the beginning

Let’s say you rebuild your scheduling tool this weekend.

It works. You connect your calendar, send a link, and book a meeting.

Then someone asks to reschedule. Someone else needs different availability for different kinds of meetings. Your team wants shared bookings. An integration changes.

Now you have a second job.

Maybe AI helps with all of it. Maybe each change takes minutes. You still have to notice the problem, explain what should happen, decide whether the result is right, and care enough to keep doing it.

That’s a different commitment from making something work once.

“Can I build this?” and “Do I want to own this?” are two different questions.

A product earns its place when the answer to the first is yes and the answer to the second is no.

The decisions are part of the product

There’s another layer here.

A useful product contains decisions you may never notice. What happens by default. Which options appear first. Which exceptions are supported. Which requests get turned down because accommodating them would make everything else harder.

Those choices give the product its shape.

This is what I explored in What You Won’t Do Defines You. Refusal is part of making something coherent. Every possible addition isn’t an improvement.

Cheap building makes that discipline more valuable. When you can produce five versions quickly, you still need a reason to choose one.

AI can help you consider the options. It can even recommend a direction. But someone still has to commit to what the product is for and accept the consequences of that choice.

As I wrote in Indecision Does Not Preserve Options, leaving a question open creates work of its own.

Being able to build every option doesn’t make every option worth building.

That doesn’t mean your business is safe

There’s a distinction worth making here.

People continuing to need software doesn’t mean they will continue to need yours.

If a competitor can reproduce your features and take the same burden off the customer for less, you have a real problem. Saying “we sell convenience” doesn’t answer it. The competitor can sell convenience, too.

And if a customer can make a version that takes almost no effort to maintain, buying yours may stop making sense.

Cheaper building changes those calculations. It puts pressure on what you charge and how much value you deliver.

But it doesn’t erase the underlying transaction.

You still have to understand a problem well enough to make useful choices about it. You still have to deliver the result reliably. You still have to keep earning the customer’s preference as their needs change.

The argument for starting a software company can’t be “people will always need software.”

It has to be: “There is something here people would gladly pay us to keep handling.”

What are you taking off their plate?

That’s the question I’d spend more time on.

How much does your customer still have to figure out? How often do they have to check your work? When something changes, who notices? Who deals with it?

A product can have a hundred features and leave most of the burden with the person using it.

Another can do one thing so completely that the customer barely thinks about it again.

That second product has a reason to exist, even in a world where its code is easy to reproduce.

The customer may know exactly how to build it.

They may even enjoy building it.

And still decide they have something else they’d rather do with their Tuesday.

That’s what you’re competing for. Not just the cost of making the software, but the value of everything your customer would rather spend their life doing.

Cheaper code changes what they can make.

Your product has to earn what they’re willing to hand off.

The Real Lesson from a Flying Tesla: Derivative Doesn’t Mean Unoriginal

Tesla shows off a Roadster that can lift itself into the air, and somewhere an aspiring entrepreneur thinks “I need to invent something even more spectacular to be successful”. I admire the ambition and showmanship. I worry about the lesson people take from it.

The ingredients have histories of their own. The Chaparral 2J used fans to generate suction and grip decades before this Roadster demo. SpaceX developed reusable rockets, tackling the engineering challenges of bringing boosters back to land. These are different applications, but they put the spectacle in context: suction, thrust, and controlled landings each have substantial work behind them.

The same pattern extends to autonomous driving. In 2004, DARPA’s Grand Challenge pushed teams to build vehicles that could navigate a desert course themselves. None finished. Government-backed experimentation helped advance technology that would later find commercial applications. Turning those advances into reliable products required substantial additional engineering and development.

Watch a Tesla demo and you see one company’s vision. Look behind it and you find a much longer history of inventions, experiments, and accumulated knowledge.

Spectacular demos compress that history into a single moment. It looks like someone invented the future from scratch. The aspiring founder walks away thinking that’s the assignment.

What gets lost is the distinction between inventing a technology and combining existing technologies into something extraordinary. Both create value. But they require different amounts of time, money, and patience.

Building on existing work doesn’t diminish what a founder accomplishes. The opportunity is knowing which pieces are ready, which solve problems people care about, and how bringing them together can create something more useful.

When your money, time, and attention are limited, every additional breakthrough your business depends on is another uncertainty you have to fund and resolve. You can spread yourself across so many invention problems that you never get to solve the customer’s problem.

I talk a lot about focus. This is one of its most consequential applications. My advice: choose where you actually need to invent, and limit it to one thing at most.

Which part of your product requires something new? Which parts can use something that already works? Where will the difference matter enough for someone to pay?

Bread and knives existed long before Otto Rohwedder’s machine made commercially sliced loaves possible in 1928. That machine took real invention and persistence. He focused that effort on a specific problem without having to reinvent everything around it.

You don’t have to invent bread and the knife to create the next sliced bread.

Indecision Does Not Preserve Options

Product bloat is often blamed on too many ideas or not enough resources to get speed. More often, it comes from too few decisions catching up with you.

A team delays deciding if a feature is required, who it is for, is the value it brings worth it to the overall product and company stage. The team hopes that time is saved by moving on to the next feature or issue will speed them up or “fix it”. The unanswered question feels like a temporary issue that doesn’t deserve time because there is not enough time for it. The work can continue. But nobody has removed the question. They have simply created compounding overhead.

So, “for now”, the feature will; to serve anyone.

It gains settings for one interpretation, exceptions for another, and workarounds for the cases nobody wanted to rule out. What began as an effort to stay flexible becomes a product that asks its users to carry the uncertainty the team did not resolve. The cost is not limited to one vague feature.

Imagine five possible directions: A, B, C, D, and E. None has been decided. It can feel as if the team has preserved five options. In reality, it has created five active futures.

Each future produces its own downstream work. Each has dependencies, edge cases, and failures. If something breaks deep inside E, the question is not only whether to fix it. The team also has to decide whether E is still important, whether it will matter later, and whether that work should take priority over something developing in A or D.

The original decision has been deferred, but the decisions it created have multiplied. This is why indecision becomes so expensive. It divides attention before it divides the product.

A person can be actively working on a branch that should never have existed. A customer can arrive with a real need for D, but the team may not notice it clearly enough because everyone is already carrying the mental load of A, B, C, D, and E. Attention is scattered across possibilities. Problems demand responses. Opportunities arrive without enough clarity to be recognized.

Eventually, the cost becomes visible in the product itself.

Flowchart illustrating the consequences of indecision with five options (A, B, C, D, E) leading to divided attention, increased work, and ultimately product bloat.

The interface gets crowded. Navigation expands. Settings multiply. A straightforward path becomes a collection of tabs, filters, alerts, roles, exceptions, and competing actions. Each piece may have a history. Each may have made sense at the moment it was added. Together, they create a product that feels fat, difficult, and strangely hard to simplify.

By then, teams often describe the problem as technical debt, feature creep, or complexity. Those descriptions are true, but they are incomplete. The product is carrying the consequences of decisions that were never made. This is also why rebuilding can be so valuable.

Starting over is usually treated as a dangerous impulse. There are real things to lose: working behavior, unusual customer needs, knowledge embedded in old systems. The fear is understandable. The first version contains too much history to dismiss casually.

But rebuilding is not about pretending that history did not happen. It is about forcing history to justify itself.

When a product is rebuilt, every feature has to earn its place again. Every workflow has to explain why it exists. The old assumptions no longer receive automatic authority simply because they have been in the product for a long time.

Some things should carry forward. Many will.

But some complexity only survives because nobody has paused long enough to ask what it was protecting. The things that melt away during a rebuild are often the most revealing. They are not necessarily mistakes. They are artifacts of uncertainty that became permanent by accident.

The first version teaches the team what it did not know. The next version has the chance to decide.

This does not mean every unanswered question needs an immediate answer. Uncertainty is real, and sometimes waiting is the right decision. But waiting should be a decision too. It should have an owner, a reason, and a point at which the team comes back to resolve it.

Otherwise, the product will resolve it for you. And it will do so by getting bigger, slowler, and unruley.

A case study in logo animation with AI

I wanted to experiment with animating a logo using agents I already had. The results surprised me. I’m glad I stuck with it, and I hope sharing the process saves you some time.

I’m not an animator, but I’ve been interested in animation for a long time. In school, video editing classes brought together my love of art, expression, and cartoons. I was fascinated by stories about how Disney and its animators thought. I made things using tools like Flash, Maya, Blender, and Final Cut. I learned that a bouncing ball is more than a ball moving up and down. Animating it meant understanding speed changes through easing and how mass affects the way a shape moves, squishes along both axes, and springs back.

Purpose built animation apps

I started by trying third-party tools that promised to make animating a logo easy. I tried Adobe, Canva, and even apps built specifically for logo animation. They were all unimpressive. At best, they added a shake or an expansion, but the motion had no character. It felt like AI’s equivalent of PowerPoint clip art. I had little control over how the animations could make use of my logo’s unique qualities. Even Lottie, a first-class web animation tool, didn’t offer a way to upload my logo and create a timeline animation for me.

Open AI

Next, I tried ChatGPT. I’d seen it do impressive things with image generation and art, so I started with a process that had worked well for me when brainstorming image ideas: a storyboard grid. You can read more about that here.

I asked for a 5-by-5 grid, with each row showing a different animation and each cell representing a keyframe: an important pose or moment in the movement.

A 5x5 grid displaying motion study concepts for logo animation, featuring visual representations for each animation idea like 'Breathe', 'Elastic bounce', and 'Magnetic assembly'.

The idea showed promise. After a few experiments, I picked my favorite animation to develop.

From there, things went downhill. GPT produced animations that bore little resemblance to the storyboard. I tried different instructions and output formats, including GIF, SVG, and Lottie. It felt like it would take a lot more turns and tokens to get to something worth celebrating.

Magnetic assembly animation with four pieces coming together and separating.

Give up or fight on

I subscribe to several AI tools because one of them may have a feature that works especially well for a particular task. I use Cursor, Grok, Claude, and OpenAI regularly. That day, after getting nowhere with the third-party tools and OpenAI, I gave Claude a try. A few weeks earlier, it had handled a tricky image request in one turn, while the others took several tries and still didn’t produce anything usable.

Claude

Claude’s first animation attempt was far better than the others. It decided on its own to use animated SVGs, and it worked. It was a unique animation made for my logo. It wasn’t ready to use, but it gave me hope. If it could get that close from a loose prompt, maybe it was worth giving it more time.

During the earlier attempts, I’d started describing parts of the logo as if they belonged to a character: “The top two nodes are hands, and the bottom two are feet. Change the right hand like this.” Then I imagined the logo getting bored from sitting in a website header all day. It might slouch, nod off, or let out a big yawn, so I asked for a yawn animation. Here it is as an animated GIF.

A stylized red plus-shaped logo with rounded edges.

Here are a few tricks I learned to take the animations further with less pain.

Storyboarding

Don’t just create a storyboard to guide the animation. Ask for one with each generated version. With the storyboard, I could point to a specific frame and annotate the change I wanted. For example, I wanted the logo’s “hands” to reach farther out and shake in frames 32 and 33.

Grid of pink shapes with text labels indicating various actions and time durations.

It also let me fix small details that would have been hard to specify. I noticed a tiny bulge between a node and its stem. I could target the frame and circle the bump so Claude could fix it. It used that feedback well, resolving the issue without regression or confusion.

Two similar abstract red shapes with a central round area, one shape has a marked detail at the top left, and the other has a marked detail at the top right.

Use reference material

When I watched behind-the-scenes footage of Disney animators, I loved seeing how they studied animals in their environment and asked actors to perform for them.

That memory came to mind when I tried to create a jumping-jack animation but wasn’t quite getting the nuances I wanted. Instead of trying to draw or verbalize what I meant in a storyboard, I showed the agent a human reference so it could understand the nuances I wanted to capture in the movement.

A sequence of three illustrations showing a person performing jumping jacks, demonstrating arm and leg movements.

Unlocking words

That was much easier than describing the change in another prompt. I also found that a broad direction like “give it life” could lead to subtle movements I might not have thought to specify.

Claude’s animation work clicked for me. Codex’s did not. The experiment reminded me how much I could contribute with visual references and an understanding of movement, even without being an expert. The agent created possibilities, and the storyboard gave me a way to direct them.

“…[political parties] are likely in the course of time and things, to become potent engines, by which cunning, ambitious, and unprincipled men will be enabled to subvert the power of the people… “

– George Washington