What Prompt Lab Is Used For

Writing a custom check in plain English is easy. Writing a custom check that actually performs well is harder, and the only real way to know is to test it against real drawings. Prompt Lab is built for exactly that.

The Core Idea

Prompt Lab lets a team take two different versions of the same check, phrased differently or scoped differently, and run them side by side on the same drawing set. Instead of picking one approach and hoping it works, the team can see both outputs directly and compare which one actually catches what it's supposed to catch.

This matters because small differences in how a check is written can produce meaningfully different results. A check phrased as "flag any beam without a connection detail" behaves differently than one phrased as "flag any beam where the connection detail doesn't match the structural notes." Both sound reasonable. Only testing them against a real set shows which one is actually doing the job.

Why Guessing Isn't Good Enough

Most firms don't have the bandwidth to run a check for three months and see how it performs before deciding it's not quite right. A check that's too narrow misses issues. A check that's too broad drowns the team in false positives. Either failure mode costs real time to discover if the only way to find out is to run it live on projects.

Prompt Lab compresses that discovery process. A team can iterate on a check's wording in an afternoon, testing variations against drawings they already know well, rather than finding out months into rollout that the check needed adjusting.

How It Pairs With Agent Playground

Prompt Lab and Agent Playground solve related but different problems. Prompt Lab is about comparison: which of two approaches performs better on this drawing set. Agent Playground is about visibility: watching a single check work in real time to understand why it behaves the way it does.

Used together, a team can use Prompt Lab to narrow down to the better-performing version of a check, then use Agent Playground to watch that version run and confirm it's reasoning through the drawings the way they expect before rolling it out firm-wide.

Who Actually Uses This

This is mostly relevant to QA leads and BIM/VDC managers building out a firm's check library, the people responsible for making sure Custom Checks actually reflect the firm's standards rather than a rough first draft of them. For a firm building its own library of checks over time, as opposed to relying only on baseline code checks, Prompt Lab is the tool that keeps that library improving instead of ossifying.

FAQ

Does Prompt Lab require technical expertise to use? No. It's built for teams writing checks in plain English, testing wording and scope differences rather than code.

How many versions of a check can be compared at once? Prompt Lab is built around comparing two approaches side by side, which keeps the comparison focused and easy to evaluate.

Is Prompt Lab a one-time step or an ongoing process? It's most useful as an ongoing process. As a firm's projects change or a check's performance drifts, Prompt Lab is the tool for testing adjustments before they go live.