Adversarial analysis, dialectical argument, Socratic questions, pre-mortems, and red teams long predate language models. Models make these exercises cheap to repeat. They do not make the result rigorous by themselves.
A model can agree too readily, invent a source, defend a weak premise, or reject a workable design when asked to be critical. The operator’s job is to turn each response into claims that evidence or tests can support or reject.
Useful Exercises
List assumptions before solutions. Ask which facts the recommendation requires, which are present in the source material, and which have been supplied by the model. An assumption should name the evidence that would test it.
Argue the strongest opposing case. Require the critique to preserve the objective and constraints. A useful opposition identifies a failed premise, counterexample, or tradeoff. Automatic disagreement is as weak as automatic agreement.
Run a pre-mortem. Assume the project failed and enumerate causal paths to that result. For each path, ask for an observable early warning, a prevention, and a recovery. This turns a list of worries into controls that can be assigned.
Find contradictions. Compare a recommendation with its requirements, another section of the same document, an earlier decision, or executable behavior. A fresh context may notice a conflict that the drafting session repeated. The conflict still needs direct confirmation.
Demand source resolution. Ask for the document, section, table, date, and applicable scope behind a factual claim. Then open the source. A plausible citation is not evidence that the model retrieved or represented it correctly.
Ask for discriminating tests. When two approaches remain plausible, request a test whose result would favor one. A benchmark, unit test, historical backtest, small deployment, or source comparison is more useful than another paragraph of advocacy.
Separate proposal from review. Use a fresh session or another model to review the artifact under the same acceptance criteria. Independence of context can reduce commitment to the original answer. It cannot remove shared training errors or substitute for execution.
Change the professional questions, not the costume. A contract deserves questions about authority, obligations, remedies, and jurisdiction. A security review deserves questions about assets, trust boundaries, capabilities, and abuse. Telling a model to “be a lawyer” or “be a security expert” does not confer professional competence.
Test missing-context behavior. Give a bounded task whose required inputs are known and observe whether the model asks for them or invents them. Use the result to design input validation; do not expose a consequential task to deliberate ambiguity merely as a test.
Request the next action and its proof. A model that understands the current state should be able to propose a specific next action, explain what uncertainty it reduces, and name the artifact or observation that would show completion. Generic advice indicates that the state or objective is underspecified.
What Multiple Models Add
Different models and contexts can produce different assumptions, source choices, and failure modes. That diversity is useful for discovery. It does not create truth by majority vote.
Suppose Claude recommends architecture A and Codex recommends B. The disagreement should produce a comparison:
| Question | Evidence |
|---|---|
| Which requirements does each design satisfy? | Trace requirements to components and tests |
| Which assumptions differ? | Name each assumption and its source |
| What does each design cost to operate? | Use a declared workload and cost model |
| How does each fail? | Exercise representative fault cases |
| Which choice is easier to reverse? | Identify migration steps and retained options |
The models help populate the comparison. A benchmark or responsible reviewer decides what the evidence supports.
The Human Failure Modes
People also prefer confirmation, momentum, and clean conclusions. A polished answer makes it easy to accept a claim we wanted to believe, skip a source check, or continue after the objective changed.
The remedy is procedural:
- define acceptance criteria before asking for implementation;
- keep source facts separate from model inferences;
- record uncertainty and unresolved disagreement;
- require observable evidence for completion;
- assign a person accountable for the decision; and
- stop when additional analysis cannot change the choice.
Prompting skill can improve an exchange, but it is not the controlling advantage. The advantage comes from formulating a testable question, rejecting unsupported answers, and checking the result at the speed the models can produce it.