02 / How it works
Answer. Review.
Check. Decide.
The current method uses prompts and saved model rounds. The planned ring brings that review into a turn-taking process with a shared draft and a claim record.
A manual review round
Start with a clear task
The round begins with a written request. Describe the input, the output needed, and any constraints. Keeping the task clear gives the responses and critiques something specific to address.
Some requirements have a direct check, such as a calculation or a required field. Others depend on judgment, such as whether an explanation is clear. That distinction matters when reviewing an answer.
Collect separate responses
Take the task to different models and collect their responses before asking them to review the other answers. This gives the review more than one starting point.
The person running the method carries the task and responses between model interfaces. The prompts and saved rounds are the working materials.
Ask for a focused review
Give models the other responses and ask them to identify issues, omissions, or disagreements. A specific request to check the task's requirements is more useful than a general request to improve the answer.
Treat each critique as something to examine. A reviewer may find a real issue, overlook one, or introduce a new mistake.
Inspect the disagreements
A person reads the responses and critiques, then decides which points affect the result. Where possible, check a disputed claim against the source material or a direct test.
A model can help explain a disagreement, but another model's preference is still a judgment to inspect. If a point cannot be settled, keep that uncertainty in the result.
Decide what to keep
Use the review to decide what to accept, change, or investigate further. The final choice rests with the person carrying out the method.
The number of model responses is not a vote on correctness. What matters is whether a proposed change has a reason that holds up against the task.
Keep the model rounds
Save the model rounds so the discussion can be revisited. The current method has prompts and saved rounds; those working records remain private.
The planned ring builds on this idea by attaching review decisions and check records to individual claims. A record of the discussion explains what happened; a separate evaluation is needed to establish how useful the method is.
The planned system
From manual rounds to a review ring
The planned system collects independent first answers, then gives one participant at a time the shared draft and its claim record. Each turn can support a claim, dispute it, or mark it unsure, with a reason and any relevant check.
A round limit bounds the process. A final review gathers the draft, checks, and open disagreements for a person to inspect. The Ring page explains the steps and the different roles.
Explore the planned ringFuture evaluation
What still needs testing
The evaluation question is whether this review helps on a defined kind of task compared with one model answering once or making repeated attempts.
A useful comparison needs clear requirements, repeated tasks, and tasks kept aside from development. It should include failures and unresolved judgments, and account for model calls, elapsed time, monetary cost, and human review.
The manual rounds and the planned ring provide ways to organize work. Their usefulness still needs to be measured against simpler approaches.
A few useful distinctions
Questions
worth asking.
Is using more models always the goal?
The goal is a result that meets the task. Another model needs a useful role and adds review work. A simpler approach should remain an option.
Can a tool settle every disagreement?
A tool can check a defined question, such as a calculation or a test case. Its result needs the right input and scope. Questions about interpretation, completeness, and suitability may still need judgment.
Why keep unresolved points?
A stopping rule ends the review process. It does not make every claim correct. Keeping open disagreements lets a person see what still needs attention.