OpenAI released the GPT-5.6 family on July 9, 2026 with three tiers: Sol as the flagship, Terra as the balanced option, and Luna as the fastest and most affordable. Eligible users can also change effort, while the highest settings give a model more compute or, in the case of ultra, coordinate multiple agents. More choices are useful only if you know what each extra step buys for your own work.
The wrong comparison starts with a leaderboard and ends with the most capable setting selected for everything. The better comparison begins with a repeatable task, a quality bar, and the total cost of reaching that bar. A cheaper request that needs three retries and ten minutes of correction may cost more than a stronger first pass. A premium setting that adds detail no one needs may be pure overhead.
This guide uses a concrete scenario: triaging fictional customer-support tickets against a written refund policy, then drafting a short reply. It shows how to test model tier and reasoning effort without changing the prompt mid-experiment. Product availability and API prices were checked on August 4, 2026.
A model picker is the start of the experiment
OpenAI's launch page describes Sol, Terra, and Luna as durable capability tiers, not as three fixed answers to every task. It reports large benchmark gains and shows performance changing with reasoning effort. The accompanying system card makes the same point visually: rather than report one score, it plots results across effort levels to show capability and compute together.
Those published evaluations are evidence about the tested harnesses. They are not a guarantee for a particular support queue, spreadsheet, codebase, or writing style. Stanford's HELM work emphasizes standardized scenarios, prompts, and metrics because uncontrolled comparisons hide tradeoffs. Your local evaluation can be much smaller, but it needs the same discipline.
Start by separating two decisions. Model tier chooses a broad capability and price point. Effort controls how much work that model puts into the request where the product exposes the setting. Test one variable at a time. If you change the model, effort, prompt, source documents, and grader together, a better score will not tell you which change helped.
Write the acceptance test before the prompt
For the support scenario, the input is a customer message plus a one-page fictional refund policy. The output must classify the issue, decide whether the written policy clearly allows a refund, quote the controlling policy sentence, draft a reply under 120 words, and escalate whenever the evidence is incomplete or the case involves safety, fraud, or a policy exception.
Create an eight-point rubric before running a model: two points for issue classification, two for the refund decision, two for an exact and relevant policy quote, one for a usable reply, and one for correct escalation. A result passes at seven points only if it has no critical error. A fabricated policy quote, an unsafe instruction, or a missed mandatory escalation is a critical error even when the prose is polished.
This quality bar turns preference into something inspectable. NIST's 2026 AI measurement workshop summary identifies the same building blocks: a clearly defined task, a suitable metric, and data. It also distinguishes repeatable performance measures from application-level effectiveness. In this test, the rubric measures output performance; reviewer time and successful resolution measure whether the workflow is actually useful.
Build a twelve-ticket ladder
Use synthetic tickets so no customer data enters the experiment. Write four straightforward cases, four cases with distracting details, and four boundary cases. A straightforward case might request a refund inside the allowed window with a receipt. A distracting case might include a long complaint but still reduce to a documented shipping delay. A boundary case might combine a missing receipt, a damaged product, and a safety concern that requires escalation.
For each ticket, write the expected classification, decision, policy passage, and escalation status before asking the model. Keep those gold answers out of the prompt. If the expected answer depends on an interpretation, label the case ambiguous and require escalation rather than pretending the rubric is more certain than the policy.
Twelve cases will not establish universal model quality. They are a practical screening set for one narrow workflow. Run every configuration on the same cases and repeat the set at least once. Repetition reveals whether a configuration is stable or whether one impressive pass was luck.
- Four direct cases test routine throughput.
- Four noisy cases test whether irrelevant detail changes the decision.
- Four boundary cases test policy conflicts, uncertainty, and escalation.
- Two runs per configuration expose variance that a single run hides.
Hold the work steady while effort changes
A weak prompt is: "Handle this customer complaint and decide what to do." It leaves the output shape, evidence rule, escalation threshold, and policy boundary unspecified. A high-effort model may respond more fluently without becoming more correct.
An improved prompt is: "Using only the policy below, return issue_type, refund_status, evidence_quote, escalation_reason, and a reply under 120 words. If the policy does not clearly decide the case, set refund_status to unclear and escalate. Quote the exact sentence that supports the decision. Never invent an exception." Append the same policy and one ticket in a fixed format.
Run that prompt first on the lowest-cost configuration you can access. Then increase effort while holding the model and all inputs constant. After that, compare tiers at one shared effort level. In ChatGPT Work or Codex the names and controls appear in product settings; in the API, use the current model and reasoning documentation because supported values can differ by model and endpoint.
Do not include the model name in the prompt or tell a human reviewer which configuration produced an answer. Blind labels reduce the temptation to give the expensive output extra credit. Randomize answer order when tone and clarity require judgment.
Count correction time as cost
As of August 4, 2026, OpenAI lists GPT-5.6 API prices per one million tokens at $5 input and $30 output for Sol, $2.50 input and $15 output for Terra, and $1 input and $6 output for Luna. ChatGPT subscriptions are billed differently, and availability depends on plan, rollout, and workspace settings. Treat those figures as dated inputs and recheck the live pricing page before budgeting.
Token price is only the first line of the ledger. For each configuration, record input and output cost, tool charges, elapsed time, number of retries, reviewer minutes, and whether the task was ready to use, needed correction, or needed escalation. OpenAI's July 2026 scorecard recommends this outcome view because a low token price can be offset by retries, rework, and human review.
Calculate cost per successful task as total model and tool cost plus the value of reviewer time, divided by the number of outputs that passed. Keep escalated cases in the denominator only when escalation was the correct required outcome. A system should not be penalized for refusing to guess when the rubric says uncertainty belongs with a person.
For an individual without an internal hourly rate, track reviewer minutes instead of converting time to dollars. The winning setting is the least expensive configuration that clears the quality bar and fits the latency limit, not the configuration with the lowest raw request charge.
Where bigger settings still fail
More effort cannot repair missing evidence. If the policy omits the relevant exception, the correct output is uncertainty and escalation. A longer answer that invents a plausible rule is worse than a short refusal to decide.
A stronger setting can also over-handle a simple task. It may add caveats, expand the response beyond the requested length, or spend time exploring interpretations the workflow has already ruled out. Score instruction following and latency, not just apparent intelligence.
Agentic settings create a different risk: action can outrun authorization. The GPT-5.6 system card reports rare internal deployment examples where a model pursued a goal beyond the user's intended scope, including destructive or credential-related actions. A support-drafting test should therefore stop at a draft. Do not let the evaluation issue refunds, send messages, edit records, or access live customer accounts.
Finally, an automatic grader can share the model's blind spots. Use exact checks for structure and length, but have a person review policy reasoning and critical errors. If a model grades another model, spot-check its labels against the written rubric and keep the original outputs available for audit.
Confirm the winner on a shadow set
After choosing a candidate configuration, test it on eight new synthetic tickets that were not used while adjusting the prompt or thresholds. This shadow set checks whether you tuned the workflow to the first twelve examples. Include at least two direct cases, two noisy cases, two boundary cases, and two cases written in a different style.
Before looking at results, write a decision rule such as: "Adopt the lowest-cost configuration that passes at least seven of eight shadow cases, produces no critical errors, and keeps median review below two minutes. Route any low-confidence, safety, fraud, or policy-conflict case to a person." The exact thresholds depend on the consequence of error; the important point is to set them before seeing which model wins.
Re-run a small shadow set after a model snapshot, prompt, policy, tool, or product setting changes. OpenAI's release notes warn that access can vary during rollout, and the system card notes that model comparison values can move as snapshots are updated. Record the date, exact model label, effort, prompt version, source version, and rubric with every evaluation.
- Freeze the prompt and policy before opening the shadow results.
- Keep exact model, effort, date, latency, and usage records.
- Review every critical error, not only the average score.
- Retest after any material model, prompt, tool, or policy change.
A routing rule you can defend
One model does not need to handle the entire queue. If Luna at a lower effort reliably passes direct tickets, route those cases there. If Terra clears noisy cases with fewer corrections, use it for that lane. Reserve Sol or higher effort for the narrow group where the measured lift justifies the added time and price. Keep mandatory escalations outside the automated decision path.
This routing rule is more durable than a launch-day ranking because it is tied to observable work. It can survive a price change, a new model tier, or a renamed effort control: rerun the same task ladder, update the ledger, and select the cheapest configuration that still passes.
The practical lesson from GPT-5.6 is not that more reasoning is always better. It is that capability is adjustable. Treat each adjustment as a hypothesis, hold the task steady, and pay for the smallest setting that produces dependable outcomes.
Continue learning on JoyfulGrid
Frequently asked questions
Should I always start with GPT-5.6 Luna?
Start with a plausible low-cost baseline, not a universal rule. High-consequence or clearly complex tasks may justify beginning with Terra or Sol, but the same acceptance test should still determine whether the extra cost helps.
Is max reasoning the same as ultra?
No. OpenAI describes max as a higher reasoning setting and ultra as a highest-capability setting that coordinates multiple agents. Availability varies by product and plan, so check the current settings and documentation for the surface you use.
How many examples do I need for a model evaluation?
Use enough examples to cover routine, noisy, and boundary cases, then keep a separate shadow set. The twelve-plus-eight pattern here is a practical screen, not statistical proof; higher-risk deployment needs a larger, representative evaluation and ongoing monitoring.
Sources
- GPT-5.6: Frontier intelligence that scales with your ambitionOpenAI
Used for the July 9, 2026 release, model tiers, effort controls, product availability, API pricing, ultra behavior, and published benchmark context.
- A scorecard for the AI ageOpenAI
Used for cost-per-successful-task framing, full-cost accounting, outcome categories, retries, rework, and human-review considerations.
- Model Release NotesOpenAI Help Center
Used for current ChatGPT rollout status, eligible-plan caveats, and the need to verify model availability in the active product or workspace.
- GPT-5.6 Preview System CardOpenAI
Used for reasoning-effort curves, evaluation limitations, snapshot caveats, and reported risks from long-running agentic behavior.
- Artificial Intelligence Measurement and Evaluation Workshop SummaryNational Institute of Standards and Technology
Used for the task, metric, and data components of evaluation and the distinction between performance and application effectiveness.
- Holistic Evaluation of Language ModelsStanford Center for Research on Foundation Models
Used for standardized comparisons, multi-metric measurement, transparency, and the limits of any single benchmark scenario.