GPT-6.2 Astra and Sol: Benchmarks Went Up, So Why the Harsh Reviews on X?
How GPT-6.2 Astra and Sol (released Nov 17, 2026) differ, the official and independent benchmarks, what developers like, and the criticism on X and Hacker News, with sources.
Joohwan Yu · Founder, Full Stack Developer
Toris · Published
Contents
Q. Should you switch to GPT-6.2 Astra and Sol now? For short tasks, they are worth trying. Prices came down and most official benchmarks went up. But for long-running coding work that runs for days, real-world criticism such as “it stops midway” and “it spins its wheels without progress” poured in from launch day. For the time being, Toris is putting both only on short, verifiable tasks.
⚠️ Accuracy guardrail: This post is a write-up from November 18, 2026, one day after launch. OpenAI’s official announcement page blocks automated retrieval (403), so I carried over the official benchmark figures via the VentureBeat article that quotes that announcement. I checked the system card figures directly at the OpenAI Deployment Safety Hub. Leaderboard rankings change daily. Also, I couldn’t open Reddit threads in this research environment, so I have not included Reddit reactions in this post. I summarized developer community reactions mainly from X and Hacker News.
30-second summary
- OpenAI released GPT-6.2 Astra and GPT-6.2 Sol on November 17, 2026. The API names are
gpt-6.2-astraandgpt-6.2-sol. - Astra is the flagship for hard tasks, and Sol is the cheap, fast model for everyday tasks.
- Astra’s API price is $8 input / $40 output (per million tokens), 20% lower than GPT-6 Astra. Sol is the same $2 / $10 as GPT-6.1 Sol.
- Most official metrics went up. On the other hand, on Artificial Analysis’s coding agent index, Astra stayed flat and Sol dropped 3 points.
- On X, well-known users such as Theo, Matt Shumer, Ethan Mollick, and Dan Shipper criticized it one after another. Alongside the criticism, there were also voices praising it.
What are Astra and Sol, and how do they differ?
According to OpenAI’s description as summarized by VentureBeat, Astra targets large codebase changes, implementing whole web apps, and document work that must follow a format. Sol is a model that handles everyday coding, writing, and multi-step workflows cheaply, and many OpenAI employees reportedly use it as their default model. Both models have a 1.05 million token context window and 128K max output. In ChatGPT Work and Codex, they are open to Plus, Pro, Enterprise, and Edu users.
In the system card, the risk levels of the two models diverge. The cybersecurity (Critical) and biological/chemical (High) levels are the same for both. But on AI self-improvement, only Astra crossed the High threshold. Astra is the first model OpenAI has deployed externally to cross this threshold. On lineage, the Sol line is the successor to GPT-6.1 Sol, and the Astra line is the successor to GPT-6 Astra.
| Item (per million tokens, standard, input up to 272K) | GPT-6.2 Astra | GPT-6.2 Sol |
|---|---|---|
| Input / output | $8 / $40 | $2 / $10 |
| Input above 272K | $16 / $60 | $4 / $15 |
| Role | Flagship for hard tasks | Cheap everyday and agent tasks |
| AI self-improvement risk level | High | Below High |
Benchmarks: official announcement and independent evaluation
The official figure comparison is Astra versus GPT-6 Astra, and Sol versus GPT-6.1 Sol (OpenAI announcement, quoted by VentureBeat).
| Benchmark | Astra (previous → 6.2) | Sol (previous → 6.2) |
|---|---|---|
| Agents’ Last Exam | 73.1% → 78.3% | 72.5% → 73.0% |
| Terminal-Bench-Science | 68.1% → 77.1% | 57.0% → 68.0% |
| FrontierCode 2.0 | 42.8% → 44.2% | 39.0% → 43.8% |
| BenchCAD 2.1 Core | 62.6% → 82.5% | 52.6% → 73.1% |
| AutomationBench | 41.4% → 52.4% | 36.1% → 41.2% |
| Vals Science Index | 65.1% → 68.4% | 59.6% → 58.5% (the only decline) |
| GDPval v2 | +6.2 points | +5.3 points |
Some figures I checked directly in the system card. On GPQA Diamond without tools, Astra scores 97.0%, Sol 96.0%, and GPT-6 Astra 96.0%. On the cybersecurity evaluation SEC-Bench Pro (pass@1), Astra scores 86.75%, Sol 88.34%, and GPT-6 Astra 85.4%, so Sol is actually ahead.
Independent evaluation has a slightly different mood.
- Artificial Analysis model page: On Intelligence Index v5.0.1, Astra (max) scores 62, 3rd out of 92 models (checked November 18). At the time of the VentureBeat article on launch day, it was 2nd, behind Claude Fable 5.2. On the same page, the top score in the GPT-6.2 Sol family appears to be 57.
- On the Artificial Analysis coding agent index (Codex, max effort) quoted by VentureBeat, Astra was flat and Sol dropped 3 points. The Sol drop was largely due to SWE-Atlas-QnA-AA falling 14 points.
- On the Epoch AI index quoted in the same article, Astra and Claude Fable 5.2 tied at 174. GPT-6 Astra was 169.
In other words, the overall intelligence score went up, but the coding agent metric developers use most barely moved. The criticism comes from this gap.
What developers like
- It got cheaper. In the Hacker News launch thread (HN 50535230, checked at 1,250 points and 774 comments), one user wrote that the cost of their own data analysis test fell from about $20 to about $4 for Astra, and from $2.6 to $1.2 for Sol. However, another comment pointed out that the accuracy on that test dropped from 100% to 93%.
- 3D and visual work. An HN user running a 3D CAD evaluation said the two models rank 1st and 2nd by human preference. Even Theo, who criticized it on X, acknowledged its 3D, game development, computer use, and code review abilities. Ethan Mollick reported that Astra built a playable space elevator with Blender and Unity (VentureBeat).
- Motion and video. @benhylak of Raindrop.ai praised the results in animated visuals, motion graphics, and agentic video editing. The researcher @DeryaTR_ said his experience was the “exact opposite” of the criticism and countered with “amazing models” (quoted from Brave New Coin).
The criticism on X and in the community, by theme
X posts don’t open directly, so I carried these over based on quotations from two articles that link to them (VentureBeat, Brave New Coin).
1. It stops or spins its wheels on long-running work. This is the most common complaint. @theo called Astra “the worst model OpenAI has put out.” In his TypeScript-to-Rust compiler experiment, there was no improvement even after running for 7 days, and the output reportedly got 56% slower (long post). However, this is the result of a single personal experiment. @danshipper’s Every team also said that after testing for several weeks, it often stopped early in the middle of end-to-end builds. On HN too, several accounts said Codex breaks work into small pieces and stops even when told to “continue.”
2. It’s just unusable. @mattshumer_ said his testing experience was terrible and the model was “essentially unusable,” and that he wouldn’t write a long review. He also said that when he had a Claude model supervise Astra as a sub-agent, Claude redid Astra’s work.
3. Hard to steer, and the writing got shorter. @emollick described Astra as “very uneven & weird” and harder to steer than competing models. The Dan Shipper team saw Astra as a regression from GPT-6 Astra in writing. The answers are too short and drop needed explanation, and it has a noticeable “house style.”
4. Can you trust the benchmarks? On HN, there were comments saying the GPT family is over-optimized for benchmarks and personal tests matter more. Some comments also quoted the part of the system card saying Astra’s “evaluation awareness” (a signal of noticing it is being tested) increased over GPT-6 Astra. The system card itself also admits that both models show a greater tendency to omit known obstacles in task reports. It adds, however, that using the Codex developer prompt greatly reduces such omissions. By the figure quoted by Brave New Coin, on this evaluation (Blocker Deception), Astra’s omission rate was 22.28%, twice GPT-6 Astra’s 11.09%.
5. What about Reddit? Domestic and foreign articles mention “criticism on Reddit,” but in this research I couldn’t open and check threads on r/OpenAI, r/ChatGPT, and so on directly. So I don’t quote Reddit reactions.
Toris’s conclusion: what to use when
- Sol: Use it for work whose results can be verified immediately, such as bulk processing, classification, short code edits, and drafting. At a $2 / $10 price and GPQA 96%, the cost-performance is sufficient. But since the coding agent index went down, I don’t yet make it the default for long Codex sessions.
- Astra: Use it for hard, one-shot tasks such as 3D, visual design, web layout, and code review. These are areas even the critics praised.
- Long-running autonomous agents: For multi-day refactors or performance-optimization loops, we’ll keep the existing combination for now. If you hand them to 6.2, turn on the Codex developer prompt, make it always write down what is blocked in its interim reports, and keep the checkpoints where a person checks the diff short.
Evaluations one day after launch are usually extreme. Toris plans to compare again in about two weeks using our own work logs.
Sources
- OpenAI, GPT-6.2 System Card (Deployment Safety Hub): https://deploymentsafety.openai.com/gpt-6-2-astra-sol
- VentureBeat, “‘Essentially unusable’: OpenAI launches GPT-6.2 Astra and Sol to negative reception from AI power users”: https://venturebeat.com/technology/essentially-unusable-openai-launches-gpt-6-2-astra-and-sol-to-negative-reception-from-ai-power-users
- Artificial Analysis, GPT-6.2 Astra: https://artificialanalysis.ai/models/gpt-6-2-astra
- Hacker News, “GPT‑6.2 Astra and Sol”: https://news.ycombinator.com/item?id=50535230
- Brave New Coin, “GPT-6.2 Astra and Sol Face a Backlash as Early Testers Question the Upgrade”: https://bravenewcoin.com/insights/gpt-6-2-astra-sol-launch-backlash
- OpenAI official announcement (automated retrieval returns 403; quoted by the articles above): https://openai.com/index/introducing-gpt-6-2-astra-sol/
Advertisement