Neon / Carbon / Argon / Hydrogen are all in the same ballpark, why did they even release four different models?
Neon doesn't seem particularly strong at coding vs Argon, and Opus 5.5 still beats both in my experience. The neurons it lost that made it incompetent at English seem to have been fully dedicated to being a freak at engineering.
Because benchmarks aren't everything? If you see Neon and Hydrogen get the same score in a benchmark you should question if the benchmark measures what we care instead of suspecting that Neon and Hydrogen are equally powerful.
Having used both Sol-6.2 and Astra-6.2 heavily over the last few days (as well as regularly using Claude Opus and Fable) they are marked improvements over the previous releases and comparable to Opus-5.5 and Fable-5.2.
I've stopped trusting any of these influencer-benchmaxxers because they generally don't run any of the workflows of interest to me - it's completely synthetic. The only benchmark that matters is how well any model works for your current use-case.
I've been finding 6.2 Sol doesn't act like an architect and doesn't doubt its instructions or existing code at all. It's back to "you're right to push back" responses when questioned about it's plans/reasoning in the slightest.
For my experience, i feel it follows instructions too well. Grey areas are completely cut out, and sure thats good sometimes. But for my usecase, i liked when the response felt smart and ready to ask questions.
Phillip has updated the leaderboard and surprisingly every new model release got substantially better, except for the OpenAi models. 6.2 Sol is 30th whereas 6.1 Sol is the 10th. Astra 6 is 9th, Astra 6.2 is 15th. By comparison Fable 5.2 became the 1st and Fable 5.1 is 7th. All the new google models, even hydrogen, are above the human baseline. By contrast, Astra 6 pro is the only openai model slightly above the human baseline at 8th place
> SimpleBench includes over 200 questions covering spatio-temporal reasoning, social intelligence, and what we call linguistic adversarial robustness (or trick questions).
We could create a utopia if we could screen voters for linguistic adversarial robustness.
6.2 is for sure the worst received update OpenAI has ever released.
What happened lol.
I wanted to update our older used 5.x models in api to 6.2 the other day (due to lower cost), asked Fable about the difference between 6.1 and 6.2 if the cost is the same and it was like "yeah don't use 6.2, it has bad feedback online, has increased laziness etc., use 6.1". Yikes.
I'm not finding it lazy -on the contrary my WTF per session has gone way up because it's doing loads of stuff I'm not asking for. About a third of my time with it over the past week has been making it strip back unwanted work it did, even when I have a clear plan and spec to work from, and when challenged every time it'll admin the extra work either wasn't specified or frequently that it directly contradicts something in the spec.
I thought you were joking, but that's actually the fourth question. Then I thought maybe it was meant to be a trick question, but none of the possible answers make sense.
Edit: I'm stupid. The question is correct. See dist-epoch's and pvitz's answers.
The trick here is that the classic version has one sister telling the truth and the other the lie. Then you would ask what the other sister would answer. In this case, you can just ask which path to take and then you take the other one.
I wonder if OP got routed to the nerfed versions. 6.2 works excellent for me. Getting better results than 6.1 and faster. On the other hand Fable and Opus are a disaster. The stupidity of those models reminds me of early Gemini.
Neon doesn't seem particularly strong at coding vs Argon, and Opus 5.5 still beats both in my experience. The neurons it lost that made it incompetent at English seem to have been fully dedicated to being a freak at engineering.
I've stopped trusting any of these influencer-benchmaxxers because they generally don't run any of the workflows of interest to me - it's completely synthetic. The only benchmark that matters is how well any model works for your current use-case.
It should be obvious that random hacker news comments are worth less than an influencer benchmark, so why waste the time commenting?
What is your goal other than to be an even worse influencer than the very one you're complaining about?
Are you a bot?
So far, my experience with 6.2 is that it is crazy fast compared the before, and much more to the point.
Is that something that can be noticed during real life use?
I really like opus 5.5, It's perfect and I haven't tried the new gemini models.
We could create a utopia if we could screen voters for linguistic adversarial robustness.
What happened lol.
I wanted to update our older used 5.x models in api to 6.2 the other day (due to lower cost), asked Fable about the difference between 6.1 and 6.2 if the cost is the same and it was like "yeah don't use 6.2, it has bad feedback online, has increased laziness etc., use 6.1". Yikes.
Generally speaking the whole 6 series is a flop to me. Only bright side is that it's more efficient/lasts longer/is cheaper than 5.6.
But for now, I'm using opus 5.5.
They were always the leaders at "common sense".
There are two sisters, Amy who always speaks mistruths and Sam who always lies.
:)
Edit: I'm stupid. The question is correct. See dist-epoch's and pvitz's answers.
There are 2 paths. You ask C) "What path leads to the treasure". Both of them lie, so you pick the other path.
Terrible funding rounds, uncompetitive models, enshitification, and insurmountably bad and incompetent PR.