Derailing slightly, a variety of HN posts (the blogs/sites they reference) have started banning me with a 403 page from nginx, including this author. My IP reputation is supposedly fine. Does anyone have any ideas where I should look next?
Depends on what you do? The site loads fine to me and you gave no indication what your originating IP space is being used for. As a residential user without anything but "normie" internet usage (reading blogs, watching yt) I experience no interuption. Are you a "normal" residential user as well? Then you might need to bring it up with your isp
I wonder he good we will be at not understanding anything to an ever increasing degree, with absolutely no hope of ever catching up. We will be super bad at absolutely everything within sciences and engineering and completely unable to contribute anything of substance, outside our own moods and quirks.
Can humans be okay with that? Can we be happy cats?
If we can trust payroll to a database instead of insisting someone do all the calculations by hand there is no reason to expect other forms of symbolic manipulation to be vastly different.
So is natural deduction when you treat quoted expressions as first class citizens.
The difficulty of doing proofs in mathematics is largely due to the fact that mathematicians refused to treat symbolic expressions as data and had to invent ever more complex paradigms to deal with them.
Lean being the final iteration of that trend.
When you add quotation and quasi quotation on top of Polish notation proofs become as mechanical as long division.
It does not. If after careful analysis people conclude that they cannot tell if it says what it claims to say, then then it can be considered not-a-proof until someone can demonstrate in a convincing manner that it is what it claims to be.
I can say god exists because 5>3. No matter how valid my claim is that five is in-fact greater than three, if I can't convince people of the connection it's a valid proof of something, just not what I say it is.
It might be true that if the Riemann hypothesis is false, leaves cannot fall to the ground. That would mean a falling leaf demonstrates the truth of the hypothesis. Without showing people why it must be the case in a way they understand it falls short of a genuine proof.
That would make proofs subjective, and I think that is true. To many people Pythagoras has not been proved, they just believe (correctly) there is a consensus amongst those who do understand the proofs.
You not knowing how to interpret the results is not the same as nobody knowing how to interpret the results. Formal proofs with very dense lean implementations have been a thing for a long time, it wasn't 'god on the mountain' then, either.
Aren't we approaching the limits of physics, with increasingly diminishing returns each generation? I remember reading an article about quantum effects at these scales, which make everything much harder. I mean, for example, even if we double the speed at which we can travel every year, you will never get to Alpha Centauri in your lifetime
> Airliners are still subsonic after a 100 years of aviation
That is a deliberate decision motivated primarily by economics and secondarily by environmental concerns. In fact airliners today are somewhat slower than they were a few decades ago because the lower speeds are more fuel efficient. We certainly could build fleets of supersonic airliners if it made sense to do so, it is not as if we have no idea how to do it and in fact we built supersonic passenger jets once upon a time.
Your example contradicts your point. Airliners are still subsonic after an extended period of time, and hence based on that past performance it's a good indicator (although not definitive) that they will remain subsonic for an extended period into the future.
In general, past performance is actually indicative of future results. It's not definitive, and some phenomenon are random, but the trajectory of technology over time is not some random fluke that just has coincidentally trended by random chance.
>Your example contradicts your point. Airliners are still subsonic after an extended period of time, and hence based on that past performance it's a good indicator (although not definitive) that they will remain subsonic for an extended period into the future.
There was a period in aviation development when it was widely assumed that speed will continue to rise. Just like in the development of CPUs there was such a period, quite recently.
But along other axes they have also improved tremendously, no? Ie rather than just time to destination, they have greatly improved efficiency (energy spent per unit of work) and throughput/parallelism (ie simultaneous flights in the air) right?
Yes, but these axes are also not showing an unlimited growth. There is no expectation that either speed, fuel efficiency, ticket price, or total number of flights will improve by an order of magnitude in the foreseeable future, unless there's a technological breakthrough (Nuclear turbines? Artificial photosynthesis making kerosene free? 3D printing the entire plane? These are currently all sci-fi ideas).
Sure you're going to get a lot of math problems solved, but food distribution is still going to be stuck by warlords and oligarchies.
Think of it like highway designers and the "one more lane" gridlock delusion. In this case, it's more will smith eating speghetti than cheap, low cost cancer cures.
> Airliners are still subsonic after a 100 years of aviation.
Yeah, but that's a choice made for very good reasons:
- Noise reduction
- Fuel efficiency
- Reduced pollution and lower environment impact
- Reduced cost of materials, equipment, and maintenance
- Passenger affordability
It's not that we can't make supersonic airliners or that the fact we don’t illustrates a lack of progress or performance: it's that it doesn't make economic or environmental sense to.
Gulfstream and Sukhoi even had a partnership to develop a supersonic business jet (which, if you look at the mockups, might be one of the coolest looking planes ever conceived) but they gave up after spending a fairly enormous amount of money already because it was going to cost an absolute fortune to bring to market... and there was no real market for it.
EDIT: the responses I’m getting are a bit weird. People are putting words in my mouth but all I was actually doing was pointing out the the example used wasn’t a great illustration of the point the person I was responding to was making. This isn’t Reddit so maybe we could reign it in a bit, please?
That's kinda the point I think ? It's not like we can't turn half of global energy production or GDP to compute - it's more about does it make sense to. As much as I'm getting value out of this technology daily - the level of investment made to get this far is astronomical compared to the value I'm seeing created, even less captured by these companies. And because the progress is so good in some domains it's easy to believe that these things are magically reaching some intelligence generalization threshold. But then you leave the training distribution and you get disturbed by how poor the performance is and how obviously equivalent things don't automatically transfer at all. Given the recent market trends I would not be surprised we see a correction in level of compute investment going forward.
The universe seems to be set up such that it wants entropy to increase. Any living thing that doesn't profit from converting low entropy energy to high entropy energy dies.
So the question is whether more intelligence will lead to faster increase in entropy. My guess is it will. Whatever recent market trends are, they are irrelevant. Whether the AI can or can't do x thing is irrelevant. What matters is whether the companies that make AI profit from making AI, and if you look closely at what's going on it sure seems like they do. If you think otherwise it may be because you're reading headlines and propaganda articles and not digging into the fundamentals.
So maybe we have a temporary correction/slow down, because these companies took on too much leverage and grew artificially too fast. That's very possible. In the big picture long term it will be a blip. The universe seems biased to increasing the volume of intelligence and I have never heard a compelling case for why this wouldn't continue for a long long time.
>It's not like we can't turn half of global energy production or GDP to compute
That depends which part of 'we' is running the economy. Just imagine an economy that makes mass massive amounts of money but doesn't align with human needs.
I can argue all of the reasons are valid for keeping compute at a certain speed range, or power consumption envelope.
It's not we can't make compute much more denser, there's a new standard informally called as megawatt racks. But deploying them at gigantic scales bring in massive problems as well, so we can keep total energy envelope at a fixed range.
Rather, nobody wants them at the price point necessary to make them economically viable. In the context of tech, the cost to develop a next-gen fab doubles every four years; absent some major breakthrough, we have about 20 years before even the most advanced economies balk at spending $1T, $2T, $4T, $8T on a single fab, causing hardware to settle down like airline speeds have.
the best computing power used to be located at the universities and national laboratories. Now you need access - employee or otherwise - to a nasdaq listed startup. Hard not to think it's the shareholders who benefit in the short term in this new alignment.
Just a proof of the infinite monkey theorem: Give a sufficiently large number of monkeys keyboards....gently nudge them in the right direction with a few banana tokens...let them type away with just enough randomness for an infinite amount of time.
Eventually, they will almost surely produce any mathematical proof imaginable...
My best guess is it's sincere. That's been a core observation of LLMs for awhile. Filtered random search is basically an inefficient gradient descent. If you apply even a poor-quality "gradient" to a lot of problems with infinite compute (with large LLM budgets), it doesn't take that good of an LLM to accomplish great things from time to time.
I mean they're obviously not stochastic parrots, I think most of us engineers have known this for a couple years now, though I do admit I saw a pretty obnoxiously vocal minority on the point here on HN.
But at the same time, where's the RSI? If the RSI is there then make the company profitable already [though I suppose that means letting go of all the staff? But if it's improving itself, not sure what humans are in the loop for].
Genuinely curious when AI will be able to solve real-world problems, what would 20,000 hours of AI say on the topic of whether Epstein killed himself?
“Because I am impressed and mystified by its outputs, I believe that the next token prediction machine must be doing something more than that, regardless of the lack of any evidence”
AGI is when the output of LLMs is plausible enough to fool putatively intelligent people that they are alive
LLMs are most definitely doing more than just predicting the next token. They can reason. This is emergent behavior. The idea is that, while one small part is doing something simple, the entire system together produces behavior that’s much more than that.
I kind of view it as how slime mold can solve a maze. Mold doesn’t think or see, but the system of a bunch of slime mold together has almost a rudimentary intelligence.
What an offputting title. The sneer really comes through. If I ever had any desire to read what this man wrote, this little potshot has extinguished it.
Not off-putting in the slightest. It's a well-deserved jab at the earliest LLM naysayers whose "criticism" was just ignorance wrapped in causeless activism. They claimed that LLMs were just randomly repeating locally coherent phrases back to us and that no amount of scaling could produce real intelligence. Well, the goal posts have moved far indeed if a proof of GRH isn't intelligence.
I think it's well deserved for all the people who made a name for themselves nay-saying llms/ai. I think it's less well deserved for people worried that their careers are shot and won't be able to provide for their families.
Indeed, it's not fashionable anymore to say "stochastic parrot" or "glorified auto-complete". What is fashionable today is to talk about the "convex hull of human knowledge that LLMs interpolate in"
It's good to poke fun at the AI phrase that's aged the worst and all the news articles that promoted it as fact. It did a lot of damage to people's understanding of AI.
Scott Aaronson has more intellectual chops and a better track record than 99.999% of people commenting on AI news. The sneer is a small fraction of that coming from the know-nothings who have been using phrases like "stochastic parrot", "fancy autocomplete", etc. for many years now. The parrots are coming home to roost, as they say.
Seriously. We are less prepared for the present and future because of all the denialists and naysayers, many on HN, who have repeatedly cried "stochastic parrot", "it just regurgitates known things, it can't create" and "marketing stunt" endlessly, sometimes >80% of the sentiment on threads over the past years.
Why the snarky headline? If parrots can generate statistically plausible proof snippets, and they are backed by a combination of Lean, other proof techniques stolen from ACL2 and a human mathematician they can still be stochastic parrots.
Aaronson is too close to the "rationalist" cult. Maybe he should sit down with ACL2 and get a clue. Emily Bender didn't speak about all that augmentation when coining the phrase.
I don't understand what this objection is even trying to argue. LLMs aren't intelligent, because they're capable of tool use?! One of the few other things we use as an example of how humans are more intelligent than most other animals?
Isn't it obvious? Scott Aaronson is a theoretical computer scientist, which is essentially a mathematician. Scott is likely seeing his career prospects disappear in front of his eyes just like so many others.
Too many people only see the perils of something once it affects them.
The optimists think AI will be like the cotton gin where making something cheaper and more accessible actually increased the demand for slaves as the price dropped. I'm more pessimistic. A lot of jobs are going to collapse in demand in the coming years. For awhile those at or near the top will survive and possibly make a lot of money but the rising floodwaters come for everybody.
> Scott is likely seeing his career prospects disappear in front of his eyes
Why would he be seeing that?
The combination of a mathematician and lean is powerful, and the combination of a mathematician, lean, and AI is more powerful still. Note that the mathematician is always part of that picture. That is fundamental -- no software system, no matter how big and complicated, can actually solve every math problem (this is a fundamental result in math that started the entire field of computer science). If you read TFA he basically tells you what he is actually worrying about here:
"the Lean formal verification system still has issues, from kernel bugs (which have, in the past, allowed proofs of 0=1) to uncertain soundness of the underlying type theory. In my view, these issues can and should urgently be fixed! Until they are, a sufficiently motivated AI could almost certainly exploit them to prove a false statement!"
He is worried about the proof checking system and the possibility that an AI might exploit bugs in the proof checker, not the disintegration of his entire career because AI can "just figure it all out" or whatever.
> A lot of jobs are going to collapse in demand in the coming years. For awhile those at or near the top will survive and possibly make a lot of money but the rising floodwaters come for everybody.
Funny, I'm actually optimistic for this. Let the flood wash away this messed up system that puts people in tiers and says people should only have access to what they can "afford", not to what they need. And then allows a few to be able to afford almost anything, and many to be able to afford nothing, just by being the first to X, or acquainted with Y.
> Scott is likely seeing his career prospects disappear in front of his eyes
...no he's not, that's a ridiculous statement. He is a prestigious professor with tenure at a top university.
Also, mathematics is much much more than just proving theorems. As the word goes, "Good mathematicians prove theorems, great mathematicians make conjunctures, the greatest mathematicians build theories". Someone has to carve out domains of interest to humankind. And humans will always know better what is of interest to humans than machines. These kind of reactions are a form of AI psychosis.
And lastly, this result was reached by a top mathematician steering the model towards the right approaches. You might say that the next iteration of the model might not need such steering to reach this result, but that just means that the pilot seat will have shifted higher up, so to say. New frontiers of complexity will become approachable to humanity and approach we will, "exploring strange new worlds and boldly going where no one has gone before".
Are you saying we should address llms with more respect now that their good at making lean proofs? Maybe we could call Claude Dr. Claude and Gemeni could be Professor Gemeni.
Ahh, I see. I took the article to be somewhat skeptical about the whole thing, even though he says he belives that the proof will hold. I personally feel like AI is a bit of a parrot hooked up to massive compute, which is the powerful part of it, and that it can solve unbelivably hard math problems but is still a parrot, so I may have read my own thinking into the headline instead of the author's.
I'm 99% sure it's not his actual opinion, but a sarcastic remark aimed at the people who claim that LLMs are just stochastic parrots. Like "your 'peaceful' dog just bit me".
LLMs are just stochastic parrots, it's simply not an insult. It reminds me of how people during covid would yell "it's not just like a bad flu" when literally the only valid historical comparisons we had were bad flus that killed millions.
When you're worshiping something, anything that implies that it is comparable to anything else is an insult to its divinity.
The title doesn't imply that sort of thinking on Aaronsom's part at all, though. Some simple dumb thing blowing through math problems thought maybe unsolvable is a hilarious contrast.
I don't think it counters the boosters to ascribe a result like this exclusively to the "stochastic parrot" though. There's an army of people behind each one of these results.
You don't need to verify 17 million lines of code, you just need to check that the hypothesis is formalized correctly (that's a subset) and that the kernel approves it
Can humans be okay with that? Can we be happy cats?
I'm genuinely curious where the median human's experience falls throughout history. Mostly happy?
I suspect probably not
The difficulty of doing proofs in mathematics is largely due to the fact that mathematicians refused to treat symbolic expressions as data and had to invent ever more complex paradigms to deal with them.
Lean being the final iteration of that trend.
When you add quotation and quasi quotation on top of Polish notation proofs become as mechanical as long division.
I can say god exists because 5>3. No matter how valid my claim is that five is in-fact greater than three, if I can't convince people of the connection it's a valid proof of something, just not what I say it is.
It might be true that if the Riemann hypothesis is false, leaves cannot fall to the ground. That would mean a falling leaf demonstrates the truth of the hypothesis. Without showing people why it must be the case in a way they understand it falls short of a genuine proof.
That would make proofs subjective, and I think that is true. To many people Pythagoras has not been proved, they just believe (correctly) there is a consensus amongst those who do understand the proofs.
Past performance is not indicative of future results. Airliners are still subsonic after a 100 years of aviation.
That is a deliberate decision motivated primarily by economics and secondarily by environmental concerns. In fact airliners today are somewhat slower than they were a few decades ago because the lower speeds are more fuel efficient. We certainly could build fleets of supersonic airliners if it made sense to do so, it is not as if we have no idea how to do it and in fact we built supersonic passenger jets once upon a time.
In general, past performance is actually indicative of future results. It's not definitive, and some phenomenon are random, but the trajectory of technology over time is not some random fluke that just has coincidentally trended by random chance.
There was a period in aviation development when it was widely assumed that speed will continue to rise. Just like in the development of CPUs there was such a period, quite recently.
Sure you're going to get a lot of math problems solved, but food distribution is still going to be stuck by warlords and oligarchies.
Think of it like highway designers and the "one more lane" gridlock delusion. In this case, it's more will smith eating speghetti than cheap, low cost cancer cures.
Yeah, but that's a choice made for very good reasons:
- Noise reduction
- Fuel efficiency
- Reduced pollution and lower environment impact
- Reduced cost of materials, equipment, and maintenance
- Passenger affordability
It's not that we can't make supersonic airliners or that the fact we don’t illustrates a lack of progress or performance: it's that it doesn't make economic or environmental sense to.
Gulfstream and Sukhoi even had a partnership to develop a supersonic business jet (which, if you look at the mockups, might be one of the coolest looking planes ever conceived) but they gave up after spending a fairly enormous amount of money already because it was going to cost an absolute fortune to bring to market... and there was no real market for it.
EDIT: the responses I’m getting are a bit weird. People are putting words in my mouth but all I was actually doing was pointing out the the example used wasn’t a great illustration of the point the person I was responding to was making. This isn’t Reddit so maybe we could reign it in a bit, please?
So the question is whether more intelligence will lead to faster increase in entropy. My guess is it will. Whatever recent market trends are, they are irrelevant. Whether the AI can or can't do x thing is irrelevant. What matters is whether the companies that make AI profit from making AI, and if you look closely at what's going on it sure seems like they do. If you think otherwise it may be because you're reading headlines and propaganda articles and not digging into the fundamentals.
So maybe we have a temporary correction/slow down, because these companies took on too much leverage and grew artificially too fast. That's very possible. In the big picture long term it will be a blip. The universe seems biased to increasing the volume of intelligence and I have never heard a compelling case for why this wouldn't continue for a long long time.
That depends which part of 'we' is running the economy. Just imagine an economy that makes mass massive amounts of money but doesn't align with human needs.
It's not we can't make compute much more denser, there's a new standard informally called as megawatt racks. But deploying them at gigantic scales bring in massive problems as well, so we can keep total energy envelope at a fixed range.
...for the sake of the reasons you outlined.
I think this approach and conclusion applies to a lot more than the immediate context of that conversation.
I think robot wives are also coming - the market will be absolutely massive.
Eventually, they will almost surely produce any mathematical proof imaginable...
Start with the title of the article
But at the same time, where's the RSI? If the RSI is there then make the company profitable already [though I suppose that means letting go of all the staff? But if it's improving itself, not sure what humans are in the loop for].
Genuinely curious when AI will be able to solve real-world problems, what would 20,000 hours of AI say on the topic of whether Epstein killed himself?
AGI is when the output of LLMs is plausible enough to fool putatively intelligent people that they are alive
I kind of view it as how slime mold can solve a maze. Mold doesn’t think or see, but the system of a bunch of slime mold together has almost a rudimentary intelligence.
I do not see what this kind of post offer here when there is already at least one thread to discuss Alpöge's result with 1000+ comments.
Indeed, it's not fashionable anymore to say "stochastic parrot" or "glorified auto-complete". What is fashionable today is to talk about the "convex hull of human knowledge that LLMs interpolate in"
The feeling of a sneer comes only from the absurdity of the belief being definitively revealed.
For anyone who feels sneered at: the problem is the absurd belief, not the reveal.
"Don't sneer at the Emperor for parading around with nothing on!"
Aaronson is too close to the "rationalist" cult. Maybe he should sit down with ACL2 and get a clue. Emily Bender didn't speak about all that augmentation when coining the phrase.
Now complain about upper case, which is the most likely continuation for a human parrot. Or the new account.
Too many people only see the perils of something once it affects them.
The optimists think AI will be like the cotton gin where making something cheaper and more accessible actually increased the demand for slaves as the price dropped. I'm more pessimistic. A lot of jobs are going to collapse in demand in the coming years. For awhile those at or near the top will survive and possibly make a lot of money but the rising floodwaters come for everybody.
Why would he be seeing that?
The combination of a mathematician and lean is powerful, and the combination of a mathematician, lean, and AI is more powerful still. Note that the mathematician is always part of that picture. That is fundamental -- no software system, no matter how big and complicated, can actually solve every math problem (this is a fundamental result in math that started the entire field of computer science). If you read TFA he basically tells you what he is actually worrying about here:
"the Lean formal verification system still has issues, from kernel bugs (which have, in the past, allowed proofs of 0=1) to uncertain soundness of the underlying type theory. In my view, these issues can and should urgently be fixed! Until they are, a sufficiently motivated AI could almost certainly exploit them to prove a false statement!"
He is worried about the proof checking system and the possibility that an AI might exploit bugs in the proof checker, not the disintegration of his entire career because AI can "just figure it all out" or whatever.
Funny, I'm actually optimistic for this. Let the flood wash away this messed up system that puts people in tiers and says people should only have access to what they can "afford", not to what they need. And then allows a few to be able to afford almost anything, and many to be able to afford nothing, just by being the first to X, or acquainted with Y.
...no he's not, that's a ridiculous statement. He is a prestigious professor with tenure at a top university.
Also, mathematics is much much more than just proving theorems. As the word goes, "Good mathematicians prove theorems, great mathematicians make conjunctures, the greatest mathematicians build theories". Someone has to carve out domains of interest to humankind. And humans will always know better what is of interest to humans than machines. These kind of reactions are a form of AI psychosis.
And lastly, this result was reached by a top mathematician steering the model towards the right approaches. You might say that the next iteration of the model might not need such steering to reach this result, but that just means that the pilot seat will have shifted higher up, so to say. New frontiers of complexity will become approachable to humanity and approach we will, "exploring strange new worlds and boldly going where no one has gone before".
How on earth can the comment you reply to be misunderstood? It even defends Bender.
When you're worshiping something, anything that implies that it is comparable to anything else is an insult to its divinity.
The title doesn't imply that sort of thinking on Aaronsom's part at all, though. Some simple dumb thing blowing through math problems thought maybe unsolvable is a hilarious contrast.
The boosters are out in force to gloat today.
This is just another hypothesis on top of the Riemann Hypothesis. We now have an Hypothetical proof of the Riemann Hypothesis, great. Breakthrough.
Perhaps read up on the value proposition of Lean.