This neat and all but I think there's a bigger issue here. Anthropic (and whoever Anthropic wants) can do this kind of work with Claude, but if you or I try to do it, we will get blocked. This is a fun exercise, but extend this to something much more nefarious and you can see a moat widening between the model providers, their government partners, and the public.
The only thing that bridges this moat is continued advancement of open models.
Maybe, but that's what the cyber program is for. Nobody knew if the seed was fine or a backdoor. If it was a backdoor, and Claude helped you find it as a normal person, Anthropic would get bad PR. If you're a verified security researcher, actually that's fantastic and Anthropic would get good PR and everybody claps.
Anthropic wants to IPO, they don't want another Fable shutdown situation.
To be clear I would prefer the cyber safeguards to be less restrictive, I think they're currently too much. I often go to open models because of it. But there are likelier reasons to restrict cyber than world domination.
Anthropic gets zero of my money because their favorite word is no. I cannot figure out how to as an individual or a small startup request elevated permissions for the purposes of anything that they think might potentially be dangerous for someone who is not me to access. I have a background in cybersecurity. I have a background in life sciences. Can I put those skills to use to benefit humanity? Who knows.
I feel like history is repeating itself - to many people AOL, Compuserve, MSN, et al seemed superior to "the web" in the early 1990s because they dumped money into making it slicker, licensed content, provided guardrails, made it easy for non-tech people... They had slick ads, were cultural phenoms, and people would take sides with the companies and think their arguments were strong and even obvious!
How do you know this to be true? I am not disagreeing; I genially need to understand why or if others are seeing blockers that I don't see or if I am seeing blockers that they do not see.
Are their guardrail benchmarks or anything of the sort that we can use to see where the bifurcation of permission/capability occurs?
There is very much a situation where people in frontier labs have access to exceptional capabilities that are being kept from the rest of us, and they're using it to strip-mine technical and scientific problems.
Broadly, the downgrading is still way too heavy-handed and clumsy. As of yesterday “How easy to crack is WPA2” downgraded to opus 4.8. Granted I could probably have phrased it differently and gotten a response, but the point is I shouldn’t have to. It can’t simultaneously be smart enough to reverse the ECDSA curve seed and not smart enough not to destroy the world by answering a trivial question with a scary word in it.
Fable 5.1 has downgraded me in the past week alone for the crimes of describing a coordinating server as "the C&C server" ([cyber]), for doing some statistical analysis on some toy Vigenère-ciphered text ([cyber]), and for asking it to help me find patterns in some semi-random text ([bio]), so it is fair to say I am not skeptical.
I have access to Claude's "defensive" cyber model and it appears to take no issue with the prompt, but I suspect that is because it uses today's publication as the basis for an answer.
In the past my attempts to work on crypto related problems have been met with needing the higher tier of "offensive" access.
> moat widening between the model providers, their government partners, and the public.
Part of me wonders whether the NSA/other TLA has either (a) an unrestricted model from a lab with less scruples like SpaceX, or (b) an in-house model, either abliterated or with data they've bought themselves.
I know Anthropic models have a lot of insane, paranoid filters, but I don't remember them blocking cryptography work last I checked (some 3 months ago now).
The cyber filter blocks really basic stuff all the time. People who are pro that bs should be forced to only eat with a spoon for the rest of their life.
“In under half an hour, Claude discovered the origins of the NIST P-192, P-224, P-256, P-384 and P-521 seeds. None of them were the results of any hash algorithm at all, but rather the output of the C library’s lrand48 random-number generator, seeded with a Unix timestamp!”
Gorgeous. I wonder if the NSA employee who did this suffered a head injury or has been quietly giggling to themselves for the last quarter century.
The two people were Bob Reiter and Jerry Solinas. Jerry wrote in a 2015 email to Daniel Bernstein that he thought it was a hash of the passphrase. I corroborated that story with 4 people, including two ex-NSA folks off the record.
Everyone was wrong!
I wasn't able to reach Bob. He was working in private industry as of 2018, but never responded.
> The fifth binary seed, B-163, fits nothing Claude has tried and remains unexplained.
The simplest explanation would be that it was generated the same way as the others, but somebody mistyped a digit somewhere along the way to publishing, and therefore a solution could not be found with the assumption that the number is correct.
Probably not quite this given the timestamps span over almost a year (but have some clustering in August 1997).
Given that the spread suggests they weren't just now() timestamps it would actually be interesting to know what the significance of the timestamps is, if any. Though it could be something boring like randomly sampling some local file creation times.
On the face if it, the big leap here was Claude autonomously envisioning and deciding to pursue the "ancient prng + date" angle, rather than working backwards from the premise of deliberately weakened seeds or some other popular starting point. To me that shows real awareness of the situation and method of the people at the NSA.
So two questions:
TFA mentions the "humourous phrase" theory, but Claude's trajectory seems to be a long way from that. To what extent was Claude explicitly prompted in the ultimately successful direction?
2. Whats with all the "hmm..." in the internal reasoning process log?
I find it so ridiculous that they (seemingly) forcefully try to state references to dedicated researchers who devoted much of their lives to the problem, while the LLM does not even know which trained papers were effectively used. That means likely many researchers not listed were heavily influencing this achievement, and Claude does not even know that they helped. And the world doesn't know.
This happens in academia all the time. You read a ton of papers, chat with tons of people about tons of ideas.
Later you come up some idea, you do the experiment, write the paper, search for relevant citations when you write the intro.
Honestly I think this is actually desirable. For a mind to work well and do things creatively and efficiently it can't be a simple search engine, it needs to blend and meld and reform the information.
And that means the output can't be cleanly deterministically linked to the input. Unless you just keep a log then cite everything you've ever read in your life (and all the conversations and talks too).
> The main contribution is the solution that Claude came up with.
That's an argument that must be infuriating to all researchers who ever published a paper on that topic, because most likely Claude was trained with all of them.
> None of them were the results of any hash algorithm at all, but rather the output of the C library’s lrand48 random-number generator, seeded with a Unix timestamp!
More and more it is the relentlessness of the models that strikes me. Leave no stone unturned! Go through ALL of these old boxes. Someone said they thought maybe it was over there somewhere >>shrug<<
While this confirms the method of production it doesn't say anything about their selection, especially since they were, apparently, generated over a period of months. Specifically you cannot see those candidates that were generated and discarded.
"It is still conceivable that these curves were selected to have a specific property in principle by searching over timestamps to the random number generator."
Yes.
"However, you would only need to do that if you were going to disclose the timestamp as some evidence that the seed was innocuously generated."
Does not logically follow when "doing and saying nothing" will completely suffice.
It doesn't prove anything about the core part of the suspicion, which is they were selected using some undisclosed criteria, which may make them weaker or stronger than some other random numbers.
You are entirely familiar with the precedent for the DES s boxes of course, where that is exactly what happened.
When they made the changes to the s boxes for differential cryptanalysis resistance the entire concept of that was essentially secret, so who knows what they know about elliptic curves.
Makes me wonder if any implementations will do the cute trick of dispensing with the seed constants and deriving them at startup from the unix timestamps. :)
"It is still conceivable that these curves were selected to have a specific property in principle by searching over timestamps to the random number generator."
Indeed! And we guess that OpenAI was gagged by the NSA because the massive proof dump on Oct. 6th did not contain cryptographic results.
I assume that Anthropic faces similar restrictions, so this seems kind of approved.
This is huge. It means the Bitcoin and Ethereum communities can now confidently ship the opcodes and precompiled contracts to check ECDSA signatures on P-256 in addition to K-256 curves, and stop worrying about possible NSA backdoors based on the chosen seeds (quantum computers are another story).
Nobody conversant with elliptic curve cryptography research believed there was a backdoor based on these seeds; for a number of reasons, best articulated by Koblitz and Menezes (who took on the question directly).
But it's nice to be able to shut the whole unproductive debate down directly.
Taking the question seriously (when I wrote this there was no /s), since I think it is still a question some people legitimately have, it is still extremely implausible that such tight dates and such a small number of iterations are all backdoored. For backdoors to be that dense would indicate some sort of serious flaw in the algorithm as a whole as well, something so serious that it should be fairly obvious. If these were initialized from dates, each separated by some billions of years and with the iteration parameters also in the millions or billions or trillions for no apparent reason it would be time to be suspicious.
Is it impossible? It is hard to prove impossible. But this is not what I would expect backdoors to look like.
In fact this makes me think that the information about how these were generated must have been lost very early in the process, or over-compartmentalized away from the wrong people. It seems like the NSA should have been positively delighted to show that these were the origins of the seeds as they are decent "nothing up my sleeve" numbers: https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number Advertising that fact should have been advantageous to the NSA if that is indeed how they were generated, so my best guess is that the parts of the NSA that should have been showing us that must not have themselves known.
Not entirely related to the topic, but did someone spent some tokens trying to figure out the origins of the "Wow! Signal" (https://en.wikipedia.org/wiki/Wow!_signal) ?
> This is not a cryptographic break, new attack, or weakness in the ECDSA curves. In fact, this makes us more confident the curves were chosen without any backdoors or intentional weaknesses.
Uh… did you talk to any cryptographers? This is EXACTLY what a weak curve would look like. Imagine that 1 in a thousand, or one in a million keys, selected at random, are vulnerable to some withheld attack. This approach lets you vary the timestamp or number of iterations until a weak key is found, then have plausible deniability afterward. This is why modern systems choose constants in a way with no free choice of parameters (e.g. sqrt of 2 for first constant, cube root for second constant, etc.).
They even note later in the article that viable curves in the generator sequence seem to have been skipped.
This is discussed in the post - since the NSA never mentioned how these seeds were created, it seems unlikely that they would develop a cover story that would not be shared and only opened to scrutiny many years later
The only thing that bridges this moat is continued advancement of open models.
Anthropic wants to IPO, they don't want another Fable shutdown situation.
To be clear I would prefer the cyber safeguards to be less restrictive, I think they're currently too much. I often go to open models because of it. But there are likelier reasons to restrict cyber than world domination.
I think the current administration would delight in harsher penalties
Are their guardrail benchmarks or anything of the sort that we can use to see where the bifurcation of permission/capability occurs?
As this is going on, Anthropic has opened up their own biology lab.
> https://www.reuters.com/world/anthropic-quietly-sets-up-biol...
There is very much a situation where people in frontier labs have access to exceptional capabilities that are being kept from the rest of us, and they're using it to strip-mine technical and scientific problems.
They sure as hell ain't stripteasing :)
In the past my attempts to work on crypto related problems have been met with needing the higher tier of "offensive" access.
If we allow these providers to promote regulatory capture they become the arbiter of what anyone is "allowed" to do
Part of me wonders whether the NSA/other TLA has either (a) an unrestricted model from a lab with less scruples like SpaceX, or (b) an in-house model, either abliterated or with data they've bought themselves.
Gorgeous. I wonder if the NSA employee who did this suffered a head injury or has been quietly giggling to themselves for the last quarter century.
https://www.youtube.com/watch?v=eOQfNT6Jdkg
Everyone was wrong!
I wasn't able to reach Bob. He was working in private industry as of 2018, but never responded.
The simplest explanation would be that it was generated the same way as the others, but somebody mistyped a digit somewhere along the way to publishing, and therefore a solution could not be found with the assumption that the number is correct.
Probably not quite this given the timestamps span over almost a year (but have some clustering in August 1997).
Given that the spread suggests they weren't just now() timestamps it would actually be interesting to know what the significance of the timestamps is, if any. Though it could be something boring like randomly sampling some local file creation times.
So two questions:
TFA mentions the "humourous phrase" theory, but Claude's trajectory seems to be a long way from that. To what extent was Claude explicitly prompted in the ultimately successful direction?
2. Whats with all the "hmm..." in the internal reasoning process log?
Later you come up some idea, you do the experiment, write the paper, search for relevant citations when you write the intro.
Honestly I think this is actually desirable. For a mind to work well and do things creatively and efficiently it can't be a simple search engine, it needs to blend and meld and reform the information.
And that means the output can't be cleanly deterministically linked to the input. Unless you just keep a log then cite everything you've ever read in your life (and all the conversations and talks too).
That's an argument that must be infuriating to all researchers who ever published a paper on that topic, because most likely Claude was trained with all of them.
Could they not be both?
Ouch. The implied judgment of us humans stings there.
"It is still conceivable that these curves were selected to have a specific property in principle by searching over timestamps to the random number generator."
Yes.
"However, you would only need to do that if you were going to disclose the timestamp as some evidence that the seed was innocuously generated."
Does not logically follow when "doing and saying nothing" will completely suffice.
You are entirely familiar with the precedent for the DES s boxes of course, where that is exactly what happened.
Are there weaker and stronger seeds for this category of elliptic curves?
When they made the changes to the s boxes for differential cryptanalysis resistance the entire concept of that was essentially secret, so who knows what they know about elliptic curves.
EDIT: ah, one of the linked authors tried that. They had no documentation on it. https://saweis.net/posts/nist-curve-seed-origins.html
"read this: https://words.filippo.io/seeds-bounty/ then find the solution to the bounty"
Indeed! And we guess that OpenAI was gagged by the NSA because the massive proof dump on Oct. 6th did not contain cryptographic results.
I assume that Anthropic faces similar restrictions, so this seems kind of approved.
>provided by the NSA with no explanation or justification.
Really? Nobody is concerned about the NSA just cheerfully dictating that? Smells very rainbow-table-y
I turned out to be right in my advocacy years ago, here is my post: https://ethereum-magicians.org/t/eip-7212-precompiled-for-se...
But it's nice to be able to shut the whole unproductive debate down directly.
Is it impossible? It is hard to prove impossible. But this is not what I would expect backdoors to look like.
In fact this makes me think that the information about how these were generated must have been lost very early in the process, or over-compartmentalized away from the wrong people. It seems like the NSA should have been positively delighted to show that these were the origins of the seeds as they are decent "nothing up my sleeve" numbers: https://en.wikipedia.org/wiki/Nothing-up-my-sleeve_number Advertising that fact should have been advantageous to the NSA if that is indeed how they were generated, so my best guess is that the parts of the NSA that should have been showing us that must not have themselves known.
Uh… did you talk to any cryptographers? This is EXACTLY what a weak curve would look like. Imagine that 1 in a thousand, or one in a million keys, selected at random, are vulnerable to some withheld attack. This approach lets you vary the timestamp or number of iterations until a weak key is found, then have plausible deniability afterward. This is why modern systems choose constants in a way with no free choice of parameters (e.g. sqrt of 2 for first constant, cube root for second constant, etc.).
They even note later in the article that viable curves in the generator sequence seem to have been skipped.
That's not how math works!
google some of the names under the "Who we are" heading