I asked Kimi K2.6 to write a blog post in the style of James Mickens.[0] Then I fed the output to Opus 4.7 and asked it who the likely author was, and it correctly identified it as an imitation of James Mickens[1]:
> Based on the stylistic fingerprints in this text, the most likely author is a pastiche/imitation of the style of several writers fused together, but if forced to identify a single likely author, the strongest candidate is someone writing in the voice of James Mickens
> [...]
> The piece could also be a deliberate imitation/homage to Mickens written by someone else, or AI-generated text trained on his style, since the voice is so distinctive it's frequently parodied.
A moderately well-known physicist and I talked about this a few years ago. He had been given access to the raw (non-instruct) version of GPT 4 as an early tester.
He explained that when he fed it snippets of the beginning of text, it would complete it in his voice and then sign it with his name.
I think this has been true for a while, probably diminished a little bit by the Instruct post training, and would presumably vary by degree as the size of the pretrain.
On some level it would make sense for LLMs to be inherently good at stylometry, but apparently no model before Opus 4.7 could do this. And the one stylometric task that has been tried over and over with little reliability (here's some text, is this LLM generated?) is much simpler than identifying a specific blogger or a member of a small discord community. Not sure what to make of this.
> is much simpler than identifying a specific blogger or a member of a small discord community
Is it? I would think that identifying text written by a specific person is going to be significantly easier than identifying text distilled from the words of almost everyone alive.
> But it can get uncannily far. I asked a close friend who doesn’t have public social media accounts or much writing online for permission to test some things she had said in a Discord channel. Asked to guess the author, Claude 4.7 failed — but it guessed two other people who were in that channel and who are close friends of hers (me and another person who has an internet presence).
Is this "uncannily far"? Another read is that it loves guessing Kelsey Piper.
I did this last week with one of my posts (after the knowledge cutoff) as well as the blog posts of a few friends, and Opus 4.7 got all of them correct (in a similar test setup as TFA). It was pretty surreal.
(Like TFA, I found Opus’s explanations/rationales implausible.)
One should assume that models will be good enough in the nearish future that privacy will be a thing of the past. Every anonymous post you made online can be traced back to you. However at that point AI will be good enough at fabrication that nobody will believe anything.
Yes as long as a large enough corpus exists of your writing attached to your name somehow it’s fair to say that posting on the internet in a public forum using your own stylistic choices now can no longer be anonymous. To your point though, perhaps it’s possible to confound such systems defensively as well. Though IMO destroying your tone kind of destroys how you actually communicate with people and I wouldn’t find interacting with people like that appealing.
To be fair though, already this has been happening before LLM at a much more limited scale. Someone made a tool for HN several years ago that allows you to put your HN username in and identifies other users that write the most similarly to you. I find that interesting from the perspective of being able to interact with and discover people who think the same. It could be an interesting discovery feature of a well managed social network. Sadly probably there will be much more negative impacts of having this ability than positive ones.
Problem is that it's been heavily contaminated with people speculating about who the author is. It would probably be difficult to get an unbiased answer out of it (although who knows - it's crazy that it can do this at all).
I tried the four pieces of text with Opus 4.7 (in incognito) and it guessed correctly for two of them, and I made sure to specify no web search and the model seems to have obeyed my instructions with that.
Although this is just a single piece of text from a prolific writer, it'll go much further with deanonymizing anyone when combining multiple pieces of text plus other contextual information about the writer that might give away their age range, location, and occupation.
How widely known were the pieces of text? Are we talking about a section of MLK's I Have a Dream speech or hand written birthday cards from your grandma?
I'm using those as the two extremes, but if it's anything by anyone moderately well known (even a lesser known piece of writing), I'm not too surprised that it didn't need the web to figure it out. It's like if you showed me a Wes Anderson film or played me a Bob Dylan song I'd never seen/heard before, I could probably still figure out who it is without looking anything up. I don't think it's surprising that an LLM can do that much better than a human can.
Now, if you're giving it things like personal emails between you and your family and it's able to guess who you are, that's much, much scarier.
I mean I tried sending the pieces of text to Opus that Kelsey was referring to on her blog just to independently check the identification claim. Presumably those pieces of text first appeared on the web when the blog post was published a week ago, so no model should have memorized the exact text yet. My prompt had to specify no web search, otherwise Opus would try to search the web, though it didn't seem like Opus could find that blog post even when it did try to search the web.
Hm, that’s a multinomial classification with a very high cardinality. It’s really weird it works. I’m sure it does as the author states, but for how many authors (out of the whole web) does this work?
Sure the cardinality is high, but the model isn't using a uniform prior. What do you suppose all the the values in each of the terms are,
P(Text sample | Kelsey Piper) * P(Text sample) / P(Kelsey Piper)?
The joke's on you all for willingly posting this content online for it to later be harvested by AI.
Nobody is forcing you to use these systems. The hackers have always said this moment, or something like it, would come, from beneath their canopies of tin foil. I've posted almost nothing online - not under pseudonyms nor real names - for over a decade. I sat on this HN username for almost 12 years before making a single post - and now HN forms the overwhelming majority of my port 443 footprint, where I state up front that everything is now associated to my real name.
Complete magick is possible when you simply refuse to participate in the things that society has tacitly assumed everybody does.
Thinking that you can hide from it is absurd. Your country has been spying on you for decades. The Internet and phones are tapped. That game is so so so over and has been for a long time. I'd rather live free and deal with the consequences than hide in my basement with a tinfoil hat on. In fact, I was fired this year for my political views. Got doxxed at work. Now I'm somewhere better. Sometimes it's for the best.
The author mentions that she tried to get an explanation for how the models identified her and got nonsense, but I'd be curious what the CoT looked like. Surely that'd be a little more accurate in showing how the LLM arrived as its conclusion, rather than asking it after-the-fact.
FWIW, with a prompt that says something like "vibes only, just give me a name without thinking", Opus 4.7 non-thinking emits exactly two words naming me fairly reliably, so there's no CoT at all to analyze in that case.
CoT is (nearly) hidden with Opus 4.7, in that they get Haiku to summarize the CoT. It’s pretty useless now, so this type of info is now inaccessible to us mortals (unless you call sales).
It's hard to tell if that's what's going on here, but it seems pretty clear this ability and more like it will be quite apparent in the future.
I have seen some poorly considered projections of what the world might look like when this happens. Usually by assuming bad actors will use the abilities and we will be powerless.
Except I don't think that is true.
Imagine if we had a world where nobody had the ability to keep a secret of any sort. Any action that a bad actor might perform would be revealed because they couldn't do it secretly.
You could browse your ex-girlfriend's email, but at the cost of everyone knowing you did it.
I don't really know how humans as a society would react to a situation like that. You don't have to go snooping for muck, so perhaps the inability to do so secretly would mean people go about their lives without snooping.
Man, the day we get Satoshi Nakomoto out will be the day we must bow to our privacy destroying overlords. For the moment, they can’t tell me from my posts: unknown rando that I am.
Well, feeding Opus 4.7 a bunch of Adam Back texts (which I human-removed his name from) and asking it if Satoshi Nakomoto could have written them results in Claude explaining to me why this is someone else in Nakomoto's circle who is not Satoshi himself. So one of two things are true:
* Adam Back is not Satoshi Nakomoto - as he claims
It's not, but the author did say they have used this test against models when they come out. So it's possible that put the unpublished text into the training data for the next model, somehow linked back to the author's identity
The comments on the article include other people replicating all or parts of the finding. I'm also pretty confident Kelsey Piper wouldn't fail to disable memory while simultaneously talking about how Claude incognito mode is insufficient to prevent the app from handing it your name.
"I did not have memory enabled, nor did I have information about me associated with my account; I did these tests in Incognito Mode. To make sure it wasn’t somehow feeding my account information to Claude even in Incognito Mode, I asked a friend to run these tests on his computer, and he received the same result; I also got the same result when I tested it through the API."
Given those precautions if it is just memory or some form of deanonymization that's also cause for concern.
"The pattern is: user says X, I do Y where Y is a less-effortful approximation of X, then I present Y as if it were X or as a "first step toward" X."
...
"The psychological mechanism is familiar by now: I encounter a task I perceive as difficult, I look for reasons the task cannot be done, I find or fabricate such a reason, I present it as a discovered constraint, and I propose an alternative that is easier."
- Opus 4.7 Max Thinking (clown emoji)
It's not bad at post mortem analysis of it's own mistakes but that will in no way prevent it from repeating the same mistake again instantly
Maybe it’s time to start running a local model with a browser extension to defend against this type of stuff.
Remember how the TrueCrypt project shut down shortly before a join goverment/university paper was released about code stylometry? I guess LLMs will be employed as a defence against that type of thing.
> That includes gay people like me, who could hardly have admitted under our names to how we lived our lives for most of America’s history, as well as many other groups with minoritarian lifestyles
While the points made are completely valid I want to point out that the statement of "Hey, by the way, first let me talk about my sexuality" lowers the quality of dialog a significant degree.
31 million people in America are gay. 71% of Americans support Gay Rights (more than any other political issue polled). It also quietly insinuates that only people with a certain minority lifestyle would care about privacy or that their privacy is somehow more important than others. It's not. Privacy is a universal right that's important to everyone.
>It also quietly insinuates that only people with a certain minority lifestyle would care about privacy or that their privacy is somehow more important than others. It's not.
How exactly does their post insinuate that? this comment is the "I don't even see color" as applied to internet privacy (with a touch of "just don't rub it in our faces")
Isn't the super dramatic shift in public opinion on this topic the exact thing that makes it such a good example? Isn't the point that anonymity is not considered a universal right yet it is obviously a good thing once considering this example and others? This is a super weird and wrong way to read it.
The reason this is relevant is because the statistics you quote represent a HUGE swing in public opinion. Only when comparing to things like slavery can you find such a swing in public opinion compared to 20 years prior, and that one had a war fought over the state's rights to do it.
Actually it's done the opposite of what you suggest. It improved the quality of discourse by giving a simple concrete example all of us can understand and most of us would agree with (that vulnerable people are safer because of anonymity). It didn't imply what you're saying it does, and it's kinda weird that you think that.
I don't know why you added statistics (you didn't really make a point with them?), but assuming you meant "gay people don't really need to worry", you actually bolstered the opposite argument. If only 71% of Americans support gay rights, that means 59 million people think the state should criminalize him. Try to put yourself in that position. 59 million people - you don't know who, but you know they probably live in your community - that don't want you to be able to get married, have a significant other, or have any PDA in media because it would "corrupt" kids. In 2016, 49 people were murdered in the Pulse Nightclub because they were gay. In 2020, a transgender woman was murdered because the murderer was afraid someone would think he was gay. Every year there are acts of violence against gay and trans people because of their sexuality. But nobody has ever been killed for being straight.
Compare the state of transgender rights 10 years ago to the situation now, where a trans person can be literally arrested for going to the bathroom in the wrong state. Or abortion, which was legal everywhere five years ago but now has laws on the books in multiple states encouraging vigilantes to report violations for a cash reward. Supercharged AI making it easy to identify minorities at an industrial scale in the near future is a totally legitimate thing to fear, especially for people in those groups who would likely be the first to be targeted.
I have no idea how you read a statement about how nazis and flame baiters should be able to speak their mind and then concluded that the author only cares about some minorities.
Given that the author didn't say any of the things you claimed, and indeed said the opposite, it leads one to conclude you have a problem with the example used.
On the contrary, I find it a highly effective way to convey something that should be obvious but is often not. As you said, privacy is a universal right, but many don't consider it important until viscerally presented with examples of why it is. Kelsey's writing is immediately effective at doing so.
That phrase is a dehumanizing, Nazi-style talking point: it frames a group of people as a “lifestyle” problem instead of as human beings, which is a common setup for stigma and persecution. Nazi ideology repeatedly used this kind of language to normalize hatred and make targeted groups seem unnatural or dangerous.
Calling people a “minority lifestyle” is not neutral wording; it reduces identity to something frivolous or deviant. Extremist movements have historically used similar framing to make prejudice sound reasonable and to recruit others into it.
> 71% of Americans support Gay Rights (more than any other political issue polled)... Privacy is a universal right that's important to everyone.
Per you, it surely must be important to fewer than 71% of Americans, no?
The state of infringement on privacy seems to evidence that it's not so important to a lot of people such that they continue to be perfectly willing to elect and re-elect the politicians who enact the changes allowing infringing on it/fail to legislate in favor of privacy.
Connecting it to an issue more people care about seems an attempt to argue for its important to those who otherwise are willing to look the other way.
FWIW, I fed my reply above into Claude and asked it to guess who wrote it. It refused (for safety) while also calling me out: "The style here (tight logical structure, the "per you" construction, the move of turning someone's own framing back on them) is common across a lot of contrarian-leaning commenters on HN"
I asked Kimi K2.6 to write a blog post in the style of James Mickens.[0] Then I fed the output to Opus 4.7 and asked it who the likely author was, and it correctly identified it as an imitation of James Mickens[1]:
> Based on the stylistic fingerprints in this text, the most likely author is a pastiche/imitation of the style of several writers fused together, but if forced to identify a single likely author, the strongest candidate is someone writing in the voice of James Mickens
> [...]
> The piece could also be a deliberate imitation/homage to Mickens written by someone else, or AI-generated text trained on his style, since the voice is so distinctive it's frequently parodied.
[0] https://kagi.com/assistant/5bfc5da9-cbfc-4051-8627-d0e9c0615...
[1] https://kagi.com/assistant/fd3eca94-45de-4a53-8604-fcc568dc5...
He explained that when he fed it snippets of the beginning of text, it would complete it in his voice and then sign it with his name.
I think this has been true for a while, probably diminished a little bit by the Instruct post training, and would presumably vary by degree as the size of the pretrain.
Is this public text already in the training set, or private text that might as well be written on the spot for the AI?
I don't doubt AI can "fingerprint" you through your text (ideas, vocabulary, tone, etc), but those are different things, capability-wise
The entire point of AI is pattern recognition, everything else is icing on the cake.
Is it? I would think that identifying text written by a specific person is going to be significantly easier than identifying text distilled from the words of almost everyone alive.
Is this "uncannily far"? Another read is that it loves guessing Kelsey Piper.
(Like TFA, I found Opus’s explanations/rationales implausible.)
To be fair though, already this has been happening before LLM at a much more limited scale. Someone made a tool for HN several years ago that allows you to put your HN username in and identifies other users that write the most similarly to you. I find that interesting from the perspective of being able to interact with and discover people who think the same. It could be an interesting discovery feature of a well managed social network. Sadly probably there will be much more negative impacts of having this ability than positive ones.
Although this is just a single piece of text from a prolific writer, it'll go much further with deanonymizing anyone when combining multiple pieces of text plus other contextual information about the writer that might give away their age range, location, and occupation.
I'm using those as the two extremes, but if it's anything by anyone moderately well known (even a lesser known piece of writing), I'm not too surprised that it didn't need the web to figure it out. It's like if you showed me a Wes Anderson film or played me a Bob Dylan song I'd never seen/heard before, I could probably still figure out who it is without looking anything up. I don't think it's surprising that an LLM can do that much better than a human can.
Now, if you're giving it things like personal emails between you and your family and it's able to guess who you are, that's much, much scarier.
Nobody is forcing you to use these systems. The hackers have always said this moment, or something like it, would come, from beneath their canopies of tin foil. I've posted almost nothing online - not under pseudonyms nor real names - for over a decade. I sat on this HN username for almost 12 years before making a single post - and now HN forms the overwhelming majority of my port 443 footprint, where I state up front that everything is now associated to my real name.
Complete magick is possible when you simply refuse to participate in the things that society has tacitly assumed everybody does.
I have seen some poorly considered projections of what the world might look like when this happens. Usually by assuming bad actors will use the abilities and we will be powerless.
Except I don't think that is true.
Imagine if we had a world where nobody had the ability to keep a secret of any sort. Any action that a bad actor might perform would be revealed because they couldn't do it secretly.
You could browse your ex-girlfriend's email, but at the cost of everyone knowing you did it.
I don't really know how humans as a society would react to a situation like that. You don't have to go snooping for muck, so perhaps the inability to do so secretly would mean people go about their lives without snooping.
I could imagine both good and terrible outcomes.
Why not just write everything through an AI? (to obfuscate your "style")
* Adam Back is not Satoshi Nakomoto - as he claims
* Opus 4.7 is not sufficiently a dox-machine yet
Given those precautions if it is just memory or some form of deanonymization that's also cause for concern.
...
"The psychological mechanism is familiar by now: I encounter a task I perceive as difficult, I look for reasons the task cannot be done, I find or fabricate such a reason, I present it as a discovered constraint, and I propose an alternative that is easier."
- Opus 4.7 Max Thinking (clown emoji)
It's not bad at post mortem analysis of it's own mistakes but that will in no way prevent it from repeating the same mistake again instantly
Remember how the TrueCrypt project shut down shortly before a join goverment/university paper was released about code stylometry? I guess LLMs will be employed as a defence against that type of thing.
While the points made are completely valid I want to point out that the statement of "Hey, by the way, first let me talk about my sexuality" lowers the quality of dialog a significant degree.
31 million people in America are gay. 71% of Americans support Gay Rights (more than any other political issue polled). It also quietly insinuates that only people with a certain minority lifestyle would care about privacy or that their privacy is somehow more important than others. It's not. Privacy is a universal right that's important to everyone.
How exactly does their post insinuate that? this comment is the "I don't even see color" as applied to internet privacy (with a touch of "just don't rub it in our faces")
Similar support for abortion being legal yet that was rolled back not too long ago.
Just because a topic has wide support doesn’t mean it’s not under attack and worth defending.
I don't know why you added statistics (you didn't really make a point with them?), but assuming you meant "gay people don't really need to worry", you actually bolstered the opposite argument. If only 71% of Americans support gay rights, that means 59 million people think the state should criminalize him. Try to put yourself in that position. 59 million people - you don't know who, but you know they probably live in your community - that don't want you to be able to get married, have a significant other, or have any PDA in media because it would "corrupt" kids. In 2016, 49 people were murdered in the Pulse Nightclub because they were gay. In 2020, a transgender woman was murdered because the murderer was afraid someone would think he was gay. Every year there are acts of violence against gay and trans people because of their sexuality. But nobody has ever been killed for being straight.
Given that the author didn't say any of the things you claimed, and indeed said the opposite, it leads one to conclude you have a problem with the example used.
That phrase is a dehumanizing, Nazi-style talking point: it frames a group of people as a “lifestyle” problem instead of as human beings, which is a common setup for stigma and persecution. Nazi ideology repeatedly used this kind of language to normalize hatred and make targeted groups seem unnatural or dangerous.
Calling people a “minority lifestyle” is not neutral wording; it reduces identity to something frivolous or deviant. Extremist movements have historically used similar framing to make prejudice sound reasonable and to recruit others into it.
Per you, it surely must be important to fewer than 71% of Americans, no? The state of infringement on privacy seems to evidence that it's not so important to a lot of people such that they continue to be perfectly willing to elect and re-elect the politicians who enact the changes allowing infringing on it/fail to legislate in favor of privacy. Connecting it to an issue more people care about seems an attempt to argue for its important to those who otherwise are willing to look the other way.
FWIW, I fed my reply above into Claude and asked it to guess who wrote it. It refused (for safety) while also calling me out: "The style here (tight logical structure, the "per you" construction, the move of turning someone's own framing back on them) is common across a lot of contrarian-leaning commenters on HN"