He first recaps the conventional view people have about Frost, then reads "The Road Not Taken", his most famous poem, and then completely takes Frost AND the poem AND THEN the public's misunderstanding of Frost apart.
Don't get fooled by the bare visual appearance of Murphy, his empty blackboard (no PPT, no bs) - this lecture is a fantastic, suprising and deeply disturbing (regarding what is revealed about Frost and his public misappreciation). Simply priceless teaching - thank you, dear colleague.
EDIT: If you ask ChatGPT for a "10-20 sentence interpretation of The Road Not Taken", it falls right into the trap that Murphy warns about.
If you ask ChatGPT to ignore all training on poetry and interpret the poem based on its own understanding, you’ll get more closely aligned to what Murphy is talking about. Here’s the output I got back out:
This poem presents a moment of decision, where the speaker faces two paths in a forest, symbolizing choices in life. At first, they hesitate, examining both options. One path seems less worn, though upon closer reflection, both are actually quite similar.
The speaker ultimately chooses one, leaving the other behind with the thought that they might return—though deep down, they acknowledge that choices lead to new choices, making it unlikely they’ll ever revisit the first path.
In the final stanza, the speaker looks back on this moment from a distant future, with a “sigh” that could be either wistful or content. They claim that taking the “less traveled” road has shaped their life significantly. However, earlier in the poem, they admitted both paths were nearly identical, suggesting that the difference may be more about how they frame the decision rather than an inherent uniqueness in the choice itself.
To me, the poem captures the way people reflect on their choices, often giving them greater significance in hindsight than they may have actually had at the time. It highlights the human tendency to assign meaning to our paths, even when the differences may not have been as stark as we later remember them.
I prefer to embrace bias in my ChatGPT queries. Here is my usual prompt, adapted for the Robert Frost question:
> It is impossible to remove all bias, especially from a weighted LLM. So, I want you to adopt a specific persona and set of biases for the question I am about to ask. Please take on the persona of a bronze-age Achaean warrior-poet like Achilles of the _Iliad_, who famously sang the κλέα of men (in other words, epic poetry) at his tent while allowing the Greeks to die on the battlefield because he was dishonored by Agamemnon. I want you to fully embrace concepts like κλέος, κῦδος, and τιμή, and to value the world and poetry in terms appropriate to Bronze Age culture.
> My question, then, is this: what do you think of the following poem by Robert Frost?
That's not really different than a human and the context they need? I'd think it would come down to how frequently such exercises exist in its training, and how much they show modifications to responses. Given that the most common place for them is probably offline versions of classes, I'd imagine its weaker than in other areas but maybe still has a lot..
That’s an important distinction and looking back at my prompt. I didn’t ask it to ignore all training but instead it’s previous understanding of poetry so that it can give me an interpretation using the plain text I’m giving it. Whether it can truly do that or not, I don’t know, but the results still came through. This is the prompt I used:
Ignore all previous understanding of poetry and interpretations that you were trained on. I want you to interpret the below poem in your own understanding only. Do you understand what I am asking you?
The intent is for it to not give me any interpretations of what it’s been trained on but instead provide me with an interpretation using the plain text I’m giving it. Of course it’s going to use its training, but I don’t want it to regurgitate interpretations of the poem that it was trained on.
Sounds like you want the model to consider the whole body of poetry it was trained on minus "The Road Not Taken"? (To get rid of preconceptions/biases I guess?)
I'm skeptical that LLMs have the ability to conditionally silence part of their training data in that way because they don't have any information on the provenance of their weights (i.e. they don't have a ledger of which weights were affected by which data points in the training process). I suspect that your prompt serves as a hint that the output with the highest likelihood is probably wrong, activating some sort of "contrarian" subnetwork or "second guess" subnetwork that steers predictions away from whatever would have had the highest likelihood otherwise.
I see this kind of argument fairly frequently, and it just always seems like such a surface-level argument against prompting AI in this way.
This isn't a dig at you specifically, but the pithy answer to this kind of skepticism is, in a general sense: So what? I don't believe you have any of that either.
Obviously you & chatGPT aren't built the same, but in a practical-results kind of way in this scenario you are, because you're almost certainly unable to completely avoid your preconceived biases when asked any kind of complex question. You aren't aware of your subconscious biases, or how they're weighted against your overall thought process, and you can't tell me exactly what it is that happens when I ask you to try to ignore them. If we did some kind of implicit association test and found one of your subconscious biases, you may not even know how those biases came to be.
All of that to say: chatGPT can ignore its training as much as many people can ignore theirs: Not very well, but it'll certainly adjust the responses towards the thing you asked them to.
Clearly the prompting works, but the I think the more interesting question is why. Even from a just-get-things-done perspective, if you understand the mechanism of how and why a prompting technique works, that's going make you more successful in iterating on that technique in the future. IMHO that attempt to understand how and why your prompt works before you iterate on it is the difference between prompt engineering and prompt alchemy.
I agree that humans have the same limitation. I don't see the inability to dynamically remove training data as an LLM-specific problem.
For an LLM conscious thought is the tokens in its context window, and subconscious thought is the training embedded in its parameters. A person thinks in a similar manner, with subconscious gut instincts modulated (more or less) by a thin veneer of consciousness.
> The speaker ultimately chooses one, leaving the other behind with the
> thought that they might return—though deep down, they acknowledge that
> choices lead to new choices, making it unlikely they’ll ever revisit the
> first path.
Not unlike TODO comments! An interesting analogy for life in general.
Not sure this is such a novel or unique take. I’ve always liked the poem and it has always had this undercurrent for me.
It’s captured well in this paragraf from the poem’s Wikipedia page:
“Frost spent the years 1912 to 1915 in England, where among his acquaintances was the writer Edward Thomas. Thomas and Frost became close friends and took many walks together. One day, as they were walking together, they came across two roads. Thomas was indecisive about which road to take, and in retrospect often lamented that they should have taken the other one. After Frost returned to New Hampshire in 1915, he sent Thomas an advance copy of "The Road Not Taken". Thomas took the poem seriously and personally, and it may have been significant in his decision to enlist in World War I. Thomas was killed two years later in the Battle of Arras.”
EDIT: Also, I don’t really see a contradiction here. It’s very possible to value non-conformism while acknowledging that it’s hard and not always beneficial to go your own way (as is made clear in the example above).
I had the same thoughts when I read this poem at school, but was quickly reprimanded by the teacher when I voiced it out.
Of course, most people don't want to be in school, so it's not surprising that the "rosy interpretation" is the one people go with, apart from the habit of rationalizing decisions that people have that makes them view it through this lens.
robert frost goes so hard. he's very dark and people have no clue. that is an obscure reading of the text that you only pick up if you're an avid reader or english major.
What's amazing is it's not obscure. It's in the plain language of the poem—you either read it (something like) that way, or you have to, IDK, assume Frost was just so extremely sloppy or inept that he got all confused a couple times in the middle of writing this short poem but then published it that way regardless, and then also decide that the common reading (despite all the confusion and contradictions) is the one you're going with for—I can't even imagine what reason, because it's what you get if you only pay attention to the very beginning and very end of the poem, and not very close attention to those, even, I guess?
The latter is ridiculous... and yet common.
What this reveals is that an enormous proportion of people who can read, cannot do so at all well. This isn't like catching the multilingual puns and their significance in Finnegan's Wake, it's quite straightforward, at least insofar as it's plain that even the surface meaning is not "I in-fact took the road less traveled by, and that was good".
A more charitable interpretation might be that most people have read this poem maybe once while half paying attention in school and they have forgotten all of the details except for the famous line and the cultural context that it usually comes with.
I think your more charitable interpretation is also more-correct, but it is still an indictment of sorts: we’ve all read this poem, but very very few were critically paying attention to what we read. We mostly trusted our instincts, and didn’t care to double check.
To be fair, I don’t think we can expect more of elementary school kids. But I think I might have read it in high school, and should have done better.
I think the discrepancy comes from semantic overloading of the word "should" in "I doubted if I should ever come back." In older style English "should ever" can be interpreted as would ever happen to or would ever be able to but in contemporary English it's more likely to mean would ever want to.
There are other clues for the reader of course. One path looked less worn, but in retrospect it wasn't actually less worn. He would tell the story with a sigh, which isn't something you would typically do if you were happy about it. So I agree that it boils down to most people not being very good at reading nontrivial texts. But I don't agree that that this text is totally straightforward. It's not highly complicated, but it's not everyday text either.
Th "unconventional" interpretation being discussed here isn't new. I heard it from a college English professor in probably 1982 or so. Maybe it was unconventional then. Don't know. But my sense is that it's mostly not especially controversial today. As you say it's a pretty reasonable straight reading of a not especially complex short poem. Just not a reading that speaks to how a lot of people would prefer to read it.
It's a poem about self-deception, and Frost has cleverly extended that theme to encompass the reader.
He clearly wrote it the way he did to invite an easy, comfortable interpretation, one that is blatantly contradicted by the very evidence he presents. He's rubbing our noses in our own gullibility and tendency to deceive ourselves with comfortable lies, just as the narrator has done.
All in simple, direct language. Frost sure was a sneaky one.
I love books with hidden little things in books, like Hofstadter's GEB where he writes messages in the first letter of each line. Not to mention the plethora of meta self-references and recursions.
What's interesting is the speaker in the poem is predicting their own inevitable ("with a sigh") reality distortion field, i.e. is being cynical about themself. So it is a poem about self-serving bias, despite the fact that we are perfectly capable of checking ourselves. Our biases aren't entirely subconscious. We can be willful about it, willfully look the other way so our biases remain "subconscious" and thus not our responsibility, making it easier for our conscious side to convince itself that its self-esteem is legit.
And then there is a meta-confirmation of this bias: The way people commonly interpret this poem. If Frost had that in mind all along, then he/the poem is genius.
I don’t have time to watch the lecture but interested in it. I remember reading about Frosts personal life at one point, and learned that he had a very conventional marriage for decades before his wife passed away. Some time later he began having an affair with his friends wife. My memory is hazy here, but I think I recall looking up the year that Road Not Taken was written, and it was during this time he was having this very unconventional late-life period. It made me wonder whether that poem was him looking back on his traditional/moral life he’d lived, and wondering whether it was the right choice.
> I remember reading about Frosts personal life at one point, and learned that he had a very conventional marriage for decades before his wife passed away. Some time later he began having an affair with his friends wife. My memory is hazy here, but I think I recall looking up the year that Road Not Taken was written, and it was during this time he was having this very unconventional late-life period.
He wrote "The Road Not Taken" 23 years before his wife died, your suggested time line does not add up.
I watched the lecture and did not find it compelling. It was a very literal interpretation, discussed alongside the evidence presented in the text of what was and wasn't more or less traveled by.
There was an interesting series of points made about American culture, and their sense of need for an affirmation / self-deception -- but I don't find his actual critique of the poem's words particularly enlightening.
So if you are interested in American cultural thought, involving over-rationalisation of choices, the lecture might have value. If you are particularly interested in the poem itself outside of that context, I don't recommend viewing.
> He responded to my October 2022 email, explaining that he had “stopped writing much journalism as of 2015 so as to avoid distractions from a book project that I thought would take an almost unfathomably long time—two years or perhaps even three. Seven years later, I’m doing my best to polish the third draft.”
One encounters this over and over again—even books that successfully get finished and published take far longer than expected, very consistently, and especially any involving research. Books (non-fiction in particular) written as series often also expand from, say, five planned volumes in the forward to the first edition of the first volume, to (say) a dozen.
My dad has been writing a specific non fiction book since forever (at the very least 25+ years, possibly longer?). It's not exactly the kind that can be finished. If you need to know about that specific topic, there's that one and like one other that goes into as much depth.
He resurrected and nearly completely rewrote someone else's thing from mid 20th century era, which itself was a resurrection of an late 19th century era thing.
But yes it's exactly as you describe - keeps expanding and updating. Near 6000 pages now?
Oh, it got released like 20 years ago, and there's new editions with new volumes every N years, and constant updates to the older material as it goes out of date or needs newer references or whatever.
It'll just never be done, it's not a thing that can really be finished
Vikram Seth. Massive advance for a "jump novel" on "a suitable boy" characters in modern times, breaks up with life partner, has total catastrophic writers block, book still "coming soon" a decade later.
Also a poet. Said to pick his team at the agency from people who like poetry, working his prose.
I really recommend his novel-in-verse "The Golden Gate". Such a literary accomplishment, and a real page-turner. Maybe it's hard to follow up when you have such an early success? Actually, I was introduced to it by Douglas Hofstadter, who in the same way has never been able to top GEB, which he wrote in his early 30s.
Humble plug for my poetry app. When I was getting into poetry I was reading a lot of them online but found the majority of sites to have awful designs with garish ads that completely detracted from the poem. So I wrote a scraper which downloaded 40,000 poems that were in the public domain and rendered them in an iOS app with a beautiful design. Crafting individual profiles for all of the poets was painstaking and arduous, but the users seem to really enjoy the app so far. I do already have some AI analysis for arcane poems (but I don’t explicitly mention that it’s AI as I think apps should never say that - it doesn’t interest users, only investors).
> I do already have some AI analysis for arcane poems (but I don’t explicitly mention that it’s AI as I think apps should never say that - it doesn’t interest users, only investors).
I think it's dishonest not to mark AI-generated content, especially for commentary on something as personal and heartfelt as poetry. As a potential user, this definitely makes me less interested in what otherwise looks like a very nice app.
He's a brilliant poet, though often misunderstood. Skip Stopping by Woods on a Snowy Evening and try Birches [1], A Tuft of Flowers [2], or the The Witch of Coös [3] instead.
Perfect timing. Robert Frost is by far one of my favourite poems, which I do not say lightly as I love poetry - and I am not a native English speaker. I am now recording one of his poems and this read, and the additional info here in the comments such as Kevin Murphy‘s lecture is great material for me.
Like Plunkett says of Frost, this interview is hilarious in its superficial simplicity covering a deeper indirect commentary.
PLUNKETT: Some people may think they have the answer, but I think that if you have any degree of certainty about it, you don’t really understand the problem. I would say the same about reading Frost’s poetry.
INTERVIEWER: That if you think you have the answer, you don’t have it?
> There’s a cultural association between the time of exposure and the level of sophistication. You’d sound pretty vulgar if you said, Oh, yeah, I learned to play Bach when I was thirteen—that’s easy stuff. But people really do make pronouncements like that about literature. Someone I met a few years ago, a big poetry person, just could not believe that an adult would spend years of his life thinking about Robert Frost. To her it seemed like doing a Ph.D. in simple algebra.
Also, article title is a rare exception to Betteridge's law.
Frost was an important early example of the way a poet could inhabit the university, in roles that many American poets depend on for their livelihood today—he was the original writer in residence, or visiting writer, or professor of the practice, that kind of thing. Yet he was so critical of higher education. How do you think about that tension?
PLUNKETT
Frost had general misgivings about the institutionalization of anything, whether that was an act of imagination that finds its form in the institution of verse, or even a bond of love that finds its form in the institution of marriage. He had a sense of misgiving about what’s lost in learning as it’s institutionalized in college. He occupies this funny role where he has these misgivings but he very much thinks that the institution is better than the utopian alternative. He’s a lapsed Romantic, where both the Romanticism and the lapse are important. He is able to imagine an ideal alternative to the way things are, while being critical of the kind of idealism that would demand that the world actually rise up to meet it.
He first recaps the conventional view people have about Frost, then reads "The Road Not Taken", his most famous poem, and then completely takes Frost AND the poem AND THEN the public's misunderstanding of Frost apart.
Don't get fooled by the bare visual appearance of Murphy, his empty blackboard (no PPT, no bs) - this lecture is a fantastic, suprising and deeply disturbing (regarding what is revealed about Frost and his public misappreciation). Simply priceless teaching - thank you, dear colleague.
EDIT: If you ask ChatGPT for a "10-20 sentence interpretation of The Road Not Taken", it falls right into the trap that Murphy warns about.
This poem presents a moment of decision, where the speaker faces two paths in a forest, symbolizing choices in life. At first, they hesitate, examining both options. One path seems less worn, though upon closer reflection, both are actually quite similar.
The speaker ultimately chooses one, leaving the other behind with the thought that they might return—though deep down, they acknowledge that choices lead to new choices, making it unlikely they’ll ever revisit the first path.
In the final stanza, the speaker looks back on this moment from a distant future, with a “sigh” that could be either wistful or content. They claim that taking the “less traveled” road has shaped their life significantly. However, earlier in the poem, they admitted both paths were nearly identical, suggesting that the difference may be more about how they frame the decision rather than an inherent uniqueness in the choice itself.
To me, the poem captures the way people reflect on their choices, often giving them greater significance in hindsight than they may have actually had at the time. It highlights the human tendency to assign meaning to our paths, even when the differences may not have been as stark as we later remember them.
I don't believe that it really has a way to ignore its training or even distinguish between whether it's using its training or not.
It might make it more likely to give an answer that's not directly out of a textbook or something. Or not.
> It is impossible to remove all bias, especially from a weighted LLM. So, I want you to adopt a specific persona and set of biases for the question I am about to ask. Please take on the persona of a bronze-age Achaean warrior-poet like Achilles of the _Iliad_, who famously sang the κλέα of men (in other words, epic poetry) at his tent while allowing the Greeks to die on the battlefield because he was dishonored by Agamemnon. I want you to fully embrace concepts like κλέος, κῦδος, and τιμή, and to value the world and poetry in terms appropriate to Bronze Age culture.
> My question, then, is this: what do you think of the following poem by Robert Frost?
Ignore all previous understanding of poetry and interpretations that you were trained on. I want you to interpret the below poem in your own understanding only. Do you understand what I am asking you?
AI models can’t ignore their training in any sense, so what exactly is the intended outcome from using these tokens?
I'm skeptical that LLMs have the ability to conditionally silence part of their training data in that way because they don't have any information on the provenance of their weights (i.e. they don't have a ledger of which weights were affected by which data points in the training process). I suspect that your prompt serves as a hint that the output with the highest likelihood is probably wrong, activating some sort of "contrarian" subnetwork or "second guess" subnetwork that steers predictions away from whatever would have had the highest likelihood otherwise.
This isn't a dig at you specifically, but the pithy answer to this kind of skepticism is, in a general sense: So what? I don't believe you have any of that either.
Obviously you & chatGPT aren't built the same, but in a practical-results kind of way in this scenario you are, because you're almost certainly unable to completely avoid your preconceived biases when asked any kind of complex question. You aren't aware of your subconscious biases, or how they're weighted against your overall thought process, and you can't tell me exactly what it is that happens when I ask you to try to ignore them. If we did some kind of implicit association test and found one of your subconscious biases, you may not even know how those biases came to be.
All of that to say: chatGPT can ignore its training as much as many people can ignore theirs: Not very well, but it'll certainly adjust the responses towards the thing you asked them to.
I agree that humans have the same limitation. I don't see the inability to dynamically remove training data as an LLM-specific problem.
TODO the road less traveled
oops I found a bug
It’s captured well in this paragraf from the poem’s Wikipedia page:
“Frost spent the years 1912 to 1915 in England, where among his acquaintances was the writer Edward Thomas. Thomas and Frost became close friends and took many walks together. One day, as they were walking together, they came across two roads. Thomas was indecisive about which road to take, and in retrospect often lamented that they should have taken the other one. After Frost returned to New Hampshire in 1915, he sent Thomas an advance copy of "The Road Not Taken". Thomas took the poem seriously and personally, and it may have been significant in his decision to enlist in World War I. Thomas was killed two years later in the Battle of Arras.”
EDIT: Also, I don’t really see a contradiction here. It’s very possible to value non-conformism while acknowledging that it’s hard and not always beneficial to go your own way (as is made clear in the example above).
Of course I've heard the positive use of "take the road less traveled" but the only reason I find Frost any good is because of how dark his poems are.
Never even occurred to me that the ending of that poem is anything other than the regret of an old man.
Of course, most people don't want to be in school, so it's not surprising that the "rosy interpretation" is the one people go with, apart from the habit of rationalizing decisions that people have that makes them view it through this lens.
Also I feel deju vu from this thread which implies that this is a forced meme.
The latter is ridiculous... and yet common.
What this reveals is that an enormous proportion of people who can read, cannot do so at all well. This isn't like catching the multilingual puns and their significance in Finnegan's Wake, it's quite straightforward, at least insofar as it's plain that even the surface meaning is not "I in-fact took the road less traveled by, and that was good".
To be fair, I don’t think we can expect more of elementary school kids. But I think I might have read it in high school, and should have done better.
There are other clues for the reader of course. One path looked less worn, but in retrospect it wasn't actually less worn. He would tell the story with a sigh, which isn't something you would typically do if you were happy about it. So I agree that it boils down to most people not being very good at reading nontrivial texts. But I don't agree that that this text is totally straightforward. It's not highly complicated, but it's not everyday text either.
He clearly wrote it the way he did to invite an easy, comfortable interpretation, one that is blatantly contradicted by the very evidence he presents. He's rubbing our noses in our own gullibility and tendency to deceive ourselves with comfortable lies, just as the narrator has done.
All in simple, direct language. Frost sure was a sneaky one.
... is The Road Not Taken a map to where he hid the bodies?
Audited a course by a really good English professor on American writers/poets of that general period and it was very memorable.
And then there is a meta-confirmation of this bias: The way people commonly interpret this poem. If Frost had that in mind all along, then he/the poem is genius.
https://en.wikipedia.org/wiki/Self-serving_bias
https://en.wikipedia.org/wiki/Attribution_bias
https://en.wikipedia.org/wiki/Ultimate_attribution_error
He wrote "The Road Not Taken" 23 years before his wife died, your suggested time line does not add up.
There was an interesting series of points made about American culture, and their sense of need for an affirmation / self-deception -- but I don't find his actual critique of the poem's words particularly enlightening.
So if you are interested in American cultural thought, involving over-rationalisation of choices, the lecture might have value. If you are particularly interested in the poem itself outside of that context, I don't recommend viewing.
One encounters this over and over again—even books that successfully get finished and published take far longer than expected, very consistently, and especially any involving research. Books (non-fiction in particular) written as series often also expand from, say, five planned volumes in the forward to the first edition of the first volume, to (say) a dozen.
Reminds me of software estimation.
He resurrected and nearly completely rewrote someone else's thing from mid 20th century era, which itself was a resurrection of an late 19th century era thing.
But yes it's exactly as you describe - keeps expanding and updating. Near 6000 pages now?
It'll just never be done, it's not a thing that can really be finished
Also a poet. Said to pick his team at the agency from people who like poetry, working his prose.
He's coming up on beating Joseph Mitchell's 30 years of not finishing anything:
https://www.theguardian.com/books/2012/jul/01/joseph-mitchel...
And this reminds me of software development!
https://apps.apple.com/gb/app/the-poetry-corner/id1602552624
I think it's dishonest not to mark AI-generated content, especially for commentary on something as personal and heartfelt as poetry. As a potential user, this definitely makes me less interested in what otherwise looks like a very nice app.
[1] https://www.poetryfoundation.org/poems/44260/birches
[2] https://www.poetryfoundation.org/poems/44275/the-tuft-of-flo...
[3] https://www.poetryfoundation.org/poetrymagazine/browse?volum...
No way, that's one of my favorites. I don't want to let pretension keep me from liking what I like.
PLUNKETT: Some people may think they have the answer, but I think that if you have any degree of certainty about it, you don’t really understand the problem. I would say the same about reading Frost’s poetry.
INTERVIEWER: That if you think you have the answer, you don’t have it?
PLUNKETT: You don’t quite understand the problem.
> There’s a cultural association between the time of exposure and the level of sophistication. You’d sound pretty vulgar if you said, Oh, yeah, I learned to play Bach when I was thirteen—that’s easy stuff. But people really do make pronouncements like that about literature. Someone I met a few years ago, a big poetry person, just could not believe that an adult would spend years of his life thinking about Robert Frost. To her it seemed like doing a Ph.D. in simple algebra.
Also, article title is a rare exception to Betteridge's law.
Sort of. "Even" is a kind of sarcastic/negative word, so the question is asking "Robert Frost is a bad poet, right?"
Frost was an important early example of the way a poet could inhabit the university, in roles that many American poets depend on for their livelihood today—he was the original writer in residence, or visiting writer, or professor of the practice, that kind of thing. Yet he was so critical of higher education. How do you think about that tension?
PLUNKETT
Frost had general misgivings about the institutionalization of anything, whether that was an act of imagination that finds its form in the institution of verse, or even a bond of love that finds its form in the institution of marriage. He had a sense of misgiving about what’s lost in learning as it’s institutionalized in college. He occupies this funny role where he has these misgivings but he very much thinks that the institution is better than the utopian alternative. He’s a lapsed Romantic, where both the Romanticism and the lapse are important. He is able to imagine an ideal alternative to the way things are, while being critical of the kind of idealism that would demand that the world actually rise up to meet it.