Training a 3.8B LLM to 0.384 CORE for $998 – Hugo Vergnes

(hugovergnes.github.io)

30 points | by Anon84 2 hours ago

2 comments

  • johnnylambada 40 minutes ago
    I don’t blame you for using an LLM to write an article about an LLM that you built. I’ve been reading so much LLM output that now I see it everywhere. I wonder if humans will start writing more like LLMs?
    • brainless 3 minutes ago
      When we hyper focus on finding something, we find it all the time.

      The world of tiny LLMs is so interesting. It is unlocking novel ways to encode information. Why focus on the style of writing instead of the subject matter?

  • vatsachak 1 hour ago
    Very cool. I wonder what's the minimum amount of money you'd need to get it to score well on AIME