Cerebras Disaggregated Inference

(cerebras.ai)

63 points | by emschwartz 5 hours ago

4 comments

  • aabdi 4 hours ago
    5x throughput more than b4, same 10x tokens/s as compared to standard.

    looks like disagg/wide ep is applicable to cerebras chips as well. kind of amazing.

    https://www.cerebras.ai/blog/how-we-deploy-trillion-paramete...

    • oivaksef 3 hours ago
      They talk about Kimi - I would love to run that on cerebras, but it's not available from them directly. Is there a provider I can use this at?
  • speedping 1 hour ago
    Cross-platform prefill/decode split is an incredible feat, kudos to them. I wonder how much time is spent transporting kv cache, and if the gains would be even higher for LLMs with ultra-compressed kv cache like the latest deepseek.
  • hephaes7us 2 hours ago
    It sounds like this might mean they could do real prefix caching?

    Cerebras would be much more interesting to me if cache hits resulted in a commensurate savings on my end.

  • augment_me 5 hours ago
    Have they solved the issue yet where they need to keep running ghost tokens to keep the board hot to prevent it from cracking, which causes massive energy draw in idle, and reduces lifetime by 3 times the estimate?

    https://culperresearch.com/wp-content/uploads/2026/10/Culper...

    • eh8 4 hours ago
      The short seller report contains a bunch of errors:

      1. The CS-3 operates on average at <65c, well below its thermal limits. Considering the actual thermal parameters of the wafer, using the short seller's own temperature equation would imply the wafer can operate at 85c for 33 years

      2. "Ghost tokens" are not unique to Cerebras and the 3x lifespan reduction claim is based on extrapolating GPU figures which is a big logical leap

      3. "since the introduction of the CS-3 in 2024, less than half of one percent of all CS-3 wafers deployed have been decommissioned."

      https://cerebras.ai/blog/what-it-took-to-build-wafer-scale-a...

      Disclaimer: I work at Cerebras

      • augment_me 4 hours ago
        Very interesting! Thanks for answering with your take, and the link.

        Questions:

        1) What is the idle power draw of the chip? Is the 80% number wrong?

        Nvidia is idling at around 180W/15% of their max for B300. So saying that "it's not unique to Cerebras" is meaningless if we don't consider the power draw itself

        2) Decommissioned is not the same as RMA:d/refurbished/redeployed, so giving this number is kind of dodging the question IMO

        • wmf 4 hours ago
          None of these expensive chips should be idling.
          • augment_me 4 hours ago
            What a non-answer. And there should be perfect latency hiding, and there should always be predictable power-of-2 workloads, and all weights should always be homogenous, etc.
            • adgjlsfhk1 3 hours ago
              It's a reasonable answer. Consumer chips are idle 95% of the time, so idle power draw matters. AI datacenters are currently under capacity, so rather than optimizing for idle power draw, just have work to do.
              • augment_me 3 hours ago
                No it's not, it does not reflect reality. I have worked with both industry and public clusters. The average is around 50% idle-time, many university clusters sit at around 80% idle-time.

                The absolute best I have seen at a hyperscalar internal with predictable, constant workloads due to own projects, and cracked routing infra is 15% idle-time.

      • latchkey 3 hours ago
        As far as I understand it, data can also be retrieved from the ghost tokens, so they are valuable information too. This offsets the cost of them.
      • lukeify 4 hours ago
        So which Cerebras statement is in error then?

        > First, in Cerebras’s preliminary IPO prospectuses filed through December 2025, the Company assigned its computer equipment useful lives of just 1.5 to 3 years – in-line with both our Arrhenius modeling and what many of the Company’s hardware customers have described to us.4 Just four months later, in the Company’s April 2026 S-1, Cerebras nearly doubled that range to 3 to 5 years – a change that significantly inflated reported margins. The Company appears to have never explained the reason for this change.

        Was the useful life of a wafer 1.5–3 years on your IPO filing or 3–5 years in your S-1?

        Hard to believe a company with such skilled engineers couldn't figure out the correct useful wafer life given existing industry data. Or (more likely) are we just looking at standard corporate BS?

        • augment_me 35 minutes ago
          Both are wrong.

          On the physics of silicon lifetime, Cerebras is right and Culper's calculation is wrong. As Cerebras blog points out, Culper took a 5-7 year GPU economic depreciation schedule and divided it by a 3x thermal power-draw ratio to claim the wafers physically die in ~2 years.

          However in the blog, Feldman writes:

          > By the next update of our S-1, we had set out to build a private cloud; the reason was that AI native customers and developers asked us to. One consequence was that the mix of fixed assets we used internally had changed to include production systems we owned for the cloud business. We therefore modified our depreciation schedule to 3-5 years.

          However the timeline is:

          Mar 2024: CS-3 launches.

          Jun 2024: First confidential IPO filing (DRS) sets equipment useful life at 1.5–3 years.

          Aug 2024: Cerebras Cloud launches (followed by $383M in 2025 cloud CapEx).

          Dec 22, 2025: Updated DRS still certifies a 1.5–3 year useful life, 16 months after launching the cloud, contradicting the statement in the blog.

          Late Dec 2025: Signs the 750 MW OpenAI cloud deal (3.7-year average contract).

          Apr 17, 2026: Public S-1 (weeks before the May 2026 IPO) doubles useful life to 3–5 years, boosting 1H 2026 gross margins from 24.0% to 29.9%.

          ---

          So the blog is not having the right causal relationship here, if launching the developer cloud in August 2024 was the reason for extending useful life to 3–5 years, that change would have been in the December 2025 filing.

          Real reasons probably touch upon their deal with OpenAI, which had to be taken on OpenAI's terms because the IPO would not have happened with a 86% UAE-based G42 (which stalled the 2024 IPO over China ties). Cerebras needed a big US anchor customer to go public, and OpenAI opted to buy cloud capacity rather than buy and run the hardware itself(you can guess why). Once you have to capitalize massive owned cloud infra for a 3.7-year contract right before an IPO, stretching hardware depreciation from 1.5–3 years to 3–5 years is kind of free because all other cloud providers use a 3–6 year schedule, so it's easy to get an auditor to sign off on it.

        • wmf 4 hours ago
          That's answered in the blog.