Sub-1-Bit LLM Compression via Latent Factorization

3 comments

• big-chungus4 15 minutes ago
  Can this produce a useful model? So far 1 bit quants have been less useful than smaller models that use the same memory
  [-]
  • GaggiX 5 minutes ago
    I recently found this 1.58-bit model for ASR and it's surprising good (and very fast), that being said it's not a LLM.

    https://huggingface.co/moondream/parakeet-redux
• badatnames 19 minutes ago
  Their paper shows this comes with huge quality loss, but that doesn't make it a negative result by any means
• nico 17 minutes ago
  Has anyone tried this on apple silicon M1-5? Any benchmarks/comps?