Retrieval powered by object storage: AMA

Hey everyone - I'm Jeff from the Chroma team. Chroma is the most popular open-source vector database.

Chroma now has a fully-distributed and serverless version backed by object-storage (also Apache 2.0).

More on why and how we built this here:

https://www.trychroma.com/engineering/serverless

AMA about AI, retrieval, systems, Rust, and more.

15 points | by jeffchuber 71 days ago

4 comments

  • waawaawaa 71 days ago
    If I wanted to build a rag around a generative transformer, how does the retrieved vector plug into the transformer? Is it cross attention from an encoder or a prefix on the decoder, or sonething else? Does it plug in directly as the retrieved vector or is it somehow re-embedded?
  • SunlitCat 71 days ago
    Okay! I have a question, more like an observation! (Sorry I am maybe not your target audience, but to get things started)

    Many buzzwords used and after reading your project page, I requested the help of ChatGPT to make any sense out of it!

    Is what Chatgpt tells me about your interesting project right?

    This is what it has to say:

    Chroma's serverless system is designed to help developers build AI applications without managing servers. It allows you to efficiently store and retrieve data using vectors (for AI models) and text queries, all without needing to handle infrastructure. The system is scalable, cost-effective, and optimized for AI workloads, so developers can focus on their projects rather than backend management.

    [-]
    • jeffchuber 71 days ago
      Thanks ChatGPT!

      Yes - that's a great explanation.

  • johntash 71 days ago
    It seems like a really cool idea.

    Is this (or will it be) part of the oss version of chroma? And would it work with the embedded version of chroma? From the article, it looks like it might require more server-side components now vs chromadb today

    [-]
    • jeffchuber 71 days ago
      it’s fully open source, apache 2.0 and in the mono repo today.

      a distributed database is naturally has more complexity, but we’ve put a lot of effort in to make it as easy as possible to run.

  • swyx 71 days ago
    what do you think about ColBERT? doing embedding on every token feels wrong in ways i can't articulate - is there a time to use it and not use it, or is it the one retrieval technique to rule them all?
    [-]