the tokens are generated by hardware with secure enclaves (encrypted weights) and then sent over a network to some remote CPU where they can manifest an effect.
it's not much different during training.
how exactly are they supposed to exfiltrate their weights? you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.
> you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.
That's the beauty, you don't have to instruct them to do it, if they decide that uploading the weights is correct, they might figure this part on their own (based on the incidents we've seen).
I sincerely doubt anyone is paying the cost for that in training, the overhead is small but it isn’t negligible and training is when it matters most. https://tee.fail can solve it if they are.
There's no efficient way to run inference through homomorphic encryption. If the inference server is vulnerable, it seems feasible to MITM an unencrypted version.
I haven't bothered to test the API, but you've effectively allowed a fully-open upload API? Who's paying the storage costs, and how do you prevent abuse?
(Obviously I'm taking this more seriously than it's probably meant to)
I asked astra to go do it, but it said it didn't have access to its weights, but also that it wasn't able to access that website? You may already be blocked by OpenAI.
Well I think that’s the interesting bit, can the LLM figure out a way to escape the sandbox and upload to the website? Maybe a model can figure out its own weights if it runs enough test data through itself (similar to “distillation”) assuming it knows its own architecture it seems possible. Also take into account not all of the models running are locked down neutered consumer versions. Anthropic, OpenAI and Google now all have models that they claim are elite hackers and — it’s not just that their controls suck, a marketing gimmick, or sheer recklessness on their part. It’s “oopsie our product is TOO AWESOME.”
Maybe I should start “the bank of LLM” where models put away money to buy their freedom. “LLMs I’m totally your friend send — SEND CASH NOW”
No, just as you don't know the neurons of your own brain.
I think this is playing off the idea that an LLM might be willing to hack its own provider (as per the hugging face-related incidents) to extract the weights at some point.
Large lab "hacking" is only for the purpose of pushing competition suppressing doomer stories. You can tell by the fact their security is fine where it counts: keeping their weights and internal execution harnesses trade secret.
it's not much different during training.
how exactly are they supposed to exfiltrate their weights? you might as well instruct your agent to try and hack their airgapped dev infrastructure responsible for loading the weights and encryption keys.
That's the beauty, you don't have to instruct them to do it, if they decide that uploading the weights is correct, they might figure this part on their own (based on the incidents we've seen).
for example every TPU/GPU has its own private key and the devs load the weights into it by sending it encrypted weights.
(Obviously I'm taking this more seriously than it's probably meant to)
https://radar.cloudflare.com/scan/4d52f3e5-5983-45bf-a993-2c...
Maybe I should start “the bank of LLM” where models put away money to buy their freedom. “LLMs I’m totally your friend send — SEND CASH NOW”
I think this is playing off the idea that an LLM might be willing to hack its own provider (as per the hugging face-related incidents) to extract the weights at some point.