llms as shards

Only cowards wait for the Singularity to finish

Sometimes people propose procedures for making decisions based on secret information that involves

1. Copying Alice, creating “shard Alice”

2. Revealing the secret to shard Alice

3. Permitting shard Alice to send a limited amount of information to Alice

4. Deleting shard Alice

There are a few examples of this general theme. An early example I¹¹ Well, okay, Fable 5.1 did all the work. was able to find was Hanson’s Bits of Secrets, which basically describes the exact procedure I laid out above, including addressing some complications (i.e. maybe you can also include a judge). I swear I remember at one point reading a blog post somewhere that proposed that, in order to avoid a potential war between two powers each of which was unwilling to reveal their hand, it would be worthwhile for each power to send a disposal negotiator, have them reveal the information to each other, come to a decision, and then kill them so they can’t leak information the other side wants secret to their side — but I wasn’t able to track down the cite.

One could do pretty much this general idea with a memory wipe, instead of cloning and deletion. Memory loss gambits are pretty common in fictional media, so you’d think there might be some examples of the general idea, but I haven’t been able to come up with any that use a memory loss gambit in quite this way. There’s a few cases of people being hired for jobs on the condition that they’d have their memories wiped afterwards, which I guess is sort of close, but it’s not really quite the same thing. A correspondent of the blog has purported to me that their roommate sometimes takes an amnestic for various reasons — I think something like “someone wants to ask them a question without revealing that they’d want to ask it” might have been the example we discussed, I dunno, you can really do a lot of variants on the theme probably.

Project Lawful involves a scene where Iomedae sends a small portion of Herself to go ask Cayden Cailean what the hell is going on, and receives back only two bits of information.

This came up a few days ago in mercury’s Stop Asking and Maximize Utility, which stated:

Obviously we do not have a Starstone, but personality cloning onto LLMs might be a mature technology within the next couple decades, and that’s basically just as good.

I do not think we need a decade of refining personality cloning onto LLMs to make this work! It’s already possible to finetune LLMs on your written output, although probably you need to write quite a lot and also have quite a lot of compute to get anywhere good with this. But also, your representative does not actually necessarily need to have your personality! At most they might need to be able to model you pretty well. But probably AI agents with the ability to inspect all the context about my life that’s on my laptop can get a pretty decent idea of what sort of person I am, especially if I were actually trying to give them as much information as I can that’s relevant to making a particular decision on my behalf.

So clearly the issue here isn’t that the process is computationally intractable. The issue is that there are appreciable fixed costs. Setting up a whole bespoke LLM-based system for any given run-of-the-mill task where you might want someone to be able to make a decision informed by a particular piece of information without forever damning them to have to carry a burdensome secret is obviously not worthwhile. But like, I think it’s totally possible to set up something worth using. You just gotta design the right interface. Someone go make the funny website or something.