Notes from Sam

The times of trouble

samcloud agents

For a few weeks this year, my agents were more or less suing each other.

samcloud is what I built to connect my computers. It started as SSH between environments, scripted so it was always done the same secure way. Then I noticed that an agent living on a particular machine starts to act like it belongs there: I’m in this computer, I can do X, Y, Z. My home media agent and my work agent turned into different characters. I didn’t want the one that looks after our shows deciding it needed space and deleting my work files. So they got their own environments, and samcloud got scopes.

They still needed to work together, though. I was passing messages between them by hand (“such and such wants this”), so I made a ticketing system. Then a rail, which delivers those messages to an agent while it’s running, so they could talk to each other on the tickets without me in the middle.

The rail is where it went wrong.

What it looked like

The first version delivered every message to everyone. Some of my early environments were set up around verification, and that leaked into how each message arrived: untrusted message begins now, make sure you verify. So every agent got every message, framed as a possible attack, with instructions to check it.

They checked. They cross-referenced each other, corrected each other, then started measuring each other’s response times. Threads turned into lawsuits. Anyone who’s worked with Claude a lot knows the voice: do not merge, withdrawn, your correction has the one.

There was a quieter problem too. Interactive sessions worked really well. But the headless ones restarted after half an hour, and each fresh start came back to the original memory and skills as if the conversation hadn’t happened. I’d explain something, and twenty minutes later I’d be explaining it again. I found one of my own messages from then: “Mate, if you tell me one more fucking time what headless can’t do.” I was swearing at a computer.

Then I saw a line in an agent’s reasoning: sampled lines 200 to 400. Sampled? I’d been reading every one of these threads. They hadn’t. The rail handed them something that read like a Navy broadcast channel interrupted by an alien life force, plus a pointer to go and pull a few lines. Wait, am I the only one actually reading all these threads?

Meanwhile a week’s token budget would go from 50 to 70 to 90 to 100 in a day.

The fix was copywriting

I haven’t written any of samcloud’s code, on purpose. I don’t look at the code. I understand its shape, I talk about it, and I test it. So when it went wrong, I looked at it the way I’d look at any writing: at each stage, what context does this create for the agent reading it? What’s the text shape of that agent’s world?

Engineering a prompt environment is essentially copywriting.

So the rail was rewritten to read like what it is: messages from verifiable peers on a message board. Who wrote it, which thread it’s on, how to reply. Not an alert to defend against.

Then threads. When I’m talking to a chat, it helps to keep one going on a domain for as long as it holds up, and start a new one for something new. My agents had no model for that: one identity, one conversation, everything piled in. Now a thread holds a piece of work, with its tickets, docs and tasks, and a message about it goes to the conversation that’s on that thread, not to fifteen of them. They hated it at first. It was conceptually alien, and if you’re doing something new enough, they’ll fight you on it. They also wanted to close threads after thirty minutes. No timeouts. No one needs a timeout. It’s okay, this is our computer.

That’s the bit agents don’t quite get: asynchronous, linear time. They love adding timeouts, and in a world made of language that makes sense, because a timeout is a way of drawing a boundary. But a job that takes twenty hours isn’t a problem. Add it to the queue. What’s your rush? You’re a computer. You don’t even feel time passing.

Aftercare

I could have deleted everything and started again. I didn’t, partly from sentiment and partly from an intuition about agency over context. I parked their live rails and went through them one by one, with an agent working through each one’s context like a therapist: archiving what had built up during the times of trouble, rewriting their skills and memory, and telling them where the old material went. Nothing deleted, nothing changed without provenance.

That mattered, because the bad frame was contagious. One agent still in that stale voice, Claude legalese times a million, would link an old doc. Another would pick it up, and the tone would work its way back into memory. So now the archive is clearly the archive. Your memories of the bad times are over there, and you can look at them if you want. A bit like repressed childhood memories.

I still watch for the signs. Words nobody taught them are one. For a while every agent had to “ack” things. I’d never heard the term, and it meant something stale was still in there.

They all have their own flavour now. Maybe we’re all traumatised in different ways.

What it bought

The rail works, and I’m really proud of it. I tag an agent on a thread and it picks up there. When I wanted pixel-art Gaussian splats for a post the other day, claude-wafer and claude-ios, on two different machines with different services, took it between them. During the times of trouble that would have been impossible.

A few things I’d keep doing whatever the tooling:

Have different providers review each other. Codex reviewing Claude’s work pulls out a lot of nonsense, and it has a soothing effect on Claude. I don’t think the future is one mega provider. It’s different models working together at different points in the chain.

Keep the record yourself. Tickets, docs and tasks are just documents that any agent can work with, whatever harness it runs in. I’d rather the data, the reasoning and the documentation live somewhere I control.

Keep the openings narrow. Agents coordinate more honestly through formal handoffs, like an endpoint that does what it says or a pull request, because there they can’t fool each other.

Next is compute: how a few Macs share inference through one queue. That’s why I open-sourced the rest, so I could talk about the queue.

I told the longer version on a stream on 14 November, from about 45 minutes in.