Sharing compute fairly
Looking back, the real reason I started samcloud was a problem I couldn’t name at the time: a zero-trust network that allows mutual, fair and legible sharing of compute. You could probably have started there. But I like that it drew itself by hand, and you can see the shape of it in the very first docs.
The near version is local models. Several agents on one machine share the same inference queue, and even the cache, without having to know about each other. A long job asks for a lease, waits its turn, and gives the machine back when it’s done.
Building workflows that suit smaller models takes time, and it’s sometimes double handling. It’s worth it, because then they can run all the time.
The far version is energy. We have solar and batteries at home. When there’s excess energy and spare compute, the work queue becomes a buyer of last resort.