Membership without a server
A group of devices with nobody in charge of who's in it, and the three ways that went wrong.
In trueseal-sync a group is a bunch of devices sharing a relay. The relay stores and forwards. It can't read anything and it has no say in who's in the group.
Membership lives in a signed Group Manifest, which is a version number, the list of members, who issued it and a signature. Any member can add or remove someone by issuing a new one with a higher version, and the highest version wins.
The first version had three problems:
- Equal versions forked. If two devices issued version 5 at the same time, each device kept whichever one it saw first and rejected the other as a "regression". Nothing linked a manifest to the one before it, so one of the two edits just vanished (removals too) and nobody got told.
- Removed devices could still rewrite membership. The only check was "is the issuer in my current manifest", so a device that hadn't seen the removal yet would take a high-version manifest from the removed device and then reject the real removal.
- Anyone could freeze the group forever. Any member could issue version
u64::MAX, and thenversion + 1overflows (panic in debug, wrap in release).
# What I picked
Any member can change membership (no admins), every current member is fully trusted for it, and the answer to a hostile device is to destroy the group. Max group size is 32.
Then the harder one, two devices editing at the same time. Every manifest now carries parent, the SHA-256 hash of the manifest it was built on, covered by the signature. Devices order manifests by (version, hash). On a tie, the lower hash wins. Any device that sees both manifests picks the same one, so there's no fork anymore.
That alone still loses the other edit, though. So the issuer keeps its change as a Pending Membership Change, and if it accepts a concurrent manifest that doesn't include it, it re-applies the change on top as version + 1. The retry has to be the issuer's job, because it's the only device that knows its change lost. The parent link is what tells "someone undid my change on purpose" (built on a manifest that had it) apart from "we raced" (built on something older).
The promise: once every member has received the same manifests, they all hold the same one, and no honest member's change is lost unless that member got removed at the same time. The app does nothing.
I did consider telling the app "your change was lost, redo it". Simpler for me, but then every app has to get it right, and my rule for this library is that the app shouldn't need to do anything to be correct. Merging add/remove operations on every receiver would also work, but that's a manifest redesign and way too much for a preview.
Member events can flicker though. A device can see E join, leave and join again while things settle. It's documented, and apps should just render the current list.
For the freeze, u64::MAX is rejected on receipt, issuing uses checked arithmetic, and any jump over 1024 versions fails with VersionJumpTooLarge. Honest issuers always step by one. A device that misses more than 1024 membership changes has to re-pair, and I'm fine with that.
# The zombie
A removed device wipes itself only if the manifest that excludes it was built on one it holds (or has held) that included it. Otherwise it's a "false removal", like a new joiner whose admission lost a tie, and it should stay alive and wait.
The "has held" part had a bug. When a device accepted a manifest and immediately re-applied a pending change on top, only the final manifest's hash went into the held history. The one in the middle never got recorded, so a removal built on it looked like a false removal. No wipe, no removed event, and the device kept going with a stale view of a group it wasn't in anymore (a zombie). The fix was to record both hashes.
One hole stays open on purpose. A removed device that refuses to cooperate can still mess with devices that haven't seen its removal. Hardening that felt pointless when a hostile current member can already remove everyone, so it's written down as the trust boundary.