buenomini

Rebuilding P2P secret sharing

Rebuilding a P2P, end-to-end encrypted secret sharing CLI from scratch.

8 min readrebuildhush, hush-clip, trueseal

I just spent last weekend building hush, a peer-to-peer, end-to-end encrypted secret sharing tool. Magic Wormhole already exists. Croc exists. So what gives?

I wanted to learn, and I also wanted a CLI we could actually use at my current job to share secrets across offices (we have a split-tunnel VPN).

# What hush does

Very basic stuff:

  • Share secrets: send text to a peer with a code like "7-Ld89GsHO-gGHps31"
  • Transfer files: send files up to 10MB, end-to-end encrypted
  • Encrypted chat: real-time P2P chat with no server in the middle

All of it works across local networks and VPNs, with zero trust in any relay server (because there isn't one). Didn't want to deal with the corporate process to spin up a STUN server.

# Why build what already exists?

Early in the project I talked about learning strategies with a coworker, focused learning vs exploratory learning. For this one I wanted to go broad and cover areas where I had absolutely no idea how things actually worked. Of course I knew how to use a lot of it, but knowing how to use something and knowing how it works are very different things, and networking, cryptography and security were always a gray box for me.

The best way for me to learn that is to build something I'd use myself, in this case to share secrets with coworkers across our VPN.

# The basic networking knowledge

I knew how to use TCP, UDP and P2P. I didn't understand them until I tried to build with them.

UDP for discovery, TCP for data is pretty straightforward but worth a quick mention. To find peers on a network you broadcast a UDP message to everyone who's listening. It's like shouting "YOOO, anyone there?" into a room, and UDP doesn't care if anyone gets the message. The good old fire and forget. Once you've found someone you could keep talking over UDP, but you probably want TCP, which is reliable, ordered, and the other side confirms it got your message with an ACK.

TCP being a byte stream is the one I didn't know... I thought I could just write() some data on one end and read() it on the other. Nope. TCP gives you a continuous stream of bytes. One read() might give you half of what you sent, or data from two different write() calls smooshed together.

That's why message framing exists. You prefix every message with its size (4 bytes for a uint32), then read exactly that many bytes. Simple, but not obvious until you hit the problem.

Then the VPN. Every VPN can be configured differently, and ours is split-tunnel (only some of the traffic gets routed through it). To figure out what was allowed, I just did a few pings to the internal IP of a coworker in a different office. Which worked! Yay! TCP was viable and UDP was not, meaning I can't auto-discover anyone outside my local network. There are workarounds, like a STUN/TURN server, but I didn't want a server. So I settled for an --ip flag that skips discovery and connects directly over TCP to an internal VPN IP. Not as elegant as automatic discovery, but it works. The IP can go over the same out-of-band channel as the code, and the CLI copies it to the clipboard for convenience.

# The cryptography learning

I'm not a cryptographer. I knew I shouldn't (and couldn't) roll my own crypto, but I wanted to understand it a bit better.

PAKE (password-authenticated key exchange) was the biggest lesson of the project, and it's not common knowledge like TCP/UDP. I didn't know how two computers could agree on a shared encryption key over an insecure network. You could use a normal shared password, but that didn't feel safe at all. Encrypting with the password directly is too weak, an attacker who intercepts the encrypted data can brute-force millions of passwords offline until one works.

PAKE protocols (like SPAKE2, which hush uses) solve this:

  1. Both parties start with the weak password (the hush code, like "7-Ld89GsHO-gGHps31")
  2. The password is shared out-of-band, meaning through a different channel than the encrypted data (Slack, Signal, in person, etc.)
  3. They exchange messages over the network (an attacker seeing them is no problem!)
  4. Through some clever math, both sides independently derive the same strong encryption key
  5. The password is never sent over the network
  6. An attacker can't brute-force offline. They'd have to try each guess against the live protocol

So an attacker would need to compromise both the channel where you share the code (like Slack) and the network where the encrypted data flows. And the code is one-time use, so even if both channels are compromised the attacker also has to be quick enough to intercept it.

Here's the flow in hush (thanks AI for the nice mermaid art):

sequenceDiagram
    participant Sender
    participant Network
    participant Receiver

    Note over Sender,Receiver: Both know password "7-Ld89GsHO-gGHps31"

    Sender->>Sender: Generate PAKE initial message
    Sender->>Network: Send initial message
    Network->>Receiver: Forward message

    Receiver->>Receiver: Process message with Update()
    Receiver->>Receiver: Generate response with Bytes()
    Receiver->>Network: Send response
    Network->>Sender: Forward response

    Sender->>Sender: Process response with Update()

    Note over Sender,Receiver: Both now have same strong key!

    Sender->>Network: Send encrypted secret
    Network->>Receiver: Forward encrypted secret
    Receiver->>Receiver: Decrypt with derived key

Even if someone intercepts every message on the network, they can't derive the key without the original password, and they can't brute-force it offline either. They'd have to take part in the protocol for every single guess, which makes it annoying.

I used the schollz/pake library (a SPAKE2 implementation) instead of implementing it myself. Because of course...

# Security from the start

Building with security in mind from the start was very interesting, and definitely not something I'm used to.

One of hush's features is sending files up to 10MB. When I first implemented receiving, I saved files with their original filename. Very reasonable for my non security oriented brain.

Well, no... An attacker could send a file named ../../.ssh/authorized_keys and overwrite critical system files. That's called a path traversal attack. And even if you sanitize the path, the filename itself can be dangerous, it can overwrite a file in your current directory if you run hush from a sensitive folder. I didn't want an annoying interactive CLI where you inspect the name and path, so I just ignore the original filename entirely. hush generates a safe name from a timestamp and the file extension, something like hush_1699747123.pdf, and renaming it is up to the user.

I'm also not a trusting person, so I wouldn't trust a random tool with my secrets. That's why there are no relay servers and no logs, and the keys are ephemeral and disappear after the transfer. Even if someone compromises the server (that doesn't exist), they can't get to your secrets. I checked every design decision against this. You don't need to trust anything because there's nothing to trust.

# How quickly Go clicked

This was my first "real" Go project, and I was shocked at how fast I got proficient. After about half a day of reading docs and writing code I was shipping features. Go's simplicity is damn awesome.

I do wonder if that simplicity runs into trouble once genuinely hard problems need solving tho. One can argue those don't exist, and that if you're a good engineer anything can be made simple. Anyone who argues that has never hit a genuinely hard problem.

Explicit error handling is great. Maybe not Rust great, and a bit verbose, but great. Combined with assertions it makes a good safety net.

# Takeaway

I didn't need to build this. Magic Wormhole exists and is probably better, and Croc is literally from the same person as the PAKE library I used, also in Go, with more and better features than hush.

But I learned way more building hush than I would have reading docs or watching videos (or chatting with everyone's beloved AI). You don't need a new idea to build something. I'd even argue the opposite, that the best projects are often the ones that help you understand old solutions, and that's going to be my next post!