Why your web service doesn't need Rust (probably)

I dislike Java.

The verbosity, the AbstractFactoryPatternManagerImpl nonsense. Every time I write Java I think it would be less verbose to write another blog post.

Zig and Rust are right there, looking at me, constantly. Manual memory management, zero-cost abstractions, no garbage collection pauses, I could write code that runs circles around the JVM. So naturally, while working on a Java microservice, I started thinking about how I could justify bringing a systems language like those two into this ecosystem.

Spoiler: I could not. But why I could not is a pretty good example of real world constraints beating theoretical performance.

# The sexy proposition

Systems languages promise bare-metal performance. Talk to the kernel, control every byte, way past what a well-tuned JVM or a "good enough for CRUD" service gives you. And when you're maintaining a service that feels slow and you're like me (you don't shy away from learning something new as long as it makes sense), that's intoxicating. You start manufacturing opportunities everywhere:

"This XML parser could be 3x faster in Zig!"
"These protobuf codecs are doing unnecessary allocations!"
"We could rewrite the database layer and eliminate JNI overhead!"

Sadly, it isn't that simple unless you're reckless. You need to do the fucking math first, and that's before the hard sell of adopting a new technology you'll have to maintain. New pipelines, new monitoring, a lot of new things.

# The case study: a Java microservice

This is a real service. A distributed user data service on a Java/JVM stack, gRPC with some REST API sprinkles, database operations, message queues. It handles tens of thousands of requests per second. It's I/O-bound (network, database, external APIs), the latency targets are in the tens to hundreds of milliseconds, and it already uses async patterns.

I went looking for places to drop in some Rust or Zig and found three candidates:

The XML/TTML parser. It parses files, and native code could make it 2-3x faster, which is about 5-10ms in absolute terms. It isn't on the critical path anyway.
The protocol buffer codecs. 400+ lines of serialization logic, already on binary protobuf (efficient). Potential gains of under 5ms, against 10-100ms of network latency.
The database access layer. This is where it gets interesting.

# The sad truth

The service uses a distributed database that's already written in C++. Every database operation goes through this flow:

Java Service Layer
      ↓ (JNI - ~100-200ns overhead)
JVM Bindings (Java wrapper)
      ↓
Native C Library
      ↓ (Network - ~1-5ms latency)
Database Server (C++)

You know what I thought? "If we wrote the data access layer in Rust, we could talk directly to that C library! We'd eliminate the JNI overhead and I for sure can make it safer" (the latter one is of course delusional).

So I did the math. Single-key lookups are the most common operation, and they take 1-5ms. JNI adds 100-200ns to that, so 0.01-0.02% of the total. Range queries (10-100 items) take 5-20ms, same 100-200ns of JNI, which is 0.001-0.004%.

Even if I made JNI infinitely fast I'd improve overall latency by less than 2%, because the network latency to the database (1-5ms) is 5,000 to 50,000 times larger than the JNI overhead.

# Two orders of magnitude or don't mention it

My new rule: only optimize a component if it's within two orders of magnitude of your dominant cost.

JNI overhead: 100-200ns
Network latency: 1-5ms (5,000-50,000ns)
Difference: 3-4 orders of magnitude

So JNI is not worth touching, and the same goes for everything else. Why would I ever start by optimizing anything other than the bottleneck...

# When yes?

Look, I'm not saying systems languages are useless for web services. They make sense when you have CPU-intensive operations at massive scale: millions of ops/sec (this service does thousands), computational kernels called constantly (crypto, compression, encoding), data processing pipelines without I/O waits. Or memory-sensitive workloads, where GC pauses directly hurt user experience, you need sub-millisecond latency (<1ms p99) and every microsecond matters.

You might be able to build a case if you already have a custom proxy server, a cache layer, an in-memory analytics engine or stuff like video encoding, I guess. None of those are waiting on I/O. A typical CRUD service hitting databases and external APIs is I/O-bound, and there the language doesn't matter.

# The sadder truth

99% of developers (including me) don't have problems that systems languages solve.

Your web service is slow because your database queries suck, or because you're not caching properly, or because you're underusing async. If I had a penny for every time I found multiple awaits in a row instead of one await-all, I would have like 2 dollars more in my bank account. It's layer 8, or it's bound to slow I/O. Java's garbage collection has nothing to do with it.

# Imagine you are in that 1% where you need it

Okay, say all the planets have aligned. You've profiled, you've done the math, a systems language makes sense, you have CPU-bound operations at massive scale and the 10x improvement is real.

You still need to put in the work to make it fast and not shoot yourself in the foot. Rust won't save you from terrible algorithms, Zig won't fix your O(n²) nested loops, and both will happily let you write slow code if you're not careful (btw, if that sounds scary, your current code could use a revision).

Then there's the organizational battle. You need to fight for adopting something entirely new in a larger organization, and onboarding a new language is never as easy as installing the compiler and creating a CI/CD pipeline. Split language expertise, split codebase, hiring gets harder, and either everyone context switches (annoying) or you end up with knowledge silos in the team.

You may not struggle with platform-specific binaries, but what about libraries? Does the obscure SaaS your company forces you to use for performance metrics have a good library for your new Rust server? Or will you have to build it yourself to get the minimum observability in prod you need to even measure whether all this was worth it?

I'd still probably say "only introduce platform complexity if it gives you a 10x improvement", or something like that. For this I/O-bound service Rust might give 1.1-1.2x on the compute portions, and compute is under 10% of request time. So under 2% overall, and that's not worth doubling operational complexity.

# My reflection

I wanted to use Zig and Rust because they're cool, because systems programming feels like "real" engineering, and because I was bored of Java. Those are preferences.

Do the math first. Profile before you rewrite, ask what's slow before you ask what could be faster. The network and the database take the same time whether you call them from Java or Rust, and external APIs respond just as slowly to your beautiful systems programming code.

Choose boring, proven technology that matches your problem, and save the systems programming energy for a toy project. You'll enjoy it more and it won't be tainted by your job requirements.