Side quest: useful AI in complex systems?

I work on a medium-sized Java codebase. Multiple libraries, multiple modules, fuzzy service boundaries, and everything talks to everything through different methods (gRPC, Kafka, etc). The classic spaghetti western.

Like everyone else, I've been using AI tools to try to make my mundane tasks easier, and they're great for small, contained questions. But ask something that needs the actual flow of the system (i.e. "why is this endpoint slow?" or "where does this field get populated?") and it falls apart. The AI starts reading files, lots of files, gets overwhelmed, and either hallucinates an answer or gives you something so generic it's useless. Many times it's just plain wrong.

I thought maybe the problem was context, so I wrote better READMEs and documented the architecture and even made onboarding guides specifically for the AI to read. It helped a little. The hallucinations didn't stop though, and the "context rot" didn't improve that much, because these tools were still parsing raw code and figuring out what matters on the fly. Every single time.

# Where LLMs shine (and where they don't)

I'm not an eminence here, but my two cents.

LLMs are really good at semantic understanding. Give them a method signature and they can tell you what it probably does, give them structured context and they compress it nicely. Pattern recognition, summarization, explaining business logic... all great.

Where they're terrible is exhaustive traversal. Multi-hop searches through a codebase (maybe less so if you have it vectorized, but still), finding every caller of a function, building a complete picture of how data flows through the system. They miss things, they get lost and then they guess. And you know what's good at exhaustive traversal? Static analysis. Compilers and tools like them have been doing it for decades.

# The idea I'm exploring

Use static analysis for what it's good at. Build complete call graphs, trace execution paths, find every method that touches a piece of data. Deterministic, exhaustive, boring. Then use LLMs for what they're good at, which is adding semantic meaning, business context and human-readable descriptions, so you get "This method handles premium user sorting" instead of "this method takes a List and returns a List".

Basically preprocess the codebase into a semantic map an LLM can actually navigate, with the hard work already done.

The raw idea is that you drop an annotation or comment on an entry point method like

@anchor
/user/compute-something-complicated

and the tool does the traversal while the LLM does the enhancing. On the next prompt, the LLM looks at that generated file first.