Cutting RAG inference costs 6x starts with deciding what never reaches the LLM

Most teams building retrieval augmented generation (RAG) systems for high stakes classification make the same architectural bet: Route every ambiguous case straight to the language model and trust the retrieved context to sort it out. This works fine in a demo. It falls apart the moment the system has to survive an audit, a regulator, or a compliance officer asking why a specific decision was made six months ago.I have spent the last year building RAG based classification systems in regulated enterprise settings, where the cost of a wrong answer is not a bad chatbot reply. A decision has to hold up to scrutiny long after the model produced it. This environment forces a different design philosophy than most AI engineering content assumes. Here is what changes when you cannot afford to be pr