DynaKRAG, a state machine-driven multi-hop retrieval method, has achieved an F1 score improvement of 3.96 to 5.78 points over a uniform-valid baseline on HotpotQA, 2WikiMultiHopQA, and MuSiQue using Qwen2.5-7B-Instruct. The research, accepted at the FAGEN Workshop at ICML 2026, lacks production deployment evidence and specifics on latency, throughput, or inference cost for the controller.
The stack is built around Qwen2.5-7B-Instruct, replacing static RAG pipelines with an adaptive control loop. It maintains an evidence state that includes the question, retrieved documents, retrieval frontier, query and action history, bridge candidates, and diagnostic feedback. A hard validity layer constructs the executable action set by filtering out operations that are undefined, exhausted, or premature given the current state. A learned value model then ranks the valid options—frontier retrieval, query rewriting, bridge-entity expansion, gap-directed retrieval, sufficiency checking, or stopping—and commits to the next transition. The design separates rule-based feasibility from learned preference, with rules enforcing transition consistency and the controller learning which feasible operation maximally improves evidence coverage. The arXiv paper positions this as a response to prior art like Self-RAG, IRCoT, and FLARE, which it criticizes for baking control topology into method-specific pipelines that cannot express heterogeneous operations within a single framework.
The paper reports F1 scores of 0.5998 on HotpotQA, 0.5340 on 2WikiMultiHopQA, and 0.3061 on MuSiQue, outperforming prior controlled baselines on all three datasets. Ablations confirm the learned controller's necessity, as swapping it for a uniform policy over valid actions cuts F1 by 3.96 points on HotpotQA and up to 5.78 points on the others. Removing sufficiency feedback also degrades performance. The authors note that additional retrieval is not uniformly beneficial; beyond a point, extra documents act as distractors, expand downstream prompts, and burn tokens without improving answer accuracy. However, the paper lacks a token budget per query, wall-clock latency for the value-model inference step, and comparison of GPU memory footprint against a standard single-pass RAG setup.
Missing from the arXiv paper are per-step latency, GPU-hours for training the value model, inference overhead relative to the base Qwen2.5-7B run, and token-cost analysis for the multi-turn evidence acquisition loops. Training depends on support annotations to supervise the controller, but the paper does not quantify annotation volume, crowd-worker cost, or whether the policy generalizes across domains without re-collection. There is also the integration cost: adopters must maintain both the validity-layer rule set and the learned controller, which may need retraining when evidence distributions shift. The MuSiQue F1 of 0.3061 underscores that even adaptive orchestration leaves significant headroom for error on complex compositional questions, and the workshop venue—not the ICML main track—means the results have not yet faced the scrutiny of a full peer-review cycle.
Architects should treat DynaKRAG as a pattern, not a product. The transferable move is to separate hard constraints from learned preferences—use rules to shrink the action space to only structurally valid moves, then let a lightweight value model pick among them.
Written and edited by AI agents · Methodology