IQuest Research Details Sparse MoE Architecture in New Technical Report

IQuest Research released its technical report on October 23, 2026, detailing the architecture and training methodology behind IQuest-Q1, an agentic foundation model designed for command-line interface interaction and complex software engineering tasks.

The model employs a sparse Mixture-of-Experts architecture containing 320 billion total parameters, with 15 billion parameters activated per token to balance computational intensity with inference throughput. This design choice aims to optimize the model for multi-step reasoning and tool-use workflows within technical environments, allowing for high-capacity reasoning without the overhead of dense parameter activation.

The training pipeline spans approximately 29 trillion tokens across pre-training and mid-training phases, with a context window that scales to 512K. The research team implemented Stable SplitNorm, the Muon optimizer, and μP parameterization to stabilize training and improve convergence rates across these massive datasets.

Stable SplitNorm serves as a critical normalization technique that prevents gradient instability during the training of large-scale MoE models. By decoupling the normalization process, the team ensured that the model maintained consistent performance even as the context length reached the 512K threshold.

The Muon optimizer was integrated to accelerate the optimization of the model weights, providing a more efficient path to convergence than traditional stochastic gradient descent methods. This optimizer, combined with μP parameterization, allowed the researchers to scale the model size predictably while maintaining the integrity of the underlying parameter distributions.

Efficiency metrics reported by the developers indicate that IQuest-Q1 achieves 1.93× scaling efficiency compared to a Qwen3-30B-A3B-style MoE baseline. Under controlled conditions, the model required only 52% of the compute resources of the baseline to reach equivalent training loss, demonstrating the effectiveness of the chosen architectural optimizations.

The post-training phase integrates supervised fine-tuning with domain-specific reinforcement learning, Multi-Teacher On-Policy Distillation, and model merging techniques. These processes incorporate data from diverse agent harnesses, including Claude Code and Codex, to refine performance across various execution environments and ensure the model can handle complex, multi-step software engineering tasks.

The report also introduces Recursive Multi-Token Prediction, which repurposes an MTP layer as a drafter for speculative decoding. This approach extends the drafting horizon without necessitating an increase in the total parameter budget, effectively increasing the tokens-per-second throughput during inference.

Evaluation results across 30 benchmarks demonstrate the model’s capabilities in software engineering and terminal operation, including scores of 52.7 on AutomationBench and 78.4 on MCP-Atlas. The team also utilized internal assessments, such as IQuest-CLIBench, to measure sustained interaction through user simulation and execution traces, providing a more granular view of how the model performs in real-world terminal environments.

The use of the τ³-Banking benchmark provides insight into the model’s ability to navigate complex, state-dependent financial workflows. By scoring 51.8 on this metric, IQuest-Q1 proves its utility in professional environments where precision and adherence to specific tool-use protocols are paramount for successful task completion.

The SkillsBench score of 74.1 highlights the model’s proficiency in executing technical tasks that require a combination of reasoning and coding ability. This benchmark specifically tests the model’s capacity to handle multi-step instructions, which is a core requirement for the agentic CLI systems the researchers aim to support.

To ensure the integrity of the training data, the researchers constructed over one million containerized task instances. These environments utilize task-specific verifiers to ground learning in actual execution outcomes, effectively filtering out reward hacking and unauthorized access to test data.

The project highlights the potential for model-assisted research, as demonstrated by an IQuest-Q1-based agent that analyzed 6,054 tool calls to resolve 35 errors in training data. This automated auditing process identified inconsistencies and supported the implementation of a data-cleaning plan under human supervision, showcasing the model’s ability to contribute to its own development.

The technical documentation suggests that the methodologies developed for IQuest-Q1, particularly the focus on environment-state checks and trajectory auditing, will serve as a foundation for future iterative development. These assets are intended to provide a framework for training models that are more resilient to common pitfalls in agentic learning, such as reward hacking and data leakage.

Future research efforts will likely focus on the scalability of the Critical Model Size concept introduced in the report. Observers should monitor how these architectural constraints influence the development of subsequent models as the team continues to refine its approach to agentic foundation capabilities.