Get it here: https://hugston.com/models/hugston-qwen-agentworldq4-k-m

Qwen-AgentWorld: Architecting Native Language World Models for General Agentic Systems

EXECUTIVE SYNTHESIS

Qwen-AgentWorld establishes the first native Language World Model (LWM) capable of simulating agentic environments across seven distinct interaction domains. By treating environment dynamics as a first-class learning objective rather than a downstream utility, the framework introduces a three-stage training pipeline (CPT → SFT → RL) that injects cross-domain world knowledge, activates explicit next-state reasoning, and sharpens simulation fidelity through hybrid rubric-and-rule reinforcement. Empirical validation demonstrates that LWM training operates as a critical scaling axis for general agents: it enables zero-shot environment scalability, controllable adversarial simulation that surpasses real-environment training, and unified foundation warm-up that instills predictive meta-reasoning capabilities without additional fine-tuning.

1.0 CORE ARCHITECTURE: UNIFIED MULTI-DOMAIN SIMULATION

Current agent frameworks prioritize policy optimization (state → action) while neglecting environment modeling (state, action → next state). Qwen-AgentWorld closes this gap by unifying seven agent interaction domains under a single textual simulation paradigm:

• Text-Based Domains: MCP Tool Use, Web Search, Terminal/CLI, Software Engineering (SWE)
• GUI-Based Domains: Android, Web Browser, Desktop OS

Unified Trajectory Schema: All domains are normalized into a single structural format consisting of a system_prompt ⊕ [turn_1, ..., turn_T]. Each turn is an (action, observation) pair. GUI observations are represented as accessibility trees and UI view hierarchies rather than pixel frames, enabling cross-domain generalization through shared textual state representations.

Data Acquisition and Processing:

• Sources: Dedicated agent infrastructure (containerized sandboxes, persistent shells, GUI VMs), open interaction traces, and internal agentic trajectory logs.
• Trajectory Expansion: Multi-turn trajectories are decomposed into turn-level prediction samples. Each turn serves as an independent supervision target conditioned on prior history.
• Intelligent Filtering: Implements retry-cycle skipping (preserving state continuity while discarding garbage output cycles) and no-change turn filtering (removing latency-induced stale states in GUI domains).
• AutoResearch Prompt Optimization: System prompts are auto-optimized via iterative propose-evaluate-refine cycles using an optimizer agent and judge model, yielding 12 distinct template variants (v0–v11) tuned for maximum prediction accuracy across domains.

2.0 TRAINING METHODOLOGY: THE CPT → SFT → RL PIPELINE

Stage 1: Continual Pre-Training (CPT)

Objective: Inject foundational environment dynamics and broad world knowledge.

• Data Composition: Environment trajectories plus specialized-domain corpora (cybersecurity, law, medicine, finance, manufacturing).
• Information-Theoretic Loss Masking: To prevent gradient noise from boilerplate turns, the pipeline computes four statistical metrics per (action, observation) pair: Overlap (OL), Novelty (Nov), Jaccard similarity (Jac), and length ratio (R). Turns are classified into seven semantic categories (e.g., retrieval, expansion, boilerplate, echo) with keep ratios ranging from 5% to 100%. Loss is computed only on turns carrying genuine environment information, decoupling "learning the next state" from "learning the next token."

Stage 2: Supervised Fine-Tuning (SFT)

Objective: Activate explicit next-state prediction as a reasoning pattern.

• Reasoning Trace Curation: Shifts from implicit next-token prediction to explicit chain-of-thought reasoning. Prompt templates are diversified across 10 variants per sample to improve generalization across system prompt formats.
• Rejection Sampling: Multi-beam rollouts from a general-purpose reasoning model are scored by an independent judge. Only trajectories exceeding a quality threshold are retained (69.2% retention rate across domains).
• Context Window: 256k tokens to accommodate long multi-step trajectories without truncation.

Stage 3: Reinforcement Learning (RL)

Objective: Sharpen simulation fidelity and prevent reward degradation.

• Algorithm: GSPO (Group Sequence Policy Optimization).
• Reward Design (Hybrid Rubric + Rule Verifier):
  • 5-Dimensional Rubric: LLM judge scores predictions on Format, Factuality, Consistency, Realism, and Quality (1–5 scale). Weighted 9:1 against rule-based verifiers.
  • Rule-Based Verifier: Executable code provides binary 0/1 correctness signals scaled to [0, 25], anchoring the reward and mitigating open-ended reward hacking.
• Stability Mitigations:
  • Reward Collapse Prevention: Restricts trajectory expansion to exactly one turn per RL sample to eliminate shared-prefix overlap.
  • Self-Praise Exploitation: Content-type classification isolates deterministic fields for exact matching. Strict tag extraction ensures thinking traces are never exposed to the judge, preventing policy inflation via qualitative affirmations.

3.0 EVALUATION FRAMEWORK: AGENTWORLDBENCH

Constructed to measure simulation fidelity under out-of-distribution conditions.

Construction Principles:

1. Widely-Used Queries: Drawn from established benchmarks (Tool Decathlon, Terminal-Bench, OSWorld-Verified, etc.).
2. Frontier-Agent Trajectories: Actions generated by Claude Opus 4.6 / Qwen-family models to stress-test complex tool-call compositions and error recovery.
3. Real Observations: Ground-truth outputs sourced directly from live environment execution.
4. Strict OoD Partitioning: Training and evaluation data are partitioned at the source level to measure generalization, not memorization.

Benchmark Composition: 2,170 turn-level evaluation samples across 7 domains. Text domains comprise 72.4% of the dataset; GUI domains comprise 9.2% each. Average context length varies by domain (e.g., MCP: ~59.3k tokens due to tool schemas; Terminal: ~12.9k tokens for self-contained sessions).

Evaluation Protocol: Reference-grounded judging using domain-aware rubrics. Content is classified into three matching tiers:

• Deterministic: Exact match required.
• Pre-existing/Environment-Specific: Format and plausibility verification only.
• Runtime Metadata: Format and range validation (timestamps, PIDs, etc.).

4.0 PERFORMANCE AND CROSS-DOMAIN GENERALIZATION

Main Results (AgentWorldBench Mean Score):

• Qwen-AgentWorld-397B-A17B: 58.71 (Surpasses GPT-5.4 at 58.25)
• Qwen-AgentWorld-35B-A3B: 56.39 (Surpasses Claude Sonnet 4.6 at 56.04)
• Domain Strengths: Text-based domains show highest gains, particularly Terminal (+4.04 over baseline) and SWE (+5.91). GUI domains lag slightly due to multimodal pre-training advantages in proprietary models, but remain highly competitive.

Cross-Domain Transfer: Training Stage 3 RL exclusively on Terminal data yields immediate gains on held-out domains:

• SWE: +11.5
• Search: +11.8
• MCP: +5.0 This confirms that RL reinforces generalizable world knowledge (state transitions, error propagation, causal chaining) rather than domain-specific formatting shortcuts.

Rule-Based Verification: Validates controllability, error handling, and long-context consistency. Qwen-AgentWorld-397B-A17B achieves 67.12% average accuracy, leading all open-weight models and ranking second overall behind GPT-5.4, with dominant performance on GUI domains.

5.0 STRATEGIC APPLICATIONS: DUAL PARADIGMS FOR AGENT ENHANCEMENT

Paradigm I: Decoupled Environment Simulator

Qwen-AgentWorld operates as a standalone simulator, providing scalability and controllability unattainable with real environments.

• Zero-Shot Environment Scaling: Synthesizes 4,000 OpenClaw environments from seed trajectories. Sim RL using the 397B model yields +4.3 on Claw-Eval and +7.1 on QwenClawBench without domain-specific adaptation.
• Controllable Perturbations: Injection of targeted environmental stressors (intermittent API failures, paginated responses, partial data retrieval, fictional world constraints) systematically exposes agent weaknesses.
  • MCPMark: +12.3 improvement over standard Sim RL.
  • WideSearch: +16.29 F1 by Item using entirely fictional, self-consistent search worlds. Agents trained in fictional environments generalize to real-world search tasks by learning query reformulation and iterative aggregation without parametric memory leakage.
• Sim RL vs. Real RL: Controllable Sim RL exceeds Real RL (50.3% vs. 45.6% F1 on WideSearch). Crucially, Sim RL increases web_extractor usage from 2.5 to 4.0 calls per trajectory, training agents to seek complete information rather than relying on superficial snippets.

Paradigm II: Unified Agent Foundation Model

The agent and world model are unified into a single framework. LWM training serves as a meta-reasoning warm-up.

• Prediction-Driven Action Refinement: The internalized world model enables mental simulation of environment responses before execution. Agents identify infeasible approaches and refine action plans within the thinking trace.
• Cross-Task Generalization: Single-turn, non-agentic LWM RL warm-up transfers to multi-turn, tool-calling agentic tasks without additional fine-tuning.
  • Terminal-Bench 2.0: +6.30
  • WideSearch: +12.79 F1 by Item
  • Claw-Eval (OoD): +11.3
  • QwenClawBench (OoD): +9.7
  • BFCL v4 (OoD): +9.0
• Quantitative Validation: Prediction accuracy on environment responses increases from 69.9% to 78.3% (+8.4%) post-training. Agents correctly predict complex causal chains (e.g., Postfix recipient validation vs. transport routing), preventing futile exploration and timeout failures.

6.0 MICRO-LEVEL FIDELITY AND REASONING PATTERNS

Beyond aggregate scores, RL training drives precision at the token and schema level:

• Multi-Step Causal Reasoning: Constructs 6+ step chains spanning package management, process lifecycle, and error semantics (e.g., curl -s failing silently due to missing Node.js → pipe receives empty input → json.tool raises specific JSONDecodeError).
• Epistemic Boundary Awareness: Recognizes computational limits (e.g., np.random.seed(42) known values vs. unknowable indices), falling back to format-only output rather than hallucinating.
• Information Leakage Prevention: In search domains, the model explicitly identifies topic mismatches and prevents answer contamination across turns.
• Character-Level Byte Arithmetic: Enumerates invisible \n bytes for exact wc -c predictions. Maintains cross-turn API schema consistency across 9 sequential Notion API calls without UUID or parent-ID drift.

7.0 PRODUCTION IMPLICATIONS AND DEPLOYMENT ROADMAP

Systemic Shift: Qwen-AgentWorld repositions agent development from reactive policy optimization to predictive simulation. By internalizing environment dynamics, agents transition from trial-and-error execution to foresight-driven planning.

Production Readiness:

• Cost/Fidelity Trade-off: Controllable simulation eliminates infrastructure bottlenecks (sandbox provisioning, GUI VM management) while enabling adversarial stress-testing.
• Training Efficiency: 128k prompt cap for RL + turn-level expansion filtering reduces compute overhead while preserving gradient quality.
• Scalability: Unified schema enables seamless integration into existing agent scaffolds without domain-specific pipeline modifications.

Future Trajectory:

1. Agent-LWM Co-Evolution: Self-play frameworks where agents discover novel states to push simulation boundaries, while LWM generates increasingly complex scenarios.
2. Multimodal Extension: Fusion of GUI screenshots with accessibility trees to unify visual and language world models for Android, Web, and OS.
3. Adaptive Sim-to-Real Routing: Per-query routers that dynamically invoke LWM simulation or live environments based on cost/fidelity requirements.
4. Dynamic Tool Synthesis: LWM-guided generation of new tool definitions at inference time, moving beyond static tool registries.

CONCLUSION

Qwen-AgentWorld establishes language world modeling as a foundational capability for general agents. The three-stage CPT → SFT → RL pipeline, unified trajectory schema, and hybrid reward design produce high-fidelity simulators that outperform frontier models on text-based domains while enabling unprecedented environmental controllability. By treating next-state prediction as a transferable meta-reasoning pattern, LWM warm-up consistently elevates downstream agent performance across seven benchmarks without additional fine-tuning. This framework provides a production-ready architecture for scaling agentic systems beyond the constraints of real-environment interaction, establishing predictive simulation as a core cognitive mechanism