Arxiv Papers of Today

生成时间: 2026-08-28 02:56:06 (UTC+8); Arxiv 发布时间: 2026-08-27 20:00 EDT (2026-08-28 08:00 UTC+8)

今天共有 30 篇相关文章

Keyword: reinforcement learning

FAMPWQ: Fisher Information-based Adaptive Mixed Precision Weight Quantization for Effective LLM Inference

FAMPWQ：基于费舍尔信息的自适应混合精度权量化，用于有效LLM推断

Authors: Gongwei Lee, Ji Liu, Juncheng Jia, Ji Wu
Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC)
Arxiv link: https://arxiv.org/abs/2608.24945
Pdf link: https://arxiv.org/pdf/2608.24945
Abstract Recent years have witnessed remarkable achievements of Large Language Models (LLMs) in multiple domains, while the excessive resource requirements of LLMs hinder the deployment on resource-constrained devices. Although model quantization stands out as an effective approach, conventional quantization approaches typically incur severe performance degradation due to uniform bit-width or simple heuristic sensitivity evaluation. In this paper, we propose a novel Fisher information-based Adaptive Mixed Precision Weight Quantization approach, i.e., FAMPWQ, which performs layer-adaptive weight quantization for effective LLM inference on commodity GPUs. First, we propose a system model with a novel Fisher information metric to measure the layer-wise sensitivity to quantization. Second, we propose a reinforcement learning-based bit-width allocator in FAMPWQ, which generates an adaptive bit-width allocation strategy based on the Fisher information sensitivity metric. Extensive experiments on 7 models and 5 benchmarks demonstrate that FAMPWQ significantly outperforms 7 baseline approaches in terms of PPL (up to 3.39 smaller), accuracy (up to 6.87% higher), and LLM-as-a-judge comparison (up to 76% win rate).
中文摘要 近年来，大型语言模型（LLM）在多个领域取得了显著成就，而LLMs过高的资源需求则阻碍了在资源受限设备上的部署。尽管模型量化是一种有效的方法，但传统量化方法通常因位宽均匀或简单的启发式敏感度评估而导致性能严重下降。本文提出了一种基于信息的自适应混合精确权重量化方法，即FAMPWQ，通过层次自适应权重量化，在商品GPU上进行有效的LLM推断。首先，我们提出一个带有新颖费舍尔信息度量的系统模型，以衡量对量子化的层级敏感性。其次，我们提出基于强化学习的 FAMPWQ 比特宽度分配器，基于 Fisher 信息敏感度指标生成自适应比特宽度分配策略。对7个模型和5个基准的广泛实验表明，FAMPWQ在PPL（最高低3.39%）、准确率（高出最高6.87%）和LLM作为评判的比较（胜率最高76%）方面显著优于7个基线方法。

Demystifying Reinforcement Learning Post-Training of Language Models

解读语言模型训练后强化学习的神秘面纱

Authors: Donovan Clay, Saket Gollapudi, Sankar Harilal, Min Jang, Jacob Morrison, Sewoong Oh, Natasha Jaques
Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Arxiv link: https://arxiv.org/abs/2608.24949
Pdf link: https://arxiv.org/pdf/2608.24949
Abstract Reinforcement learning (RL) post-training has emerged as a powerful framework for enhancing the capabilities of large language models (LLMs), enabling impressive reasoning, math, and coding capabilities. Yet for many researchers and practitioners, the principles behind classical RL remain a "black box". In this work, we deconstruct the RL post-training algorithm, investigating each step to clarify what is actually happening beneath the surface. By isolating the mechanics of RL with Verifiable Rewards in a controlled and simplified environment, we examine how RL outcomes are shaped by the base model's prior distribution, the granularity of the reward signal, the diversity of the prompt distribution, and model scale. We use the entropy of the policy's output distribution as a lens to compare the distributions learned through pretraining, SFT, and RL post-training, revealing how each stage shapes model certainty. Our investigation sheds light on how these choices interact to affect post-training success. For example, we show that the effect of so-called 'spurious rewards' depends on the prompt distribution used for post-training. We also provide insight into why the success of RL post-training depends on whether the base model already places sufficient probability mass on the desired behavior, linking it to the classical concept of exploration in RL. Ultimately, we provide this primer as a resource to those in the NLP community wishing to incorporate RL as a tool in their toolbox.
中文摘要 强化学习（RL）后训练已成为增强大型语言模型（LLM）能力的强大框架，使推理、数学和编码能力更为突出。然而，对于许多研究者和实践者来说，经典强化学习背后的原理仍然是“黑箱”。在本研究中，我们拆解了强化学习的后训练算法，逐一步探讨，以澄清表面下实际发生的事情。通过在受控且简化的环境中分离可验证奖励的强化学习机制，我们考察了基础模型先验分布、奖励信号的粒度、提示分布多样性以及模型规模如何塑造强化学习的结果。我们以策略输出分布的熵为视角，比较通过预训练、SFT和强化学习后学习到的分布，揭示每个阶段如何塑造模型确定性。我们的研究揭示了这些选择如何相互作用，影响培训后的成功。例如，我们表明所谓“虚假奖励”的效果取决于训练后所用的即时分布。我们还深入探讨了为何强化学习训练后的成功取决于基础模型是否已经对期望行为赋予足够的概率质量，并将其与强化学习中的经典探索概念联系起来。最终，我们为希望将强化学习纳入工具箱的自然语言处理社区成员提供这本入门指南。

AFDBench: A Reasoning-First AI Scientist for NationalWeather Service Forecast Discussions

AFDBench：一位以推理为先的人工智能科学家，参与国家气象局预报讨论

Authors: Manmeet Singh, Somnath Luitel, Prabhjot Singh, Manraaj Banga, Naveen Sudharsan, Josh Durkee
Subjects: Subjects: Machine Learning (cs.LG); Atmospheric and Oceanic Physics (physics.ao-ph)
Arxiv link: https://arxiv.org/abs/2608.24954
Pdf link: https://arxiv.org/pdf/2608.24954
Abstract Large language models (LLMs) hallucinate numerical values when generating high-stakes meteorological text, posing risks for weather communication. We present AFDBench, an AI meteorologist that generates professional Area Forecast Discussions (AFDs) by reasoning through structured AI weather forecast data from Google's WeatherNext 2. We introduce AFDBench, the first benchmark for evaluating generative meteorological reasoning, comprising 7,732 expert written discussions from 13 National Weather Service (NWS) offices paired with real AI weather forecast inputs, and three complementary metrics: Met-Align (numerical accuracy), Style-Align (professional dialect adherence), and Input-Grounding (fidelity to source weather data). Zero-shot evaluations reveal that open-source LLMs achieve low Style-Align (~0.33) and moderate Input-Grounding (~0.88), failing to write in the professional NWS register or faithfully use their input data. We apply Group Relative Policy Optimization (GRPO) with domain-specific rewards targeting temperature accuracy, synoptic correctness, and format compliance. On 1,033 held-out samples from two unseen NWS offices, GRPO nearly doubles Style-Align from 0.318 to 0.619 and improves Input-Grounding from 0.881 to 0.940, demonstrating that reinforcement learning teaches a 7B-parameter model to write like a professional meteorologist and faithfully interpret AI weather data.
中文摘要 大型语言模型（LLMs）在生成高风险气象文本时会产生数值幻觉，这对天气通信构成风险。我们介绍AFDBench，一款AI气象学家，通过推理来自谷歌WeatherNext 2的结构化AI天气预报数据，生成专业的区域预报讨论（AFD）。我们引入了AFDBench，这是评估生成气象推理的首个基准，包含来自13个国家气象局（NWS）办公室的7,732篇专家书面讨论，配合真实的AI天气预报输入，以及三个互补指标：Met-Align（数值准确性）、Style-Align（专业方言遵循度）和Input-Grounding（对原始天气数据的忠实度）。零样本评估显示，开源LLM的样式对齐率较低（~0.33）和中等的输入接地（~0.88），未能写入专业的NWS寄存器或忠实使用其输入数据。我们应用群相对策略优化（GRPO），并以特定领域奖励为目标，目标是温度准确性、天气正确性和格式合规性。在两个未曾见过的国家气象局办公室的1033个样本中，GRPO几乎将Style-Align从0.318提升至0.619，输入接地从0.881提升至0.940，证明强化学习能教会7B参数模型像专业气象学家一样书写并忠实解读AI气象数据。

Towards Reliable, Generalizable, and Specific In-Context Knowledge Editing via Multi-Objective Reinforcement Learning

通过多目标强化学习实现可靠、可概括且具体的上下文知识编辑

Authors: Xuzhong Wang, Maiqi Jiang, Tejal Nair, Girija Bhusal, Yanfu Zhang, Haipeng Chen
Subjects: Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Arxiv link: https://arxiv.org/abs/2608.25100
Pdf link: https://arxiv.org/pdf/2608.25100
Abstract Large Language Models (LLMs) are powerful but limited by static parametric knowledge that becomes outdated once pretraining ends. Knowledge editing addresses this problem by updating model behavior on target facts without full retraining. In particular, in-context knowledge editing has gained attention because it is training-free and readily applicable to black-box LLMs. Recent reinforcement learning (RL)-based approaches improve over fixed retrieval strategies by adapting prompt construction to the quantity-quality trade-off. Despite initial success, they fail to model the prompt as a structured entity under the distinct and often competing objectives of reliability, generality, and specificity. Previous methods largely optimize a single objective and make decisions over only part of the prompt construction process, thereby overlooking both the balance of different objectives and the global organization of demonstrations. We propose Multi-Objective In-context Knowledge Editing (MO-IKE), a multi-objective RL algorithm that formulates prompt construction for in-context knowledge editing as a Constrained Markov Decision Process. MO-IKE trains a dynamic retriever to optimize competing objectives in knowledge editing, enabling more balanced and globally coherent prompt construction. On Llama-3.2, MO-IKE improves edit success (reliability) from 85.0% to 92.0%, paraphrase consistency (generality) from 77% to 79%, while increasing retention rate (specificity) by 23.0% compared to prior RL-based methods.
中文摘要 大型语言模型（LLM）功能强大，但受限于静态参数化知识，一旦预训练结束，这些知识就会过时。知识编辑通过在目标事实上更新模型行为来解决这一问题，而无需完全重新训练。特别是，上下文知识编辑因其无需培训且易于应用于黑箱大型语言模型而备受关注。基于强化学习（RL）的最新方法通过调整即时构建以适应数量与质量的权衡，优于固定检索策略。尽管初期取得成功，但他们未能将提示建模为一个结构化的实体，目标是可靠性、通用性和具体性。以往方法主要优化单一目标，仅在提示构建过程的部分做出决策，从而忽视了不同目标的平衡以及演示的整体组织。我们提出了多目标上下文知识编辑（MO-IKE），这是一种多目标强化学习算法，将上下文知识编辑的提示构造方式呈现为受限马尔可夫决策过程。MO-IKE 训练动态检索器，优化知识编辑中的竞争目标，实现更平衡且全局一致的提示构建。在Llama-3.2上，MO-IKE将编辑成功率（可靠性）从85.0%提升至92.0%，改写一致性（概括性）从77%提升至79%，同时保持率（特异性）比以往基于强化学习的方法提高了23.0%。

Bayesian Flow Networks for Offline Trajectory Planning

贝叶斯流网络用于离线轨迹规划

Authors: Ludvig Killingberg, Helge Langseth
Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Arxiv link: https://arxiv.org/abs/2608.25163
Pdf link: https://arxiv.org/pdf/2608.25163
Abstract Offline reinforcement learning (RL) leverages static datasets to learn decision policies without real-time environment interaction. While recent sequence-modeling approaches rely on continuous diffusion models for trajectory synthesis, applying these methods to discrete planning tasks requires a categorical formulation rather than the standard Gaussian construction. We present BFN-RL, a unified generative modeling framework for offline RL based on Bayesian Flow Networks (BFNs). By iteratively evolving distribution parameters rather than noisy data instances, BFN-RL natively models both discrete and continuous trajectory spaces within a single probabilistic formulation. The categorical planner generates future state sequences, and a learned inverse-dynamics model converts consecutive generated states into actions. Evaluations in discrete planning and continuous control show that BFN-RL can generate effective trajectories across both categorical and continuous state spaces. Our results establish BFNs as a versatile generative foundation for offline trajectory planning across data modalities.
中文摘要 离线强化学习（RL）利用静态数据集学习决策策略，无需实时环境交互。虽然近期序列建模方法依赖连续扩散模型进行轨迹合成，但将这些方法应用于离散规划任务则需要范例表述，而非标准的高斯构造。我们介绍BFN-RL，这是一个基于贝叶斯流网络（BFN）的统一离线强化生成建模框架。通过迭代演化分布参数而非噪声数据实例，BFN-RL 能够在单一概率表述中原生建模离散和连续轨迹空间。分类规划器生成未来状态序列，学习的逆动力学模型将连续生成状态转换为动作。离散规划与连续控制的评估表明，BFN-RL能够在类别和连续状态空间中生成有效的轨迹。我们的研究结果确立了BFN作为跨数据模式离线轨迹规划的多功能生成基础。

Simulating Cognitive Smart Freight Corridors with Agent-Based Models and Reinforcement Learning

利用基于代理的模型和强化学习模拟认知智能货运走廊

Authors: Madelaine Martinez-Ferguson, Chun Wang, Mustafa Can Camur, Xueping Li
Subjects: Subjects: Emerging Technologies (cs.ET); Machine Learning (cs.LG)
Arxiv link: https://arxiv.org/abs/2608.25193
Pdf link: https://arxiv.org/pdf/2608.25193
Abstract Smart freight corridors offer a practical pathway for connected and automated vehicle (CAV) deployment in freight transportation, but physical experimentation is expensive and existing approaches rely on predefined control policies that cannot capture adaptive behaviors. This paper presents an agent-based modeling (ABM) framework coupling a physical infrastructure layer, a connectivity layer (V2X), and a decision layer integrating reinforcement learning (RL) and multi-agent reinforcement learning (MARL) for platoon formation and charging coordination. We evaluate three scenarios (Baseline, Assisted, and Cognitive) using throughput, congestion, energy, emissions, and robustness metrics. Preliminary results indicate that the Cognitive scenario achieves higher throughput and lower congestion than the baseline, while the Assisted scenario delivers meaningful energy savings per kilometer through platooning. Sensitivity analysis shows that the throughput advantage of the smart corridor widens under conditions with high demand and that MARL coordination extracts greater utilization from fixed charging capacity than rule-based assignment.
中文摘要 智能货运走廊为货运中互联和自动化车辆（CAV）的部署提供了实用路径，但物理实验成本高昂，现有方法依赖于预设的控制策略，无法捕捉适应性行为。本文提出了一个基于主体建模（ABM）的框架，结合了物理基础设施层、连接层（V2X）和整合强化学习（RL）和多智能体强化学习（MARL）的决策层，用于排队编队和冲锋协调。我们利用吞吐量、拥堵、能源、排放和稳健性指标评估三种场景（基线、辅助和认知）。初步结果表明，认知模式比基线实现更高的通量和更低的拥堵，而辅助模式通过分队行进每公里实现了显著的节能。敏感度分析显示，在高需求条件下，智能走廊的吞吐量优势会扩大，MARL协调比基于规则的分配更能从固定充电容量中提取更高的利用率。

From Memorization to Absorption: Mixed-Policy RL for Continual Knowledge Injection

从记忆到吸收：混合策略强化学习用于持续知识注入

Authors: Zhibo Hou, Fan Zhao, Zhiyu An, Wan Du
Subjects: Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
Arxiv link: https://arxiv.org/abs/2608.25243
Pdf link: https://arxiv.org/pdf/2608.25243
Abstract Continual knowledge injection is essential for keeping large language models up-to-date in a fast-evolving world. Existing methods rely on supervised fine-tuning (SFT), which memorizes injected facts in their training format but fails to generalize across paraphrasing, document combinations, and reasoning. To address this, we propose Golden-GRPO Injection (GRIN), a three-stage self-learning framework for continual knowledge injection. Golden-GRPO is a mixed-policy reinforcement learning algorithm designed specifically for knowledge injection, which injects a golden answer to provide learning signal even when on-policy rollouts fail on novel facts. We further introduce Blank and Counter, two document-level benchmarks targeting novel acquisition and counterfactual overwrite respectively, each evaluating single-fact recall, multi-source retrieval, and inferential reasoning. Our experiments establish a clear empirical claim: mixed-policy reinforcement learning enables knowledge absorption beyond what supervised fine-tuning can achieve. GRIN substantially outperforms SFT and mixed-policy RL baselines on the harder question types while matching them on basic fact recall.
中文摘要 持续的知识注入对于在快速变化的世界中保持大型语言模型的更新至关重要。现有方法依赖监督微调（SFT），该方法在训练格式中记忆注入的事实，但未能在意译、文档组合和推理中进行推广。为此，我们提出了Golden-GRPO注入（GRIN），一种三阶段的持续知识注入自学框架。Golden-GRPO是一种专为知识注入设计的混合策略强化学习算法，即使在新颖事实的策略推广失败时，也能注入黄金答案，提供学习信号。我们还进一步介绍了空白和反对，这两个文档级基准分别针对新颖获取和反事实覆盖，分别评估单一事实回忆、多源检索和推理推理。我们的实验确立了一个明确的实证主张：混合策略强化学习能够实现超出监督微调所能实现的知识吸收。GRIN在较难题型上显著优于SFT和混合策略强化学习基线，同时在基本事实回忆方面也相当。

PointRL: Learning Point-Level Vision-Language Grounding from Verifiable Annotation Evidence

PointRL：从可验证的注释证据中学习点级视觉语言基础

Authors: Jingyang Su, Pu Cao, Xiuze Jin, Longyue Zhang, Qing Song, Lu Yang
Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV)
Arxiv link: https://arxiv.org/abs/2608.25299
Pdf link: https://arxiv.org/pdf/2608.25299
Abstract Vision-language models (VLMs) increasingly rely on point coordinates as a compact and executable interface for visual grounding in GUI interaction, robotic manipulation, and interactive visual systems. However, learning reliable pointing behavior remains difficult because the supervision space is inherently non-unique: many coordinates may be valid within the same target region, while multi-instance instructions require target coverage, count consistency, and duplicate suppression. This work presents PointRL, a verifiable reinforcement learning framework that learns point-level grounding from existing heterogeneous annotation evidence. PointRL converts bounding boxes, masks, and instance labels into pointing instructions, while retaining their target supports, instance membership, and set constraints as hidden verifier evidence, i.e., annotations kept outside the prompt and used by a deterministic checker to score predictions. The proposed reward evaluates parseability, point validity, instance coverage, cardinality consistency, and redundant or missing predictions. On PointArena, PointRL improves the overall accuracy of Qwen3.5-4B from 56.11% to 65.58%. Further evaluations on RoboSpatial, BLINK, and Ref-Adv show same-backbone gains on the evaluated external benchmarks, suggesting that verifiable point-level feedback may benefit spatial grounding in these settings.
中文摘要 视觉语言模型（VLM）越来越依赖点坐标作为一种紧凑且可执行的界面，用于图形界面交互、机器人操作和交互式视觉系统中的视觉基础。然而，学习可靠的指向行为仍然困难，因为监督空间本质上是非唯一性的：许多坐标可能在同一目标区域内有效，而多实例指令则需要目标覆盖、计数一致性和重复抑制。本研究提出了PointRL，一种可验证的强化学习框架，能够从现有的异构注释证据中学习点级基础。PointRL 将边界框、掩码和实例标签转换为指向指令，同时保留其目标支持、实例成员资格，并设置约束作为隐藏的验证证据，即注释被提示外保存，供确定性检查器用于对预测进行评分。拟议的奖励评估可解析性、点有效性、实例覆盖率、基数一致性以及冗余或缺失的预测。在PointArena上，PointRL将Qwen3.5-4B的整体准确率从56.11%提升到65.58%。对RoboSpatial、BLINK和Ref-Adv的进一步评估显示，外部基准在同等骨干上均有提升，表明可验证的点级反馈可能有助于这些环境下的空间基础。

Beyond Pairwise Feedback: Listwise Vision-Language Supervision for Preference-Based Reward Learning

超越成对反馈：基于偏好的奖励学习中的列表式视觉-语言监督

Authors: Srivalli Katkuri, Maxwell Kawada, Juan Wachs
Subjects: Subjects: Machine Learning (cs.LG); Robotics (cs.RO)
Arxiv link: https://arxiv.org/abs/2608.25350
Pdf link: https://arxiv.org/pdf/2608.25350
Abstract Vision-language models (VLMs) have emerged as a powerful source of supervision for reinforcement learning, enabling agents to leverage rich semantic knowledge during training. Inspired by the success of preference-based reward learning (PbRL) in reinforcement learning from human feedback (RLHF), vision-language model generated image-based preferences provide an effective source for learning reward functions. This can be done by visually comparing two outcomes through the Bradley-Terry (BT) model. However, this pairwise formulation utilizes only two observations at a time, despite VLMs being capable of ranking multiple candidates. The Plackett-Luce (PL) formulation can shape a reward model with listwise rankings as opposed to pairwise preferences, allowing for a more suited use of a VLM based ranking. In this work, to our knowledge, we introduce the first framework that combines VLM-generated preferences with the Plackett-Luce model for reward learning. We evaluate our approach on Meta-World manipulation tasks and show that Plackett-Luce (PL) reward models can train robotic policies from VLM-generated rankings as effectively as pairwise Bradley-Terry, $K$-wise Bradley-Terry, and RL-VLM-F baselines. Across all environments, at least one PL ranking size ($K \in {3,4,5}$) consistently performs with or outperforms other methods in mean success rate. Unlike pairwise methods, which are restricted to $K=2$, PL supports different ranking sizes and can therefore be adapted to the environment and desired feedback format. Our best PL configuration achieves an 86% mean final success rate and matches the Oracle baseline on Drawer Open. Overall, these results demonstrate that listwise VLM preference supervision is a competitive and flexible approach to reward learning for reinforcement learning.
中文摘要 视觉语言模型（VLMs）已成为强化学习的强大监督来源，使智能体能够在训练过程中利用丰富的语义知识。受基于偏好的奖励学习（PbRL）在人类反馈强化学习（RLHF）中的成功启发，视觉语言模型生成的基于图像的偏好为学习奖励函数提供了有效的资源。这可以通过布拉德利-特里（BT）模型进行视觉比较来实现。然而，这种两对方式一次只使用两个观测值，尽管VLM能够对多个候选人进行排名。Plackett-Luce（PL）表述可以构建一个基于列表排名而非成对偏好的奖励模型，从而更适合基于VLM的排序。据我们所知，本研究引入了首个将VLM生成偏好与Plackett-Luce模型结合用于奖励学习的框架。我们评估了对元世界操作任务的方法，并证明Plackett-Luce（PL）奖励模型能够从VLM生成的排名中训练机器人策略，与成对Bradley-Terry、$K$级Bradley-Terry和RL-VLM-F基线同样有效。在所有环境中，至少有一个PL排名规模（$K \3,4,5}$）在平均成功率上始终稳定地与其他方法相匹配或优于其他方法。与限制在$K=2$的成对方法不同，PL支持不同的排名大小，因此可以根据环境和期望的反馈格式进行调整。我们最好的PL配置平均最终成功率为86%，并且与Oracle在Drawer Open上的基线相当。总体而言，这些结果表明，列表式VLM偏好监督是一种竞争性且灵活的强化学习奖励学习方法。

Where vs What: Decomposing Structural and Content Failures in LLM-Generated Structured Outputs

哪里与什么：分解LLM生成结构化输出中的结构性和内容失败

Authors: Yiwei Zhang, Chengke Wu, Li Wang, Jianqiang Li
Subjects: Subjects: Artificial Intelligence (cs.AI)
Arxiv link: https://arxiv.org/abs/2608.25358
Pdf link: https://arxiv.org/pdf/2608.25358
Abstract Structured outputs such as JSON and tables are central to modern LLM-based systems, yet generation failures are evaluated monolithically, conflating two distinct error modes: placement errors (correct values at wrong positions) and value errors (wrong values at intended positions). We introduce Structure-Content Decomposition (SCD), a framework that independently measures structural fidelity and content accuracy. Applying SCD to nested JSON and table tasks across six models (7B to frontier), we uncover a consistent phenomenon: structural fidelity degrades earlier and more sharply than content accuracy as complexity increases. At the highest complexity, even DeepSeek-V4-Flash (with reasoning) misplaces 35% of recalled values, while Qwen2.5-7B misplaces 74%. Controlled ablations suggest that this pattern is associated with reliance on semantic shortcuts rather than topological understanding of output structure. Based on these findings, we propose SA-RLVR, converting SCD metrics into verifiable rewards for reinforcement learning via GRPO. SA-RLVR successfully optimizes structural addressing across distinct topologies: it lifts JSON Value Placement Accuracy (VPA) from 26% to 63% while generalizing to held-out schemas; moreover, it consistently drives VPA improvements in the table domain, demonstrating that structure-aware rewards can directly enhance multi-domain structural positioning.
中文摘要 结构化输出如JSON和表格是现代基于LLM系统的核心，但生成失败却被整体评估，混淆了两种截然不同的错误模式：放置错误（错误位置的正确值）和错误位置的值错误（预期位置错误的值）。我们介绍结构-内容分解（SCD），这是一个独立测量结构忠实度和内容准确性的框架。将SCD应用于六个模型（7B到Frontier）的嵌套JSON和表任务，我们发现一个持续的现象：结构忠实度随着复杂度增加，比内容准确性更早且更剧烈地下降。在最高复杂度下，即使是DeepSeek-V4-Flash（带推理）也会丢失35%的召回值，而Qwen2.5-7B则丢失了74%。受控消融表明，这种模式与对语义捷径的依赖有关，而非对输出结构的拓扑理解。基于这些发现，我们提出了SA-RLVR，将SCD指标转化为通过GRPO进行强化学习的可验证奖励。SA-RLVR成功优化了不同拓扑的结构寻址：它将JSON值置放准确率（VPA）从26%提升到63%，同时推广到保留的模式;此外，它持续推动VPA在表格领域的改进，证明结构感知奖励可以直接提升多域结构定位。

Traffic-Adaptive Per-Hop Multipath Routing in Multi-Hop UAV Networks

多跳无人机网络中的流量自适应每跳多径路由

Authors: Zhenyu Zhao, Tiankui Zhang, Xiaoxia Xu, Yuanpeng Zheng, Junjie Li, Wenjuan Xing
Subjects: Subjects: Networking and Internet Architecture (cs.NI)
Arxiv link: https://arxiv.org/abs/2608.25383
Pdf link: https://arxiv.org/pdf/2608.25383
Abstract In uncrewed aerial vehicle (UAV)-relayed mobile edge computing (MEC) networks, computation tasks generate traffic with diverse latency requirements and data sizes. Routing decisions therefore need to adapt to both traffic characteristics and changing network conditions. Compared with single-path routing, multipath routing is better suited to such heterogeneous traffic because it provides multiple forwarding options and enables flexible traffic splitting. However, conventional multipath routing usually splits traffic over predefined end-to-end paths, making it difficult to respond quickly to link fluctuations and topology changes in UAV networks. To address this issue, we propose a traffic-adaptive per-hop multipath routing method for multi-hop UAV networks, in which each UAV dynamically distributes traffic among multiple candidate next hops. We formulate the routing problem to improve the on-time packet delivery ratio while reducing the packet loss ratio, and model it as a decentralized partially observable Markov decision process (Dec-POMDP). To solve this problem, we develop a multi-agent reinforcement learning (MARL) algorithm, termed Multi-Agent Proximal Policy Optimization with Dirichlet Modeling (MAPPO-DM). MAPPO-DM follows the centralized-training-and-decentralized-execution framework and models continuous traffic-splitting actions using a Dirichlet distribution. Simulation results show that MAPPO-DM outperforms the baseline methods and maintains robust performance under various network conditions.
中文摘要 在无人机（UAV）中继移动边缘计算（MEC）网络中，计算任务产生的流量具有不同的延迟需求和数据量。因此，路由决策需要适应流量特性和不断变化的网络状况。与单径路由相比，多径路由更适合这种异构流量，因为它提供了多种转发选项并支持灵活的流量分割。然而，传统的多径路由通常将流量分配到预设的端到端路径，这使得快速响应无人机网络中的链路波动和拓扑变化变得困难。为解决这一问题，我们提出了一种流量自适应的每跳多路径路由方法，适用于多跳无人机网络，其中每架无人机动态将流量分配到多个候选下一跳。我们提出路由问题以改善按时数据包传递率，同时降低丢包率，并将其建模为去中心化部分可观测马尔可夫决策过程（Dec-POMDP）。为解决此问题，我们开发了一种多智能体强化学习（MARL）算法，称为多智能体近端策略优化与狄利克雷建模（MAPPO-DM）。MAPPO-DM遵循集中式训练与去中心化执行框架，并利用狄利克雷分布建模连续的流量分割动作。仿真结果表明，MAPPO-DM在各种网络条件下表现优于基线方法，并保持稳健性能。

Lightweight AI for UAV-Mounted RIS: An Overview

无人机安装RIS的轻量级人工智能概述

Authors: Sherief Hashima, Kohei Hatano, Eiji Takimoto, Mohamed Rihan, Basem.M. Elhalawany, Hamada Rizk
Subjects: Subjects: Networking and Internet Architecture (cs.NI)
Arxiv link: https://arxiv.org/abs/2608.25402
Pdf link: https://arxiv.org/pdf/2608.25402
Abstract Unmanned Aerial Vehicles (UAV)-mounted Reconfigurable Intelligent Surfaces (RIS) have emerged as a promising architecture for enhancing wireless coverage, spectral efficiency, and energy performance in 6G networks. By combining programmable electromagnetic wave manipulation with aerial mobility, UAV-RIS systems enable dynamic blockage mitigation, adaptive beamforming, and flexible deployment across terrestrial, maritime, and satellite-integrated environments. However, joint optimization of UAV trajectory, RIS phase configuration, and resource allocation incurs high computational complexity, which is incompatible with the strict energy and onboard processing constraints of UAV platforms. Lightweight AI techniques offer practical solutions to this challenge. Hence, this paper provides a comprehensive overview of lightweight AI techniques for UAV-mounted RIS systems, including Reinforcement Learning (RL), meta-learning, Federated Learning (FL), Multi-Armed Bandits (MAB), and energy-aware optimization. We present a detailed taxonomy and comparative analysis of existing work, highlight computational-energy trade-offs, and identify open research challenges for scalable, energy-efficient airborne intelligent surfaces. Furthermore, we present a case study demonstrating the effect of MAB schemes on throughput and energy efficiency in UAV-mounted RIS.
中文摘要 无人机（UAV）安装的可重构智能表面（RIS）已成为提升6G网络无线覆盖、频谱效率和能源性能的有前景架构。通过将可编程电磁波操控与空中机动性结合，无人机-RIS系统实现了动态阻断缓解、自适应波束形成以及在地面、海上及卫星集成环境中的灵活部署。然而，无人机轨迹、RIS阶段配置和资源分配的联合优化带来了较高的计算复杂度，这与无人机平台严格的能源和机载处理限制不兼容。轻量级人工智能技术为这一挑战提供了切实可行的解决方案。因此，本文全面概述了无人机安装RIS系统的轻量级AI技术，包括强化学习（RL）、元学习、联邦学习（FL）、多臂强盗（MAB）和能量感知优化。我们提出了详细的分类学和现有工作的比较分析，突出计算与能源的权衡，并识别可扩展、节能空中智能表面的开放研究挑战。此外，我们还展示了一个案例研究，展示了MAB方案对无人机安装RIS中吞吐量和能效的影响。

LAC: Linear and Angular Compliance for Humanoid Whole-body Control

LAC：人形全身控制的线性和角度顺应性

Authors: Yang Liu, Zhongkai Gu, Wei Zhu, Mitsuhiro Hayashibe
Subjects: Subjects: Robotics (cs.RO)
Arxiv link: https://arxiv.org/abs/2608.25405
Pdf link: https://arxiv.org/pdf/2608.25405
Abstract Real-world humanoid tasks involve physical interaction with objects and humans, yet current controllers either reject external forces as disturbances or restrict compliance to limited body links while ignoring angular effects. We present LAC, a general whole-body controller that simultaneously realizes commanded Linear and Angular Compliance for wrenches applied to the upper body. First, we synthesize whole-body compliant responses into a large-scale augmented dataset. Sampled force and couple events are imposed on contact frames extracted from human interaction data. At each contact link, the external force and a virtual torque from the passively yielding kinematic chain drive a virtual admittance under the commanded stiffness. Subsequently, teacher-student reinforcement learning trains a single policy to track the compliant motions under external wrenches. Finally, extensive simulation and real-world experiments demonstrate whole-body compliant responses to wrenches across the upper body, monotonic modulation over the full range of both stiffness commands, and applicability to teleoperated loco-manipulation tasks. Project website: this https URL
中文摘要 现实中的类人机器人任务涉及与物体和人类的物理互动，但当前的控制器要么将外部力量视为干扰而拒绝，要么限制对有限身体连接的顺从，同时忽略角度效应。我们介绍了LAC，一款通用的全身控制器，能够同时实现施加在上半身的扳手的指令线性和角度顺应性。首先，我们将全体一致的响应综合到一个大规模增强数据集中。采样的力和耦合事件被施加在从人类交互数据中提取的接触框上。在每个接触链节处，外力和被动产生的运动链的虚扭矩驱动在命令刚度下的虚拟导纳。随后，师生强化学习训练单一策略，跟踪在外部扳手下合规的动作。最后，广泛的模拟和实际实验证明，扳手横跨上半身时具有全车体响应，在刚度指令的全范围内实现单调调制，并适用于远程操作机车操作任务。项目网站：此 https URL

BVR Sim: An Open and High-Throughput Environment for Heterogeneous Air-Combat Reinforcement Learning

视距模拟：一个开放且高通量的异构空战增援学习环境

Authors: Haocheng Sun (Beijing University of Posts and Telecommunications), Mulai Tan (Air Force Engineering University)
Subjects: Subjects: Multiagent Systems (cs.MA); Machine Learning (cs.LG)
Arxiv link: https://arxiv.org/abs/2608.25419
Pdf link: https://arxiv.org/pdf/2608.25419
Abstract Beyond-visual-range (BVR) air combat is a challenging reinforcement-learning domain characterized by partial observability, long-horizon decision making, energy management, and limited weapons. We present BVR Sim, an open-source Gymnasium-style environment designed for heterogeneous air-combat reinforcement learning. BVR Sim supports multiple JSBSim aircraft models, including the F-15, F-16, F/A-18, and F-22, with configurable weapons, sensors, controllers, and opponents. A unified tactical action interface specifies desired heading, altitude, speed, and weapon release above aircraft-specific inner-loop controllers, enabling policies to operate across heterogeneous platforms. The environment provides interchangeable Python and accelerated C++ backends, entity-oriented observations, compositional rewards, scripted opponents, replay and visualization, and adapters for multi-agent learning frameworks. At a 0.4-s decision interval, the C++ backend achieves 104 simulated seconds per wall-clock second in 1-vs-1 and remains practical through 10-vs-10 scenarios. A policy trained only on the F-16 transfers without retraining to four unseen aircraft, reaching a 45.5% mean win rate with aircraft-specific controller adaptation. MAPPO and HAPPO experiments further verify end-to-end compatibility with standard multi-agent reinforcement-learning pipelines.
中文摘要 超视距（BVR）空战是一个具有挑战性的强化学习领域，其特点是部分可探测性、长视野决策、能量管理和有限的武器。我们介绍BVR Sim，一个开源的Gymnasium风格环境，专为异构空战强化学习设计。BVR 模拟支持多种JSBSim飞机型号，包括F-15、F-16、F/A-18和F-22，配备可配置武器、传感器、控制器和对手。统一的战术动作界面指定了目标航向、高度、速度和武器释放，超过了飞机特定的内环控制器，使政策能够跨异构平台运行。该环境提供了可互换的Python和加速C++后端、面向实体的观察、组合奖励、脚本化对手、重放和可视化，以及多智能体学习框架的适配器。在0.4秒决策间隔下，C++后端在1对1中每墙钟秒内可实现104秒的模拟效率，并且在10对10的场景中依然实用。该政策仅针对F-16转移进行训练，未重新训练为四架未被发现的飞机，采用特定飞机控制适应后平均胜率达到45.5%。MAPPO和HAPPO实验进一步验证了与标准多智能体强化学习流水线的端到端兼容性。

VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality

VGA-BenchV2：扩展的统一基准测试和多模型框架，用于评估视频美学和生成质量

Authors: Longteng Jiang, DanDan Zheng, Qianqian Qiao, Heng Huang, Huaye Wang, Yihang Bo, Bao Peng, Jingdong Chen, Jun Zhou, Xin Jin
Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Arxiv link: https://arxiv.org/abs/2608.25452
Pdf link: https://arxiv.org/pdf/2608.25452
Abstract We introduce VGA-BenchV2, an extended human-aligned benchmark and optimization framework for jointly evaluating and improving video generation quality and aesthetic value. Built upon VGA-Bench, VGA-BenchV2 preserves the original fine-grained taxonomy with two primary dimensions-Aesthetic and Generation-and 52 sub-dimensions. Guided by this taxonomy, we curate 1,016 diverse prompts and collect over 60,000 videos generated by 12 mainstream video generation models. More importantly, VGA-BenchV2 substantially expands human-labeled supervision by adding 36,000 task-level annotations, including 16,200 for aesthetic quality, 13,200 for aesthetic tagging, and 6,600 for generation quality, corresponding to 13.46x, 11.15x, and 1.55x scale-ups over VGA-Bench, respectively. Leveraging this enlarged annotation corpus, we develop a hybrid evaluator architecture consisting of VAQA-Net for continuous aesthetic scoring and two Qwen-based Large Vision-Language Model evaluators, VTag-Net and VGQA-Net, for aesthetic tagging and generation quality assessment. Extensive experiments demonstrate strong alignment with human judgments across diverse generation models. Beyond evaluation, VGA-BenchV2 further introduces an evaluation-to-optimization pipeline, where the learned aesthetic evaluator serves as a reward model for reinforcement learning-based generator fine-tuning. This closes the loop from benchmark construction and human supervision to automated evaluation and model optimization, enabling video generators to improve not only in realism but also in aesthetic quality and human preference alignment. Resources are available at this https URL.
中文摘要 我们介绍VGA-BenchV2，这是一个扩展的人类基准和优化框架，用于共同评估和提升视频生成质量及美学价值。基于VGA-Bench，VGA-BenchV2保留了原始细粒度分类法，包含两个主要维度——美学和生成——以及52个子维度。在这一分类法的指导下，我们策划了1016个多样化的提示词，并收集了12种主流视频生成模型生成的6万多段视频。更重要的是，VGA-BenchV2大幅扩展了人工标记监督，增加了36,000个任务级注释，其中包括16,200个美观标注、13,200个美观标记和6,600个生成质量标注，分别对应VGA-Bench的13.46倍、11.15倍和1.55倍放大。利用这一扩展的注释语料库，我们开发了一个混合评估器架构，包括用于连续美学评分的VAQA-Net和基于Qwen的两个基于Qwen的大型视觉语言模型评估器VTag-Net和VGQA-Net，用于美学标记和生成质量评估。大量实验显示，跨越多种世代模型，与人类判断高度契合。除了评估，VGA-BenchV2 还引入了评估到优化的流程，学习到的美学评估器作为基于强化学习生成器微调的奖励模型。这实现了从基准构建和人工监督到自动评估和模型优化的环节，使视频生成器不仅提升真实性，还能提升美学质量和人类偏好一致性。相关资源可在此 https 网址获取。

Training Alignment Auditors via Reinforcement Learning

通过强化学习培训对齐审计员

Authors: Paul Rosu, Rowan Wang
Subjects: Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Arxiv link: https://arxiv.org/abs/2608.25460
Pdf link: https://arxiv.org/pdf/2608.25460
Abstract Alignment auditing of frontier models increasingly relies on LLM auditors to surface undesirable behaviors at scale, but current automated auditors can struggle with coherent investigation and audit realism. In this work, we improve LLM auditors with reinforcement learning. In our best training environment, the policy investigates target models that potentially possess hidden behaviors planted via their system prompt. An LLM judge, which knows whether the target has a hidden behavior, holistically compares the policy's investigation to a reference investigation to determine the reward. With systematic ablations, we find that pairwise rewards yield more robust training compared to pointwise rewards, and that adding targets without planted behaviors helps maintain a low false positive rate. Training improves investigation quality against targets with planted behaviors, the rate of concerning behaviors surfaced in unmodified production models, and audit realism, while false-positive rates stay below 1%. Furthermore, auditing capabilities generalize across scaffolds: performance on AuditBench's adversarially fine-tuned targets substantially improves [Sheshadri et al., 2026].
中文摘要 前沿模型的对齐审计越来越依赖LLM审计员大规模发现不良行为，但现有自动化审计人员在连贯调查和审计真实性方面可能存在困难。在这项工作中，我们通过强化学习提升LLM审计员的能力。在我们最佳的训练环境中，该策略会调查那些可能通过系统提示植入隐藏行为的目标模型。LLM评判者知道目标是否有隐藏行为，会整体比较策略调查与参考调查以确定奖励。通过系统消融，我们发现成对奖励比点对奖励更能带来更稳健的训练，且添加无固定行为的目标有助于保持低假阳性率。培训提升了针对植入行为目标的调查质量，未修改生产模型中令人担忧行为的出现率和审计真实性，而误报率保持在1%以下。此外，审计能力在各个支架上具有普遍性：在AuditBench对抗性微调目标上的表现显著提升[Sheshadri 等，2026]。

AERIS: Offline Policy Improvement for Multi-UAV Integrated Sensing and Communication

AERIS：多无人机综合感测与通信的离线政策改进

Authors: Ziyuan Wang, Yifan Sui, Wei Wei, Wenjie Xin, Zekai Zhang, Xiangwang Hou, Xiao-Ping (Steven)Zhang
Subjects: Subjects: Networking and Internet Architecture (cs.NI); Machine Learning (cs.LG)
Arxiv link: https://arxiv.org/abs/2608.25477
Pdf link: https://arxiv.org/pdf/2608.25477
Abstract Unmanned aerial vehicle (UAV)-enabled integrated sensing and communication (ISAC) is a promising 6G paradigm, but dynamic multi-UAV ISAC control must jointly balance communication quality, sensing reliability, and flight safety under stochastic mobility. Existing optimization methods often require repeated global non-convex solving, while online reinforcement learning (RL) depends on risky trial-and-error flights that may cause sensing loss or collision-risk events. This paper proposes AERIS, an offline policy improvement framework for multi-UAV ISAC. AERIS learns from fixed flight logs under centralized training and decentralized execution, so each UAV acts from local histories while training uses logged global information to assess team-level effects. We further design STAR-CRDT, an offline multi-agent RL algorithm that performs support-aware local action rectification and distills only trusted improvements into the decentralized actor. We prove an offline-support policy improvement guarantee. Experiments show that STAR-CRDT improves the main ISAC objective return by 29.3% over the strongest baseline. It further improves communication sum rate, sensing pass rate, and sensing margin by 3.4%, 4.8%, and 69.1%, while reducing collision-risk events by 54.2%. On unseen real-road maps built from OpenStreetMap data, STAR-CRDT still obtains the best return.
中文摘要 无人机（UAV）支持的综合感测与通信（ISAC）是一种有前景的6G范式，但动态多无人机ISAC控制必须在随机机动性下共同平衡通信质量、传感可靠性和飞行安全。现有优化方法通常需要反复进行全局非凸求解，而在线强化学习（RL）依赖于高风险的试错飞行，可能导致感测丢失或碰撞风险事件。本文提出了AERIS，一个针对多无人机ISAC的离线政策改进框架。AERIS通过集中培训和分散执行从固定飞行日志中学习，因此每架无人机根据本地历史数据行动，而训练则利用全球记录信息评估团队层面的影响。我们还进一步设计了STAR-CRDT，这是一种离线多智能体强化学习算法，能够执行支持意识的本地动作纠正，并将可信改进内容提炼进去中心化的行为体。我们证明了离线支持政策改进的保证。实验显示，STAR-CRDT比最强基线提高了主要ISAC客观回报29.3%。它进一步提升了通信加和率、感应通过率和感应余距分别提升了3.4%、4.8%和69.1%，同时将碰撞风险事件降低了54.2%。在基于OpenStreetMap数据构建的未公开真实道路图上，STAR-CRDT仍获得最佳回报。

Agentic Game Development as a Verifiable Trajectory Data Engine for Scaling World Models

代理游戏开发作为可验证轨迹数据引擎用于缩放世界模型

Authors: Pengfei Zhou, Hexin Wang, Zhengfeiyang Zhang, Yixing Ma, Zhenglin Wan, Kaipeng Zhang, Wangbo Zhao, Yang You
Subjects: Subjects: Artificial Intelligence (cs.AI)
Arxiv link: https://arxiv.org/abs/2608.25518
Pdf link: https://arxiv.org/pdf/2608.25518
Abstract A common strategy for scaling world models is to train on more crawled video with more compute. We argue that this strategy is inefficient: scaling world models also requires a recursive data engine that offers grounded reward signals. The success of code agents illustrates why this matters. As code is executable, compilers and runtimes can provide high-quality rewards for Reinforcement Learning (RL) post-training of LLMs. By contrast, spatial generation still relies largely on fuzzy proxies such as CLIP scores. These signals are fuzzy and biased, making them hard to support RL post-training. Compared with these, game development provides a missing reward environment for spatial world models. A scene encoded by a game engine is an executable world specification: the engine can efficiently check collision, physics, navigability and bounded playability, while the developer provides the global verification signal by judging whether the scene should be accepted. Game development also provides real-world long-horizon trajectory data for RL post-training. We therefore propose Reinforcement Learning with Human-Engine Verification (RLHEV), a post-training paradigm that combines dense engine signals with implicit human acceptance feedback from the development process.
中文摘要 扩展世界模型的常见策略是用更多爬取的视频和更多计算量训练。我们认为这种策略效率低下：扩展世界模型还需要一个递归数据引擎，提供基于基础的奖励信号。代码代理的成功说明了这一点的重要性。由于代码可执行，编译器和运行时可以为强化学习（RL）后训练LLM提供高质量奖励。相比之下，空间生成仍然主要依赖于模糊的代理指标，如CLIP分数。这些信号模糊且有偏见，使得训练后难以支持强化学习。相比之下，游戏开发为空间世界模型提供了缺失的奖励环境。由游戏引擎编码的场景是一个可执行的世界规范：引擎可以高效检查碰撞、物理、可导航性和可玩性，而开发者则通过判断场景是否应被接受来提供全局验证信号。游戏开发还为强化学习后提供现实世界的长期轨迹数据。因此，我们提出了带人机验证的强化学习（RLHEV），这是一种训练后范式，结合了密集的引擎信号与开发过程中隐含的人类接受反馈。

ClueWeaver: Reward-Guided Dual-Agent Evidence Reasoning for Compact LLMs on Literary Long Narratives

ClueWeaver：基于文学长篇叙事的精简大型语言模型的奖励引导双代理证据推理

Authors: Jihao Zhu, Zhiwei Yang, Wenxiao Zhang, Junqian Zhao, Qi You, Fangqi Wang, Zheyuan Deng, Hanzhe Yang, Yu Liu, Jin B. Hong
Subjects: Subjects: Computation and Language (cs.CL)
Arxiv link: https://arxiv.org/abs/2608.25531
Pdf link: https://arxiv.org/pdf/2608.25531
Abstract Humanities and social science research requires close reading of long narrative materials such as novels, scripts, archives, and case reports, yet many users have limited access to costly proprietary long-context models. Compact, locally deployable language models are a practical alternative, but directly feeding them an entire long context remains costly, hard to inspect, and prone to missing sparse evidence. We present ClueWeaver, an evidence-aware dual-agent framework for long-narrative question answering with compact local models. A Finder identifies passages containing answer-critical clues through retrieval-guided segmentation, while an Interpreter derives the answer from the selected evidence, produces rationales with paragraph-ID citations, and applies an internal self-calibration pass for high-risk questions. Both agents are optimized with reward-guided reinforcement learning: Finder rewards emphasize evidence retention and faithful paragraph-ID references, and Interpreter rewards emphasize correctness, grounding, and concise explanations. This decomposition makes evidence selection and reasoning more inspectable than end-to-end prompting. Experiments across multiple long-context narrative question answering and claim verification settings show that ClueWeaver substantially improves local end-to-end language models while providing evidence coverage and paragraph-referenced reasoning traces. Code is available at this https URL.
中文摘要 人文和社会科学研究需要仔细阅读长篇叙事材料，如小说、剧本、档案和案例报告，但许多用户对昂贵的专有长语境模型的访问有限。紧凑、可本地部署的语言模型是一个实用的替代方案，但直接向它们提供完整的长上下文仍然成本高昂、难以检查，且容易遗漏稀疏的证据。我们介绍ClueWeaver，一种基于证据感知的双代理框架，用于长叙述性问答，采用紧凑的局部模型。查找器通过检索引导分割识别包含关键答案线索的段落，而解释者则从所选证据中推导答案，生成带有段落ID引用的理由，并对高风险问题进行内部自我校准。两者均采用奖励引导强化学习优化：发现者奖励强调证据保留和忠实的段落ID引用，解释者奖励强调正确性、扎根性和简洁解释。这种分解使得证据的选择和推理比端到端的提示更易于检查。在多个长上下文叙述问答和主张验证环境中的实验表明，ClueWeaver 在提供证据覆盖和段落引用推理痕迹的同时，显著提升了本地端到端语言模型。代码可在此 https URL 访问。

AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research

AdaVDR：视频深度研究中的自适应工具使用与反思

Authors: Xintong Zhang, Xiaomeng Fan, Shilin Yan, Ekko He, Zicheng Liu, Zijian Zou, Guannan Zhang, Yuwei Wu, Zhi Gao, Hongwei Xue
Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Arxiv link: https://arxiv.org/abs/2608.25559
Pdf link: https://arxiv.org/pdf/2608.25559
Abstract Video deep research answers complex questions by jointly understanding video content and retrieving external knowledge from the open Web. However, diverse questions and videos require different tool-use strategies, and inappropriate tool calls can produce incorrect results. Uncertain grounding and retrieval also make unnecessary interactions costly and error-prone, increasing latency and reasoning errors. To address these challenges, we propose AdaVDR, an adaptive video deep research agent with adaptive tool invocation and reflection. AdaVDR selects tools according to the task and its capabilities, and backtracks only when unreliable intermediate results require correction. To enable these capabilities, we develop a video deep research data construction pipeline. We first discover retrieval-relevant events and entities in diverse videos and acquire detailed information through grounding and external retrieval to construct high-quality QA pairs. For each QA, task-specific prompts organize the information acquisition process into a tool-use trajectory, allowing different question and video types to follow different grounding and retrieval strategies. We further introduce model-conditioned tool necessity filtering, which evaluates tool calls against the target model's video understanding and internal knowledge, removing tools or tool chains the model can bypass. This yields trajectories tailored to the target model's video understanding capability and knowledge. Using this pipeline, we construct training data and VDR-EE, a benchmark covering entity-centric and event-centric questions. We perform supervised fine-tuning followed by reinforcement learning with a redundancy-aware reward to strengthen adaptive tool invocation and reflection. Experiments show that our method performs best among the evaluated open-source models on VDR-EE and substantially improves over its base models on VideoDR.
中文摘要 视频深度研究通过共同理解视频内容并从开放网络获取外部知识，回答复杂的问题。然而，不同问题和视频需要不同的工具使用策略，不当的工具调用可能导致错误的结果。不确定的接地和检索也使不必要的交互成本高昂且易出错，增加延迟和推理错误。为应对这些挑战，我们提出了AdaVDR，一款自适应视频深度研究代理，具备自适应工具调用和反思功能。AdaVDR根据任务及其能力选择工具，只有在需要纠正的不可靠中间结果时才会回溯。为实现这些能力，我们开发了视频深度研究数据构建流程。我们首先在多样化的视频中发现与检索相关的事件和实体，并通过接地和外部检索获取详细信息，构建高质量的质量保证对。对于每个质量保证，针对任务的提示将信息获取过程组织成工具使用轨迹，使不同类型的问题和视频能够遵循不同的基础和检索策略。我们还进一步引入了模型条件工具必要性过滤，它将工具调用与目标模型的视频理解和内部知识进行评估，去除模型可绕过的工具或工具链。这会生成符合目标模型视频理解能力和知识的轨迹。利用该流程，我们构建了训练数据和VDR-EE，这是一个涵盖实体中心和事件中心问题的基准测试。我们先进行监督微调，随后进行带有冗余意识奖励的强化学习，以强化自适应工具调用和反思能力。实验显示，我们的方法在VDR-EE上被评估的开源模型中表现最佳，且相较VideoDR基础模型有显著提升。

V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning

V评分标准：通过基于评分标准的强化学习实现视觉忠实

Authors: Shulin Tian, Minglun Li, Yuhao Dong, Hao Ding, Jiarui Yao, Haiwen Diao, Jingkang Yang, Hongyuan Zhu, Ziwei Liu
Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Arxiv link: https://arxiv.org/abs/2608.25580
Pdf link: https://arxiv.org/pdf/2608.25580
Abstract Vision-language models can produce fluent answers that are insufficiently grounded in the visual evidence: a single unsupported object, chart value, or intermediate inference can undermine an otherwise plausible response. We argue that this is a credit-assignment failure in multimodal post-training. Scalar outcome rewards indicate whether an answer is acceptable, but do not identify which visual facts are grounded, which reasoning steps are valid, or which instruction constraints are missed. We introduce Visual Rubrics-Based Reinforcement Learning, which decomposes reference responses into atomic propositions and scores generated answers along Visual Faithfulness (VF), Reasoning Consistency (RC), and Instruction Following (IF). The resulting rubric items provide structured partial credit and localize rubric credit when supporting evidence spans are available. We first obtain an SFT checkpoint by fine-tuning Qwen3-VL-8B-Instruct on the public OpenMMReasoner-SFT-874K corpus, adapting OpenMMReasoner's cold-start data recipe. We construct V-Rubrics 50K, a 50,248-example training set from 17 visually grounded sources, by applying rule-based filters before deriving example difficulty from rejection-sampling scores and then annotating every example with Gemini-3-Pro under the same structured prompt and protocol. We train our model based on the same SFT checkpoint using component-wise, prefix-localized rubric credit. Experiments show that our rubricbased GRPO improves over both the shared SFT baseline and answer-only GRPO, with the largest gains on knowledge-oriented and visually grounded reasoning benchmarks. The results show rubrics as a useful reward abstraction for visual post-training.
中文摘要 视觉语言模型可能产生流畅的回答，但这些回答缺乏视觉证据的基础：单个无依据的对象、图表值或中间推断可能破坏本应合理的回答。我们认为这是多模态后培训中的学分分配失败。标量结果奖励表明答案是否可接受，但未指出哪些视觉事实是有根据的，哪些推理步骤有效，哪些指令约束被遗漏。我们引入基于视觉评分标准的强化学习，将参考响应分解为原子命题，并根据视觉忠实度（VF）、推理一致性（RC）和指令跟随（IF）对生成的答案进行评分。由此产生的评分标准项目提供结构化的部分学分，并在有支持证据时对评分标准进行本地化。我们首先通过微调公开 OpenMMReasoner-SFT-874K 语料库中的 Qwen3-VL-8B-Ininstruction，并采用 OpenMMReasoner 的冷启动数据配方，获得 SFT 检查点。我们构建了V-Rubrics 50K，这是一套由17个视觉基础来源组成的50,248个示例训练集，方法是先从拒绝抽样分数中应用基于规则的过滤器，再根据拒绝抽样分数推导示例难度，然后在同一结构化提示和协议下用Gemini-3-Pro注释每个示例。我们基于同一SFT检查点，使用按组件、前缀局部化的评分标准来训练模型。实验显示，基于评分标准的GRPO在共享SFT基线和仅答案GRPO上均有进步，在知识导向和视觉基础推理基准测试上取得最大提升。结果显示，评分标准作为视觉训练后奖励抽象的有用工具。

AutoVerifier: Residual-Guided Non-Parametric Optimization for Reference-Based Answer Verification

AutoVerifier：基于参考的答案验证的残差引导非参数优化

Authors: Zebei Zhao, Zhihao Shi, Minqi Shi
Subjects: Subjects: Computation and Language (cs.CL)
Arxiv link: https://arxiv.org/abs/2608.25637
Pdf link: https://arxiv.org/pdf/2608.25637
Abstract Reference-based verifiers are important for evaluating reasoning models and providing accurate outcome rewards in reinforcement learning with verifiable rewards. To improve verification accuracy, prior work has explored rule-based, model-based, and tool-augmented verifiers for checking answer equivalence across diverse answer forms. However, the equivalence of answer forms such as $1+3.14$ and $1+\pi$ may depend on the question and scoring criterion. We frame such implicit assumptions as verifier inductive biases. To address this challenge, we propose AutoVerifier, a residual-guided non-parametric optimization method that learns these biases from recurring verifier errors. Specifically, AutoVerifier records these biases in rule cards and promotes them to code modules or prompt guidance only after replay validation detects no direct regressions, keeping accepted updates auditable, editable, and reusable. Experiments on four verifier benchmarks demonstrate that AutoVerifier outperforms state-of-the-art verifiers by a large margin.
中文摘要 基于引用的验证器对于评估推理模型和提供可验证奖励的准确结果奖励非常重要。为提高验证准确性，先前研究过基于规则、基于模型和工具增强的验证器，用于检查不同答案表格间的答案等价性。然而，答案形式如$1+3.14$和$1+\pi$的等价性可能取决于问题和评分标准。我们将此类隐含假设框架为验证者归纳偏差。为应对这一挑战，我们提出了AutoVerifier，这是一种残差引导的非参数优化方法，能够从验证器反复出现的错误中学习这些偏差。具体来说，AutoVerifier 在规则卡中记录这些偏差，只有在重放验证检测到无直接回归后，才将其推广为代码模块或提示指导，保持可接受的更新可审计、可编辑和可重用。四个验证器基准测试的实验表明，AutoVerifier 远远优于最先进的验证器。

TailSFT: Filtered Fine-Tuning Improves Post-Training Performance

TailSFT：过滤微调提升训练后表现

Authors: Sadhika Malladi, Samy Jelassi, Dylan Foster, Jordan T. Ash, Akshay Krishnamurthy
Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Arxiv link: https://arxiv.org/abs/2608.25756
Pdf link: https://arxiv.org/pdf/2608.25756
Abstract Reinforcement learning post-training drives reasoning and agentic capabilities in modern AI systems, yet a growing body of work shows that it is most effective when used to fine-tune an already capable base model. We question whether existing pipelines yield models that are most suitable for reinforcement learning. Building on prior work highlighting the role of coverage and pass@K as predictors of post-RL performance, we design a simple modification to supervised fine-tuning, TailSFT, which filters out already fit sequences during training, thereby focusing learning on under-modeled regions, or the tail, of the data distribution. We justify and validate the design choices in TailSFT, particularly the specific filtering criteria, through a combination of controlled experiments and theoretical analysis. On OLMo-3 7B, TailSFT often improves pass@16 performance on math and coding evaluations, with gains up to 17% absolute, while incurring minimal computational overhead. These higher-coverage checkpoints consistently translate to up to 4% absolute pass@1 gains in subsequent GRPO runs, demonstrating that TailSFT checkpoints serve as better initializations for RL. We further introduce a lightweight diagnostic for identifying settings where TailSFT is most likely to help. More broadly, our results motivate a principled, stage-aware approach to model development, in which intermediate checkpoints are judged by how effectively they support subsequent training.
中文摘要 训练后强化学习推动了现代人工智能系统的推理和代理能力，但越来越多的研究表明，当它用于微调已经具备能力的基础模型时效果最佳。我们质疑现有的流水线是否产生了最适合强化学习的模型。基于之前强调覆盖和pass@K作为后强化学习表现预测因素的作用的工作，我们设计了对监督微调的简单修改——TailSFT，该技术在训练过程中过滤掉已拟合序列，从而将学习重点放在数据分布的欠模区域或尾部。我们通过受控实验和理论分析相结合，验证并验证了TailSFT中的设计选择，特别是具体的过滤标准。在OLMo-3 7B上，TailSFT常常提升数学和编程评估的pass@16性能，绝对提升最高可达17%，同时计算开销极低。这些高覆盖度检查点在后续GRPO运行中持续转化为最高4%的绝对 pass@1 收益，表明 TailSFT 检查点是更优的强化初始化。我们还引入了轻量级诊断工具，用于识别TailSFT最有可能帮助的环境。更广泛地说，我们的结果推动了一种有原则、阶段意识的模型开发方法，通过中间检查点对后续培训的支持程度来评判。

Cooperative Multi-Agent Reinforcement Learning for Adaptive Aggregation in Semi-Supervised Federated Learning with non-IID Data

半监督联合学习中非IID数据中的合作多智能体强化学习，用于自适应聚合

Authors: Rene Glitza, Luca Becker, Rainer Martin
Subjects: Subjects: Machine Learning (cs.LG); Distributed, Parallel, and Cluster Computing (cs.DC); Sound (cs.SD); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)
Arxiv link: https://arxiv.org/abs/2608.25794
Pdf link: https://arxiv.org/pdf/2608.25794
Abstract Federated Learning (FL) enables distributed training of machine learning models while preserving data privacy. However, FL struggles with heterogeneous, non-IID client data distributions, resulting in sub-optimal and biased global models. In this paper, we propose pFedMARL, a novel approach leveraging Multi-Agent Reinforcement Learning (MARL) with Twin Delayed Deep Deterministic Policy Gradient (TD3) to dynamically adapt aggregation strategies in FL settings. Our method employs a server-side agent adjusting client contributions to optimize global model robustness and client-side agents balancing global and local updates to personalize models effectively without pre-training. We demonstrate superior performance of pFedMARL for training a semi-supervised audio spectrogram transformer, matching or outperforming FedAvg, Ditto, and local training approaches across multiple non-IID scenarios and in the presence of adversarial clients. Our results indicate that pFedMARL actively improves accuracy, robustness, and fairness, making it suitable for real-world deployments.
中文摘要 联邦学习（FL）实现了机器学习模型的分布式训练，同时保护数据隐私。然而，FL在异构、非IID客户端数据分布方面存在困难，导致全球模型不够优且存在偏见。本文提出了pFedMARL，一种利用多智能体强化学习（MARL）结合双延迟深度确定性策略梯度（TD3）的创新方法，在FL环境中动态调整聚合策略。我们的方法采用服务器端代理调整客户端贡献以优化全局模型的稳健性，客户端代理则平衡全局和本地更新，以有效个性化模型，无需预训练。我们展示了pFedMARL在训练半监督音频频谱变压器时的优异表现，在多种非IID场景及对抗客户面前，能够匹配甚至超过FedAvg、Ditto和本地训练方法。我们的结果表明，pFedMARL积极提升了准确性、鲁棒性和公平性，使其适合实际部署。

Answer Is Cheap, Show Me the Evidence! Augmenting Automated Vulnerability Assessment with Evidence

答案很廉价，给我看证据！以证据增强自动化脆弱性评估

Authors: Shengyi Pan, Zelong Zheng, Jiayuan Zhou, Xing Hu, Xin Xia, Shanping Li
Subjects: Subjects: Software Engineering (cs.SE)
Arxiv link: https://arxiv.org/abs/2608.25905
Pdf link: https://arxiv.org/pdf/2608.25905
Abstract Software vulnerability (SV) assessment helps prioritize remediation by characterizing reported vulnerabilities. Existing automated methods predict assessment results from SV reports (SVRs), but often overlook information in rich text, such as screenshots and code snippets, as well as contextual information about vulnerable projects. They also focus on prediction accuracy without providing explanations or supporting evidence, limiting their practical use when analysts must validate imperfect predictions. We propose EAVA, a framework that uses large language models (LLMs) to assess SVs and provide supporting evidence. EAVA employs specialized LLM agents to process rich-text content and project information, and builds a dedicated assessment model through a two-stage training pipeline. It first uses supervised instruction tuning on automatically annotated reasoning trajectories to inject domain knowledge, and then applies reinforcement learning to improve intrinsic reasoning. EAVA also retrieves similar historical vulnerabilities as supplementary evidence. Experiments on a newly collected SVR dataset show that EAVA outperforms the strongest baseline by 5.3 to 35.2 percent across multiple metrics. Ablation studies confirm the effectiveness of assessment-specific model training and information enrichment. A user study with security experts further demonstrates that the evidence provided by EAVA is useful and practical for real-world SV assessment.
中文摘要 软件漏洞（SV）评估通过描述报告的漏洞，帮助优先处理修复工作。现有的自动化方法能够预测SV报告（SVR）中的评估结果，但常常忽略了富文本中的信息，如截图和代码片段，以及关于易受攻击项目的上下文信息。它们还专注于预测准确性，却不提供解释或支持证据，限制了分析师在验证不完美预测时的实际应用。我们提出了EAVA框架，利用大型语言模型（LLMs）评估SV并提供支持证据。EAVA使用专业的LLM代理处理富文本内容和项目信息，并通过两阶段培训流程构建专用评估模型。它首先在自动注释的推理轨迹上使用监督式指令调优来注入领域知识，然后应用强化学习来提升内在推理能力。EAVA还检索了类似的历史漏洞作为补充证据。对新收集的SVR数据集的实验显示，EAVA在多个指标上优于最强基线5.3%至35.2%。消融研究证实了评估特异性模型训练和信息丰富的有效性。一项与安全专家合作的用户研究进一步表明，EAVA提供的证据对于现实世界SV评估具有实用性和实用性。

One Symptom, Three Levers: A Critical Review of On-Policy Self-Distillation

一个症状，三把杠杆：政策自我提炼的批判性回顾

Authors: Justin Robert, Raheel Qader
Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Arxiv link: https://arxiv.org/abs/2608.25936
Pdf link: https://arxiv.org/pdf/2608.25936
Abstract On-policy distillation trains a language model on its own generations while a teacher scores them token by token. It combines the dense supervision of imitation learning with the on-policy sampling of reinforcement learning. But it requires a second, larger model to act as teacher. On-Policy Self-Distillation (OPSD) removes that cost. The teacher is the model itself, conditioned on privileged information the student will not have at test time, such as a reference solution, a plan, or environment feedback. The teacher is no stronger than the student, only better informed. Early results were promising, with accuracy comparable to reinforcement learning at a fraction of the generated tokens. But the same asymmetry that produces the signal also biases it. One failure mode now dominates the field: collapse, the progressive narrowing of the set of reasoning paths the model can produce. Collapse is not specific to OPSD, though privileged information aggravates it. This review treats collapse as a symptom governed by three levers: (i) where the signal is applied, that is, how tokens are weighted; (ii) what the teacher is shown, that is, the nature of the privileged information; and (iii) when the signal changes, that is, the teacher's dynamics and the decay of guidance. We restrict our scope to mathematical reasoning, where the method originated and where its failure modes are best documented. We report no new experiments. The contribution is structural: a shared vocabulary for phenomena named differently across papers, and a clear line between what is settled and what is still disputed.
中文摘要 政策提炼是在语言模型自身的代际上训练，而教师则按代币给模型评分。它结合了模仿学习的密集监督和强化学习的策略抽样。但这需要第二个更大型号来作为教师。政策上自我蒸馏（OPSD）消除了这一成本。教师本身就是模型，基于学生在考试时不会拥有的特权信息，如参考解答、计划或环境反馈。老师并不比学生强，只是信息更充分。早期结果令人鼓舞，准确度可与生成代币的一小部分强化学习相当。但产生信号的同样不对称性也会影响信号。目前，一种失败模式主导了整个领域：崩溃，即模型能够产生的推理路径的逐步缩小。崩溃并非OPSD独有，尽管特权信息会加剧这种情况。本综述将崩溃视为由三个杠杆控制的症状：（i）信号的应用位置，即代币权重的分配;（ii）教师所见内容，即特权信息的性质;以及（iii）信号发生变化，即教师动态变化和引导衰减。我们将研究范围限制在数学推理，也就是方法起源及失效模式最有记录的地方。我们报告没有新实验。这种贡献是结构性的：为不同论文中不同命名的现象提供了共享词汇，并且明确界定了已定论和仍有争议的界限。

VISA: Agentic Self-Evolving Data Synthesis for Multimodal Instruction Following

VISA：多模态教学跟踪的代理自我演化数据综合

Authors: Min Zeng, Guanxin Tan, Libin Cen, Yawei Wen, Rui Hu, Liuyang Bian, Xiaolong Chen, Xiaoxin Chen
Subjects: Subjects: Computation and Language (cs.CL)
Arxiv link: https://arxiv.org/abs/2608.26013
Pdf link: https://arxiv.org/pdf/2608.26013
Abstract Multimodal instruction-following models require training data that is accurate, diverse, verifiable, and challenging. Existing synthesis pipelines typically follow a one-pass generate-and-filter paradigm, discarding feedback from failed samples, verifier outcomes, and target-model errors. We present VISA (Visual Instruction Synthesis Agent), an agentic framework that reformulates multimodal instruction synthesis as a self-evolving loop. At each round, VISA analyzes an image to filter incompatible constraints and discover new verifiable ones, samples diversity- and difficulty-aware constraint sets from persistent memory, generates candidate instructions, and verifies the resulting samples with executable tools and structured large language model judges. Failed samples trigger diagnostic-guided recovery, while accepted samples are probed against the target model to estimate difficulty. The resulting verifier signals and target-model failure profiles are written back to memory, allowing subsequent rounds to adaptively expand the constraint space, reduce template repetition, and focus on unresolved model weaknesses. The same verifier contracts further provide reward signals for reinforcement learning without a separately trained reward model. Experiments on MM-IFEval show that VISA consistently improves multimodal instruction following over strong baselines, while preserving general multimodal capability across seven public benchmarks.
中文摘要 多模态指令跟随模型需要准确、多样化、可验证且具有挑战性的训练数据。现有的综合流程通常采用单遍生成和过滤的范式，丢弃来自失败样本、验证者结果和目标模型错误的反馈。我们介绍了VISA（可视化指令合成代理），这是一个能动框架，将多模态指令合成重新表述为一个自我演化的循环。在每一轮，VISA分析图像以过滤不兼容的约束并发现新的可验证约束，从持久内存中抽样多样性和难度感知约束集，生成候选指令，并使用可执行工具和结构化大型语言模型评判验证所得样本。失败的样本触发诊断引导恢复，而接受的样本则通过目标模型进行探测以估算难度。由此产生的验证信号和目标模型失效配置文件会被写回内存，使后续轮次能够自适应扩展约束空间，减少模板重复，并聚焦于未解决的模型弱点。相同的验证合同还在没有单独训练的奖励模型的情况下，为强化学习提供奖励信号。MM-IFEval实验显示，VISA在强基线条件下持续提升多模态指令跟踪能力，同时保持七个公共基准的通用多模态能力。

Prefix Sliding for efficient test-time scaling

前缀滑动以实现高效的测试时间缩放

Authors: Niklas Muennighoff, Zhengyang Wang, Zeyi Chen, Weijia Shi, Binyuan Hui, John Yang, Dapeng Jiang, Mika Senghaas, Fares Obeid, Johannes Hagemann, Sami Jaghouar, Ludwig Schmidt, Percy Liang, Jason Wei, Andrew Y. Ng, Luke Zettlemoyer, Yejin Choi, Mike Lewis
Subjects: Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Arxiv link: https://arxiv.org/abs/2608.26070
Pdf link: https://arxiv.org/pdf/2608.26070
Abstract Test-time scaling uses extra test-time compute to improve performance, such as letting language models reason longer when solving a problem. As models keep the entire reasoning trace in memory via full attention, hard tasks that need long thinking can be prohibitively expensive. However, we find most intermediate reasoning tokens lose importance as the model continues reasoning. This calls into question whether retaining them is worth the cost. Based on this insight, we propose Prefix Sliding, which discards tokens during reasoning that are not part of the prefix or the window of the last few thousand tokens. The prefix has key instructions and tools available to the model, while the most recent tokens are the current reasoning the model is working on. This caps the total memory requirement regardless of how long the model reasons, allowing for efficient long-horizon test-time scaling. Without training, Prefix Sliding can make existing models 3x faster while maintaining performance. Training with Prefix Sliding using reinforcement learning can achieve better performance by enabling scaling to reasoning traces beyond a hundred thousand tokens. Ablations show Prefix Sliding outperforms summarizing intermediate tokens or vanilla sliding window. Our code is at this https URL
中文摘要 测试时间缩放利用额外的测试时间计算来提升性能，比如让语言模型在解决问题时可以更长时间地推理。由于模型通过全注意力将整个推理轨迹保存在记忆中，需要长时间思考的艰难任务成本可能高得令人望而却步。然而，我们发现随着模型继续推理，大多数中间推理代币的重要性会逐渐降低。这也让人质疑保留这些人是否值得付出这样的代价。基于这一见解，我们提出了前缀滑动，即在推理过程中丢弃不属于前缀或最近几千个令牌窗口的标记。前缀包含模型可用的关键指令和工具，而最新的标记则是模型当前正在研究的推理。无论模型推理多长，这都限制了总内存需求，从而实现高效的长视野测试时间缩放。无需培训，前缀滑动可以让现有模型速度提升3倍，同时保持性能。使用前缀滑动训练，利用强化学习实现推理轨迹超过十万个，从而实现更好的性能。消解显示，前缀滑动在总结中间代币或普通滑动窗口时表现优于。我们的代码是这个 https URL

VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning

VBVR-Pro：一套可扩展且可验证的原生视觉推理套件

Authors: Junxiang Xu, Ruisi Wang, Fanyi Pu, Maijunxian Wang, Ran Ji, Tongxi Zhou, Chenyang Gu, Jing Zuo, Hongcan Xiao, Yimeng Geng, Wanqi Yin, Wei Chen, Oscar Qian, Zhengan Yan, Ziqi Huang, Haiwen Diao, Liang Pan, Bo Li, Xiangyu Fan, Dezhi Luo, Fengyuan Yu, Zehong Zhao, Qingying Gao, Tinghui Zhu, Yilan Zhang, Jingqi Tong, Pinyuan Feng, Zhengze Jiang, Letian Wang, Ziyu Guo, Renrui Zhang, Jieneng Chen, Sonia Joseph, Constantin Venhoff, Saman Motamed, Mengyue Yang, Chandra Sripada, Alan Yuille, Philip Torr, Lvmin Zhang, Vikash Kumar, Daniel Khashabi, Nikolaus Kriegeskorte, Raphaël Millière, Vincent C. Müller, Anyi Rao, Quan Wang, Ziwei Liu, Dahua Lin, Lei Yang, Hokin Deng, Zhongang Cai
Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Multimedia (cs.MM); Robotics (cs.RO)
Arxiv link: https://arxiv.org/abs/2608.26105
Pdf link: https://arxiv.org/pdf/2608.26105
Abstract Native visual reasoning treats visual generation as the medium of reasoning itself: visual states (i.e. images and videos) are not merely inputs to be understood or outputs to be rendered, but first-class substrates for problem solving beyond language. Yet progress remains bottlenecked by the lack of scalable training tasks, reliable feedback, and controlled comparisons across generative substrates. In this work, we introduce VBVR-Pro, a closed-loop testbed that makes native visual reasoning through generation trainable, verifiable, optimizable, and experimentally controllable. 1) Task scaling. VBVR-Pro turns visual reasoning into a controlled task space of 300 procedurally generated tasks. Models trained on VBVR-Pro show strong transfer beyond the proposed suite across seven external visual reasoning benchmarks such as RISE-Video, MME-CoF-Pro, and BabyVision. 2) Verifiable rewards. VBVR-Pro provides verifiable reward scorers for task-grounded evaluation. Through a systematic study of leading MLLMs as judges, we identify recurring failure modes of the prevalent VLM-as-a-judge paradigm. In contrast, the proposed scorers are grounded in deterministic, task-specific rules, achieve fine-grained alignment with human judgments. Importantly, they serve as reliable reward signals for large-scale multi-task reinforcement learning and demonstrate stronger post-RL performance across visual reasoning tasks. 3) Mechanism study. VBVR-Pro enables controlled modality studies across more than 30 image, video, and interleaved generators. Our analysis shows that video generation remains strongest for tasks requiring persistent spatiotemporal state tracking, while interleaved generation provides a compute-efficient alternative. Critically, ablations and probing suggest the presence of vision-native trajectories that are crucial to visual reasoning. We release all data, models, scorers, and code.
中文摘要 原生视觉推理将视觉生成视为推理的媒介：视觉状态（即图像和视频）不仅仅是需要理解的输入或需要渲染的输出，而是超越语言解决问题的一流基础。然而，进展仍受限于可扩展的训练任务、可靠反馈以及生成式基底间的受控比较。在本研究中，我们介绍了VBVR-Pro，一种闭环测试平台，使生成过程中的原生视觉推理可训练、可验证、可优化且可实验控制。1）任务扩展。VBVR-Pro 将视觉推理转化为一个由 300 个程序生成任务组成的受控任务空间。在VBVR-Pro上训练的模型在RISE-Video、MME-CoF-Pro和BabyVision等七个外部视觉推理基准测试中展现出强劲的迁移能力。2）可验证的奖励。VBVR-Pro 提供可验证的奖励评分器，用于基于任务的评估。通过系统研究主要MLLM作为法官，我们识别出盛行的VLM即法官范式中反复出现的失败模式。相比之下，拟议的评分器基于确定性、任务特定规则，实现与人类判断的细致对齐。更重要的是，它们作为大规模多任务强化学习的可靠奖励信号，并在视觉推理任务中展现了更强的强化后表现。3）机制研究。VBVR-Pro支持在30多种图像、视频和交错生成器上进行受控模态研究。我们的分析显示，视频生成在需要持续时空状态跟踪的任务中依然最强，而交错生成则提供了一种计算效率高的替代方案。关键的是，消融和探测暗示了视觉本源轨迹的存在，这对视觉推理至关重要。我们发布所有数据、模型、评分器和代码。

Keyword: diffusion policy

VISTA: Visually Inferred Spatial ConTact Attention for Contact-Rich Manipulation

VISTA：视觉推断空间接触感知，用于丰富接触操作

Authors: Jiayi Chen, Wenlong Dong, Yan Huang, Xianglin Chen, Zijian Lin, Jiaqi Yin, Yushan Liu, Wenbo Ding
Subjects: Subjects: Robotics (cs.RO)
Arxiv link: https://arxiv.org/abs/2608.25872
Pdf link: https://arxiv.org/pdf/2608.25872
Abstract Contact-rich manipulation requires precise interaction feedback. While vision-centric imitation learning is prevalent, external visual observations provide indirect and ambiguous cues about contact states, particularly under occlusion or subtle object--gripper interactions; dedicated tactile or force sensors can provide rich contact information but introduce additional hardware complexity, calibration requirements, and deployment costs. To bridge this gap, we propose VISTA-Policy, an imitation learning paradigm that utilizes the Visual Deformation Field (VDF), a 3D displacement representation of a compliant gripper, as high-dimensional visuo-physical feedback. The framework integrates: 1) a Physics-Aware Encoding Engine for real-time VDF decoding; 2) an Energy Aggregation Denoising Mechanism to isolate true interaction signals; and 3) a Deformation-Augmented Policy Network with incremental gripper actions for precise closed-loop correction. Extensive evaluations on Cross-Scale Object Grasping, Cap Unscrewing, and Calligraphy Writing demonstrate that VISTA-Policy outperforms the strong pure-vision baseline 3D Diffusion Policy and the tactile baseline. VISTA-Policy further demonstrates substantial out-of-distribution generalization to unseen object scales and robustness against dynamic disturbances, offering a durable and cost-effective route toward general-purpose fine-grained manipulation in unstructured environments. Project videos and supplementary materials are available at: this https URL.
中文摘要 接触丰富的操作需要精确的交互反馈。虽然以视觉为中心的模仿学习普遍存在，但外部视觉观察提供了间接且模糊的接触状态线索，尤其是在遮挡或微妙物体-握手交互下;专用的触觉或力传感器可以提供丰富的接触信息，但会带来额外的硬件复杂度、校准要求和部署成本。为弥合这一空白，我们提出了VISTA-Policy，一种模仿学习范式，利用视觉变形场（VDF）——一种顺应抓钳的三维位移表示——作为高维视觉物理反馈。该框架集成了：1）用于实时VDF解码的物理感知编码引擎;2）一种能量聚合去噪机制，用于分离真实的相互作用信号;以及3）具有增量抓力作用的变形增强政策网络，实现精确闭环修正。对跨尺度物体抓取、帽盖拧开和书法书写的广泛评估表明，VISTA-Policy 优于强的纯视觉基线 3D 扩散策略和触觉基线。VISTA-Policy 进一步展示了对看不见对象尺度的显著非分布推广能力，以及对动态扰动的鲁棒性，为非结构化环境中实现通用细粒度操作提供了持久且经济的途径。项目视频和补充材料可在以下网址获取：https URL。