生成时间: 2026-08-21 16:40:57 (UTC+8); Arxiv 发布时间: 2026-08-21 20:00 EDT (2026-08-22 08:00 UTC+8)

今天共有 21 篇相关文章

Keyword: reinforcement learning

Learning the Right Abstraction: Neural Reduced Dynamics for Complex Robot Control

学习正确的抽象：复杂机器人控制的神经约化动力学

• Authors: Harry Zhang, Dan Negrut
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2608.19375
• Pdf link: https://arxiv.org/pdf/2608.19375
• Abstract High-fidelity embodied AI simulators provide realistic evaluation of complex robotic systems, but their computational cost limits their direct use for large-scale reinforcement learning campaigns. We advocate the use of less accurate but more expeditious simulations, which might draw on data-driven, e.g., neural dynamics, models. This contribution argues that the practical value of a neural dynamics model for complex robot control lies in learning the \emph{right abstraction}: a reduced state that preserves the control-relevant physics of the high-fidelity system while enabling high-throughput policy learning. We develop a neural reduced dynamics (NRD) framework that separates the state the model propagates from what can be supplied as an input or recovered analytically, trains policies entirely inside the frozen learned model, and validates them back in the high-fidelity simulator. Two case studies instantiate it across three control tasks: terrain-aware HMMWV trajectory tracking on rigid, bumpy and deformable Continuum Representation Model (CRM) terrain; and goal reaching for a stock tracked vehicle and its front-mounted articulated arm. Every policy transfers back to the high-fidelity simulator. A single policy trained inside the terrain-conditioned dynamics model, and given no terrain input of its own, attains lower median and mean tracking error than both single-terrain specialists on all three terrains, including zero-shot bumpy terrain. Quantitatively, the tracked vehicle reaches 100 of 100 goals and the arm 97 of 100, with zero contacts or joint-limit violations. The NRD models advance roughly four orders of magnitude faster in simulated time than the high-fidelity simulator scenes they replace, making iterative on-policy learning practical and supporting neural reduced dynamics as a bridge between accurate but expensive physics simulation and scalable robot learning.
• 中文摘要 高精度具象人工智能模拟器能够真实评估复杂机器人系统，但其计算成本限制了其直接用于大规模强化学习活动的应用。我们主张使用精度较低但更高效的仿真，这些仿真可能借助数据驱动的，例如神经动力学模型。本贡献认为，神经动力学模型在复杂机器人控制中的实际价值在于学习\emph{右抽象}：一种简化状态，保持高保真系统的控制相关物理，同时实现高通量策略学习。我们开发了一个神经简化动力学（NRD）框架，将模型传播的状态与可作为输入提供或分析恢复的状态分离，完全在冻结学习模型中训练策略，并在高保真模拟器中验证。两个案例研究将其应用于三个控制任务：在刚性、颠簸和可变形的连续体表示模型（CRM）地形上进行地形感知HMMWV轨迹跟踪;以及目标伸手，准备使用原厂履带车辆及其前置铰接臂。每项保单都会回归高保真模拟器。在地形条件动力学模型中训练的单一策略，且自身不输入地形，在三种地形（包括零射击崎岖地形）上，其中位数和平均跟踪误差均值低于单一地形专家。从数量上看，履带车辆达到100个目标中的100个目标，机械臂达到100个目标中的97个目标，且无接触或关节极限违规。NRD模型在模拟时间内的进展速度比它们所替代的高保真模拟场景快约四个数量级，使迭代策略学习变得可行，并支持神经简化动力学，作为准确但昂贵的物理模拟与可扩展机器人学习之间的桥梁。

SSR-GRPO: Integrating Supervision and Semantic IDs into Reinforcement Learning for Dense Retrieval in E-commerce

SSR-GRPO：将监督和语义ID整合进强化学习，实现电子商务中的密集检索

• Authors: Guangxin Song, Xing Fang, Mingmin Jin, Jing Wang, Bokang Wang, Zhentao Song, Junjie Bai, Jianbo Zhu
• Subjects: Subjects: Information Retrieval (cs.IR)
• Arxiv link: https://arxiv.org/abs/2608.19595
• Pdf link: https://arxiv.org/pdf/2608.19595
• Abstract Embedding-based retrieval (EBR) is pivotal in e-commerce search but often struggles with complex semantics. While recent methods often fine-tune large language models (LLMs) for representation learning, they typically lack robust mechanisms for handling complex and implicit semantics. While Retrieval-GRPO (R-GRPO) recently introduced reinforcement learning to dense retrieval, it suffers from noisy top-K candidates due to limited batch sampling and biased relevance assessments caused by using similarly trained LLMs as reward models. To tackle these issues, we propose Supervised Retrieval-GRPO with Semantic Identifiers (SSR-GRPO). Specifically, our method first proposes a dual-perspective framework for relevance assessment. It leverages both Semantic Identifiers (SIDs) produced by quantization learning and dense representation vectors to generate more unbiased relevance scores. Furthermore, leveraging the hierarchical similarity relationships of the generated SIDs, we mine a set of hard negative samples that serve two purposes: (1) to design a masking function integrated into R-GRPO, effectively filtering intra-group noisy samples; and (2) to construct a Retrieval-DPO task composed of positive and negative sample pairs, enabling the model to capture fine-grained semantic distinctions from a pair-wise perspective. By integrating these optimization strategies, we propose SSR-GRPO. Extensive offline and online experiments validate SSR-GRPO's effectiveness, and it has been deployed on a large-scale e-commerce platform.
• 中文摘要 基于嵌入的检索（EBR）在电子商务搜索中至关重要，但常常面临复杂的语义难题。虽然最新方法常常对大型语言模型（LLMs）进行微调以进行表示学习，但它们通常缺乏处理复杂隐式语义的稳健机制。虽然检索-GRPO（R-GRPO）最近引入了强化学习以实现密集检索，但由于批次抽样有限且使用训练相似的大型语言模型作为奖励模型，导致相关性评估偏颇，导致顶K候选数据噪声较大。为解决这些问题，我们提出了带语义标识符的监督检索-GRPO（SSR-GRPO）。具体来说，我们的方法首先提出了一个双视角的相关性评估框架。它利用量化学习产生的语义标识符（SID）和密集的表示向量，生成更无偏的相关性评分。此外，利用生成SID的层级相似关系，我们挖掘了一组硬负样本，具有两个目的：（1）设计集成在R-GRPO中的掩蔽函数，有效过滤组内噪声样本;以及（2）构建由正负样本对组成的检索-DPO任务，使模型能够从成对视角捕捉细致语义区分。通过整合这些优化策略，我们提出了SSR-GRPO方案。广泛的线下线下实验验证了SSR-GRPO的有效性，并已部署在大型电子商务平台上。

Scaffolding Minds: Optimizing Latent Visual Target Representations for Multimodal Reasoning

支架思维：优化潜在视觉目标表征以实现多模态推理

• Authors: Haoqiang Kang, Yinpeng Chen, Luyang Liu, Jesper Sparre Andersen, Abhijit Ogale, Baochen Sun, Lichan Hong, Ed H. Chi
• Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2608.19669
• Pdf link: https://arxiv.org/pdf/2608.19669
• Abstract Latent reasoning has advanced multimodal reasoning through a two-stage training paradigm: (1) a helper image is encoded into latent tokens to teach visual chain-of-thought during a supervised fine-tuning (SFT) stage, and (2) these latent tokens are further refined with reward feedback during a reinforcement learning (RL) stage. In this paper, we identify two key limitations of this framework, one in each stage. First, the SFT stage typically relies on an off-the-shelf vision encoder to encode the helper image, yielding suboptimal latent representations that may not be well aligned with the downstream reasoning task. Second, existing RL methods treat the latent component only through deterministic regularization, which constrains policy drift but does not create alternative latent trajectories for exploration. To address these limitations, we propose Scaffolding Minds. Our approach learns a dedicated scaffolding encoder that provides an optimized target in latent space, and learns both the mean and variance of the RL sampler. We further show that these two improvements are complementary, together yielding substantial gains over strong baselines. Empirically, our method improves over the strongest latent-reasoning baseline by +9.5% on FrozenLake spatial planning, with the gain widening to +19% at 32x32 grid map, and by +5.2% on average across nine visual-centric reasoning benchmarks.
• 中文摘要 潜能推理通过两阶段训练范式推进了多模态推理：（1）辅助图像被编码到潜在标记中，用于在监督微调（SFT）阶段教授视觉思维链;（2）这些潜在标记在强化学习（RL）阶段通过奖励反馈进一步完善。本文指出该框架的两个关键局限性，分别在每个阶段。首先，SFT阶段通常依赖现成的视觉编码器来编码辅助图像，导致潜在表现不佳，可能与后续推理任务不匹配。其次，现有的强化学习方法仅通过确定性正则化处理潜在成分，这限制了政策漂移，但不创造可供探索的替代潜在轨迹。为了解决这些局限性，我们提出了“脚手架式思维”。我们的方法学习一个专用的支架编码器，在潜空间中提供优化目标，并学习强化学习采样器的均值和方差。我们还进一步证明，这两项改进是互补的，共同带来了相较于强基线的显著提升。实证上，我们的方法在FrozenLake空间规划中较最强潜能基线提升+9.5%，在32x32网格地图上增益扩大至+19%，在九个视觉中心推理基准测试中平均提升+5.2%。

RIPE++: Reinforced Keypoint Learning from Positive Pairs Only

RIPE++：仅从正对强化关键点学习

• Authors: Johannes Künzel, Peter Eisert, Anna Hilsmann
• Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2608.19693
• Pdf link: https://arxiv.org/pdf/2608.19693
• Abstract Sparse keypoint extraction and matching underpin core tasks in geometric computer vision, including structure-from-motion, visual SLAM, augmented reality, and medical image registration. Learning robust local feature representations, however, typically requires accurate camera poses or depth supervision, which are often unavailable in real-world settings. Reinforcement learning (RL) has recently emerged as a promising alternative, requiring only the information if two images show the same scene or not. However, existing RL formulations such as RIPE rely on coarse binary rewards and carefully constructed negative training pairs, limiting training stability and descriptor discriminability. In this paper, we revisit RL-based keypoint learning and propose a reward that fully exploits the geometric consistency signal, deriving both reward and penalty from a single positive pair without contrasting against negatives. This richer signal provides sufficient supervisory contrast to learn discriminative detectors and descriptors from positive image pairs alone, enabling representation learning under extremely limited supervision. Furthermore, we show that the same RL objective can be extended to the matching stage by adapting LightGlue, raising AUC@5 on MegaDepth1500 from 56.58 to 59.65 and enabling weakly-supervised training of the full sparse matching pipeline from image pairs with partial visual overlap. We validate our approach on established benchmarks, demonstrating competitive results compared to fully-supervised methods. We further show that the method can be even trained on low texture medical video sequences, where camera poses are usually unavailable and standard SfM pipelines often fail. Code and data are available at this https URL .
• 中文摘要 稀疏关键点提取和匹配是几何计算机视觉的核心任务基础，包括从运动结构、可视化SLAM、增强现实和医学图像注册。然而，学习稳健的局部特征表示通常需要准确的相机姿态或深度监督，而这些在现实环境中往往无法实现。强化学习（RL）最近作为一种有前景的替代方案出现，只需提供两张图像是否显示相同场景的信息。然而，现有的强化学习（RL）表述如RIPE依赖粗糙的二元奖励和精心构建的负训练对，限制了训练稳定性和描述子的可辨别性。本文重新审视基于强化学习的关键点学习，提出了一种完全利用几何一致性信号的奖励，能够从单一正对中推导出奖励和惩罚，而不与负对比。这种更丰富的信号提供了足够的监督对比，使得仅从正图像对中学习判别检测器和描述符，从而实现在极其有限的监督下进行表征学习。此外，我们展示了通过调整LightGlue，将同一RL目标扩展到匹配阶段，将MegaDepth1500的 AUC@5从 56.58 提升至 59.65，并实现对部分视觉重叠图像对的完整稀疏匹配流水线进行弱监督训练。我们基于既定基准验证方法，展示与完全监督方法的竞争效果。我们还进一步证明，该方法甚至可以在低纹理的医疗视频序列上训练，因为在这些场景中，摄像机姿势通常不可用，标准的SfM流水线也常常失效。代码和数据可在此 https 网址获取。

MileGPO: Milestone Inference with Local Evidence for Graph-Based Policy Optimization of Long-Horizon LLM Agents

MileGPO：基于图的策略优化的里程碑推断，支持长视野LLM代理的图化策略优化

• Authors: Bo Qian, Yuting Wu, Shuang Zeng, Huaiyu Wan, Dalin Zhang, Jiqiang Liu
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
• Arxiv link: https://arxiv.org/abs/2608.19803
• Pdf link: https://arxiv.org/pdf/2608.19803
• Abstract Credit assignment is challenging in long-horizon agentic reinforcement learning, where supervision often comes only from final rewards. Existing methods refine trajectory-level signals into step-level credits through step grouping or graph-based advantage estimation, but can overlook meaningful intermediate milestones. We propose MileGPO (Milestone Inference with Local Evidence for Graph-Based Policy Optimization), which derives process-level credit from grouped on-policy rollouts through three designs. Milestone Discovery identifies candidate milestones on successful rollouts and recurring traps on failed ones. Reliability-Calibrated Shaping (RCS) weights these candidates by outcome-based confidence, strengthening reliable milestones and traps while down-weighting uncertain ones. Progress-Contrastive Calibration (PCC) further tests whether a candidate reflects local progress and whether its incoming ansition outperforms observed alternatives from the same this http URL requires neither auxiliary models nor additional environment interaction. Experiments on ALFWorld and WebShop show state-of-the-art performance and a small in-distribution to out-of-distribution gap on ALFWorld. Ablations and credit diagnostics indicate that reliability weighting, local progress, and same-state branch evidence complement milestone discovery and resolve ambiguous intermediate credit.
• 中文摘要 在长期能动强化学习中，学分分配具有挑战性，因为监督往往仅来自最终奖励。现有方法通过步数分组或基于图的优势估计，将轨迹级信号细化为步级积分，但可能忽视有意义的中间里程碑。我们提出了MileGPO（基于图的政策优化的本地证据里程碑推断），通过三种设计从分组的政策上推广中获得流程层级的信用。里程碑发现在成功推广中识别候选里程碑，在失败推广中识别反复出现的陷阱。可靠性校准塑形（RCS）通过基于结果的置信度对候选者进行权重，强化可靠的里程碑和陷阱，同时降低不确定的权重。进展对比校准（PCC）进一步测试候选选物是否反映局部进展，以及其输入的对比是否优于同类的观察到的替代方案。该http网址无需辅助模型或额外的环境交互。ALFWorld和WebShop上的实验显示，ALFWorld上表现最先进，且存在小幅的分发与非分发差距。消融和学分诊断表明，可靠性加权、地方进展和同州分行证据补充了里程碑发现，并解决了模糊的中间学分。

ADAPT: Physics-Aware Diffusion-based World Models for Adaptive Predictive Transferable HVAC Control

ADAPT：基于物理感知扩散的世界模型，用于自适应预测可转移暖通空调控制

• Authors: Xu Yang, Kailai Sun, Dianyu Zhong, Qianchuan Zhao
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2608.19804
• Pdf link: https://arxiv.org/pdf/2608.19804
• Abstract Buildings account for roughly one-third of global energy consumption and CO$_2$ emissions. Optimizing indoor climate systems plays a critical role for urban climate mitigation aligned with UN Sustainable Development Goals 11 and 13. However, indoor delayed thermodynamic responses and partial observability severely hinder existing methods, which are primarily limited by implicit thermal inertia, occupancy dynamic prediction, and cumulative prediction errors, especially for out-of-distribution environments. In practice, these challenges are further exacerbated by the high cost and privacy burden of dense indoor sensing, forcing operators to collect only limited data in a single operating regime while expecting controllers to generalize reliably across unseen seasons and climate regions. To address this problem, we propose ADAPT, a physics-aware conditional diffusion indoor environmental world model for HVAC control. The model predicts a short-horizon held-action thermal baseline to capture the latent thermal inertia of the buildings. The diffusion backbone utilizes the robustness of generative models, while a learnable multi-zone heat-balance regularizer constrains generated trajectories to satisfy transferable building thermodynamics without requiring known building geometry or manually calibrated thermal parameters. A credit assignment is then design for the downstream reinforcement learning. Extensive experiments on SemibuildingSim and Sinergym demonstrate that ADAPT reduces HVAC energy consumption by 7.3\% and occupant discomfort by 30.2\% compared with state-of-the-art baselines under IID control. Under OOD control scenarios spanning unseen seasons and climate regions, ADAPT maintains robust performance with only marginal degradation relative to its IID performance, substantially outperforming existing methods in transfer robustness.
• 中文摘要 建筑物约占全球能源消耗的三分之一，二氧化碳$_2的排放量。优化室内气候系统在符合联合国可持续发展目标11和13的城市气候缓解中起着关键作用。然而，室内延迟热力学响应和部分可观测性严重阻碍了现有方法，这些方法主要受隐性热惯性、占用动态预测和累计预测误差的限制，尤其是在分布外环境下。实际上，密集室内传感的高成本和隐私负担进一步加剧了这些挑战，迫使操作员在单一操作模式下只能收集有限数据，同时期望控制人员能够可靠地跨越未见的季节和气候区域进行泛化。为解决这一问题，我们提出了ADAPT，一种物理感知的条件扩散室内环境世界模型，用于HVAC控制。该模型预测一个短视距的保持作用热基线，以捕捉建筑物的潜在热惯性。扩散主干利用生成模型的鲁棒性，而可学习的多区热平衡正则化器则约束生成轨迹以满足可转移的建筑热力学，而无需已知建筑几何形状或人工校准热参数。然后设计学分作业用于后续强化学习。在SemibuildingSim和Sinergym上的大量实验表明，与IID控制下的先进基线相比，ADAPT能将暖通空调能耗降低7.3%，居住者不适减少30.2%。在跨越未见季节和气候区域的OOD控制场景下，ADAPT保持了稳健的性能，相较于其IID性能仅有边际降解，在转移鲁棒性方面远超现有方法。

Adaptive Probabilistic Shielding by Learning MDPs for Safe Reinforcement Learning

通过学习型MDP实现自适应概率屏蔽，实现安全强化学习

• Authors: Astrid Horn Brorholt (1), Maris F. L. Galesloot (2), Nils Jansen (2) (3), Kim Guldstrand Larsen (1), Christian Schilling (1) ((1) Aalborg University, Aalborg, Denmark (2) Radboud University, Nijmegen, Netherlands (3) Ruhr University Bochum, Bochum, Germany)
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Logic in Computer Science (cs.LO)
• Arxiv link: https://arxiv.org/abs/2608.19836
• Pdf link: https://arxiv.org/pdf/2608.19836
• Abstract Probabilistic shielding is a technique for safe reinforcement learning (RL). Typically, a static observer -- called the shield -- constrains the learning agent's actions to those for which acting safely remains feasible. Traditionally, the shield is computed from the transition probabilities of the underlying Markov decision process (MDP). Thus, this technique is not applicable when the MDP model is not given a priori, which, unfortunately, is the case in typical RL applications. In this paper, we study the problem of computing a shield in the setting where the transition graph of the MDP is known, but the transition probabilities are unknown. Our approach integrates probabilistic shielding with online model learning: as the RL agent explores the environment, we estimate the transition probabilities. From this estimate, we compute a shield. While the shield may be conservative initially, it adapts as the model estimate becomes more precise. Thus, the shield improves in tandem with the RL agent. This paradigm of adaptive probabilistic shielding raises a number of challenges, such as when to recompute the shield and how to balance between exploration and safety during learning. We empirically evaluate multiple variants of this paradigm across several environments.
• 中文摘要 概率屏蔽是一种安全强化学习（RL）技术。通常，静态观察者——称为盾牌——将学习主体的行为限制在仍可安全行动的范围内。传统上，屏蔽是根据底层马尔可夫决策过程（MDP）的转移概率计算的。因此，当MDP模型没有先验信息时，这种技术不适用，而这在典型的强化学习应用中很遗憾。本文研究在已知MDP转移图但转移概率未知的背景下计算屏蔽问题。我们的方法将概率屏蔽与在线模型学习相结合：当强化学习代理探索环境时，我们估算转变概率。基于这个估计，我们计算出一个护盾。虽然盾牌起初可能较为保守，但随着模型估算的更精确，它会相应调整。因此，屏蔽与强化学习代理同步提升。这种自适应概率屏蔽范式带来了诸多挑战，比如何时重新计算屏蔽，以及如何在学习过程中平衡探索与安全。我们通过实证方式评估了该范式在多个环境中的多个变体。

SAPO: Single-Rollout Autoregressive Policy Optimization for Agentic Reinforcement Learning

SAPO：针对代理强化学习的单次推出自回归策略优化

• Authors: Dayang Liang, Lang Feng, Bo An, Yunlong Liu
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2608.19842
• Pdf link: https://arxiv.org/pdf/2608.19842
• Abstract Agentic reinforcement learning (RL) has become a critical stage in the post-training of large language models. Existing critic-free, group-relative methods estimate policy advantages from multiple rollouts, avoiding the substantial memory overhead of conventional proximal policy optimization (PPO) and achieving strong performance on long-horizon interactive tasks. Despite their success, recent studies revealed three limitations: (1) Lack explicit value generalization and effective temporal credit assignment; (2) Suffer from potential advantage collapse in long-horizon complex tasks; (3) Require a costly trade-off between sampling budget and policy performance. In this work, we propose Single-rollout Autoregressive Policy Optimization (SAPO), a low-memory and compute-efficient framework in which the policy and value functions share a single autoregressive backbone. SAPO exploits the autoregressive structure of LLMs to produce policy and value predictions at distinct causal boundaries with shared parameters, while independently optimizing the PPO objectives and auxiliary on-policy SARSA objectives. To robustly estimate the contribution of each turn, we further introduce a trajectory-level generalized advantage estimator that combines lambda-returns with batch normalization. Experiments across ALFWorld and WebShop with Qwen2.5-1.5B/7B show that SAPO trains stably and outperforms PPO and GRPO by mean +15.1 and +12.1 percentage points, respectively, while eliminating the memory cost of a separate critic model and reducing per-iteration runtime by 33.2% over PPO.
• 中文摘要 代理强化学习（RL）已成为大型语言模型后训练中的关键阶段。现有无批评、群体相对的方法能够从多次推广中估算策略优势，避免了传统近端策略优化（PPO）带来的巨大内存开销，并在长期交互任务中实现了强劲的性能。尽管取得了成功，近期研究揭示了三个局限性：（1）缺乏明确的价值推广和有效的时间归属分配;（2）在长期复杂任务中存在潜在优势崩溃;（3）要求在抽样预算与政策执行之间做出昂贵的权衡。在本研究中，我们提出了单次推出自回归策略优化（SAPO），这是一种低内存和低计算效率的框架，其中策略函数和价值函数共享单一的自回归骨干。SAPO利用LLMs的自回归结构，在不同因果边界和共享参数下生成策略和价值预测，同时独立优化PPO目标和辅助策略内SARSA目标。为了稳健估计每回合的贡献，我们进一步引入了轨迹级广义优势估计器，结合了λ收益与批次归一化。在ALFWorld和WebShop上对Qwen2.5-1.5B/7B的实验显示，SAPO训练稳定，分别平均为+15.1和+12.1个百分点，同时消除了独立批评模型的内存成本，且每次迭代运行时间比PPO减少33.2%。

EnvHarness: Awakening Static Worlds for Agent Learning

EnvHarness：为代理学习唤醒静态世界

• Authors: Chengsong Huang, Zifeng Wang, Rujun Han, Jun Yan, Yanfei Chen, Zoey CuiZhu, Ke Jiang, Peng Xia, Han Yu, Yufan Zhuang, Yifei Ming, Jiaqi Pan, Bhavana Dalvi Mishra, Jiaxin Huang, Burak Gokturk, Tomas Pfister, Chen-Yu Lee
• Subjects: Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2608.19880
• Pdf link: https://arxiv.org/pdf/2608.19880
• Abstract LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent's weaknesses, and quickly left behind as it improves. While recent environment generation methods attempt to address this, they require domain-specific pipelines, rely on expensive or unreliable verifiers, and still produce static environments. To alleviate the engineering burden of rebuilding environments from scratch, we propose Environment Harness (EnvHarness), a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic. Operating through standard interfaces, EnvHarness applies across diverse domains while ensuring every reshaped environment retains its original verifier. To automate this process, we introduce EnvRigger, which treats the target policy as a black box, observing its execution trajectories to synthesize EnvHarness components targeting diagnosed flaws, and validating them via fresh rollouts. Across five benchmarks in four domains, EnvHarness outperforms both original environments and domain-specific environment generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. Furthermore, EnvHarness provides a superior optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of the policy and its environment.
• 中文摘要 LLM代理通过与环境互动来学习，但这些环境是手工构建且静态的：对代理的弱点视而不见，随着智能体的提升，很快就会被忽视。虽然近期的环境生成方法尝试解决这个问题，但它们需要域特定的流水线，依赖昂贵或不可靠的验证器，并且仍然能生成静态环境。为了减轻从零重建环境的工程负担，我们提出了环境约束（EnvHarness），这是一种可编程的插件组件层，可以包裹静态环境，重塑其行为，而无需修改底层逻辑。通过标准接口运行，EnvHarness 适用于多个领域，同时确保每个重塑环境都保留其原始的验证者。为了自动化这一过程，我们引入了EnvRigger，它将目标策略视为黑箱，观察其执行轨迹，综合针对已诊断缺陷的EnvHarness组件，并通过全新部署进行验证。在四个领域的五个基准测试中，EnvHarness 表现优于原始环境和特定领域环境生成流水线，在保留实例上提升了高达 9.0 分，执行步骤减少了 9.8%。此外，EnvHarness为强化学习提供了更优越的优化信号，使策略及其环境能够持续、有针对性地共同演化。

EXIMO: VLM Guided Exploration of VLA Policies

EXIMO：VLM引导探索VLA政策

• Authors: Bhavya Sukhija, Oliver Groth, Mohit Shridhar, Tim Hertweck, Michael Bloesch, Markus Wulfmeier, Abbas Abdolmaleki, Martin Riedmiller
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2608.19891
• Pdf link: https://arxiv.org/pdf/2608.19891
• Abstract How to efficiently finetune robot policies to learn new tasks on the fly? State of the art robotic manipulation policies are based on behaviour cloning of large vision-language-action (VLA) models with billions of parameters on huge teleoperation datasets. While this simple approach has enabled significant advances for robotic manipulation, finetuning of VLA policies for learning new tasks still remains an open problem. In particular, collecting teleoperation datasets requires hundreds of hours of expensive human labour and the alternative, reinforcement learning (RL), can be notoriously sample-inefficient especially for long-horizon tasks. In addition, RL with VLAs imposes several challenges due to the model's size and architectural design. In this work, we propose EXIMO, an efficient algorithm for finetuning of VLA policies. EXIMO operates in three stages: explore, imitate, and optimize. During the explore phase, EXIMO equips the VLA with a vision language model (VLM) that acts as a planner. The VLM thinks and breaks down challenging long-horizon problems into shorter ones for the VLA. The VLM, together with the VLA, is used to collect an orchestrated dataset on new tasks. During the imitate phase, the VLA is finetuned with the orchestrated data. Finally, during the optimize stage, we use residual off-policy RL to further finetune the policy. In our experiments, we ablate all three stages of EXIMO and show that it outperforms existing approaches significantly in terms of sample-efficiency and final performance.
• 中文摘要 如何高效地微调机器人策略，以便能够实时学习新任务？最先进的机器人操作策略基于行为克隆，这些模型拥有数十亿参数，覆盖庞大的远程操作数据集。虽然这种简单的方法推动了机器人操作的重大进展，但学习新任务时微调VLA策略仍是一个悬而未决的问题。特别是，收集远程操作数据集需要数百小时昂贵的人力劳动，而另一种方法——强化学习（RL）则在长期任务中样本效率极低。此外，带有VLA的强化学习由于模型的规模和架构设计带来了诸多挑战。在本研究中，我们提出了EXIMO，一种高效的VLA策略微调算法。EXIMO分为三个阶段：探索、模仿和优化。在探索阶段，EXIMO为VLA配备了视觉语言模型（VLM），作为规划工具。VLM思考并将具有挑战性的长期问题拆解为VLA的短期问题。VLM与VLA一起用于收集新任务的编排数据集。在模拟阶段，VLA会根据编排数据进行微调。最后，在优化阶段，我们使用残余非策略强化学习（residual off-policy RL）进一步微调策略。在我们的实验中，我们对EXIMO的三个阶段进行了消融，并证明其在样品效率和最终性能方面显著优于现有方法。

MILD: Tractable Terrain Modeling for Learning Improved Bipedal Locomotion on Deformable Surfaces

MILD：可处理地形建模，用于学习改进的双足行走在可变形表面上

• Authors: Zeren Luo, Jiahui Zhang, Zhe Xu, Wanyue Li, Xinqi Li, Xuechao Chen, Zhangguo Yu, Annan Tang, Peng Lu
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2608.19955
• Pdf link: https://arxiv.org/pdf/2608.19955
• Abstract Enabling robots to walk on yielding terrain is vital for applications ranging from disaster response to planetary exploration. While bipedal robots hold immense potential, their locomotion on deformable surfaces remains limited as current simulators fail to capture the spatiotemporal heterogeneity of such yielding substrates. We present MILD, featuring a physics-grounded discrete-element contact solver that accurately simulates spatially varying foot-terrain interactions. Complementing this model, we train a terrain-aware locomotion controller via deep reinforcement learning with latent modulation and proprioceptive estimation. Quantitative comparisons against state-of-the-art methods show our approach generates more diverse and realistic contact scenarios during training, resulting in controllers that exhibit natural adaptation on real deformable surfaces. Through hardware experiments, we demonstrate the system's capability for online terrain identification and adaptation across a wide range of surface stiffness.
• 中文摘要 让机器人能够在变形地形上行走，对于从灾害响应到行星探测等多种应用至关重要。虽然双足机器人潜力巨大，但由于当前模拟器未能捕捉此类屈服基底的时空异质性，其在可变形表面上的运动能力仍然有限。我们介绍MILD，配备基于物理的离散元件接触求解器，能够准确模拟空间变化的足部与地形相互作用。作为该模型的补充，我们通过深度强化学习训练一个地形感知的运动控制器，结合潜在调制和本体感觉估计。与最先进方法的定量比较表明，我们的方法在训练过程中产生了更丰富且真实的接触场景，从而使控制器在真实可变形表面上展现出自然适应性。通过硬件实验，我们展示了系统在广泛表面刚度范围内在线识别和适应地形的能力。

Learning Highly Dynamic Skills Transition for Quadruped Jumping Through Constrained Space

学习四足跳跃在有限空间中的高度动态技能过渡

• Authors: Zeren Luo, Jiahui Zhang, Yimin Han, Ji Ma, Minghao Lu, Ioannis Havoutis, Peng Lu
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2608.19977
• Pdf link: https://arxiv.org/pdf/2608.19977
• Abstract Although legged animals are capable of performing explosive motions while traversing confined spaces, replicating this behavior in quadrupedal robots has been a longstanding challenge. Here, we propose a hierarchical reinforcement learning pipeline that empowers the robots to perform aggressive locomotion through constrained obstacles--a narrow gate. The imitation learning technique is used to train the low-level policy, which mimics the behaviors of real animals and forms a set of diverse skills. The high-level controller, having an awareness of the capability of low-level skills and acquiring the gate information via vision-based detection, determines the suitable maneuvers with collision-free trajectories to traverse it dynamically. Notably, we also verify that this framework can be extended to other highly dynamic tasks. This is one of the first works that perform autonomous and agile aerial gate traversal tasks on ground-walking robots, extending the lifelike agility of legged robots to match that of their biological counterparts.
• 中文摘要 虽然有腿动物能够在狭小空间内进行爆炸性动作，但在四足机器人中复制这种行为一直是长期的挑战。在这里，我们提出了一种分层强化学习流程，使机器人能够通过受限障碍物——狭窄的门——进行激进的移动。模仿学习技术用于训练低层次策略，该策略模拟真实动物的行为，形成一套多样化的技能。高级控制员了解低级技能的能力，并通过基于视觉的探测获取星门信息，确定合适的机动，确保无碰撞轨迹以动态穿越。值得注意的是，我们还验证了该框架可以扩展到其他高度动态的任务。这是最早在地面行走机器人上实现自主且灵活空中门穿越任务的作品之一，将有腿机器人的逼真敏捷性扩展到与生物机器人相当。

End-to-end Early Classification of Time Series in Non-Stationary Environments

非定流环境中时间序列的端到端早期分类

• Authors: Aurélien Renault, Alexis Bondu, Antoine Cornuéjols, Vincent Lemaire
• Subjects: Subjects: Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2608.20044
• Pdf link: https://arxiv.org/pdf/2608.20044
• Abstract Early Classification of Time Series (ECTS) requires making accurate decisions as early as possible in inherently online and evolving environments. Yet, most existing methods assume stationarity and rely on separable designs, where classification and triggering are optimized independently, an assumption that fundamentally limits their adaptability under drift. In this work, we challenge this paradigm and study ECTS under non-stationary conditions. We provide the first systematic comparison between separable and end-to-end approaches across controlled drifting scenarios. Building on Reinforcement Learning, we introduce DQeND, a unified architecture that jointly learns representation, classification, and triggering decisions, while remaining directly comparable to state-of-the-art separable baselines. Across a wide range of drifts, DQeND demonstrates strong robustness across various non-stationary scenarios, consistently outperforming separable baselines. An ablation study further highlights that jointly updating representation and decision modules is critical to these gains. Overall, our results indicate that end-to-end learning can offer improved adaptation capabilities for ECTS in dynamic environments, and motivate further investigation of alternatives to separable designs.
• 中文摘要 时间序列的早期分类（ECTS）要求在本质上在线且不断发展的环境中尽早做出准确决策。然而，大多数现有方法假设平稳性，依赖可分离设计，即分类和触发独立优化，这一假设从根本上限制了它们在漂移下的适应性。在本研究中，我们挑战这一范式，研究非平稳条件下的ECTS。我们首次系统地比较了可分离方法与端到端方法在受控漂移场景中的应用。基于强化学习，我们引入了DQeND，这是一种统一架构，能够共同学习表示、分类和触发决策，同时保持与最先进的可分离基线直接比较。在广泛的漂移范围内，DQeND在多种非静态场景下展现出强的鲁棒性，持续优于可分基线。一项消融研究进一步强调，联合更新表征模块和决策模块对这些收益至关重要。总体而言，我们的结果表明端到端学习能够提升动态环境中ECTS的适应能力，并激励对可分离设计替代方案的进一步探索。

Reward-Guided Autoregressive Graph Generation for Efficient Multi-Agent Communication Topology Design

奖励引导自回归图生成，用于高效多智能体通信拓扑设计

• Authors: Poomphob Suwannapichat, Boonyarit Changaival, Caesar Wu, Pascal Bouvry
• Subjects: Subjects: Multiagent Systems (cs.MA); Computation and Language (cs.CL); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2608.20099
• Pdf link: https://arxiv.org/pdf/2608.20099
• Abstract LLM-based Multi-Agent Systems (MAS) achieve strong performance on complex reasoning tasks by coordinating multiple agents, but at the cost of substantial token consumption. Recent work on automatic topology design, ARG-Designer, has reframed this problem as autoregressive graph generation. However, its training objective provides no explicit incentive for the model to generate sparse and efficient topologies. We address this limitation by introducing a Reward-Guided Autoregressive Graph Generation (RGA-Designer) inspired by Reinforcement Learning from Human Feedback (RLHF). We train a reward model that jointly captures task correctness and structural compactness, and then fine-tune the pretrained graph generator using the reward model as feedback. Our method preserves task accuracy at the level of ARG-Designer while reducing token consumption by an average of 20.5%.
• 中文摘要 基于LLM的多代理系统（MAS）通过协调多个代理在复杂推理任务中表现出色，但代价是大量令牌消耗。最近关于自动拓扑设计的研究ARG-Designer将这一问题重新定义为自回归图生成。然而，其训练目标并未明确激励模型生成稀疏高效拓扑。我们通过引入受人类反馈强化学习（RLHF）启发的奖励引导自回归图生成（RGA-Designer）来解决这一局限。我们训练一个奖励模型，结合任务的正确性和结构紧凑性，然后利用奖励模型作为反馈，微调预训练的图生成器。我们的方法在ARG-Designer层面保持任务准确性，同时平均减少20.5%的代币消耗。

Multi-Agent Orchestration with the Common-Sense Reasoning Capabilities of LLMs for Autonomous Driving

多智能体编排结合LLMs的常识推理能力，用于自动驾驶

• Authors: Mehdi Azarafza, Faezeh Pasandideh, Ali Ehteshami Bejnordi, Stefan Henkler, Achim Rettberg
• Subjects: Subjects: Multiagent Systems (cs.MA); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2608.20129
• Pdf link: https://arxiv.org/pdf/2608.20129
• Abstract Autonomous vehicles require robust perception and decision-making capabilities to operate in diverse and unseen scenarios. While reinforcement learning and rule-based methods can provide effective control and safety mechanisms, their performance may degrade in situations requiring contextual reasoning. Large Language Models (LLMs) have demonstrated strong capabilities in understanding multimodal information and generating contextual reasoning, however, their use for direct vehicle control can introduce latency and hallucination risks. To address these limitations, a hybrid framework is proposed. This system uses an orchestrator to coordinate PPO-trained reinforcement learning and PID control, with LLM common-sense reasoning applied throughout the framework. LLM reasoning is further employed iteratively to refine the RL reward function for dynamic driving environments. The proposed framework is evaluated in highly randomized CARLA scenarios under diverse environmental and traffic conditions. The results demonstrate the potential of integrating LLM-based reasoning with conventional autonomous driving methods while retaining structured control and safety mechanism.
• 中文摘要 自动驾驶车辆需要强大的感知和决策能力，以便在多样且未被预见的场景中运行。虽然强化学习和基于规则的方法可以提供有效的控制和安全机制，但在需要情境推理的情况下，其表现可能会下降。大型语言模型（LLMs）已展现出理解多模态信息和生成上下文推理的强大能力，然而，其用于直接车辆控制可能带来延迟和幻觉风险。为解决这些限制，提出了一个混合框架。该系统使用编排器协调PPO训练的强化学习和PID控制，框架内贯穿LLM常识推理。LLM推理进一步迭代应用于动态驾驶环境下的强化学习奖励函数。该框架在高度随机的CARLA场景下，在不同环境和交通条件下进行评估。结果展示了将基于LLM的推理与传统自动驾驶方法整合的潜力，同时保留结构化控制和安全机制。

DARS: Dual-Level Credit Assignment RL with Structured Reasoning for Instruction-Based Image Editing

DARS：双层次学分作业强化学习，结构化推理用于基于指令的图像编辑

• Authors: Haoxiang Cao, Jiajiong Cao, Xuanpu Zhang, Changqian Yu, Chaoqun Wang
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2608.20161
• Pdf link: https://arxiv.org/pdf/2608.20161
• Abstract Instruction-based image editing uses a planner-renderer pipeline: a vision-language model (VLM) first converts the instruction into an edit plan, and a diffusion model then executes that plan. Training such systems with only final-image rewards is inefficient because a poor edit does not reveal whether additional optimization should place more emphasis on the planner or the renderer, and even planner-dominant cases remain difficult to localize within a free-form reasoning trace. We present DARS, a reinforcement learning framework for dual-level credit assignment in this two-stage setting. Across modules, multi-plan multi-render rollouts estimate between-plan and within-plan reward variability for soft module routing, while rollout mean rewards provide hardness estimates for an adaptive curriculum. Within the planner, a four-field structured reasoning output enables a prefix-gated reward and token-level advantage reweighting, turning outcome-level feedback into localized supervision. Experiments on five benchmarks show that DARS outperforms a Joint~RL baseline with the same backbone, data, reward model, and rollout budget, with the largest gains on reasoning-intensive edits.
• 中文摘要 基于指令的图像编辑使用规划器-渲染器流水线：视觉语言模型（VLM）首先将指令转换为编辑计划，然后扩散模型执行该计划。仅用最终图像奖励训练此类系统效率低下，因为编辑不佳无法判断额外优化应如何更重视规划器还是渲染器，即使是以规划器为主的情况也难以在自由推理轨迹中定位。我们介绍DARS，一种用于双级学分作业的强化学习框架，适用于该两阶段的教学环境。跨模块，多计划多渲染推广估计软模块路由的计划内及计划内奖励变异，而推广平均奖励则为适应性课程提供难度估计。在规划器内部，四字段结构化推理输出实现了前缀门槛奖励和代币级优势重权重调整，将结果层级反馈转化为局部监督。五个基准测试的实验显示，DARS在采用相同骨干、数据、奖励模型和推广预算时，表现优于Joint~RL基线，且在推理密集编辑上获得最大收益。

RoMAN-Flow: Taming Autoregressive Normalizing Flows for Offline Reinforcement Learning in Robotic Manipulation

RoMAN-Flow：驯服自回归归一化流，用于机器人操作中的离线强化学习

• Authors: Shaoxuan Wang, Guangting Zheng, Rui Huang, Zhipeng Tang, Sha Zhang, Jiajun Deng, Yanyong Zhang
• Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2608.20208
• Pdf link: https://arxiv.org/pdf/2608.20208
• Abstract Offline reinforcement learning improves robotic policies using previously collected data without further environment interaction. Yet prevalent diffusion- and flow-matching robot policies lack tractable likelihoods, limiting their use in likelihood-based offline RL post-training. AR-NFs offer both expressive action modeling and exact likelihood evaluation, but their sequential sampling incurs substantial sampling overhead during policy optimization and deployment. We present RoMAN-Flow (Robotic Manipulation with Autoregressive Normalizing Flows), an offline reinforcement learning framework that makes AR-NF policies practical for robotic manipulation by addressing this sampling bottleneck in both stages. During policy optimization, RoMAN-Flow employs a sampling-free, advantage-weighted likelihood objective that assigns higher likelihood to high-advantage actions from the offline dataset without sampling from the autoregressive policy. For efficient deployment, it distills the optimized autoregressive policy into a one-step action generator, enabling low-latency action prediction. Experiments across multiple simulated manipulation benchmarks and real-world robotic platforms demonstrate that RoMAN-Flow achieves competitive policy performance while substantially reducing inference latency. Code is available at this https URL.
• 中文摘要 离线强化学习利用先前收集的数据改进机器人策略，无需额外环境交互。然而，普遍存在的扩散和流量匹配机器人策略缺乏可处理的似然度，限制了它们在基于似然的离线强化学习训练后应用。AR-NF既提供表达式动作建模，也提供精确似然评估，但其顺序抽样在策略优化和部署过程中会产生较大的采样开销。我们介绍RoMAN-Flow（自回归归一化流程的机器人操作），这是一种离线强化学习框架，通过解决两个阶段的采样瓶颈，使AR-NF策略在机器人操作中变得切实可行。在策略优化过程中，RoMAN-Flow采用无抽样、优势加权似然目标，将离线数据集中高优势行为赋予更高似然，而无需从自回归策略中抽样。为了高效部署，它将优化后的自回归策略提炼为一步动作生成器，实现低延迟的动作预测。在多个模拟操作基准测试和现实机器人平台上的实验表明，RoMAN-Flow在显著降低推断延迟的同时，实现了具有竞争力的策略性能。代码可在此 https URL 访问。

Learning When to Think: Adaptive Reasoning for Test-Time Compute Allocation

学习何时思考：测试时间计算分配的自适应推理

• Authors: Gijs Kassenaar, Zhao Yang, Vincent François-Lavet
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2608.20256
• Pdf link: https://arxiv.org/pdf/2608.20256
• Abstract Reasoning language models trained with reinforcement learning typically operate under a fixed token budget rather than an explicitly adaptive one, which can lead to over-computation on easy problems and insufficient computation on difficult ones. We study whether a model can learn to allocate its own reasoning effort by choosing, as the first token of its response, one of three modes: \textsc{NoThink} (answer as quickly as possible), \textsc{Short} (brief reasoning), or \textsc{Long} (extended reasoning). The choice is learned inside Group Relative Policy Optimization (GRPO) with no separate router, through a shaped reward that makes each mode worthwhile at a different response length, together with hard per-mode token caps that keep the modes distinct. On a 1.5B distilled model trained on MATH, the three modes emerge without collapsing to a single choice, and the brief modes end up more accurate than \textsc{Long}, which shows that the router sorts problems by difficulty rather than at random. Averaged over three seeds, the resulting policy stays close to the base model's accuracy on the held-out MATH500 ($0.782$ vs.\ $0.796$) while cutting the mean response length from $4{,}796$ to $2{,}811$ tokens (a $41\%$ reduction). Interestingly, it also transfers to other benchmarks without retraining, with the largest savings where problems are easier, with for instance 76\% token reduction on GSM8K and at higher accuracy than the baselines at similar response length. In short, we build a reasoning model that adaptively chooses how much to reason for each problem.
• 中文摘要 通过强化学习训练的推理语言模型通常采用固定的令牌预算，而非显式自适应预算，这可能导致简单问题过度计算，困难问题计算不足。我们研究模型是否可以通过选择三种模式中的一种作为其响应的第一个标记来分配自己的推理努力：\textsc{NoThink}（尽快回答）、\textsc{Short}（简短推理）或\textsc{Long}（扩展推理）。选择是在群相对策略优化（GRPO）中学习的，没有单独的路由器，通过一个形状奖励，使每种模式在不同响应长度下有价值，同时有严格的每个模式代币上限，保持模式的区别。在一个用 MATH 训练的 1.5 B 精炼模型中，三种模式都不会归结为单一选择，且简短模式比 \textsc{Long} 更准确，后者表明路由器是按难度而非随机排序问题。在三个种子中平均后，最终的策略在保留MATH500上保持接近基础模型的准确性（$0.782$对$0.796$），同时将平均响应长度从$4{，}796$缩短到$2{，}811$（减少$41\%$）。有趣的是，它也能迁移到其他基准测试，且在问题较易的地区节省最大，例如GSM8K在相似响应长度下，令牌减少了76%且准确率高于基线。简而言之，我们构建了一个推理模型，能够自适应地选择每个问题推理的程度。

MidTool: Mid-training Data Synthesis for Agentic Tool Use

MidTool：用于代理工具的中训练数据综合

• Authors: Fengqing Jiang, Yite Wang, Boyi Liu, Zhaoyang Wang, Canwen Xu, Zhewei Yao, Radha Poovendran, Yuxiong He
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2608.20314
• Pdf link: https://arxiv.org/pdf/2608.20314
• Abstract Mid-training is increasingly recognized as a critical stage for shaping the capabilities of large language models. Recent work has shown that targeted mid-training can strengthen reasoning-intensive abilities such as math and science, and can also improve agentic capabilities in software-engineering settings. In this work, we study the parallel but less explored agentic capability: general tool use. We present MidTool, an open corpus construction pipeline for agentic tool-use mid-training that combines large-scale web, PDF, and code data with synthesized supervision from real-world tool APIs, MCP skills, and document-grounded workflows. MidTool is designed to teach models how to recognize tool affordances, ground arguments from context, compose tool call workflow, and recover from incomplete information. We mid-train Qwen3-4B-Base and Qwen3-8B-Base on MidTool-Mix, and then apply follow-up post-training with both supervised fine-tuning and reinforcement learning. Compared with baselines, MidTool-Mix consistently improves downstream performance under both SFT and RL on BFCL, tau2-Bench, and MCP Universe. These results suggest that general tool use, like other important LLM capabilities, benefits from dedicated mid-training rather than being left entirely to post-training.
• 中文摘要 训练中期越来越被视为塑造大型语言模型能力的关键阶段。最新研究表明，针对性的中期训练可以强化推理密集型能力，如数学和科学，同时也能提升软件工程环境中的代理能力。在本研究中，我们研究了平行但较少被探讨的代理能力：一般工具的使用。我们介绍MidTool，一个面向代理工具使用培训的开放语料库构建流程，结合了大规模的网页、PDF和代码数据，并结合了现实工具API、MCP技能和基于文档的工作流程的综合监督。MidTool 旨在教授模型识别工具的可有性、从上下文中建立论据、构建工具调用工作流程以及从不完整信息中恢复。我们在MidTool-Mix上进行Qwen3-4B-Base和Qwen3-8B-Base的中期培训，然后在培训后进行监督微调和强化学习的后续跟进。与基线相比，MidTool-Mix在BFCL、tau2-Bench和MCP Universe上，在SFT和RL下持续提升下游性能。这些结果表明，像其他重要的大型语言模型功能一样，一般工具的使用需要在培训中专门进行，而非完全留给训练后进行。

G-CARL: Grounded Checklist-Aligned Reward Learning for Patient-Oriented Medical Report Interpretation

G-CARL：基于核对清单的奖励学习，用于患者导向的医疗报告解读

• Authors: Shiao Xie, Siyu Chen, Jianwei Lv, Bo Yuan, Yujin Wang, Xiandong Li
• Subjects: Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2608.20331
• Pdf link: https://arxiv.org/pdf/2608.20331
• Abstract Personalized interpretation of medical reports has emerged as an increasingly important need among patients. Addressing this need requires both evidence-grounded medical factuality and context-dependent patient communication, yet existing medical vision-language tasks do not adequately capture these dual requirements. To bridge this gap, we introduce Patient-oriented Medical Report Interpretation (PMRI), a novel open-ended multimodal generation task that requires models to explain medical reports in accurate and accessible language based on a user's query and dialogue history. These two objectives differ fundamentally in their verifiability, yet remain tightly coupled, making them difficult to optimize jointly under conventional supervised fine-tuning and holistic reinforcement learning paradigms. To address this challenge, we propose G-CARL, a grounded, checklist-aligned reinforcement learning framework that combines multi-source retrieval for atomic claim verification with context-aware, instance-specific weighted checklists for response coverage, providing structured supervision for factuality, user-demand satisfaction, and expression quality without constraining response diversity. We further construct MMedReport, a real-world PMRI benchmark, along with a clinician-designed three-dimensional evaluation protocol. Extensive experiments demonstrate that G-CARL consistently outperforms existing post-training baselines in overall quality, claim-level precision, and checklist recall. Pairwise preference evaluation by clinicians further confirms that G-CARL produces interpretations that are more accurate and better aligned with patient needs.
• 中文摘要 个性化解读医疗报告已成为患者日益重要的需求。满足这一需求既需要基于证据的医学事实性，也需要依赖情境的患者沟通，但现有的医疗视觉语言任务未能充分捕捉这些双重需求。为弥合这一空白，我们引入了面向患者医疗报告的解释（PMRI），这是一种新型开放式多模态生成任务，需要模型根据用户的查询和对话历史，用准确且易懂的语言解释医疗报告。这两个目标在可验证性上有根本差异，但彼此紧密耦合，使得在传统的监督微调和整体强化学习范式下难以联合优化。为应对这一挑战，我们提出了G-CARL，一个基于基础、符合检查表的强化学习框架，结合了多源检索以验证原子主张，同时采用上下文感知、实例特定加权检查表来覆盖回应覆盖，提供结构化监督，提升事实性、用户需求满意度和表达质量，同时不限制反应多样性。我们还进一步构建了MMedReport，一个真实世界的PMRI基准，以及临床医生设计的三维评估方案。大量实验表明，G-CARL在整体质量、索赔级精度和检查表回忆率方面，始终优于现有的培训后基线。临床医生的两对偏好评估进一步证实，G-CARL能够产生更准确、更符合患者需求的解释。

Swift-Image: Exploring the Performance Frontier of Compact Unified Image Generation Models

Swift-Image：探索紧凑统一图像生成模型的性能前沿

• Authors: Taihang Hu, Zhao Wang, Zuan Gao, Tao Liu, Hao Yan, Zhengze Xu, Yuhang Yu, Yongchao Du, Xingjian Wang, Jun Zheng, Qinye Zhou, Zhengrui Chen, Chao Lin, Yefeng Shen, Zhengtao Wu, Ge Wu, Xiaoli Xu, Denghui Yang, Huayu Zhang, Mingzhou Zhang, Mengting Chen
• Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2608.20334
• Pdf link: https://arxiv.org/pdf/2608.20334
• Abstract We present Swift-Image, a compact unified model for text-to-image generation, single-image editing, and multi-image editing. Our goal is to explore how far a relatively small visual generator can be pushed through systematic training engineering under a constrained computational budget. Swift-Image adopts an efficient 6B single-stream DiT and a progressive training pipeline that evolves from broad semantic coverage to higher resolution, stronger visual quality, and unified generation-editing supervision. For post-training, we employ parallel expert reinforcement learning followed by multi-teacher on-policy distillation to alleviate interference among heterogeneous objectives. We further decouple high-level reasoning from pixel-level rendering with a Prompt Enhancer that translates user requests into generator-aligned visual specifications. For efficient deployment, structural pruning and few-step distillation produce 3B and accelerated variants. Swift-Image achieves leading aggregate performance among evaluated open-source models with only 6B parameters and 243K GPU training hours; the compressed 3B model incurs nearly no loss, while few-step distillation further improves aggregate editing performance with substantially fewer sampling steps. Our study also summarizes practical lessons for architecture, data curriculum, post-training, prompt enhancement, and model compression.
• 中文摘要 我们介绍Swift-Image，一个紧凑统一的模型，用于文本对图像生成、单图编辑和多图编辑。我们的目标是在有限的计算预算下，探索一个相对较小的视觉生成器在系统化训练工程中能被推到多远。Swift-Image 采用高效的 6B 单流 DiT 和渐进式训练流程，从广泛的语义覆盖逐步发展到更高分辨率、更强的视觉质量和统一的生成编辑监督。在培训后期，我们采用并行专家强化学习，随后多教师进行政策提炼，以减少异质目标之间的干扰。我们进一步将高层推理与像素级渲染分离，使用提示增强器将用户请求转化为与生成器对齐的视觉规范。为了高效部署，结构修剪和少步蒸馏可产生3B及加速型变体。Swift-Image 在仅有 6B 参数和 24.3 万 GPU 训练小时的评估开源模型中实现了领先的综合性能;压缩后的3B模型几乎没有损失，而少步蒸馏进一步提升了聚合编辑性能，采样步骤显著减少。我们的研究还总结了架构、数据课程、后期培训、提示增强和模型压缩的实用经验。

Keyword: diffusion policy

There is no result