生成时间: 2026-09-17 21:08:34 (UTC+8); Arxiv 发布时间: 2026-09-17 20:00 EDT (2026-09-18 08:00 UTC+8)

今天共有 38 篇相关文章

Keyword: reinforcement learning

Composite-Gradient Learning for Shared Control Authority Between Deep Reinforcement Learning and Model Predictive Control

深度强化学习与模型预测控制共享控制权威的复合梯度学习

• Authors: Giray Önür, Azita Dabiri, Bart De Schutter
• Subjects: Subjects: Machine Learning (cs.LG); Systems and Control (eess.SY)
• Arxiv link: https://arxiv.org/abs/2609.17697
• Pdf link: https://arxiv.org/pdf/2609.17697
• Abstract Integrated deep reinforcement learning (DRL) and model predictive control (MPC) methods are increasingly used to control autonomous systems by combining their complementary capabilities. DRL learns control policies through interaction with the environment. MPC uses a system model to optimize control inputs while accounting for constraints. In DRL-MPC frameworks with shared control authority, both the DRL agent and the MPC controller each determine part of the control inputs. However, common learning formulations treat MPC as part of the environment and therefore do not explicitly account for MPC's contribution to control or its interaction with the DRL agent. This paper proposes a novel composite-gradient learning (CGL) method that integrates the MPC controller into the learning process by representing the DRL and MPC control inputs as a joint action and accounting for their interaction when updating the DRL agent during training. CGL is evaluated on two multi-class freeway traffic networks with different strengths of interaction between the DRL and MPC control inputs and it is compared with alternative methods that treat MPC as part of the environment or that only partially incorporate MPC into learning. The results show that CGL offers limited benefit under weak interaction, but learns higher-performing control policies than the alternative methods in a subset of training runs under strong interaction, although the average control-performance gains remain modest.
• 中文摘要 集成深度强化学习（DRL）和模型预测控制（MPC）方法日益被用于通过结合互补能力来控制自主系统。DRL通过与环境交互学习控制策略。MPC利用系统模型优化控制输入，同时考虑约束因素。在具有共享控制权的DRL-MPC框架中，DRL代理和MPC控制器各自决定控制输入的一部分。然而，常见的学习表述将MPC视为环境的一部分，因此未明确考虑MPC对控制的贡献或其与DRL代理的交互。本文提出了一种新颖的复合梯度学习（CGL）方法，将MPC控制器整合进学习过程，将DRL和MPC控制输入表示为联合动作，并在训练期间更新DRL代理时考虑它们的交互。CGL在两个多类别高速公路交通网络上评估，这些网络在DRL与MPC控制输入交互强度不同，并将其与将MPC视为环境一部分或仅部分将MPC纳入学习的替代方法进行比较。结果显示，CGL在弱交互下效果有限，但在强交互下的部分训练运行中，CGL学习的控制策略优于其他方法，尽管平均控制性能提升仍属有限。

REVERSAL-BENCH: A Reversibility Axis and Reset Oracle for Measuring the Reset-Free RL Cliff

反转工作台：用于测量无重置强化逻辑悬崖的可逆轴和重置神谕

• Authors: Riyaaz Shaik, Chandru Venkataraman
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.17745
• Pdf link: https://arxiv.org/pdf/2609.17745
• Abstract A central goal of autonomous reinforcement learning is continuous policy training without external resets. However, existing paradigms largely depend on underlying environmental reversibility, a property absent in real world manipulation, where events such as pushing objects off tables or spilling granular substances cannot be undone. We introduce REVERSAL-BENCH, a benchmark that controls reversibility via a continuous parameter $\rho \in [0, 1]$ and provides a reset oracle, a ground-truth verification mechanism to test state recoverability across eight manipulation settings in five physics engines. Evaluating a broad spectrum of policy architectures, including standard actor-critic algorithms, safe RL, and specialized reset-free frameworks, reveals a sharp reversibility cliff: reset-free agents are consistently absorbed into irrecoverable states as $\rho$ increases, whereas episodic agents maintain steady learning. We see this failure mode across autonomous reset-free baselines and constrained RL. Because reset-free agents lack external resets, any transition into an irrecoverable state results in permanent absorption, leaving the agent trapped where further learning halts. We show that this absorption phenomenon persists in full physics simulations under learned manipulation policies. By evaluating against geometrically identical reversible counterparts, we confirm that this breakdown is causally driven by irreversibility rather than obstacle complexity. We release the benchmark suite, a large multi-simulator dataset labeled with recoverability and a reset oracle. We also evaluate a safety shield that intervenes before irreversible failures occur, showing that while recoverability can be predicted accurately, active recovery primarily succeeds only when the agent can physically steer clear of the trap
• 中文摘要 自主强化学习的核心目标是实现无外部重置的连续策略训练。然而，现有范式在很大程度上依赖于潜在的环境可逆性，而现实世界操作中缺乏这一特性，比如将物体从桌面上推下或颗粒状物质溢出等事件无法被逆转。我们引入了REVERSAL-BENCH，这是一个通过连续参数$\rho \in [0， 1]$控制可逆性的基准测试工具，并提供了重置oracle，这是一种基于地面真实的验证机制，用于测试五个物理引擎中八种操作设置中的状态恢复性。评估包括标准actor-critic算法、安全强化学习和专门无重置框架在内的广泛策略架构，揭示了一个陡峭的可逆性悬崖：无重置智能体随着$\rho$增加，持续被吸收到不可恢复状态，而情节智能体则保持稳定学习。我们在自主无重置基线和受限强化学习中都能看到这种失效模式。由于无重置智能体缺乏外部重置，任何进入不可恢复状态的转换都会导致永久吸收，使智能体被困在学习停止的地方。我们证明，这种吸收现象在学习操作策略下的全物理模拟中依然存在。通过与几何上相同的可逆对应物进行评估，我们确认这种崩溃是由不可逆性驱动的，而非障碍复杂性。我们发布了基准测试套件，这是一个标注可恢复性和重置预言机的大型多模拟器数据集。我们还评估了一个在不可逆失效发生前介入的安全屏障，表明虽然可准确预测可恢复性，但主动恢复主要成功前提是智能体能够物理避开陷阱

Learning Market Competition in Shared Spectrum: A Multi-Agent Reinforcement Learning Approach

共享频谱中的学习市场竞争：多智能体强化学习方法

• Authors: Qixuan Zai, Randall Berry
• Subjects: Subjects: Computer Science and Game Theory (cs.GT)
• Arxiv link: https://arxiv.org/abs/2609.17754
• Pdf link: https://arxiv.org/pdf/2609.17754
• Abstract This paper investigates market competition among wireless service providers (SPs) that serve customers using shared spectrum. Prior work has analyzed such markets through models of competition with congestible resources, capturing both the congestion-sensitive nature of wireless spectrum and the effects of spectrum sharing on service quality. These models typically assume that the market demand function is known, enabling SPs to optimize pricing or quantity decisions under either Bertrand or Cournot competition. In contrast, we consider a setting in which the demand function is initially unknown and must be learned over time. We model this learning process using multi-agent reinforcement learning (MARL), allowing competing SPs to learn market dynamics while adapting their competitive strategies. Although MARL has shown strong performance in a variety of economic settings, recent work has demonstrated that it can also give rise to tacit collusion among self-interested agents. We therefore examine whether similar collusive behavior emerges in shared-spectrum markets and how its prevalence depends on the mode of competition (price versus quantity) and the choice of MARL algorithm. Our results provide insight into the interaction between learning dynamics, market structure, and spectrum sharing, with implications for both wireless market design and the deployment of learning-enabled decision-making systems.
• 中文摘要 本文探讨了使用共享频谱服务客户的无线服务提供商（SP）之间的市场竞争。此前的研究通过资源拥挤的竞争模型分析了此类市场，既捕捉了无线频谱对拥塞的敏感性质，也反映了频谱共享对服务质量的影响。这些模型通常假设市场需求函数已知，使SPS能够在贝特朗或库诺竞争下优化定价或数量决策。相比之下，我们考虑需求函数最初未知且需随时间学习的情境。我们用多代理强化学习（MARL）模拟这一学习过程，使竞争的SP能够在调整竞争策略的同时学习市场动态。尽管MARL在多种经济环境中表现出色，但近期研究表明，它也可能引发自利代理之间的默契合谋。因此，我们考察了共享频谱市场中是否会出现类似的串通行为，以及其普遍性如何取决于竞争模式（价格与数量）和MARL算法的选择。我们的结果揭示了学习动态、市场结构与频谱共享之间的相互作用，并对无线市场设计和学习驱动决策系统的部署具有启示意义。

CALOS: Control-Affine Lyapunov On-manifold Safety Layer for Safe Deep Reinforcement Learning for Quadrotors

CALOS：Control-仿射李雅普诺夫流管安全层，用于四旋翼的深度强化安全学习

• Authors: Fabrizio Cesareo, Sebastiano Mengozzi, Nicola Mimmo, Andrea Acquaviva
• Subjects: Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2609.17758
• Pdf link: https://arxiv.org/pdf/2609.17758
• Abstract Deep Reinforcement Learning has demonstrated remarkable capability in quadrotor control, yet learned policies offer no guarantee of respecting safety constraints during training or deployment. We present CALOS (Control-Affine Lyapunov On-manifold Safety), a runtime safety layer that enforces attitude constraints on a quadrotor without modifying the underlying learning algorithm. CALOS formulates four tilt-angle inequalities and a Lyapunov descent condition as a single quadratic program whose solution is the minimum-norm correction to the nominal torque output of the policy. The quadratic program is solved exactly via active-set enumeration over the three-dimensional torque space, with a computational cost low enough to enforce constraints in real time across thousands of parallel simulation environments, as required by modern massively parallel Deep Reinforcement Learning training. Evaluated on trajectory-tracking tasks in NVIDIA Isaac Lab, CALOS reduces lateral tracking error by 55-60% relative to an unconstrained Proximal Policy Optimization baseline while achieving zero attitude-constraint violations on the training trajectory. By restricting exploration to safe regions of the state space, the safety layer also accelerates training convergence and improves data efficiency without producing suboptimal policies.
• 中文摘要 深度强化学习在四旋翼控制方面展现了卓越的能力，但学习策略无法保证在训练或部署过程中遵守安全约束。我们提出了CALOS（控制-仿射里雅普诺夫开流形安全），这是一个运行时安全层，可在不修改底层学习算法的情况下强制四旋翼的姿态约束。CALOS将四个倾斜角不等式和一个李雅普诺夫下降条件设为单一二次程序，其解为策略标称扭矩输出的最小范数修正。二次程序通过三维扭矩空间上的主动集枚举精确求解，计算成本足够低，能够在数千个并行模拟环境中实时强制约束，符合现代大规模并行深度强化学习训练的要求。在NVIDIA Isaac实验室的轨迹跟踪任务中，CALOS相较于无约束的近端策略优化基线，横向跟踪误差降低了55-60%，同时在训练轨迹上实现零姿态约束违规。通过限制探索在状态空间的安全区域，安全层还加快了训练收敛，提高了数据效率，同时不产生次优策略。

AgenTeeth: A Model-Agnostic Framework for Suppressing Hallucination in Frozen Vision-Language Models on Dental X-Rays via Tool Evidence Injection

AgenTeeth：一种模型无关框架，通过工具证据注入，在牙科X光片上抑制冻结视觉语言模型中的幻觉

• Authors: Ahmed Rafid, Fariya Ahmed, Rumman Adib, Mehedi Ahamed, Ajwad Abrar, Tareque Mohmud Chowdhury
• Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2609.17800
• Pdf link: https://arxiv.org/pdf/2609.17800
• Abstract Vision-language models (VLMs) remain largely unreliable on panoramic dental radiographs and can rely on learned anatomical priors rather than evidence in the image. This is particularly problematic for tooth localization and spatial reasoning, and fine-tuned dental VLMs can retain the same spatial biases. We present AgenTeeth, a model-agnostic, tool-augmented framework that grounds frozen VLMs using seven specialized dental vision experts. A question-aware orchestrator selects the relevant tools, whose detections are mapped to FDI tooth numbers or anatomical regions and returned as structured findings together with annotated image overlays. A fresh synthesis call then answers the question using this evidence, without fine-tuning the underlying VLM. On MMOral-OPG-Bench, AgenTeeth improves four backbone VLMs by 12.9-23.0 percentage points over their baselines. Our strongest configuration reaches 65.66% on open-ended VQA, compared with 45.35% for OralGPT-Plus. The advantage also holds at matched scale: a frozen Qwen2.5-VL-7B-Instruct with AgenTeeth reaches 48.11%, exceeding OralGPT-Plus built on the same backbone after supervised fine-tuning and reinforcement learning for tool use. We release the framework, all seven expert models, and a dentist-annotated dataset for alveolar bone-loss detection in panoramic radiographs.
• 中文摘要 视觉语言模型（VLMs）在全景牙科X光片上仍然大多不可靠，且可能依赖已学到的解剖先验而非图像中的证据。这对牙齿定位和空间推理尤其困难，而精细调优的牙科VLM可能仍保留相同的空间偏向。我们介绍AgenTeeth，一个模型无关、工具增强的框架，利用七位专业牙科视觉专家为冻结VLM提供基础。一个有问能力的编排器选择相关工具，其检测结果映射到FDI牙齿数量或解剖区域，并以结构化发现及注释图像叠加返回。然后通过新的综合调用，利用这些证据回答问题，而无需微调底层VLM。在MMOral-OPG-Bench上，AgenTeeth比基线提升了四个骨干VLM12.9-23.0个百分点。我们在开放式VQA中最强配置达到65.66%，而OralGPT-Plus为45.35%。这一优势在匹配尺度上同样适用：冷冻的Qwen2.5-VL-7B-Instruct配合AgenTeeth可达48.11%，超过基于同一骨干的OralGPT-Plus，经过监督微调和工具使用强化学习。我们发布了框架、全部七个专家模型及一个由牙医注释的全景X光片齿槽骨流失检测数据集。

The Free Inference Dimension: Complexity Measure for Zero-Collision Navigation under Hypothesis Mixtures

自由推断维度：假设混合下零碰撞导航的复杂度量

• Authors: Luiz Carlos Castro Guedes, Edward Hermann Haeusler
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2609.17816
• Pdf link: https://arxiv.org/pdf/2609.17816
• Abstract Solomonoff induction frames prediction as a mixture over computable hypotheses, typically leading to identification of the true environment. In a finite meta-reinforcement learning setting with nested constraint families, in our previous work, we observe a different regime: a value-mixture (VM) agent achieves near-optimal, zero-collision navigation without identifying the true environment, a phenomenon we call Free Inference. This regime persists up to a sharp density threshold, beyond which performance degrades and posterior-mode selection (PMS) becomes preferable. We formalize this behavior via the Free Inference dimension dFI(S,N), a combinatorial measure of the environmental complexity a VM agent can handle while preserving trajectory coherence. We prove dFI is strictly smaller than the VC-dimension and relates to the Natarajan dimension up to a path-length factor, capturing the cost of non-decomposable loss. A PAC-style relaxation yields generalization bounds driven by dFI^(epsilon,delta). We also define a complementary PMS identification dimension and show that a hybrid strategy---averaging until the first collision, then switching to selection---is optimal, with links to Littlestone-type dimensions supported by grid-world experiments.
• 中文摘要 所罗门诺夫归纳将预测框架为可计算假设的混合，通常导致真实环境的识别。在带有嵌套约束族的有限元强化学习环境中，我们之前的工作观察到不同的模式：值混合（VM）代理在不识别真实环境的情况下实现近乎最优的零碰撞导航，我们称之为自由推断。该状态持续到一个锐利的密度阈值，超过该阈值性能下降，后验模式选择（PMS）变得更优。我们通过自由推断维数 dFI（S，N）形式化此行为，这是 VM 智能体在保持轨迹一致性的环境复杂度的组合度量。我们证明dFI严格小于VC维数，并且在路径长度因子下与Natarajan维数相关，捕捉了不可分解损失的代价。PAC式松弛产生由dFi^（epsilon，delta）驱动的推广界限。我们还定义了一个互补的PMS识别维度，并证明混合策略---平均直到第一次碰撞，然后切换为选择性---是最优的，并结合网格世界实验支持的Littlestone类型维度。

Adaptive hybrid coupling with operator inference, the overlapping Schwarz alternating method and reinforcement learning

自适应混合耦合与算子推断、重叠施瓦茨交替法与强化学习

• Authors: Trishit Mondal, Irina Tezaur, Anthony Gruber
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Mathematical Physics (math-ph)
• Arxiv link: https://arxiv.org/abs/2609.17837
• Pdf link: https://arxiv.org/pdf/2609.17837
• Abstract Hybrid domain decomposition methods provide a flexible framework for coupling full order models (FOMs) and reduced order models (ROMs), but typically assume the model assigned to each subdomain is fixed throughout a simulation. This is limiting for transient problems in which localized features propagate through the domain and the regions requiring high-fidelity resolution change over time. We introduce a reinforcement learning (RL)-based approach for online adaptation of FOM-ROM models coupled via the overlapping Schwarz alternating method (O-SAM), an iterative domain decomposition method that solves subdomain-local problems while exchanging solution information through transmission boundary conditions on overlapping interfaces. Deep Q-networks (DQNs) are trained offline to select among subdomain-local FOMs and pre-trained Operator Inference (OpInf) ROMs using a reward balancing accuracy, cost, and model-switching frequency. Once trained, the policies are deployed predictively on problem instances not seen during training, without requiring a reference FOM solution. We demonstrate the approach on two examples: a 1D advection-diffusion problem with a moving front, and a 3D linear elastic wave propagation problem implemented in the this http URL solid mechanics code. For the advection-diffusion benchmark, the learned policy dynamically allocates high-fidelity resolution as the front propagates and outperforms static FOM/ROM assignments; letting the agent also adapt the domain decomposition provides no further benefit. For the elastic wave benchmark, learned policies for two and three subdomain decompositions track the propagating wave by assigning FOMs to subdomains containing the wave and ROMs elsewhere, as expected. Our results demonstrate the potential of RL to enable predictive online adaptation of model fidelity within Schwarz-based hybrid simulations.
• 中文摘要 混合域分解方法为耦合全序模型（FOM）和降序模型（ROM）提供了灵活框架，但通常假设分配给每个子域的模型在整个模拟过程中是固定的。这对瞬态问题存在限制，即局部特征会在域内传播，以及需要高保真分辨率随时间变化的区域。我们引入了基于强化学习（RL）的方法，用于通过重叠施瓦茨交替法（O-SAM）联动的FOM-ROM模型在线适配，这是一种迭代域分解方法，通过重叠接口上的传输边界条件交换解信息，解决子域-局部问题。深度Q网络（DQN）通过离线训练，利用奖励平衡精度、成本和模型切换频率，在子域本地FOM和预训练的操作员推断（OpInf）ROM之间进行选择。训练完成后，策略可预测地部署在训练中未见的问题实例，无需参考FOM解。我们在两个例子中演示了该方法：一个带有移动锋面的一维平流-扩散问题，以及一个在这 http URL 固态力学代码中实现的三维线性弹性波传播问题。对于平流-扩散基准测试，学习的策略动态分配高保真分辨率，因为前锋传播并优于静态 FOM/ROM 分配;允许代理同时适应域分解则无额外益处。对于弹性波基准测试，针对二和三子域分解的学习策略通过将 FOM 分配给包含波和 ROM 的其他子域来跟踪传播波，符合预期。我们的结果展示了强化学习在基于施瓦茨的混合模拟中实现模型忠实度的预测性在线适配的潜力。

SFT or RL for Tool-Calling Agents? A Controlled Study Across Data, Method, and Scale

工具调用代理的SFT还是RL？一项跨数据、方法和规模的受控研究

• Authors: Md Tahmid Rahman Laskar, Xue-Yong Fu, Shashi Bhushan TN
• Subjects: Subjects: Computation and Language (cs.CL)
• Arxiv link: https://arxiv.org/abs/2609.17848
• Pdf link: https://arxiv.org/pdf/2609.17848
• Abstract Limited controlled evidence exists on how training data, adaptation method, and model scale jointly affect tool-calling performance in language-model agents. We evaluate supervised fine-tuning (SFT) with LoRA, reinforcement learning (RL) via Group Relative Policy Optimization (GRPO), and SFT followed by GRPO across six Qwen3 models from 0.6B to 32B parameters, covering both in-distribution performance and cross-dataset transfer. SFT with LoRA is the strongest in-distribution method throughout the 0.6B-32B range and best in 15 out of 18 experimental settings. On cross-dataset transfer, the methods are closer: GRPO wins 29 out of 54 settings where training and test datasets differ, but its margin over SFT averages under one point, and SFT->GRPO is rarely strongest in either comparison. Dataset mixing gives consistently strong transfer while staying close to specialized in-distribution training, regardless of method. Additional analysis further confirms that LoRA outperforms full-parameter fine-tuning, demonstrating that LoRA better preserves pretrained agentic behavior.
• 中文摘要 目前有限的受控证据表明训练数据、适应方法和模型规模如何共同影响语言-模型代理的工具调用性能。我们评估了LoRA的监督微调（SFT）、通过群相对策略优化（GRPO）进行强化学习（RL），以及涵盖分布内性能和跨数据集转移的六个Qwen3模型中的SFT及GRPO。在0.6B至32B范围内，使用LoRA的SFT是分布内最强的方法，在18个实验场景中15个中表现最佳。在跨数据集转移方面，两方法更接近：GRPO在54个训练与测试数据集差异的环境中赢得29个，但其相较于SFT平均值的差距低于1个，SFT->GRPO在任何比较中都很少最强。无论采用何种方法，数据集混合都能持续提供强力转移，同时保持接近专门的分布内训练。进一步分析进一步证实，LoRA的表现优于全参数微调，表明LoRA更能保留预训练的代理行为。

Anchoring What Matters: A Dual-Level Learning Framework for Visually-Grounded Multimodal Reasoning

锚定重要因素：一个基于视觉基础的多模态推理的双层次学习框架

• Authors: Xinxin Song, Siyuan Li, Tingxiong Xiao, Jinli Suo
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2609.18057
• Pdf link: https://arxiv.org/pdf/2609.18057
• Abstract Reinforcement learning with verifiable rewards (RLVR) has significantly improved the reasoning capabilities of large vision-language models (LVLMs). However, standard on-policy RLVR algorithms face a critical optimization bottleneck in preserving and reinforcing visually grounded reasoning behaviors: valuable visually-grounded reasoning trajectories are discarded after a single update, while uniform token advantage allocation prevents the model from reinforcing critical perception or reasoning steps. To bridge this gap, we propose PIVOT, a dual-level learning framework that anchors policy optimization around informative visual reasoning signals. Specifically, PIVOT introduces a self-calibrated experience replay mechanism, which selectively collects and replays visually-grounded historical experiences as stable reference anchors for policy optimization. Building upon this, we further design a vision-guided advantage allocation mechanism to allocate additional vision-aware advantages to tokens based on their local visual support and impact on downstream reasoning. Extensive experiments across diverse benchmarks demonstrate that PIVOT achieves highly competitive performance in enhancing the multimodal reasoning capabilities of LVLMs.
• 中文摘要 带有可验证奖励的强化学习（RLVR）显著提升了大型视觉语言模型（LVLM）的推理能力。然而，标准的策略上RLVR算法在保留和强化视觉基础推理行为方面面临关键优化瓶颈：有价值的视觉基础推理轨迹在一次更新后被丢弃，而统一的令牌优势分配则防止模型强化关键感知或推理步骤。为弥合这一差距，我们提出了PIVOT这一双层学习框架，将策略优化锚定于信息性视觉推理信号。具体来说，PIVOT引入了自校准的体验回放机制，选择性地收集和重放基于视觉的历史经验，作为策略优化的稳定参考锚点。基于此，我们进一步设计了一种视觉引导优势分配机制，根据代币的局部视觉支持和对后续推理的影响，为其分配额外的视觉感知优势。跨越多种基准测试的广泛实验表明，PIVOT在提升LVLM多模态推理能力方面取得了极具竞争力的性能。

Mask 2D-3D: Adaptive Dual-Masked Autoencoder Network for Image-to-Point Cloud Registration

掩码2D-3D：用于图像到点云注册的自适应双掩盖自编码网络

• Authors: Zhixin Cheng, Jiacheng Deng, Xiaotian Yin, Baoqun Yin, Richang Hong, Tianzhu Zhang
• Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2609.18088
• Pdf link: https://arxiv.org/pdf/2609.18088
• Abstract Detection-free methods for image-to-point cloud registration are prone to erroneous correspondences caused by domain and modality discrepancies, limited sensitivity of feature extractors, and the presence of non-overlapping regions. The Masked Autoencoder (MAE) has shown strong performance in visual representation for images and point clouds. It may be helpful to apply this approach to image-to-point cloud registration, a task that requires unified feature extraction and accurate cross-modal correspondences. Standard MAE's random masking may overlook key regions due to limited camera views, reducing registration effectiveness. To address this, we propose the Intermodal Dual-MAE Framework (ID-MAE) with a Similarity-based RL Masking Strategy (SRLM), which adaptively masks informative positions by leveraging cross-modal similarity and reinforcement learning, thus narrowing the modality gap. Our method enhances cross-modal representation learning by enforcing representation consistency during feature extraction, thereby enabling more reliable 2D-3D correspondence estimation. Experiments on RGB-D Scenes v2 and 7-Scenes benchmarks show that our method achieves state-of-the-art performance in image-to-point cloud registration.
• 中文摘要 无检测的图像到点云配准方法容易因域域和模态差异、特征提取器灵敏度有限以及非重叠区域存在而产生错误对应。掩盖自编码器（MAE）在图像和点云的视觉表现方面表现出优异表现。将该方法应用于图像到点云配对可能有帮助，该任务需要统一特征提取和准确的跨模态对应。标准MAE的随机遮罩可能因摄像机视角有限而忽略关键区域，降低配准效果。为此，我们提出了采用基于相似度的强化学习掩蔽策略（SRLM）的模态双重MAE框架（ID-MAE），通过利用跨模态相似性和强化学习自适应掩蔽信息位置，从而缩小模态差距。我们的方法通过在特征提取过程中强制表示一致性，增强了跨模态表示学习，从而实现更可靠的二维-三维对应估计。RGB-D Scenes v2 和 7-Scenes 基准测试的实验表明，我们的方法在图像到点云配准方面实现了最先进的性能。

Fetch My Beer: Synthetic-to-real Hierarchical Policy for Smooth Pick-and-place

“取我的啤酒”：合成到真实的层级政策，实现顺畅的挑选与放置

• Authors: Yingyue Li, Chenyangguang Zhang, Ruida Zhang, Bowen Fu, Guangyao Zhai, Xiangyang Ji
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18119
• Pdf link: https://arxiv.org/pdf/2609.18119
• Abstract Many real-world robotic applications require dynamically sensitive manipulation, where success depends not only on reaching a target state but on maintaining stable object dynamics throughout execution. We study the stable transport of liquid-filled containers, where a robot must move objects to target locations while suppressing sloshing and preventing spillage. Unlike conventional pick-and-place, this task imposes stringent requirements on motion smoothness and trajectory-level stability, exposing clear limitations in existing systems. Specifically, fluid simulation remains too costly for online reinforcement learning; human teleoperation introduces unintended accelerations that induce sloshing during imitation learning; and current policy pipelines optimize for task completion rather than dynamic stability. We propose a synthetic-to-real framework coupling physically validated data generation with a hierarchical, diffusion-based controller. The scalable data pipeline synthesizes grasps, filters unstable poses via a vision-language model, and validates transport trajectories through fluid simulation. The policy is organized with a high-level module that translates language and visual observations into SE(3) control targets, and a latent diffusion controller that first plans efficiently in a compact latent space and then decodes dense action chunks, enabling the high control frequency needed for smooth and stable motion. Extensive experiments show our system outperforms state-of-the-art manipulation policies in transport smoothness and dynamic stability. Our project page: this https URL
• 中文摘要 许多现实世界的机器人应用需要动态敏感操作，成功不仅取决于达到目标状态，还要在执行过程中保持稳定的物体动力学。我们研究液体容器的稳定运输，机器人必须在抑制晃动和防止溢出的同时将物体移动到目标位置。与传统的拾取放位不同，该任务对运动的平滑性和轨迹级稳定性有严格要求，暴露了现有系统的明显局限。具体来说，流体仿真对在线强化学习来说仍然过于昂贵;人类远程操作引入了非预期的加速，导致模拟学习过程中出现晃动;当前的策略流水线更注重任务完成而非动态稳定性。我们提出了一个合成到实际的框架，将物理验证的数据生成与基于扩散的分层控制器结合起来。可扩展的数据流水线综合抓取，通过视觉语言模型过滤不稳定姿态，并通过流体仿真验证传输轨迹。该策略由一个高级模块组织，将语言和视觉观测数据转换为SE（3）控制目标，以及一个潜在扩散控制器，先在紧凑的潜在空间中高效规划，然后解码密集动作块，实现实现平稳运动所需的高控制频率。大量实验表明，我们的系统在传输平滑性和动态稳定性方面优于最先进的操作策略。我们的项目页面：此链接

DualSQL: Text-to-SQL with Multi-Agent Reinforcement Learning

DualSQL：多智能体强化学习的文本转SQL。

• Authors: Shijie Chen, Yu Gan, Yeounoh Chung, Jiani Zhang, Quannan Li, Sravan Babu Bodapati, Cody J. Greer, Yu Su, Fatma Ozcan
• Subjects: Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Databases (cs.DB)
• Arxiv link: https://arxiv.org/abs/2609.18135
• Pdf link: https://arxiv.org/pdf/2609.18135
• Abstract State-of-the-art Text-to-SQL systems are typically multi-agent pipelines centered around two fundamental tasks: schema linking and SQL generation. However, existing work trains separate models for each task, failing to leverage the synergy between these interrelated tasks. In this work, we propose DualSQL, a new Text-to-SQL system consisting of two agents powered by a single model backbone. The agents share the same model weights and agentic scaffold, enabling joint optimization through a robust multi-agent reinforcement learning (RL) framework. We design three database access tools to facilitate effective multi-step reasoning grounded to interactions with the databases. To improve training and avoid model collapse, we introduce a set of rollout guardrail mechanisms that stabilizes multi-agent RL training, supporting DualSQL to keep improving during training. We also introduce a new SQL correctness metric, robust execution match (REX), to more accurately judge SQL correctness and assign reward signals. Being trained on only 3755 examples, DualSQL-4B achieves an impressive 68.0% execution accuracy on the BIRD development set, matching previous 7B models. DualSQL-8B further improves to 71.1%, outperforming previous state-of-the-art single-model solutions with 32B parameters. These results demonstrate the strength of joint multi-agent reinforcement learning for building high performance Text-to-SQL pipelines.
• 中文摘要 最先进的文本转SQL系统通常是围绕两个基本任务：模式链接和SQL生成的多代理管道。然而，现有工作为每个任务训练独立模型，未能充分利用这些相互关联任务之间的协同效应。本研究提出DualSQL，一种由两个代理组成、由单一模型骨干驱动的文本转SQL系统。这些代理共享相同的模型权重和代理支架，通过强大的多代理强化学习（RL）框架实现联合优化。我们设计了三种数据库访问工具，以促进基于数据库交互的多步有效推理。为改善训练并避免模型崩溃，我们引入一套部署护栏机制，稳定多智能体强化学习，支持DualSQL在训练过程中持续改进。我们还引入了新的SQL正确性指标——稳健执行匹配（REX），以更准确地判断SQL正确性并分配奖励信号。仅基于3755个样本训练，DualSQL-4B在BIRD开发集上实现了令人印象深刻的68.0%执行准确率，与之前的7B模型相当。DualSQL-8B进一步提升至71.1%，优于以往采用32B参数的先进单模型解决方案。这些结果展示了联合多代理强化学习在构建高性能文本转SQL流水线方面的优势。

Characterizing Replay Retention Under Dynamics Shift in Model-Based Reinforcement Learning

模型型强化学习中动态转变下的重放保留特性

• Authors: Everest Yang, Skye Thompson, George D. Konidaris
• Subjects: Subjects: Robotics (cs.RO); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2609.18167
• Pdf link: https://arxiv.org/pdf/2609.18167
• Abstract Adapting to changes in robot dynamics requires learning from new data without discarding experience that may still be useful. In continual model-based reinforcement learning (RL), replay collected before a dynamics change can slow adaptation, while removing it unnecessarily reduces available training data and can be especially costly if earlier dynamics return. We study when recent transitions are preferable to the full replay history. Two quantities characterize this trade-off: change magnitude and age-staleness area under the curve (AUC), measuring how well transition age separates stale from fresh data. Forgetting stale data helps after large permanent shifts but hurts when dynamics recur and older data becomes useful again. Choosing a replay strategy therefore depends on predicting when older data will help or hurt. We test these effects across two locomotion morphologies, two model-based RL algorithms, and Real-World RL benchmark perturbations. Because ground-truth staleness labels are unavailable on deployed robots, we evaluate whether an estimator built from interaction data can still provide the quantities needed to choose a replay strategy after permanent changes. Our results show that replay retention depends on change magnitude and on how the dynamics evolve.
• 中文摘要 适应机器人动力学变化需要从新数据中学习，同时不舍弃仍可能有用的经验。在基于模型的持续强化学习（RL）中，动态变化前收集的回放会减缓适应速度，同时移除回放会不必要地减少可用训练数据，且如果早期动态回归，代价尤其高昂。我们研究何时近期的转变优于完整回放历史。两个量表征这种权衡：变化幅度和曲线下的年龄陈旧面积（AUC），衡量过渡年龄如何区分陈旧与新数据。忘记陈旧数据有助于在长时间永久性变化后出现，但当动态反复出现且旧数据再次有用时则有害。因此，选择重放策略依赖于预测旧数据何时会帮助或有害。我们在两种运动形态、两种基于模型的强化学习算法和现实现实强化学习基准扰动中测试这些效应。由于部署机器人无法提供地面真实性陈旧标签，我们评估基于交互数据构建的估计器是否仍能提供选择重放策略所需的量，尤其是在永久性变化之后。我们的结果表明，重放保留取决于变化的大小以及动态的演变。

Reinforcement Learning for Real-Time Vision-Language-Action Policies

实时视觉-语言-行动政策的强化学习

• Authors: Perry Dong, Kuo-Han Hung, Dorsa Sadigh, Chelsea Finn
• Subjects: Subjects: Robotics (cs.RO); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2609.18207
• Pdf link: https://arxiv.org/pdf/2609.18207
• Abstract Reinforcement learning fine-tuning on top of large, pretrained Vision-Language-Action (VLA) models offers promise for highly reliable robot deployment. However, because of their scale, modern VLA models suffer from high inference latency, so the observation used to select an action is often stale by execution time, creating a distribution shift that can substantially degrade reliability and performance. Prior work has explored asynchronous policy execution to reduce the effect of latency, but these methods are mostly built on imitation learning and offer no mechanism for moving beyond the training distribution toward higher reliability. We close this gap by enabling RL fine-tuning that meets the real-time control requirements of dynamic real-world manipulation. Our approach builds on EXPO-FT, a framework for sample-efficient, reliable VLA fine-tuning with reinforcement learning, and decouples slow, expressive action generation from fast, reactive action edits: a large pretrained VLA proposes action chunks using its strong behavior prior, while a lightweight edit policy performs fast, reactive decision-making by editing actions in response to changes in state, conditioned on the latest observation. We instantiate this as Real-Time EXPO-FT, an RL framework for finetuning real-time VLA policies. On the Kinetix benchmark, Real-Time EXPO-FT enables a delayed policy to achieve the best performance among delayed and non-delayed methods in 10 out of 10 environments. On four dynamic real-world tasks, robot object passing, ball balancing, table soccer kicking, and dynamic object picking, with online robot data capped at 10 minutes, Real-Time EXPO-FT improves average policy performance from 42% to 97%, all without human intervention, demonstrating rapid, sample-efficient adaptation to challenging real-world dynamics. Website: this https URL
• 中文摘要 基于大型预训练的视觉-语言-动作（VLA）模型进行强化学习微调，有望实现高度可靠的机器人部署。然而，由于规模较大，现代VLA模型存在较高的推理延迟，因此用于选择动作的观察常因执行时间而变得陈旧，导致分布偏移，显著降低可靠性和性能。此前的研究曾探索异步策略执行以减少延迟影响，但这些方法大多基于模仿学习，没有提供超越训练分布、追求更高可靠性的机制。我们通过实现符合动态现实操作实时控制需求的强化学习微调来弥补这一差距。我们的方法基于EXPO-FT框架，该框架用于样本高效、可靠的VLA微调，并结合强化学习，将缓慢、表达性的动作生成与快速反应性编辑解耦：大型预训练VLA利用其强先验行为提出动作块，而轻量级编辑策略则通过根据最新观察条件，响应状态变化来快速、反应性地做出决策。我们将此实例化为Real-Time EXPO-FT，一个用于微调实时VLA策略的强化学习框架。在Kinetix基准测试中，Real-Time EXPO-FT使延迟策略在10种环境中在延迟和非延迟方法中表现最佳。在四项动态现实任务中：机器人物体传球、球平衡、桌上足球踢球和动态物体挑选，在线机器人数据限制为10分钟，实时EXPO-FT将平均政策性能从42%提升至97%，且均无需人工干预，展现了对复杂现实动态的快速且高效适应能力。网站：此 https URL

APGEM: Adaptive Policy-Guided Error Mitigation for Quantum Reinforcement Learning on a Real-World CVRP Case Study

APGEM：基于真实世界CVRP案例研究的自适应策略引导错误缓解量子强化学习

• Authors: Shabir Ahmad Sofi, Bisma Majid, Mir Mohammad Yousuf
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Emerging Technologies (cs.ET)
• Arxiv link: https://arxiv.org/abs/2609.18219
• Pdf link: https://arxiv.org/pdf/2609.18219
• Abstract Quantum Reinforcement Learning (QRL) represents policies as variational quantum circuits (VQCs), making it attractive for combinatorial optimization such as the Capacitated Vehicle Routing Problem (CVRP). On noisy intermediate-scale quantum (NISQ) hardware, however, decoherence degrades fidelity and destabilizes learning, and conventional error mitigation is applied statically without regard to the learning context. We introduce Adaptive Policy-Guided Error Mitigation (APGEM), a controller that selects among Zero-Noise Extrapolation (ZNE), Probabilistic Error Cancellation (PEC), Clifford Data Regression (CDR), and Readout Error Mitigation (REM) online, driven by a fidelity, entropy, and cost aware utility function and an epsilon-greedy rule over temporal-difference Q-scores. We evaluate on a realistic urban-logistics testbed, a Delhi-based CVRP over real landmarks with geodesic inter-node costs, exercised across five noise families and four severity levels. On this instance, the QRL agent outperforms constructive heuristics and approaches metaheuristics, while mitigation restores approximation ratios from 0.84-0.87 to 0.92-0.94 under high noise. The controller shifts from a CDR-dominated regime under short training horizons to a balanced deployment across all four techniques under longer horizons, indicating genuine regime-dependent selection. These preliminary results position adaptive, learning-aware mitigation as a practical route to noise-resilient QRL.
• 中文摘要 量子强化学习（QRL）将策略表示为变分量子电路（VQC），因此非常适合如电容车辆路由问题（CVRP）等组合优化。然而，在噪声高的中级量子（NISQ）硬件上，退相干会降低准确度并使学习不稳定，传统错误缓解则静态应用，不考虑学习上下文。我们介绍了自适应策略引导错误缓解（APGEM），这是一种控制器，在线选择零噪声外推（ZNE）、概率错误消除（PEC）、Clifford数据回归（CDR）和读出错误缓解（REM），由保真度、熵和成本感知的效用函数以及对时间差Q分数的ε贪婪规则驱动。我们在一个现实的城市物流测试平台——位于德里的CVRP上，对具有测地线节点间成本的真实地标进行了评估，该测试涵盖五个噪声家族和四个严重程度级别。在此情况下，QRL代理优于建设性启发式方法，采用元启发式方法，而缓解在高噪声下将近似比从0.84-0.87恢复到0.92-0.94。控制器在短训练视野下从CDR主导的状态转变为在较长视野下四种技术均均衡部署，表明存在真正的依赖性选择。这些初步结果使得自适应、学习意识的缓解成为实现噪声韧性QRL的实用途径。

MiST: Mid-Training LLMs for Cybersecurity

MiST：网络安全中期培训LLMs

• Authors: Oded Ovadia, Elad Ben Zaken, Elad Guttman, Orly Moreno Kadosh
• Subjects: Subjects: Cryptography and Security (cs.CR); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2609.18496
• Pdf link: https://arxiv.org/pdf/2609.18496
• Abstract Cybersecurity combines high-stakes analysis with complex technical language, making it an impactful and challenging domain for LLMs. We present MiST (Mid-trained Security Transformer), a suite of 8B and 32B models that achieve strong performance on public cybersecurity benchmarks. We use mid-training as an intermediate adaptation stage between general pre-training and cybersecurity training. Rather than performing continual pre-training over large volumes of raw domain text, we curate a compact, expert-vetted seed corpus, and transform it into high-quality domain-specific synthetic training data. The final MiST checkpoints improve mean cybersecurity accuracy by +13.1 and +8.6 absolute percentage points over the corresponding Qwen baselines for 8B and 32B, respectively, corresponding to relative gains of +27.0% and +15.8%. Ablation results further show that these cybersecurity gains arise in the mid-training and supervised fine-tuning stages through a combination of the synthetic data generation flows. Furthermore, we show that MiST provides a stronger initialization for downstream task-specific fine-tuning adaptation and reinforcement learning.
• 中文摘要 网络安全将高风险分析与复杂的技术语言相结合，使其成为大型语言模型（LLM）具有影响力且具有挑战性的领域。我们展示了MiST（中训安全转换器），这是一套8B和32B模型，在公共网络安全基准测试中表现出色。我们将中期培训作为通用预训练与网络安全培训之间的过渡阶段。我们不再对大量原始领域文本进行持续的预训练，而是精心策划一个紧凑、专家审核的种子语料库，并将其转化为高质量的领域特定合成训练数据。最终的MiST检查点分别比8B和32B的Qwen基线提升平均网络安全准确率+13.1和+8.6绝对百分点，分别提升了+27.0%和+15.8%。消融结果进一步表明，这些网络安全提升主要发生在中期训练和监督微调阶段，通过合成数据生成流程的组合实现。此外，我们证明MiST为下游任务特定的微调适应和强化学习提供了更强的初始化。

GroundingVLN: Reasoning and Acting with Grounding for Vision-Language Navigation

GroundingVLN：基于接地的推理与行动，进行视觉语言导航

• Authors: Kailing Li, Yu Han, Tianwen Qian, Yuqian Fu, Jingyu Gong, Jiangming Shi, Xiaoling Wang
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18581
• Pdf link: https://arxiv.org/pdf/2609.18581
• Abstract Although vision-language models (VLMs) possess strong visual understanding and reasoning capabilities, existing vision-and-language navigation (VLN) agents struggle to connect semantic reasoning with spatial execution. Two coupled gaps remain in this connection, as intermediate reasoning is not explicitly anchored to visual evidence and high-level decisions lack precise spatial goals to guide low-level motion. Cognitive science suggests that human navigation bridges these levels hierarchically by anchoring cognition to relevant landmarks and guiding locomotion toward spatial goals. Motivated by this principle, we propose GroundingVLN, which uses visual grounding as a shared interface between reasoning and action. GroundingVLN first reasons with grounding by anchoring task-relevant visual evidence to precise image locations throughout structured reasoning. It then acts through grounding by predicting a progress-aligned pixel goal that a geometric planner translates into primitive actions. To learn these capabilities, we construct GroundingCOTVLN-188K, a dataset of temporally aligned grounded reasoning traces, and introduce Grounded and Execution-Aware Reinforcement Learning (GEAR), which aligns grounded reasoning and spatial decisions with downstream execution. Experiments demonstrate that GroundingVLN achieves state-of-the-art performance (69.9% SR on R2R-CE and 75.1% SR on RxR-CE) with high sample efficiency, using just 0.9% as much training data as the strongest baseline. It also generalizes strongly across datasets, attaining 59.9% SR on RxR-CE when trained solely on R2R, a gain of 20.1% over the strongest baseline.
• 中文摘要 尽管视觉语言模型（VLMs）具备强大的视觉理解和推理能力，现有的视觉与语言导航（VLN）智能体在将语义推理与空间执行连接起来时遇到了困难。这一联系中存在两个耦合的空白，中间推理未明确锚定于视觉证据，且高层决策缺乏精确的空间目标来指导低层次运动。认知科学认为，人类导航通过将认知锚定于相关地标并引导移动朝空间目标方向，在层级上架起这些桥梁。基于这一原则，我们提出了GroundingVLN，利用视觉接地作为推理与行动之间的共享接口。GroundingVLN首先通过将任务相关的视觉证据锚定到结构化推理中的精确图像位置来推理。然后通过接地来预测一个与进展对齐的像素目标，几何规划者将其转化为原始动作。为了解这些能力，我们构建了GroundingCOTVLN-188K，一个时间对齐的基准推理痕迹数据集，并引入了基于基准且执行感知强化学习（GEAR），使基准推理和空间决策与下游执行对齐。实验表明，GroundingVLN在使用仅为最强基线0.9%的训练数据时，采样效率高，实现了最先进的性能（R2R-CE为69.9%，RxR-CE为75.1%）。它在不同数据集间具有强烈的泛化性，仅在R2R训练时，RxR-CE的SR提升了59.9%，比最强基线提升20.1%。

Label-free steering: Compressing test-time reinforcement learning into bias-only subspaces

无标签引导：将测试时间强化学习压缩到仅有偏见的子空间

• Authors: Naveen Vakada, Mingyuan Li, Shaoxiong Ji
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2609.18587
• Pdf link: https://arxiv.org/pdf/2609.18587
• Abstract Test-time reinforcement learning (TTRL) enables models to improve their reasoning without relying on labeled training data, but existing approaches typically optimize a large fraction of the model parameters. This raises a natural question: can effective test-time adaptation emerge when both the reward signal and the optimization space are severely restricted? We answer this question with label-free bias-only TTRL, which uses majority-vote pseudo-labels as rewards and optimizes only approximately 100K bias parameters while keeping the pretrained backbone frozen. On MATH-500, our approach reaches 76.67% accuracy, slightly exceeding our own labeled bias-steering reproduction while optimizing 76,000x fewer parameters than full-parameter TTRL. The same training procedure improves performance across vision-language and audio reasoning tasks, including MathVista, AI2D, LogicVista, and MMAU. We further show that the learned steering vectors transfer to 4,500 held-out MATH problems, indicating that the adaptation is not limited to the problems used during test-time optimization. Finally, we analyze why this highly restricted adaptation can work, showing that majority-vote reliability improves with rollout consensus and that bias subspaces with greater accessible gradient energy exhibit stronger downstream trainability. These results demonstrate that substantial test-time adaptation can emerge from optimizing a tiny bias-only subspace using entirely label-free rewards.
• 中文摘要 测试时强化学习（TTRL）使模型能够在不依赖带标签训练数据的情况下改进推理能力，但现有方法通常优化了大量模型参数。这自然引出了一个问题：当奖励信号和优化空间都受到严重限制时，有效的测试时间适应能否实现？我们用无标签偏见仅用的TTRL来回答这个问题，该方法使用多数票伪标签作为奖励，仅优化约10万个偏见参数，同时保持预训练骨干保持冻结。在MATH-500中，我们的方法准确率达到76.67%，略高于我们自身的带标签偏向引导再现，同时优化的参数数量比全参数TTRL少76,000倍。相同的训练过程提升了视觉-语言和音频推理任务的性能，包括MathVista、AI2D、LogicVista和MMAU。我们进一步证明，所学到的引导向量可转移至4500个未完成的数学问题，表明适应性不限于测试时间优化期间的问题。最后，我们分析了这种高度受限的适应为何可行，表明多数票可靠性随着展开共识而提升，且具有更大梯度能量的偏置子空间展现出更强的下游可训练性。这些结果表明，利用完全无标签的奖励优化极小的仅有偏差子空间，可以带来显著的测试时间适应。

CoRe-MARL: Cooperative Redistribution Under Unknown Dynamics Using Recurrent Multi-Agent Reinforcement Learning

CoRe-MARL：利用循环多智能体强化学习在未知动力学下的合作再分布

• Authors: Naimur Rahman Chowdhury, Shatabdi Sen Prapti, Md. Salehin Seyam, Limon Bin Hossain
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2609.18639
• Pdf link: https://arxiv.org/pdf/2609.18639
• Abstract Emergency management assistance programs, such as relief distribution, are essential for delivering necessary supplies to affected communities. However, these programs operate in a decentralized network of local centers that face uncertain local demand and supply dynamics, resulting in inconsistent avail- ability of local services. Redistribution of supplies among these local centers reduces these imbalances, but the centers often make decisions independently, with limited information and disrupted transportation. This study develops CoRe-MARL, a cooperative multi-agent reinforcement learning (MARL) framework, by formulating a decentralized partially observable Markov decision process (Dec-POMDP). We treat each center as an agent that learns a redistribution policy to improve the service in the worst-case region and reduce the service gap across regions while protecting network-wide service. We incorporate a recurrent network that captures evolving supply and demand dynamics without direct observation, while multi-agent proximal policy optimization (MAPPO) enables centralized training and decentralized execution (CTDE). We evaluate the framework in a simulated environment with diverse trajectories, where exact dynamics are not observed by actors and the MAPPO critic. We compare the recurrent MAPPO with the recurrent independent PPO (IPPO) and a local only heuristic, and find that MAPPO reduces the service gap across local centers and enhances service for the worst-served center while maintaining competitive network-wide service. The recurrent MAPPO also shows consistent performance across diverse trajectory patterns, demonstrating its ability to adapt to evolving dynamics. The findings demonstrate the capability of cooperative learning for decentralized redistribution and improving equitable service under uncertain and evolving dynamics.
• 中文摘要 应急管理援助项目，如救援分发，对于向受影响社区运送必要物资至关重要。然而，这些项目运行在分散的本地中心网络中，面临不确定的本地需求和供给动态，导致本地服务的可用性不一致。物资在这些地方中心之间重新分配减少了这些不平衡，但这些中心通常独立决策，信息有限且交通受阻。本研究通过制定去中心化、部分可观测的马尔可夫决策过程（Dec-POMDP），开发了CoRe-MARL，一种合作式多智能体强化学习（MARL）框架。我们将每个中心视为一个代理，学习重新分配政策以改善最坏情况的服务，减少区域间服务差距，同时保护网络范围的服务。我们采用了循环网络，捕捉不断变化的供需动态，无需直接观察，而多代理近端策略优化（MAPPO）则实现集中训练和去中心化执行（CTDE）。我们在模拟环境中评估该框架，环境轨迹多样，参与者和MAPPO批评者未观察到精确动态。我们将重复MAPPO与循环独立PPO（IPPO）及仅本地启发式比较，发现MAPPO缩小了本地中心间的服务差距，提升了服务最差中心的服务，同时保持了竞争性的全网络服务。循环MAPPO还在多样化轨迹模式中表现出稳定表现，展示了适应不断变化动态的能力。研究结果展示了协作学习在去中心化再分配和改善不确定和动态变化下公平服务的能力。

STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolution

拓展边界：一个统一的自学框架，促进渐进式大型语言模型演进

• Authors: Yajie Yu, Mark Lee, Yue Feng
• Subjects: Subjects: Computation and Language (cs.CL)
• Arxiv link: https://arxiv.org/abs/2609.18642
• Pdf link: https://arxiv.org/pdf/2609.18642
• Abstract Large language models (LLMs) often suffer from capability stagnation in self-improvement training because fixed difficulty levels fail to adapt to their evolving proficiency. To address this issue, we propose STRETCH (Self-Taught Reasoning Evolution via Targeted CHallenge), a unified framework inspired by cognitive scaffolding theory. STRETCH introduces a dynamic Stretch Zone mechanism that continuously aligns question difficulty with the model's solving capability. Within a single parameter space, the model alternates between a Scaffolder that generates adaptive, boundary-pushing challenges and a Learner that that optimizes its solving trajectories through reinforcement learning. This dual-loop co-evolution effectively stabilizes training, mitigates reward hacking and promote progressive reasoning growth. Experiments on both negotiation and operation research benchmarks demonstrate that STRETCH consistently outperforms strong prompting and domain-specific baselines. Further scaffolder configuration analysis shows that dynamic difficulty alignment is critical for sustained capability improvement and synchronized reasoning evolution.
• 中文摘要 大型语言模型（LLM）在自我提升训练中常常面临能力停滞的问题，因为固定难度等级无法适应其不断演进的熟练度。为解决这一问题，我们提出了STRETCH（通过目标挑战实现自学推理演化），这是一个受认知支架理论启发的统一框架。STRETCH引入了动态的拉伸区机制，持续将问题难度与模型的求解能力对齐。在单一参数空间内，模型交替出现了产生自适应、突破边界挑战的支架和通过强化学习优化其解决轨迹的学习者。这种双循环共进化有效稳定了训练，减轻了奖励黑客现象，并促进了渐进式推理的成长。在协商和运筹学基准测试上的实验表明，STRETCH始终优于强提示和领域特定基线。进一步的支架构型分析表明，动态难度对持续能力提升和同步推理演进至关重要。

FIERCE: From Generalist Robot Policies to Fast Specialists via Progress-Failure Feedback

Fierce：从通才机器人政策到通过进步-失败反馈快速专家

• Authors: Runjia Tan, Yuang Tu, Yujie Yan, Lan Yu, Xuesong Tian, Chen Lv
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18651
• Pdf link: https://arxiv.org/pdf/2609.18651
• Abstract Generalist robot policies offer useful initialization, but refining compact specialists through limited physical interaction requires informative learning feedback. We present FIERCE, a generalist-initialized reinforcement learning framework centered on a unified, task-adaptive progress-failure evaluator. Its architecture shares an observation-language representation between an observed-progress head and an action-conditioned latent predictor whose past and current predictions feed a causal sequence head for task-failure estimation. Joint supervision from progress and preference labels, synchronized commands and observations, and terminal outcomes trains the evaluator; target-task rollouts support adaptation and calibration. Fixed evaluator snapshots provide progress shaping and failure-risk penalties alongside independently verified terminal rewards, while evaluator and policy updates alternate as new experience is collected. Refinement requires neither continued generalist action queries nor a dedicated target-task simulator or manually annotated dense rewards. Only the compact specialist is retained at deployment. The evaluation separates feedback quality, policy-learning efficiency, and deployment cost across simulation and two contact-rich real tasks. Code, model weights, and data-restoration tools are released at this https URL.
• 中文摘要 通用机器人策略提供了有用的初始化，但通过有限的物理互动来优化紧凑型专家需要信息性学习反馈。我们介绍FIERCE，这是一个以统一任务自适应进度-失败评估器为核心的通用初始化强化学习框架。其架构共享观察语言表示，分别是观察进展头和动作条件潜在预测器，过去和当前的预测为因果序列头提供任务失败估计。进度和偏好标签、同步指令与观察以及终端结果的联合监督培训评估者;目标任务推广支持适应和校准。固定评估者快照提供进展塑造和失败风险惩罚，同时提供独立验证的终端奖励，评估者与策略更新交替进行。精细化既不需要持续的通用动作查询，也不需要专用的目标任务模拟器或手动注释的密集奖励。部署时仅保留紧凑型专家。评估将反馈质量、策略学习效率和部署成本区分给仿真和两个接触丰富的真实任务。代码、模型权重和数据恢复工具均发布于此 https URL。

Learning to Program Adaptive Non-Local Observables for Machine Learning

学习编程自适应非局域可观测量以用于机器学习

• Authors: Yu-Ting Lee, Samuel Yen-Chi Chen, Huan-Hsin Tseng
• Subjects: Subjects: Machine Learning (cs.LG); Quantum Physics (quant-ph)
• Arxiv link: https://arxiv.org/abs/2609.18655
• Pdf link: https://arxiv.org/pdf/2609.18655
• Abstract Quantum neural networks (QNNs) are typically built from variational quantum circuits (VQCs), which are limited by local measurements. Adaptive non-local observables (ANO) address this by jointly optimizing circuit parameters and multi-qubit measurements. However, existing ANO-based VQCs learn only a single static observable that remains invariant across all inputs. We propose QFWP-ANO, a novel architecture which employs a classical hypernetwork to dynamically program VQC parameters and/or non-local observables conditioned on each input. On multivariate time-series forecasting across four ETT datasets, QFWP-ANO achieves the lowest MSE in 16 of 20 settings and second-lowest in the remaining four, surpassing ANO-based and other strong baselines. On reinforcement learning tasks, QFWP-ANO consistently surpasses ANO-VQCs. Our results establish input-conditioned ANO as an effective approach for enhancing QNNs.
• 中文摘要 量子神经网络（QNN）通常由变分量子电路（VQC）构建，VQC受限于局部测量。自适应非局部可观测量（ANO）通过联合优化电路参数和多量子比测量来解决这一问题。然而，现有基于ANO的VQC仅学习一个静态可观测量，且在所有输入中保持不变。我们提出了QFWP-ANO架构，这是一种新颖架构，利用经典超网络动态编程VQC参数和/或非局部可观测量，条件为每个输入。在跨4个ETT数据集的多变量时间序列预测中，QFWP-ANO在20个设置中有16个实现了最低的MSE，在剩余4个设置中排名第二，超过了基于ANO的及其他强基线。在强化学习任务中，QFWP-ANO持续超过ANO-VQCs。我们的结果确立了输入条件抗性向比诺作为增强QNNs的有效方法。

M$^3$P-R1: Reinforcement Learning for Large Language Model Guided Multi-Modal Motion Planning via MIP Code Generation

M$^3$P-R1：通过MIP码生成实现大型语言模型引导多模态运动规划的强化学习

• Authors: Xingpeng Sun, Zherong Pan, Kai Cheng, Xindi Tang, Syed Talha Bukhari, Aniket Bera
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18669
• Pdf link: https://arxiv.org/pdf/2609.18669
• Abstract Multi-Modal Motion Planning (M$^3$P) requires joint reasoning over continuous motions and discrete mode transitions, making it difficult to solve efficiently. For instance, a bipedal robot may walk to a target location and then use its arms to grasp an object. This scenario captures both mode transitions and continuous dynamics, yielding feasible paths that neither purely discrete nor continuous planners can handle. While Mixed-Integer Programming (MIP) offers a principled framework, constructing tractable formulations for non-convex problems is typically manual and domain-specific, especially in the approximate, discretization-based MIP regime needed for non-convex robotic tasks. We propose M$^3$P-R1, a reinforcement learning method that fine-tunes large language models (LLMs) to decompose M$^3$P tasks into MIP variables, constraints, and objectives. Instead of directly outputting answers, which are often prone to hallucination, the model generates executable Python code using MIP optimization libraries and constraint interfaces. This enables solver-backed execution for robust and verifiable solutions. Trained with an outcome-driven reward against the solver, M$^3$P-R1 learns to compose modality-level discretization primitives and synthesize cross-modal coupling constraints, producing executable MIP programs for complex M$^3$P tasks.
• 中文摘要 多模态运动规划（M$^3$P）需要对连续运动和离散模式转换进行联合推理，这使得高效求解变得困难。例如，一台双足机器人可能走到目标位置，然后用手臂抓住物体。该场景既捕捉了模式转变，也包含连续动力学，产生了纯离散或连续规划者都无法处理的可行路径。虽然混合整数规划（MIP）提供了一个有原则的框架，但构建非凸问题的可处理表述通常是手工且领域特定的，尤其是在非凸机器人任务所需的近似离散化MIP体系中。我们提出了M$^3$P-R1，一种强化学习方法，通过微调大型语言模型（LLM）将M$^3$P任务分解为MIP变量、约束和目标。模型不直接输出容易产生幻觉的答案，而是利用MIP优化库和约束接口生成可执行的Python代码。这使得求解器支持的执行实现了稳健且可验证的解。在以结果驱动的奖励下，M$^3$P-R1学习组合模态级离散化原语并合成跨模态耦合约束，生成复杂M$^3$P任务的可执行MIP程序。

Voice of Reason: Reinforcement Learning for Spoken Math

理性之声：口语数学的强化学习

• Authors: Timothée Weisselberger, Edouard Graves, Alexandre Défossez
• Subjects: Subjects: Computation and Language (cs.CL)
• Arxiv link: https://arxiv.org/abs/2609.18677
• Pdf link: https://arxiv.org/pdf/2609.18677
• Abstract Speech language models enable richer spoken interactions between humans and machines than cascaded systems, allowing access to paralinguistic information and lower latency. However, their accuracy on mathematical reasoning benchmarks has lagged behind those of text models. Reinforcement learning (RL) with verifiable rewards has been instrumental in extending text models' capabilities for solving complex problems and limiting hallucinations. In this work, we explore applying RL to the GLM-4-Voice speech model (Zeng et al., 2024) to bridge the gap between textual and spoken mathematical problem solving. We first adapt the model to the domain using supervised fine-tuning on synthesized spoken question-answering data. We then show that, even without extra reasoning tokens, RL improves the accuracy on GSM8K beyond levels previously achieved for speech models only with supplementary reasoning traces. When combined with existing streaming reasoning techniques, we show further gains to 74.8% free-form accuracy. This establishes a new state-of-the-art for mathematical spoken abilities with speech-native models.
• 中文摘要 语音语言模型比级联系统更能实现人机间的口语互动，允许访问副语言信息并降低延迟。然而，它们在数学推理基准测试中的准确性落后于文本模型。带有可验证奖励的强化学习（RL）在扩展文本模型解决复杂问题和限制幻觉的能力方面发挥了重要作用。本研究中，我们探讨将强化学习应用于GLM-4语音语音模型（Zeng 等，2024），以弥合文本与口语数学问题解决之间的差距。我们首先通过对合成口语问答数据进行监督微调，将模型适配到该领域。随后我们展示了，即使没有额外的推理标记，RL也能在GSM8K上提升到超越以往仅靠补充推理痕迹的语音模型所达到的水平。结合现有的流式推理技术，我们进一步提升了74.8%的自由形式准确率。这为数学口语能力与语音原生模型建立了新的领先水平。

WeaveRL: Weaving Reconstruction into Scene-Aware Fabrics for Perceptive Reinforcement Learning

WeaveRL：将重建编织进场景感知织物中，以实现感知强化学习

• Authors: Remo Steiner, Vikram Ramasamy, David Tingdahl, Sam Mady, Karl Van Wyk, Nathan Ratliff, David Recasens Lafuente, Soha Pouya, Tuur Stuyck, Alex Millane
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18685
• Pdf link: https://arxiv.org/pdf/2609.18685
• Abstract Reinforcement learning allows robots to acquire complex skills, but producing policies for geometrically complex manipulation remains difficult. A promising approach is to learn on top of collision-avoidant controllers, such as geometric fabrics. However, these approaches have relied on static, hand-specified representations of the scene. Integrating active, online 3D perception into massively parallel RL training has so far been inaccessible. We introduce a GPU-accelerated method that reconstructs the scene as a collection of surfels across thousands of parallel simulation instances during active rollouts. This lets policies operate over sensor-derived, rather than hand-specified, geometry. On a suite of collision-dense manipulation tasks, our surfel fabrics enable policies to tackle geometrically complex scenes where primitive-based baselines fail, while maintaining sim-to-real transfer. Furthermore, policies learned with a scene-aware fabric are more robust to the introduction of novel geometry at test time, improving collision-free task completion under unseen obstacles from 35% to 61%. We release our reconstruction system, training code and test dataset to spur research in this direction.
• 中文摘要 强化学习使机器人能够获得复杂技能，但制定几何复杂操作策略仍然困难。一种有前景的方法是基于碰撞避免控制器（如几何织物）进行学习。然而，这些方法依赖静态、手工指定的场景表示。将主动在线3D感知整合进大规模并行强化学习训练迄今为止尚不可及。我们引入了一种GPU加速方法，在主动部署期间将场景重建为数千个并行模拟实例中的多个冲动单元集合。这使得策略能够在传感器衍生的几何体上运行，而非手工指定的几何体。在一系列碰撞密集的操作任务中，我们的冲波结构能够应对原始基线失效的几何复杂场景，同时保持模拟到实的传输。此外，采用场景感知结构学习的策略对测试时引入新几何结构更具韧性，将无碰撞任务完成率从35%提升至61%。我们发布重建系统、训练代码和测试数据集，以推动相关研究。

Rethinking Critic Learning in PPO: Understanding and Mitigating Value Flattening

重新思考PPO中的批评者学习：理解并缓解价值平坦化

• Authors: Yizhuo Li, Jianhao Yan, Yun Luo, Zhi Wang, Futing Wang, Rong-Xi Tan, Kanghui Tian, Ganqu Cui, Ning Ding, Peilin Zhao, Yafu Li, Yu Cheng
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2609.18708
• Pdf link: https://arxiv.org/pdf/2609.18708
• Abstract In reinforcement learning for large language models, Proximal Policy Optimization (PPO) commonly uses a critic to estimate state values and reduce the variance of policy updates. However, we uncover a systematic failure mode in PPO critics, which we call Value Flattening: state values, estimated from multiple Monte Carlo continuations, change sharply across intermediate states while critic predictions remain comparatively flat. We further observe this phenomenon in a controlled FrozenLake environment and find that it becomes more pronounced as the state space grows. Our theoretical and empirical analyses relate Value Flattening to an implicit variance penalty in the critic loss and redundant updates from temporally correlated states with similar gradients. Motivated by these findings, we introduce SParse Proximal Policy Optimization (SP$^3$O), which applies the value loss to only a few well-separated states in each response to mitigate both effects. Experiments on Qwen3-Base show that SP$^3$O with only three states supervised per response can mitigate Value Flattening and consistently improve the learned policy across model sizes and evaluation suites. Together, our results identify Value Flattening as an important yet overlooked failure mode of critic learning in standard PPO and show that a simple sparse supervision strategy can mitigate it.
• 中文摘要 在大型语言模型的强化学习中，近端策略优化（PPO）通常使用批判者来估计状态值并减少策略更新的方差。然而，我们发现PPO批评者存在一种系统性的失败模式，我们称之为“价值平坦化”：通过多个蒙特卡洛延拓估计的状态值在中间状态间发生剧烈变化，而批判者预测则相对平稳。我们在受控的FrozenLake环境中观察到这一现象，发现随着状态空间的增长，这种现象变得更加明显。我们的理论和实证分析将价值平坦化与批评者损失的隐性方差惩罚以及来自具有相似梯度的时间相关状态的冗余更新联系起来。基于这些发现，我们引入了SParse Proximal策略优化（SP$^3$O），该方法仅将价值损失应用于每个响应中少数几个分离良好的状态，以减轻这两种影响。Qwen3-Base上的实验显示，SP$^3$O且每个响应仅监督三个状态，可以减轻价值平坦化，并持续提升模型规模和评估套件中学习策略的质量。综合来看，我们的结果将价值平坦化识别为标准PPO中一个重要但被忽视的失败模式，并表明一个简单的稀疏监督策略可以缓解该问题。

CERA-MoA: Co-Evolving Routing Mechanisms with Continually Learning LLM Agents

CERA-MoA：与持续学习的大型语言模型代理共同演进的路由机制

• Authors: Jiaxuan Jiang, Liyuan He, Zhixuan Fang
• Subjects: Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2609.18779
• Pdf link: https://arxiv.org/pdf/2609.18779
• Abstract Current Mixture-of-Agents (MoA) paradigms generally treat query routing and agent fine-tuning as separate processes, limiting their ability to respond to evolving agent capabilities. This disconnect prevents routing strategies from adapting to evolving agent capabilities during post-training and prevents agents from achieving synergistic data-driven specialization. To resolve this, we introduce CERA-MoA (Co-Evolving Router with continually learning Agents for Mixture-of-Agents), an iterative reinforcement learning framework where the dynamic router and independent agent policies co-evolve. We design a predictive familiarity estimator that leverages mid-layer hidden states to evaluate semantic competence among agents, avoiding the overhead of full rollouts. Based on these familiarity scores, a cumulative-threshold adaptive routing mechanism dynamically activates a tailored minimal agent subset, achieving a trade-off between task performance and efficiency. By proactively allocating targeted training samples to agents based on their evolving competence, CERA-MoA promotes capability differentiation. Extensive experiments across various domains demonstrate that CERA-MoA outperforms state-of-the-art static-agent routing and fix-workflow fine-tuning baselines.
• 中文摘要 当前的代理混合（MoA）范式通常将查询路由和代理微调视为独立过程，限制了它们对代理能力演变的响应能力。这种脱节阻碍了路由策略在训练后适应代理能力演变，也阻碍代理实现协同数据驱动的专业化。为解决这一问题，我们引入了CERA-MoA（与持续学习的代理混合代理共演化路由器），这是一种迭代强化学习框架，动态路由器与独立代理策略共同演化。我们设计了一个预测熟悉度估计器，利用中层隐藏状态评估代理间的语义能力，避免全面部署的开销。基于这些熟悉度评分，一个累积阈值自适应路由机制动态激活定制的最小代理子集，实现任务性能与效率之间的权衡。通过主动根据代理不断发展的能力分配有针对性训练样本，CERA-MoA促进了能力差异化。跨多个领域的大量实验表明，CERA-MoA优于最先进的静态代理路由和修复工作流程微调基线。

KINO: A Keyframe Interface for VLM Planning and Whole-Body Control in Humanoid Loco-Manipulation

KINO：用于人形机车操控中VLM规划和全身控制的关键帧接口

• Authors: Sitong Chen, Fatemeh Zargarbashi, Jin Cheng, Tianxu An, Stelian Coros
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18869
• Pdf link: https://arxiv.org/pdf/2609.18869
• Abstract Humanoid loco-manipulation requires robots to interpret task instructions and scene semantics while executing coordinated whole-body motions. We propose a hierarchical framework that uses motion keyframes as an intermediate representation between Vision-Language Model (VLM) planning and Reinforcement Learning (RL) control. Each keyframe specifies a target whole-body robot pose and, when applicable, an object pose. Given a language instruction, scene observations, and execution feedback, the VLM selects successive task-relevant keyframes from a predefined library. The selected keyframes are retargeted to the current scene to account for object poses and dimensions. A keyframe-conditioned whole-body policy then generates joint-level actions to reach these goals. We introduce a saliency-based keyframe sampling strategy for low-level policy training that improves end-to-end task success rate from 44% to 92% when using sparse VLM keyframes. We evaluate our framework on object pickup, transport, and placement tasks in simulation and on a Unitree G1 humanoid. The system successfully performs both one- and two-handed manipulation and generalises to placement locations beyond the training reference data.
• 中文摘要 类人机车操作要求机器人在执行协调的全身运动时，解释任务指令和场景语义。我们提出了一个层级框架，使用运动关键帧作为视觉语言模型（VLM）规划与强化学习（RL）控制之间的中间表示。每个关键帧指定一个目标全身机器人姿势，并在适用时指定一个对象姿态。给定语言指令、场景观察和执行反馈后，VLM从预定义库中选择连续的任务相关关键帧。选定的关键帧会重新定向到当前场景，以考虑物体姿势和尺寸。随后，关键帧条件的整体策略生成联合层级动作以实现这些目标。我们引入基于显著性的关键帧采样策略，用于低层策略训练，使用稀疏VLM关键帧时，端到端任务成功率从44%提升至92%。我们在模拟和Unitree G1人形机器人中评估了该框架在物体拾取、传输和放置任务上的应用。该系统成功完成单手和双手操作，并能推广到训练参考数据之外的放置位置。

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator

使用双足移动机械臂学习整体全身运动操作

• Authors: Zhongyu Chen, Yuxuan Nai, Qian Chen, Yidong Zhu, Chen Jing, Qihan Wang, Xudong Li, Zhizhan Li, Leixin Chang, Liangjing Yang, Hua Chen
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18930
• Pdf link: https://arxiv.org/pdf/2609.18930
• Abstract Bipedal loco-manipulation enables robots to interact with objects beyond the nominal workspace of their arms by coordinating locomotion and manipulation. Realizing this capability requires a low-level whole-body controller that translates task-level manipulation goals into coordinated arm and leg motions while maintaining balance. We present a unified whole-body controller trained with reinforcement learning that directly maps 6-DoF end-effector targets to coordinated actions for the bipedal base and robotic arm. Given only an end-effector target, the learned controller autonomously coordinates reaching, postural adaptation, and stepping without explicit base-velocity or footstep commands. A reward-gating strategy regulates the trade-offs among end-effector tracking, locomotion, and balance during training, while a temporal context estimator combines windowed Transformer encoding, recurrent GRU memory, and auxiliary dynamics prediction to extract dynamics-relevant information from observation history. Real-robot experiments demonstrate that the same controller supports reaching, postural adaptation, and stepping under commands from VR teleoperation, a learned diffusion policy, and scripted trajectories, providing a common end-effector interface for diverse manipulation tasks.
• 中文摘要 双足操作使机器人能够通过协调运动和操作，与手臂名义工作空间之外的物体互动。实现这一能力需要一个低级别的全身控制器，将任务级操作目标转化为协调的手臂和腿部动作，同时保持平衡。我们展示了一个经过强化学习训练的统一全身控制器，能够直接将6度的末端执行器目标映射到双足基部和机械臂的协调动作。仅给定一个端效器目标，学习后的控制器能够自主协调伸手、姿势适应和步进，无需明确的基底速度或脚步指令。奖励门控策略调节训练期间末端执行器跟踪、运动和平衡之间的权衡，而时间上下文估计器结合了窗口变换器编码、循环GRU记忆和辅助动力学预测，从观察历史中提取与动力学相关的信息。真实机器人实验表明，同一控制器支持VR遥控操作、学习扩散策略和脚本轨迹下的伸手、姿势适应和步进，为多种操作任务提供了通用的终端执行器接口。

FedGuide: Diffusion Prior Alignment and Value Baseline Guidance for Heterogeneous Federated Reinforcement Learning

FedGuide：异构联合强化学习的扩散先验对齐与价值基线指导

• Authors: Zhilin He, Gauri Joshi
• Subjects: Subjects: Machine Learning (cs.LG); Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18964
• Pdf link: https://arxiv.org/pdf/2609.18964
• Abstract Federated Reinforcement Learning (FRL) enables collaborative policy learning across distributed agents with heterogeneous environments. While recent methods based on variance reduction, divergence penalization, and momentum optimization improve FRL under heterogeneous settings, they still primarily synchronize policy or value-network parameters and do not explicitly address distributional mismatch among heterogeneous clients. Therefore, we propose \textbf{FedGuide}, a FRL framework that uses diffusion priors as behavior models to provide personalized data supported distributions for heterogeneous local policy learning. Instead of directly averaging local policies, FedGuide aggregates those diffusion priors through Optimal-Transport Mixture-of-Experts (OT-MoE), preserving heterogeneous behavior modes in distribution space. It further develops a Distribution Correction Estimation (DICE) value baseline to provide low-variance, return-aware guidance for local policy improvement. Experiments across heterogeneous environments show that FedGuide outperforms representative FRL methods in client-average returns, final-round performance, and worst-round robustness, while maintaining stable learning under stronger heterogeneity.
• 中文摘要 联合强化学习（FRL）使分布式代理间的协作策略学习成为可能，且环境异质。虽然基于方差减少、发散惩罚和动量优化的最新方法在异构环境中改进了FRL，但它们仍主要同步策略或价值网络参数，并未明确解决异构客户端之间的分布不匹配。因此，我们提出了\textbf{FedGuide}，这是一个FRL框架，利用扩散先验作为行为模型，为异构局部政策学习提供个性化数据支持的分布。FedGuide不直接平均局部策略，而是通过最优传输专家混合（OT-MoE）聚合这些扩散先验，保持分布空间中的异质行为模式。它进一步开发了分布修正估计（DICE）价值基线，提供低方差、回报意识的本地政策改进指导。跨异构环境的实验显示，FedGuide在客户平均回报、最终轮表现和最差轮稳健性方面优于代表性的FRL方法，同时在更强异质性下保持稳定学习。

Loco-Loco-RL: Low-Cost Terrain Mapping for Humanoid Locomotion with Reinforcement Learning

Loco-Loco-RL：低成本的类人机动地形映射与强化学习

• Authors: Jordan Dowdy, Gryffin Reizian, Jean Chagas Vaz
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.19041
• Pdf link: https://arxiv.org/pdf/2609.19041
• Abstract Informative terrain perception is important for robust reinforcement learning policies in humanoid locomotion. Still, common sensors such as depth cameras and LiDARs incur high cost, power, and processing overhead while often producing redundant, high-resolution data. This work uses a low-cost time-of-flight sensor to provide a compact 3D local terrain representation for humanoid locomotion. To efficiently use this sparse exteroceptive input, we introduce a token-compressed temporal transformer policy. Proprioceptive and terrain observations are tokenized and processed by a self-attention multi-head transformer to capture within-timestep relationships between observation terms. The attended tokens are then compressed through an MLP-based latent-space token compression module before being stored in a rolling 15-timestep history. A second cross-attention multi-head transformer extracts temporal locomotion features from this compact history for policy learning. By compressing tokens before temporal aggregation, the architecture preserves important terrain-observation structure while limiting the dimensional growth of attention over observation histories. We validate our method through sim-to-real transfer on physical hardware using a terrain-based locomotion benchmark, demonstrating robust humanoid terrain walking with low-cost local terrain sensing.
• 中文摘要 信息型地形感知对于人形运动中稳健的强化学习策略至关重要。然而，常见传感器如深度相机和激光雷达（LiDAR）存在高成本、功耗和处理开销，且常产生冗余的高分辨率数据。该工作利用低成本飞行时间传感器为类人机动提供紧凑的3D局部地形表示。为高效利用这种稀疏的外感知输入，我们引入了令牌压缩的时间变换器策略。本体感知和地形观测被自我注意多头变换器分区并处理，以捕捉观测项之间的时间步长关系。受关注的令牌随后通过基于MLP的潜空间令牌压缩模块压缩，并存储在滚动的15步时间步历史中。第二个交叉注意多头变换器从该紧凑历史中提取时间运动特征，用于策略学习。通过在时间聚合前压缩标记，该架构保留了重要的地形观测结构，同时限制了注意力对观测历史的维度增长。我们通过在物理硬件上使用基于地形的移动基准测试进行模拟到现实传输验证了我们的方法，展示了低成本的本地地形感测下，实现了稳健的人形地形行走。

Integrated Optimization of Automated Warehouse Operations and Last-Mile Transport for Differentiated On-Demand Delivery

自动化仓库运营和最后一公里运输的综合优化，实现差异化的按需配送

• Authors: Xiaozhu Sun, Bilal Farooq
• Subjects: Subjects: Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2609.19048
• Pdf link: https://arxiv.org/pdf/2609.19048
• Abstract In the context of differentiated on-demand goods delivery services, this study proposes an integrated optimization method for automated guided vehicles (AGVs) based smart warehouse operations and the last-mile multi-modal transport. A deep reinforcement learning algorithm for multi-objective joint scheduling is designed to establish a dynamic connection between two systems, solving key challenges such as achieving high-throughput continuous order scheduling, meeting competing requirements, and improving the overall system sensitivity and adaptability. For warehouse optimization within this framework, we propose an improved algorithm based on multi-objective, Multi-Reward Machines-A* Guided Deep Q-Network (MORM-AGDQN), which combines service level, system cost, and external transportation demand. For external optimization, we propose an improved algorithm based on a Multi-Reward, Multi Head attention-Heterogeneous Capacity Vehicle Routing Problem (MRMH-HCVRP) framework, which incorporates the optimized scheduling order sequence and grouping, combined with customer location, demand, and priority, vehicle capacity, speed, and service range. The results show that the proposed framework significantly outperforms traditional methods, achieving 100% on-time delivery rate for warehousing operations. After joint optimization, the average delivery time for the last mile was reduced by 29.3% to 53.2%, the total transportation distance was reduced by 46.4%, the high-priority service rate was increased to over 92%, and a balance was maintained between operating costs and customer satisfaction.
• 中文摘要 在差异化按需货物配送服务的背景下，本研究提出了一种基于自动导引车辆（AGV）的智能仓库运营和最后一公里多模式运输的综合优化方法。设计了一个用于多目标联合调度的深度强化学习算法，旨在建立两个系统之间的动态连接，解决实现高通量连续订单调度、满足竞争需求以及提升整体系统敏感性和适应性等关键挑战。在该框架下进行仓库优化，我们提出了基于多目标多奖励机-A*引导深度Q网络（MORM-AGDQN）的改进算法，结合了服务水平、系统成本和外部运输需求。在外部优化方面，我们提出了基于多奖励多头注意力异构容量车辆路由问题（MRMH-HCVRP）框架的改进算法，结合优化的调度顺序和分组，结合客户位置、需求与优先级、车辆容量、速度和服务范围。结果显示，所提出的框架显著优于传统方法，实现仓储作业的100%准时交付率。联合优化后，最后一英里的平均配送时间减少了29.3%，降至53.2%，总运输距离缩短了46.4%，高优先级服务率提升至92%以上，运营成本与客户满意度保持平衡。

RLLBC-Lib: An Educational Code Library for Reinforcement Learning and Learning-Based Control

RLLBC-Lib：一个用于强化学习和基于学习控制的教育代码库

• Authors: Bernd Frauenknecht, Emma Cramer, Artur Eisele, Paul Kruse, Lukas Kesper, Jonas Hertrampf, Ramil Sabirov, Jyotirmaya Patra, Johannes Berger, Paul Brunzema, Friedrich Solowjow, Sebastian Trimpe
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2609.19074
• Pdf link: https://arxiv.org/pdf/2609.19074
• Abstract Reinforcement learning (RL) is an exciting concept as well as a remarkable success story worth sharing. However, RL builds on rather complex interactions between different objects that play out over several cycles. Such dynamics are often best explained with an easily accessible implementation. We present RLLBC-Lib, a carefully crafted code library with the goal of lowering the entry barrier for students and other learners of RL in the context of learning-based control. At its heart, RLLBC-Lib comprises a comprehensive library of tabular RL approaches to enforce a clear understanding of the theoretical foundations. A deep RL library follows the same design principles, underscoring the parallels between simple tabular and state-of-the-art deep RL approaches. Additionally, RLLBC-Lib provides a collection of implementations illustrating core RL principles and contrasting RL to other learning-based control approaches. Finally, RLLBC-Lib provides an ideal basis for creating programming assignments with automated grading.
• 中文摘要 强化学习（RL）是一个令人兴奋的概念，也是一个值得分享的非凡成功故事。然而，强化学习建立在不同对象之间相当复杂的交互之上，这些交互会在多个周期内展开。这种动态通常最好用易于理解的实现来解释。我们介绍RLLBC-Lib，一个精心设计的代码库，旨在降低学生和其他学习者在基于学习的控制环境中的入门门槛。RLLBC-Lib的核心包含一个全面的表格式强化学习方法库，旨在强化对理论基础的清晰理解。深度强化学习库遵循相同的设计原则，强调简单表格式与最先进深度强化学习方法之间的相似性。此外，RLLBC-Lib提供了一系列实现，展示了核心强化学习原则，并对比强化学习与其他基于学习的控制方法。最后，RLLBC-Lib为自动评分的编程作业提供了理想的基础。

ScienceIDE: Turning World's Scientific Codebase into Agent Learnable Environments

ScienceIDE：将全球科学代码库转变为可智能体学习环境

• Authors: Hejia Geng, Zesen Huang, Haoyang Li, Wenbin Li, Koutian Wu, Zihan Zhou, Yuanbo Pang, Weihao Liu, Zigong Xu, Zhiping Li, Zongzheng Zhang, Chuanfei Dong, Jiankai Sun, Tianzhe Zheng, Fengyu Xie, Yue Ma, Yueheng Shi, Tong Xie, Zonglin Di, Xianrong Liu, Qucheng Gao, Yimin Liu, Jiaming Pan, Sheng Huang, Xiao-Han Ma, Lanqing Yuan, Zhenlin Zhu, Ziang Liu, Ziyang Xu, Junkai Wang, Kangkai Liang, Jiayi Xian, Zehong Zhao, Liuwei Xu, Jingxu Xie, Peijin Zhang, Qiang Gao, Chengyi Xing, Zhe Zhao, Xi Wang, Yaopeng Xing, Xing Meng, Zhenfei Yin, Yingcheng Wu, Ling Yang
• Subjects: Subjects: Computation and Language (cs.CL); Computers and Society (cs.CY)
• Arxiv link: https://arxiv.org/abs/2609.19134
• Pdf link: https://arxiv.org/pdf/2609.19134
• Abstract Scientific code repositories encode decades of human knowledge in executable models, methods, and tools. Yet fragmented toolchains, implicit domain conventions, and specialized correctness criteria make this knowledge difficult to convert into reliable learning experience-a challenge we call the scientific experience bottleneck. We introduce ScienceIDE, infrastructure for turning the world's scientific code into programmable environments for scientific agents. Guided by expert-defined scientific cases and acceptance criteria, agents transform repositories into executable environments that support task generation, execution, and scientific verification. These environments provide a shared foundation for supervised fine-tuning, reinforcement learning, and evaluation. Using verified interaction trajectories, we train PhAI-IDE-72B, PhAI-IDE-9B, and PhAI-IDE-4B. The model family shows gains in held-out scientific-code repair and across selected general-purpose benchmarks in code, reasoning, and knowledge, providing evidence of positive transfer from scientific experience to broader capabilities. ScienceIDE lays the foundation for an integrated workspace for agent learning and scientific practice, making humanity's scientific software a shared substrate for developing scientific intelligence. Code: this https URL
• 中文摘要 科学代码仓库将数十年的人类知识编码在可执行模型、方法和工具中。然而，碎片化的工具链、隐式领域惯例和专业的正确性标准使这些知识难以转化为可靠的学习体验——我们称之为科学体验瓶颈。我们引入了ScienceIDE，将全球科学代码转化为科学代理可编程环境的基础设施。在专家定义的科学案例和验收标准指导下，代理将仓库转化为支持任务生成、执行和科学验证的可执行环境。这些环境为监督式微调、强化学习和评估提供了共享基础。利用经过验证的交互轨迹，我们训练PhAI-IDE-72B、PhAI-IDE-9B和PhAI-IDE-4B。该模型家族在保留的科学代码修复以及部分通用基准中取得了进步，体现了从科学经验向更广泛能力的积极转移。ScienceIDE为智能体学习与科学实践的集成工作空间奠定了基础，使人类的科学软件成为科学智能发展的共享基底。代码：此 https URL

Keyword: diffusion policy

Vision-Language Grounded Task-Context-Aware Imitation Learning for Robotic Disassembly

视觉语言基础任务上下文感知模仿学习用于机器人拆解

• Authors: Jeon Ho Kang, Igal Tamarkin, Ethan Niu, Ian Novales, Satyandra K. Gupta
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.17714
• Pdf link: https://arxiv.org/pdf/2609.17714
• Abstract Real-world robotic disassembly requires long-horizon execution, where robots must perform ordered sequences of manipulation tasks across multiple parts within a single scene. Multiple valid task goals and diverse assembly configurations make it difficult for imitation policies to infer the intended skill from raw observations alone, particularly when training data cannot cover the combinatorial diversity of real-world configurations and part geometries. We show that incorporating task context through language alleviates these challenges by providing explicit structure for skill selection and associating language-specified tasks with their corresponding manipulation targets in the visual scene. The proposed framework combines hierarchical task selection with task-context-aware imitation learning to ground language instructions in spatial visual representations for robotic disassembly. The resulting framework generalizes across diverse connector geometries and assembly configurations without requiring explicit object annotations. Our method improves end-to-end task success by 35 percentage points over the baseline diffusion policy and by 75 percentage points over the previous task-context-aware baseline.
• 中文摘要 现实世界的机器人拆解需要长视野执行，机器人必须在单一场景的多个部件上执行有序的操作任务序列。多个有效任务目标和多样化的组装配置使模仿策略难以仅凭原始观察推断预期技能，尤其是在训练数据无法覆盖现实世界配置和零件几何的组合多样性时。我们展示了通过语言整合任务上下文，通过提供技能选择的明确结构，并将语言指定任务与其在视觉场景中的相应操作目标关联，从而缓解了这些挑战。提出的框架结合了层级任务选择与任务上下文感知的模仿学习，将语言指令扎根于空间视觉表示中，用于机器人拆解。最终框架能够推广到多种连接器几何和装配配置，而无需显式对象注释。我们的方法使端到端任务成功率比基线扩散策略高出35个百分点，较之前的任务上下文感知基线提升75个百分点。

Missing Bridges: Composition-Aware Active Imitation Learning

缺失的桥梁：构图感知主动模仿学习

• Authors: Maxwell J. Jacobson, Ahmed H Qureshi, Yexiang Xue
• Subjects: Subjects: Artificial Intelligence (cs.AI); Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18004
• Pdf link: https://arxiv.org/pdf/2609.18004
• Abstract Active imitation learning reduces expert effort by allowing a learner to request the demonstrations it needs. Existing methods typically select these requests for their expected information gain about the expert policy. In structured multi-task domains, however, the number of start-goal tasks may grow combinatorially despite their solutions sharing reusable behavior. This makes composable behaviors especially valuable, since a single demonstration may help solve many tasks at once. Prior methods do not explicitly account for this value when selecting which demonstration to request. We introduce Adaptive Agents via Latent Topologies (AALT), which requests demonstrations that maximize expected gains in start-goal connectivity. We further show that this objective is formally tied to information gain about task reachability. AALT organizes existing demonstrations into a topology of latent hub states connected by learned behaviors, identifies high-value bridge demonstrations that are likely to enable many tasks at once, and grounds each to an expert query. At inference, it plans through the resulting topology and conditions a diffusion policy on each successive hub transition. In a simulated UR5e robot ordered-retrieval domain with 72 tasks, AALT improved from 42/72 to 72/72 (100%) successful tasks consistently using only 3 demonstrations totaling 5 transitions beyond the initial dataset. After 20 demonstrations, the strongest baseline averaged 88.6% success using 98 transitions.
• 中文摘要 主动模仿学习通过允许学习者请求所需的演示，减少专家的工作量。现有方法通常选择这些请求以获得对专家策略的预期信息获取。然而，在结构化多任务领域，尽管起始-目标任务的解决方案共享可重用行为，但数量可能组合增长。这使得可组合行为尤为有价值，因为单一演示可能帮助一次性解决多个任务。以往方法在选择请求演示时未明确考虑该值。我们引入了通过潜在拓扑的自适应代理（AALT），该方法请求最大化起始-目标连接性预期收益的演示。我们还进一步证明，这一目标形式上与任务可达性的信息获取挂钩。AALT将现有演示组织为由学习行为连接的潜在枢纽状态拓扑，识别可能同时支持多个任务的高价值桥接演示，并对每个任务进行专家查询。推断时，它通过生成的拓扑和条件规划每个后续枢纽转换的扩散策略。在一个模拟UR5e机器人有序检索域中，包含72个任务，AALT成功率从42/72提升到72/72（100%），且仅使用3次演示，总计超过初始数据集5次转换。经过20次演示，使用98次转换，最强基线平均成功率为88.6%。

A Comprehensive Review of Generative Physical Artificial Intelligence

生成式物理人工智能的全面综述

• Authors: Satyam Gaba, Krutiksinh Rana, Siva Sai, Vinay Chamola, Dusit Niyato
• Subjects: Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2609.18111
• Pdf link: https://arxiv.org/pdf/2609.18111
• Abstract The integration of large-scale foundation models with physical embodiments has led to significant advancements in robotics known as Generative Physical Artificial Intelligence (GPAI). These agentic AI systems autonomously perceive, reason, and act in complex real-world situations. This survey comprehensively analyzes GPAI systems, focusing on their architectural foundations, current applications, and key limitations. We introduce a taxonomy of five distinct approaches: Robot Foundation Models (RFMs) for cross-platform skill transfer; Vision-Language Action (VLA) models for end-to-end multi-modal perception and control; Large Behavior Models (LBMs) for human-like movement generation; Diffusion Policy Models (DPMs) for diffusion model-based temporally coherent action generation; and World Foundation Models (WFMs) for physics-compliant simulation and data generation. We examine how these approaches complement each other: WFMs generate training data for VLAs and DPMs, RFMs enable cross-platform deployment of learned policies, while LBMs provide motion priors for natural behavior. Through examples across autonomous vehicles, industrial automation, healthcare robotics, and humanoid systems, we identify significant performance improvements and summarize promising research directions in data-efficient learning, sim-to-real transfer, edge-compatible architectures, and safety frameworks. These insights advance embodied AI for IoT-connected environments where intelligent agents interact with networked sensors, actuators, and edge devices.
• 中文摘要 大规模基础模型与物理实体的整合，推动了被称为生成式物理人工智能（GPAI）的机器人技术取得重大进展。这些智能人工智能系统能够自主感知、推理并在复杂的现实世界中行动。本综述全面分析了GPAI系统，重点关注其架构基础、当前应用及关键局限性。我们介绍了五种不同方法的分类法：机器人基础模型（RFM），用于跨平台技能转移;视觉语言动作（VLA）模型，用于端到端多模态感知与控制;大型行为模型（LBM），用于类人运动生成;扩散政策模型（DPMs），用于基于扩散模型的时间连贯动作生成;以及世界基础模型（WFMs），用于物理合规的仿真和数据生成。我们考察了这些方法如何互补：WFM生成VLA和DPM的训练数据，RFM支持跨平台部署所学策略，而LBM则提供自然行为的运动先验。通过跨自动驾驶、工业自动化、医疗机器人和类人系统，我们识别出显著的性能提升，并总结了数据高效学习、模拟到现实传输、边缘兼容架构和安全框架的有前景研究方向。这些洞见推动了物联网连接环境中的具身人工智能，智能代理与网络传感器、执行器和边缘设备交互。

Learning Holistic Whole-Body Loco-Manipulation with a Bipedal Mobile Manipulator

使用双足移动机械臂学习整体全身运动操作

• Authors: Zhongyu Chen, Yuxuan Nai, Qian Chen, Yidong Zhu, Chen Jing, Qihan Wang, Xudong Li, Zhizhan Li, Leixin Chang, Liangjing Yang, Hua Chen
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2609.18930
• Pdf link: https://arxiv.org/pdf/2609.18930
• Abstract Bipedal loco-manipulation enables robots to interact with objects beyond the nominal workspace of their arms by coordinating locomotion and manipulation. Realizing this capability requires a low-level whole-body controller that translates task-level manipulation goals into coordinated arm and leg motions while maintaining balance. We present a unified whole-body controller trained with reinforcement learning that directly maps 6-DoF end-effector targets to coordinated actions for the bipedal base and robotic arm. Given only an end-effector target, the learned controller autonomously coordinates reaching, postural adaptation, and stepping without explicit base-velocity or footstep commands. A reward-gating strategy regulates the trade-offs among end-effector tracking, locomotion, and balance during training, while a temporal context estimator combines windowed Transformer encoding, recurrent GRU memory, and auxiliary dynamics prediction to extract dynamics-relevant information from observation history. Real-robot experiments demonstrate that the same controller supports reaching, postural adaptation, and stepping under commands from VR teleoperation, a learned diffusion policy, and scripted trajectories, providing a common end-effector interface for diverse manipulation tasks.
• 中文摘要 双足操作使机器人能够通过协调运动和操作，与手臂名义工作空间之外的物体互动。实现这一能力需要一个低级别的全身控制器，将任务级操作目标转化为协调的手臂和腿部动作，同时保持平衡。我们展示了一个经过强化学习训练的统一全身控制器，能够直接将6度的末端执行器目标映射到双足基部和机械臂的协调动作。仅给定一个端效器目标，学习后的控制器能够自主协调伸手、姿势适应和步进，无需明确的基底速度或脚步指令。奖励门控策略调节训练期间末端执行器跟踪、运动和平衡之间的权衡，而时间上下文估计器结合了窗口变换器编码、循环GRU记忆和辅助动力学预测，从观察历史中提取与动力学相关的信息。真实机器人实验表明，同一控制器支持VR遥控操作、学习扩散策略和脚本轨迹下的伸手、姿势适应和步进，为多种操作任务提供了通用的终端执行器接口。