生成时间: 2026-07-08 18:20:55 (UTC+8); Arxiv 发布时间: 2026-07-08 20:00 EDT (2026-07-09 08:00 UTC+8)

今天共有 24 篇相关文章

Keyword: reinforcement learning

Catalyst Papers in Artificial Intelligence Research: A Landscape on ICLR from 2017 to 2025

《2017年至2025年ICLR的人工智能研究催化剂论文：一幅全景》

• Authors: Fan Huang
• Subjects: Subjects: Digital Libraries (cs.DL); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2607.05401
• Pdf link: https://arxiv.org/pdf/2607.05401
• Abstract A small number of methodological contributions, including word2vec, the Transformer, large-scale pre-training, and reinforcement learning from human feedback, have reshaped NLP and AI research over the past decade. OpenReview now makes numeric reviewer scores and accept/reject decisions public for every ICLR submission. Whether such review signals identify trajectory-changing papers at submission time, however, remains untested at corpus scale. We answer this question on $36{,}113$ papers from ICLR 2017--2025, identifying \emph{catalysts}: papers whose descendants measurably redirect future research. We compare four disruptiveness measures (the Consolidation/Destabilization (CD) index, node2vec, the direction-aware Embedding Disruptiveness Measure (EDM), and an LLM-based semantic rater) and define a five-type operational catalyst taxonomy (topic initiator, topic bridge, within-topic redirector, simultaneous, and recognition-misaligned). EDM leads at identifying highly cited ICLR papers (AUC $0.83$ vs.\ $0.60$ for CD, $0.49$ for node2vec, and $0.42$ for the LLM rater). Topic initiators precede a $7.55{\times}$ topic-share growth and topic bridges precede an $11.52{\times}$ growth in cross-topic citation flow versus year-matched controls. We found that the peer review scores are essentially orthogonal to future disruptiveness ($|\rho|{\leq}0.005$; accepted and rejected papers have indistinguishable mean EDM, $p{=}0.11$).
• 中文摘要 过去十年中，包括word2vec、Transformer、大规模预训练和基于人类反馈的强化学习在内的少数方法论贡献，重塑了NLP和AI研究。OpenReview 现在对每份 ICLR 提交的评审员评分和接受/拒绝结果都公开了。然而，这些综述信号是否能在提交时识别出改变轨迹的论文，目前尚未在语料库规模上进行验证。我们通过ICLR 2017-2025年价值36{，}113美元论文，识别\emph{catalysts}：其后代显著转向未来研究的论文，回答了这个问题。我们比较了四种破坏性指标（整合/不稳定化（CD）指数、node2vec、方向感知嵌入破坏性指标（EDM）以及基于LLM的语义评分器），并定义了五类操作催化剂分类法（主题发起者、主题桥、主题内重定向、同时和识别错位）。EDM在识别高被引ICLR论文方面领先（AUC为0.83美元，CD为0.60美元，node2vec为0.49美元，LLM评分者为0.42美元）。主题发起者在跨主题引用流增长中领先于$7.55{\times}$，主题桥接则在跨主题引用流与年份匹配对照组中增长$11.52{\times}$之前。我们发现同行评审评分与未来的颠覆性基本成正交（$|\rho|{\leq}0.005$;被接受和被拒绝的论文的平均EDM不可区分，$p{=}0.11$）。

KAT-Coder-V2.5 Technical Report

KAT-Coder-V2.5 技术报告

• Authors: Bo Huang, Fengxiang Li, Hao Xu, Haoyang Huang, Hongyi Fu, Jinhua Hao, Kun Yuan, Minglei Zhang, Pengcheng Xu, Shiyang Liu, Wenhao Zhuang, Yuze Shi, Zongxian Feng, Chao Wang, Cheng He, Chongling Rao, Deyu Cao, Fan Yang, Gang Xiong, Haochen Liu, Jiabao Li, Jian Liang, Jinghui Jia, Jingwen Chang, Jun Du, Junyu Shi, Min Li, Mingqi Wu, Qiang Gao, Shangpeng Yan, Shaotong Qi, Shu Xu, Shuo Zhou, Tiankuo Xu, Tong Zheng, Weilun Zhao, Xiancheng Meng, Xianda Sun, Xiaoyu Jiang, Xunhao Jia, Yao Xia, Yimeng Xu, Yinghan Cui, Yingpeng Chen, Yiwen Ning, Yong Wang, Yuxuan Sun, Zhongsheng Liu, Ming Sun, Cheng Luo, Chen Yang, Han Li, Kun Gai
• Subjects: Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2607.05471
• Pdf link: https://arxiv.org/pdf/2607.05471
• Abstract We present KAT-Coder-V2.5, a coding-focused agentic model trained to act autonomously inside real, executable repositories rather than as a single-turn code generator. Its capability is bottlenecked less by model scale than by the scarcity of reproducible environments, verifiable rewards, and high-value trajectories, which we address with an end-to-end agentic post-training framework. AutoBuilder reconstructs multilingual repositories into sandboxed environments with fail-to-pass and pass-to-pass verification at scale, from which we regenerate self-contained task specifications, recover near-miss trajectories, and distill supervision through process-aware filtering, while KwaiClawEnv synthesizes large-scale tool-use trajectories from executable services and real task seeds. We further scale reinforcement learning with harness randomization, a reliability-hardened sandbox, an asymmetric actor--critic PPO with hindsight-augmented value estimation, and a harness-oriented reward framework, and unify SWE, Agent-Claw, and WebCoding experts via Multi-Teacher On-Policy Distillation. Across six software-engineering and agentic benchmarks, KAT-Coder-V2.5 delivers the best agentic tool-use result on PinchBench and ranks second only to the frontier Opus 4.8 on repository-level software engineering. Our service is available at this https URL.
• 中文摘要 我们介绍KAT-Coder-V2.5，一个以编码为核心的代理模型，训练为在真实可执行仓库中自主行动，而非单回合代码生成器。其能力的瓶颈并非模型规模，而是可重复环境、可验证奖励和高价值轨迹的稀缺，我们通过端到端的代理后培训框架来解决这些问题。AutoBuilder 将多语言仓库重建为沙盒环境，支持大规模的失败通过和通过验证，从中我们重新生成自包含的任务规格，恢复近距离未中轨迹，并通过过程感知过滤提炼监督，而 KwaiClawEnv 则从可执行服务和真实任务种子中综合大规模工具使用轨迹。我们进一步通过利用随机化、一个具可靠性的沙盒、一个非对称行为者-批判性PPO和事后诸葛亮增强价值估计，以及一个基于引导的奖励框架来扩展强化学习，并通过多教师策略提炼统一软件工程师、代理爪和WebCod专家。在六个软件工程和代理基准测试中，KAT-Coder-V2.5 在 PinchBench 上提供了最佳代理工具使用结果，在仓库级软件工程方面仅次于 Opus 4.8 的前沿版本。我们的服务可在此 https 网址访问。

Self-Review Reinforcement Learning (SRRL) with Cross-Episode Memory and Policy Distillation

自评强化学习（SRRL）结合跨章节记忆与策略提炼

• Authors: Muhammad Zain Amin, Kibele Sebnem Yildirim
• Subjects: Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2607.05541
• Pdf link: https://arxiv.org/pdf/2607.05541
• Abstract Reinforcement Learning is commonly used to train large language models using environmental feedback. In applied settings, the environment usually provides sparse or delayed feedback. This makes it difficult for the model to pinpoint which actions in its reasoning led to success or failure. So, learning effectively from these signals is hard because the model must determine how each failure should inform meaningful behavioral corrections in subsequent iterations. We introduce a training framework, Self-Review Reinforcement Learning, that embeds an explicit self-review step into each RL episode. When a first-pass response fails, the model generates a self-review to identify what went wrong, which conditions an improved second attempt. Unlike inference-time reflection approaches, such as Reflexion, the framework optimizes self-review with policy gradients and internalizes improvements into the base policy via selective distillation, ensuring they persist across future episodes. A cross-episode memory keeps successful self-reviews for reuse when encountering similar tasks in future episodes during training. We evaluate SRRL against a standard RLVR baseline using the GRPO optimizer across two language models, Qwen 3-4B and OLMo-3- 7B, on GSM8K benchmark. SRRL consistently outperforms the RLVR in final reward performance and achieves greater learning efficiency by successfully transforming feedback into behavioral improvement.
• 中文摘要 强化学习常用于利用环境反馈训练大型语言模型。在应用环境中，环境通常提供稀疏或延迟的反馈。这使得模型难以准确指出推理中哪些行为导致了成功或失败。因此，从这些信号中有效学习很难，因为模型必须决定每次失败如何指导后续迭代中的有意义行为纠正。我们引入了一个培训框架——自我评价强化学习，在每个强化学习集中嵌入明确的自我评估步骤。当第一次响应失败时，模型会生成自我审查以识别错误，从而决定改进第二次尝试。与推理时间反思方法（如Reflexion）不同，该框架通过策略梯度优化自我审查，并通过选择性提炼将改进内容内化到基础政策中，确保改进在未来事件中持续存在。跨集记忆可以保留成功的自我回顾，以便在未来培训中遇到类似任务时再利用。我们利用GRPO优化器在GSM8K基准测试上，针对两个语言模型Qwen 3-4B和OLMo-3-7B，对SRRL进行标准RLVR基线评估。SRRL在最终奖励表现上持续优于RLVR，并通过成功将反馈转化为行为改进，实现更高的学习效率。

Federated Physics-Grounded Reinforcement Learning for Distributed Stability Control in Smart Grids

智能电网分布式稳定控制的联合物理基础强化学习

• Authors: Omar Al-Refai, Ibrahim Shahbaz, Adam Ali Husseinat, Eman Hammad
• Subjects: Subjects: Machine Learning (cs.LG); Systems and Control (eess.SY)
• Arxiv link: https://arxiv.org/abs/2607.05553
• Pdf link: https://arxiv.org/pdf/2607.05553
• Abstract Transient stability control in smart grids requires rapid post-fault damping of generator frequency and rotor angle deviations to prevent cascading failures. This paper proposes FedPPO-PG, a Federated Multi-Agent Proximal Policy Optimization framework with Physics-Grounded neighborhoods, which reformulates transient stability control as a cooperative multi-agent reinforcement learning problem optimized directly against closed-loop stability objectives. Each generator hosts an independent local actor augmented with the frequency deviations of its two most strongly coupled electrical neighbors, identified from the post-fault Kron-reduced susceptance matrix. A guided policy initialization phase warm-starts all actors from the classical decentralized controller, while a centralized critic guides advantage estimation under the centralized training--decentralized execution (CTDE) paradigm. Evaluated on a simulation of the IEEE 39-bus benchmark system across five training and three unseen fault contingencies, FedPPO-PG achieves 100% stabilization in all 24 trials, reduces mean stability time by 72.4%, and cuts the control power by 7-14 times compared to the centralized baseline. Each actor executes independently with no central coordinator at deployment, and the per-actor inference latency satisfies the IEEE/IEC 60255-118-1-2018 real-time reporting requirements.
• 中文摘要 智能电网中的瞬态稳定控制需要对发电机频率和转子角度偏差进行快速故障后阻尼，以防止级联故障。本文提出了FedPPO-PG，这是一个结合物理基础邻域的联邦多智能体近端策略优化框架，将瞬态稳定控制重新表述为一个合作式多智能体强化学习问题，直接针对闭环稳定性目标进行优化。每个发生器都承载一个独立的本地行为者，并由其两个最强耦合的电邻的频率偏差增强，这些偏差来自故障后Kron简化的感知矩阵。引导式策略初始化阶段从经典去中心化控制器热启动所有参与者，而集中式批评阶段则在集中训练-去中心化执行（CTDE）范式下指导优势估计。在对 IEEE 39 总线基准系统的五次训练和三种未见故障突发情况的模拟下评估，FedPPO-PG 在所有 24 项试验中实现了 100% 的稳定性，平均稳定时间减少了 72.4%，控制能力比集中基线降低了 7-14 倍。每个参与者在部署时独立执行，没有中央协调员，每个演员的推断延迟满足IEEE/IEC 60255-118-1-2018实时报告要求。

Dynamics and Convergences for Markov Coevolutionary Opinion Formation Games in Dynamic Social Networks

动态社交网络中马尔可夫共进化意见形成游戏的动态与收敛

• Authors: Po-An Chen, Chi-Jen Lu, Chuang-Chieh Lin, Jim Shi, Chih-Chieh Hung
• Subjects: Subjects: Computer Science and Game Theory (cs.GT)
• Arxiv link: https://arxiv.org/abs/2607.05580
• Pdf link: https://arxiv.org/pdf/2607.05580
• Abstract While deterministic variants of the coevolutionary opinion formation games such as the K-Nearest Neighbor (K-NN) game, e.g., in Bhawalkar et al., in a dynamic social network can sometimes be shown to stabilize using potential functions or localized smoothness arguments, introducing stochasticity fundamentally changes the mathematical landscape. In the "K-NN Markov game", network topologies evolve via a time-varying, randomized selection process. Proving whether such a system, as a special case of general-sum Markov games, converges to an equilibrium is a profoundly non-obvious and challenging theoretical question. Multiagent reinforcement learning has been shown to derive Nash (minimax) equilibria in two-player zero-sum Markov games and Markov potential games (along with some price-of-anarchy types of results). In recent work, optimistic dynamics are shown to converge to correlated equilibria in general-sum Markov games while the price-of-anarchy bounds are unknown. We thus analyze playing specific no-regret algorithms in general-sum Markov games for convergence to a stricter set than correlated equilibria. We integrate the convergence analysis techniques from multi-agent reinforcement learning in works of Wei et al. and online learning in a recent work of Anagnostides et al.. Specifically in (general-sum) Markov games, since the regret of the optimistic gradient ascent algorithm would have extra positive terms coming from Q-values, taking care of these terms requires non-trivial extra work setting an appropriate range of our learning rate and deriving the threshold on the number of iterations for convergence or a bounded price of anarchy, significantly different from those in the assumption in a main technical theorem of Anagnostides et al.. We analyze a weaker sense of convergences to approximate Nash equilibria by playing optimistic gradient ascents in general-sum Markov games.
• 中文摘要 虽然共进化意见形成博弈的确定性变体，如Bhawalkar等人的K-最近邻（K-NN）博弈，在动态社交网络中有时可以通过潜在函数或局部光滑性论证证明稳定，但引入随机性则从根本上改变了数学格局。在“K-NN马尔可夫游戏”中，网络拓扑通过时间变化的随机选择过程演变。证明这样一个系统作为一般和马尔可夫博弈的特例，是否收敛到均衡是一个极其不显而易见且具有挑战性的理论问题。多智能体强化学习已被证明能在两人零和马可夫博弈和马尔可夫势博弈中推导出纳什（极小极大）均衡（以及一些无政府价格类型的结果）。在近期研究中，乐观动力学在一般和马尔可夫博弈中收敛到相关均衡，而无政府状态的代价界限尚未知。因此，我们分析在一般和马尔可夫博弈中采用特定的无遗憾算法，以寻找收敛到比相关均衡更严格的集合。我们将Wei等人工作中多智能体强化学习中的收敛分析技术和Anagnostides等人近期研究中的在线学习相结合。具体在（一般和）马尔可夫博弈中，由于乐观梯度上升算法的遗憾会有来自Q值的额外正项，处理这些项需要额外工作，比如设置适当的学习率范围，并推导收敛或无政府状态价格的迭代次数阈值， 与Anagnostides等人主要技术定理中的假设有显著不同。我们通过在广义和马尔可夫博弈中进行乐观梯度上升，分析较弱的收敛感以近似纳什均衡。

Deep Reinforcement Learning for Dynamic Battery Management of Autonomous Order Pickers

自主拣货者动态电池管理的深度强化学习

• Authors: Taniya Shaji, Abhay Sobhanan, Christof Defryn
• Subjects: Subjects: Machine Learning (cs.LG); Multiagent Systems (cs.MA); Optimization and Control (math.OC)
• Arxiv link: https://arxiv.org/abs/2607.05683
• Pdf link: https://arxiv.org/pdf/2607.05683
• Abstract Battery charging of Autonomous Mobile Robots (AMRs) in warehouses is a critical operational challenge that heavily impacts both order processing times and throughput. In this study, we address the dynamic AMR charging problem under stochastic order arrivals, where robots must learn optimal charging decisions. Traditional fixed-rule heuristics often prove suboptimal in dynamic environments and fail to account for multi-AMR coordination, leading to severe resource inefficiencies. To overcome these limitations, we propose a Proximal Policy Optimization (PPO)-based Deep Reinforcement Learning (DRL) framework designed for multi-block warehouses with fixed charging stations. Our model dynamically learns two key decisions: charging station selection and optimal charging duration, explicitly accounting for anticipated queuing times at the stations. Extensive numerical experiments benchmark the proposed model against state-of-the-art DRL and traditional heuristic approaches. Results demonstrate that our PPO framework increases order-completion rates by up to 6\% compared to the strongest baseline, while significantly reducing the total time dedicated to recharging operations. Furthermore, we validate the model's robustness across diverse warehouse configurations and stochastic arrival rates. Finally, we interpret the learned DRL policy, offering valuable operational insights into its superiority over standard benchmarks.
• 中文摘要 仓库中自主移动机器人（AMR）的电池充电是一个关键的运营挑战，严重影响订单处理时间和吞吐量。本研究探讨了随机顺序到达下的动态AMR充电问题，机器人必须学习最优充电决策。传统的固定规则启发式在动态环境中往往表现不佳，且未能考虑多AMR协调，导致资源效率极低。为克服这些限制，我们提出了基于近端策略优化（PPO）的深度强化学习（DRL）框架，专为多区块仓库和固定充电站设计。我们的模型动态学习两个关键决策：充电站选择和最佳充电时长，明确考虑充电站预期排队时间。大量的数值实验将所提模型与最先进的日程学习（DRL）和传统启发式方法进行对比。结果显示，我们的PPO框架相比最强基线提高了订单完成率高达6%，同时显著减少了充值操作的总时间。此外，我们还验证了该模型在不同仓库配置和随机到达率下的稳健性。最后，我们解读了已学到的DRL政策，提供了其优于标准基准的宝贵运营见解。

FourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models

FourTune：迈向扩散模型的完全4位高效后期训练

• Authors: Bowen Xue, Zihan Min, Xingyang Li, Zhekai Zhang, Haocheng Xi, Lvmin Zhang, Maneesh Agrawala, Jun-Yan Zhu, Song Han, Yujun Lin, Muyang Li
• Subjects: Subjects: Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2607.05711
• Pdf link: https://arxiv.org/pdf/2607.05711
• Abstract Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. However, post-training of large diffusion models is still challenging due to the prohibitive memory footprints and slow training speed, which existing parameter-efficient fine-tuning methods only partially address. To overcome these limitations, we propose FourTune, an efficient post-training framework for diffusion models based on an end-to-end W4A4G4 paradigm. FourTune introduces a triple-branch hybrid pipeline that augments the standard LoRA architecture with a frozen numerical stabilizer to isolate quantization-sensitive outliers, enabling stable training under native 4-bit computation. In addition, FourTune employs hardware-efficient block-wise quantization and customized fused kernels to support efficient quantized backpropagation and reduce memory bandwidth overhead. Across customization, reinforcement learning, and distillation tasks, FourTune matches the quality of full-precision fine-tuning. On FLUX.1-dev (12B), FourTune reduces memory overhead by 2.25$\times$ and increases end-to-end training throughput by 2.27$\times$ compared to BF16 LoRA.
• 中文摘要 扩散模型已成为高质量生成建模的主导范式，而后期训练对于将其适应到多样化的后续应用至关重要。然而，由于内存占用巨大且训练速度缓慢，大型扩散模型的后训练仍然具有挑战性，现有参数高效的微调方法只能部分解决这些问题。为克服这些限制，我们提出了FourTune，这是一个基于端到端W4A4G4范式的高效扩散模型后训练框架。FourTune引入了三分支混合流水线，在标准LoRA架构基础上加入冻结数值稳定器，以隔离量化敏感的离群值，实现原生4位计算下的稳定训练。此外，FourTune采用硬件高效的分块量化和定制融合核，支持高效的量化反向传播并降低内存带宽开销。在定制、强化学习和蒸馏任务中，FourTune 能够媲美全精度微调的质量。在FLUX.1开发版（12B）中，FourTune相比BF16 LoRA减少了2.25美元\时间$，端到端训练吞吐量提升了2.27美元\时间美元。

From Passive Retrieval to Active Memory Navigation: Learning to Use Memory as a Structured Action Space

从被动检索到主动记忆导航：学习将记忆作为结构化行动空间

• Authors: Yue Xu, Yutao Sun, Yihao Liu, Mengyu Zhou, Jiayi Qiao, Lu Ma, Kai Tang, Wenjie Wang, Xiaoxi Jiang, Guanjun Jiang
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2607.05794
• Pdf link: https://arxiv.org/pdf/2607.05794
• Abstract Long-term user memory is essential for personalized conversational agents, yet many memory systems still expose memory through passive retrieval interfaces, making the model a consumer of pre-selected evidence. We introduce NapMem, a framework for learning to use long-term user memory as a structured action space rather than passively retrieved context. NapMem organizes user history into a linked multi-granularity memory pyramid, where raw conversations, typed memory records, topic tracks, and user profiles are connected through provenance relations, and exposes these levels through memory tools. The agent is trained to select memory according to the query and intermediate evidence, allowing it to inspect different memory granularities before answering. Experiments on PersonaMem-v2, LongMemEval, and LoCoMo show that a NapMem agent trained with memory-tool reinforcement learning is competitive across diverse memory-intensive tasks, while evaluations on non-memory tasks suggest that the learned policy largely preserves general reasoning and tool-use abilities. Additional analyses examine storage, inference cost, tool-use behavior, and ablations over navigation, memory granularity, and RL training. Our results suggest that long-term user memory benefits from coupling structured storage with a learned policy for using memory at the appropriate granularity.
• 中文摘要 长期用户记忆对于个性化对话代理至关重要，但许多记忆系统仍通过被动检索接口暴露记忆，使模型成为预先选择证据的消费者。我们介绍了NapMem，一个学习将长期用户记忆作为结构化动作空间而非被动检索上下文的框架。NapMem 将用户历史组织成一个连接的多粒度内存金字塔，原始对话、输入的内存记录、主题轨迹和用户配置文件通过来源关系相互连接，并通过内存工具暴露这些层级。代理被训练为根据查询和中间证据选择记忆，从而在回答前检查不同的记忆粒度。对PersonaMem-v2、LongMemEval和LoCoMo的实验显示，经过记忆工具强化学习训练的NapMem代理在多种记忆密集型任务中具有竞争力，而对非记忆任务的评估则表明，所学策略在很大程度上保留了一般推理和工具使用能力。其他分析还涉及存储、推断成本、工具使用行为以及导航、记忆粒度和强化学习训练的消融。我们的结果表明，长期用户记忆受益于将结构化存储与学习到的在适当粒度内存使用策略相结合。

SCOPE: Leveraging Subgoal Critiques for Code Generation

范围：利用子目标批评来生成代码

• Authors: Yueke Zhang, Yifan Zhang, Zihan Fang, Kevin Leach, Wei Zhang, Yu Huang
• Subjects: Subjects: Software Engineering (cs.SE)
• Arxiv link: https://arxiv.org/abs/2607.05810
• Pdf link: https://arxiv.org/pdf/2607.05810
• Abstract Code generation with large language models (LLMs) remains unreliable because generated programs can appear correct while still violating key semantic requirements in the natural language specification. Existing feedback-based methods improve over coder-only generation, but they often rely on unstructured critique or execution signals that do not explicitly identify what the code is semantically missing. We present SCOPE, a prover-initialized subgoal critic for code generation. SCOPE adapts a Lean-oriented prover model to produce three parseable feedback fields for downstream code generation: subgoals, gap analysis, and a robustness checklist. Our approach combines supervised fine-tuning, process-aligned reinforcement learning (RL), and feedback-guided inference, with two complementary rewards during RL: a dense reward for structured critique quality and a sparse reward based on whether the critique improves the coder's execution score. Experiments show that SCOPE improves over the compared feedback baselines. On LiveCodeBench V6, SCOPE achieves 39.4% pass@1, compared with 36.6% for Reflexion and 20.6% for the coder-only baseline. On BigCodeBench (Hard), it reaches 42.6%, surpassing Reflexion at 36.5% and coder-only generation at 34.5%. Further analysis shows that SCOPE's gains are concentrated in tasks with concrete semantic constraints and that its code corrections are more localized than Reflexion's.
• 中文摘要 使用大型语言模型（LLM）生成代码仍然不可靠，因为生成的程序可能看起来正确，但仍然违反自然语言规范中的关键语义要求。现有的基于反馈的方法比仅编码者生成有所改进，但它们通常依赖非结构化的批评或执行信号，这些信号并未明确指出代码语义上缺失的内容。我们介绍SCOPE，一个经过验证的子目标代码生成批评器。SCOPE采用精益导向的校验模型，生成三个可解析的反馈字段用于下游代码生成：子目标、差距分析和鲁棒性检查表。我们的方法结合了监督式微调、过程对齐强化学习（RL）和反馈引导推理，在强化学习期间提供两种互补奖励：结构化批评质量的密集奖励和基于批评是否提升程序员执行分数的稀疏奖励。实验显示，SCOPE相较于对比反馈基线有所提升。在LiveCodeBench V6中，SCOPE实现了39.4%的 pass@1，而 Reflexion 为 36.6%，仅编码员基线为 20.6%。在BigCodeBench（困难）中，生成率达到42.6%，超过Reflexion的36.5%和仅编码者生成的34.5%。进一步分析显示，SCOPE的收益集中在具有具体语义约束的任务中，其代码修正比Reflexion更为局部化。

Mitigating Factual Hallucination in Large Reasoning Models via Mixed-Mode Advantage Regularization

通过混合模式优势正则化减轻大型推理模型中的事实幻觉

• Authors: Kaishen Wang, Tong Zheng, Xuehao Cui, Ruibo Chen, Tianyi Xiong, Heng Huang
• Subjects: Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2607.05861
• Pdf link: https://arxiv.org/pdf/2607.05861
• Abstract Large reasoning models (LRMs) improve language model capabilities by generating explicit thinking traces before final answers. In factuality-oriented question answering (QA), such thinking often improves overall performance by helping the model recover relevant knowledge and refine its answers. However, we find that this benefit is not uniform at the instance level: explicit thinking can also overturn correct non-thinking answers and lead to factual drift. We refer to this failure mode as \emph{thinking-induced hallucination}. To explain this phenomenon, we formulate explicit thinking in factuality QA as a thinking residual over the model's direct-answer tendency, which can either recover missing knowledge or introduce unsupported associations. Based on this formulation, we propose MARGO, \underline{\textit{M}}ixed-Mode \underline{\textit{A}}dvantage \underline{\textit{R}}egularization for \underline{\textit{G}}rounded \underline{\textit{O}}ptimization, a reinforcement learning framework that uses non-thinking rollouts as same-model references in advantage estimation. By constructing mixed-mode rollout groups with both thinking and non-thinking trajectories, MARGO evaluates whether explicit thinking adds factual value beyond direct answering, thereby suppressing hallucination-prone thinking while preserving beneficial thinking behaviors. Experiments across multiple factuality-oriented QA benchmarks demonstrate that MARGO improves factual reliability over strong baselines, while evaluations on mathematical benchmarks show that it preserves general reasoning ability.
• 中文摘要 大型推理模型（LRM）通过在最终答案前生成显性思维痕迹来提升语言模型的能力。在以事实为导向的问题答覆（QA）中，这种思维通常通过帮助模型恢复相关知识和完善答案来提升整体表现。然而，我们发现这种好处在实例层面并不一致：显性思考也可能推翻正确的非思考答案，导致事实漂移。我们称这种失败模式为\emph{思维诱发的幻觉}。为解释这一现象，我们将事实性质检的显式思维表述为模型直接回答倾向的思维残余，这种残余可以恢复缺失的知识或引入无支持的关联。基于此表述，我们提出了MARGO，\underline{\textit{M}}ixed-Mode \underline{\textit{A}}dvantage \underline{\textit{R}}egularization for \underline{\textit{G}}rounded \underline{\textit{O}}ptimization，这是一种强化学习框架，利用非思考的rollout作为优势估计中的同模型参考。通过构建包含思考轨迹和非思考轨迹的混合模式展开组，MARGO评估显性思维是否除了直接回答外还能增加事实价值，从而抑制易产生幻觉的思维，同时保持有益的思维行为。多项以事实为导向的质量保证基准测试实验表明，MARGO在强基线下提升了事实可靠性，而数学基准测试的评估则显示其保持了一般推理能力。

Strategic Bargaining in Multi-Buyer Markets: Reinforcement Learning from Verifiable Rewards for LLM Negotiations

多买家市场中的战略谈判：从可验证奖励中学习强化的LLM谈判

• Authors: Shuze Daniel Liu, Claire Chen, Jiabao Sean Xiao, Xin Chen, David Simchi-Levi
• Subjects: Subjects: Machine Learning (cs.LG); Computer Science and Game Theory (cs.GT)
• Arxiv link: https://arxiv.org/abs/2607.05863
• Pdf link: https://arxiv.org/pdf/2607.05863
• Abstract Negotiation is a fundamental strategic interaction in management science, characterized by agents attempting to reach agreements while protecting private information, such as reservation costs and hidden valuations. A prevalent yet complex scenario involves a single seller negotiating concurrently with multiple buyers, each possessing heterogeneous, private budgets. In such settings, constrained by a limited number of communication turns, the seller must balance exploring the broader market to discover the highest valuation with concentrating sufficient turns on a single target buyer to secure the best possible outcome. Our analysis reveals a significant gap in standard Large Language Models (LLMs): while these models are linguistically proficient, they fail to act as effective economic decision-makers. Specifically, they exhibit a failure to explore the buyer pool, often fixating on the current highest bid rather than strategically investigating the market to discover latent high valuations. In this paper, we propose a specialized training recipe using Reinforcement Learning from Verifiable Rewards (RLVR). By anchoring the reward function to objective economic outcomes, the strategic balance between market discovery and surplus extraction emerges natively through the learning process. Our results demonstrate that the trained seller undergoes a multi-stage strategic evolution, learning to leverage price anchoring and strategic probing to identify more profitable counterparties. The agent extracts a substantially higher surplus than frontier models by both improving its persuasive bargaining skills and consistently closing deals with high-value buyers. Finally, we show that our seller strategies generalize robustly to unseen buyer negotiation styles and budget distributions.
• 中文摘要 谈判是管理科学中一种基本的战略互动，其特点是代理人试图在保护私人信息（如预留成本和隐藏估值）的同时达成协议。一种常见但复杂的情景是单个卖家同时与多个买家谈判，每个买家拥有异质且私人的预算。在这种受限于有限沟通回合的情境下，卖方必须在探索更广泛的市场以寻找最高估值与集中足够时间对单个目标买家进行之间取得最佳结果之间取得平衡。我们的分析揭示了标准大型语言模型（LLMs）存在显著差距：虽然这些模型在语言学上具备能力，但它们未能作为有效的经济决策者发挥作用。具体来说，他们未能探索买家群体，常常专注于当前最高报价，而非战略性地调查市场以发现潜在的高估值。本文提出了一种基于可验证奖励强化学习（RLVR）的专业训练方案。通过将奖励函数锚定于客观的经济结果，市场发现与剩余开采之间的战略平衡自然地通过学习过程自然显现。我们的结果表明，受过培训的卖方会经历多阶段的战略演进，学习利用价格锚定和战略探查来识别更具盈利性的交易对手方。该经纪人通过提升说服谈判技巧和持续与高价值买家达成交易，获得了远高于前沿模型的盈余。最后，我们展示了卖方策略能够强有力地推广到看不见的买方谈判风格和预算分配。

Progressive Reasoning with Primitive Correction for Compositional Zero-Shot Learning

基于原始修正的渐进推理，用于合成零元学

• Authors: Ziyi Chen, Haoyan Shi, Sunhan Xu, Congyan Lang
• Subjects: Subjects: Computer Vision and Pattern Recognition (cs.CV)
• Arxiv link: https://arxiv.org/abs/2607.05911
• Pdf link: https://arxiv.org/pdf/2607.05911
• Abstract Compositional Zero-Shot Learning (CZSL) aims to combine known attributes and objects as primitives for recognizing previously unseen attribute-object pairs. Prior works either predict attributes and objects independently, missing their strong contextual dependency, or use unidirectional conditional modeling (e.g., object-guided attribute prediction), which is prone to error propagation. We propose PRPC, a Progressive Reasoning framework with Primitive Correction, which explicitly models the bidirectional dependency between attributes and objects via step-wise inference. PRPC performs mutual correction of primitives to suppress prediction errors in earlier steps. Specifically, we formulate CZSL as structured, Q&A-style Chain-of-Thought reasoning process and constrain the MLLM to follow predefined semantic steps to generate intermediate decisions. To further enhance the reliability and logical consistency of intermediate reasoning, we introduce reinforcement learning post-training with a GRPO-based objective, providing step-level rewards aligned with the progressive inference procedure. Extensive experiments on three CZSL benchmarks demonstrate that PRPC achieves state-of-the-art performance, validating the effectiveness of progressive reasoning and bidirectional correction for robust compositional generalization.
• 中文摘要 合成零样本学习（CZSL）旨在将已知属性和对象作为识别此前未见过的属性-对象对的原语结合起来。以往的研究要么独立预测属性和对象，忽略其强烈的上下文依赖性，要么使用单向条件建模（例如对象引导属性预测），后者容易出现错误传播。我们提出了PRPC，一种带有原始纠正的渐进推理框架，通过逐步推理明确建模属性与对象之间的双向依赖关系。PRPC对原语进行相互校正，以抑制早期步骤中的预测误差。具体来说，我们将CZSL表述为结构化的问答式思维链推理过程，并限制MLLM遵循预设语义步骤以生成中间决策。为进一步提升中级推理的可靠性和逻辑一致性，我们在训练后引入基于GRPO目标的强化学习，提供与渐进推理过程相符的步骤级奖励。在三个CZSL基准测试上的大量实验表明，PRPC实现了最先进的性能，验证了渐进推理和双向校正对稳健组合推广的有效性。

Intercepting an Agile Target with Net-Carrying Drones using Competitive Multi-Agent Reinforcement Learning

利用竞争性多智能体强化学习，利用网络携带无人机拦截敏捷目标

• Authors: Timothée Gavin (ENAC-LAB, LAAS-RIS), Murat Bronz (ENAC)
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2607.05939
• Pdf link: https://arxiv.org/pdf/2607.05939
• Abstract This article presents a solution to intercept an agile drone by a team of agile drone carrying catching nets. We formulate the problem as a competitive Multi-Agent Reinforcement Learning (MARL) task. To address the problem of nonstationarity and catastrophic forgetting of agents overfitting to the current opponent strategy, we train the pursuers and the evader using Multi-Agent Proximal Policy Optimization (MAPPO) with Prioritized Fictitious Self Play (PFSP). We train the agents in a high-fidelity simulator using low-level control commands, collective thrust and body rates (CTBR), to achieve agile flights for both the pursuers and the evader. We compare the performance of the trained policies in terms of catch rate, time to catch and crash rates, against heuristic baselines and show that our solution outperforms them. Ablation studies show that PFSP lead to more robust policies that can adapt to different opponent strategies, and that a low-level control commands are crucial for learning performing strategies in the pursuit-evasion task. Finally, a qualitative analysis of the learned behaviours highlights the emergence of cooperative tactics among the pursuers.
• 中文摘要 本文提出了一个由携带捕捉网的敏捷无人机团队拦截敏捷无人机的解决方案。我们将问题提出为一项竞争性多智能体强化学习（MARL）任务。为解决代理过拟合当前对手策略的非平稳性和灾难性遗忘问题，我们使用多代理近端策略优化（MAPPO）配合优先虚构自玩（PFSP）训练追捕者和规避者。我们在高保真模拟器中训练特工，使用低级控制指令、集体推力和体速（CTBR），以实现追击者和逃避者双方的敏捷飞行。我们将训练有素的政策在捕捉率、捕获时间和崩溃率方面与启发式基线进行比较，结果显示我们的方案表现优于它们。消融研究表明，PFSP带来了更稳健的策略，能够适应不同的对手策略，低层次控制指令对于学习追击-规避任务中的执行策略至关重要。最后，对学习行为的定性分析强调了追捕者之间合作策略的出现。

Delay-Aware Active Triangulation with Uncertainty-Driven Multi-Agent Reinforcement Learning for Counter-UAS

延迟感知主动三角定位与不确定性驱动的多智能体强化学习，用于反无人机系统

• Authors: Seungwook Lee, David Hyunchul Shim
• Subjects: Subjects: Robotics (cs.RO); Multiagent Systems (cs.MA)
• Arxiv link: https://arxiv.org/abs/2607.05957
• Pdf link: https://arxiv.org/pdf/2607.05957
• Abstract Multi-agent active visual triangulation enables precise 3D localization of aerial targets by coordinating mobile observers with controllable cameras. However, existing methods assume instantaneous state feedback, ignoring cumulative latency from detection, communication, and decision propagation. We present a delay-aware, uncertainty-driven multi-agent reinforcement learning framework for target localization in Counter-UAS applications. Our contributions are: (1) a Dec-POMDP formulation with Age-of-Information (AoI) augmented observations enabling delay-aware coordination -- AoI improves triangulation validity by 10.6 percentage points; (2) a controlled comparison showing that perception-consistent rewards outperform privileged clean-state rewards (0.547 m vs.0.633 m RMSE, 27% fewer track losses) -- both policies are trained through identical observation noise but differ in what they are optimized for, producing a stability-robustness tradeoff; and (3) multi-source analytical covariance propagation incorporating pixel, pose, gimbal, and intrinsics uncertainties -- restricting to angular noise alone causes 2.8-fold RMSE degradation. Experiments with MAPPO in 4096 parallel environments achieve 0.547 +- 0.217 m RMSE with 78.1% triangulation validity, while MLP policies achieve near-zero validity (0.7%), confirming recurrent memory as essential for delay compensation.
• 中文摘要 多智能体主动视觉三角定位通过协调可控摄像头的移动观察者实现对空中目标的精确三维定位。然而，现有方法假设状态反馈为瞬时，忽略了检测、通信和决策传播带来的累积延迟。我们提出了一个延迟感知、不确定性驱动的多智能体强化学习框架，用于反无人机应用中的目标定位。我们的贡献包括：（1）带有信息年龄（AoI）增强观测的Dec-POMDP表述，实现延迟感知协调——AoI使三角剖分的效度提升了10.6个百分点;（2）受控比较显示，感知一致性奖励优于特权清洁状态奖励（0.547 m 对 0.633 m RMSE，减少27%的轨道丢失）——两种策略都通过相同的观测噪声训练，但优化对象不同，产生稳定性与鲁棒性之间的权衡;以及（3）多源解析协方差传播，包含像素、姿态、万向节和内在不确定性——仅限于角噪声会导致RMSE的2.8倍劣化。在4096个并行环境中的MAPPO实验实现了0.547 +- 0.217米的RMSE，三角剖分有效度为78.1%，而MLP策略的有效度接近零（0.7%），证实了循环记忆对延迟补偿至关重要。

6G Sensing Security: Distributed Game-Theoretic RL for Urban Beamforming and Attacker Detection

6G传感安全：分布式博弈论强化学习用于城市波束成形和攻击者检测

• Authors: Parmida Geranmayeh, Onur Günlü
• Subjects: Subjects: Information Theory (cs.IT); Cryptography and Security (cs.CR); Computer Science and Game Theory (cs.GT); Machine Learning (cs.LG); Networking and Internet Architecture (cs.NI)
• Arxiv link: https://arxiv.org/abs/2607.06115
• Pdf link: https://arxiv.org/pdf/2607.06115
• Abstract In next-generation networks, communication systems will no longer be limited to data transmission and will be expected to acquire awareness of the surrounding environment. This leads to the concept of integrated sensing and communication (ISAC), where the same wireless infrastructure is used for both communication and environmental sensing. Thus, ISAC enables the system to transmit information efficiently and observe and interpret channel variations and user behavior. Motivated by this capability, this work focuses on detecting an active attacker in an urban environment scenario, where the attacker intentionally manipulates beamforming directions to increase interference and mislead the transmitter into allocating the main lobe of beam toward itself instead of legitimate users. We apply game-theoretic approaches to model the interaction between legitimate users and the attacker, and integrate the resulting utility-based formulation into a reinforcement learning (RL) framework. Simulation results demonstrate that the proposed method effectively addresses security challenges in dynamic 6G ISAC systems.
• 中文摘要 在下一代网络中，通信系统将不再局限于数据传输，而是需要获得对周围环境的感知。这引出了综合感测与通信（ISAC）的概念，即通信和环境感测同时使用相同的无线基础设施。因此，ISAC使系统能够高效传输信息，并观察和解读信道变化及用户行为。基于这一能力，本研究重点在城市环境中检测主动攻击者，攻击者故意操控波束形成方向以增加干扰，误导发射器将主波瓣分配给自身而非合法用户。我们运用博弈论方法模拟合法用户与攻击者之间的互动，并将由此产生的基于效用的表述整合进强化学习（RL）框架中。模拟结果表明，所提方法有效解决了动态6G ISAC系统中的安全挑战。

CurateEvo: Data-Curation Evolving for Agentic Post-Training

CurateEvo：数据策展为代理后培训而演进

• Authors: Dingzirui Wang, Xuanliang Zhang, Keyan Xu, Qingfu Zhu, Wanxiang Che
• Subjects: Subjects: Computation and Language (cs.CL)
• Arxiv link: https://arxiv.org/abs/2607.06140
• Pdf link: https://arxiv.org/pdf/2607.06140
• Abstract Large language model (LLM) agents require post-training methods that can improve long-horizon decision making from environment feedback. However, existing agentic post-training pipelines often treat data curation as a fixed preprocessing step, focusing mainly on data augmentation while neglecting filtering, refinement, and adaptation to downstream failures. We propose CurateEvo, a failure-driven dynamic evolution framework for agentic post-training data curation. CurateEvo represents the curation strategy as executable code and iteratively rewrites it using failed trajectories from a held-out development set. At each epoch, the evolved strategy transforms a fixed raw corpus into supervised fine-tuning data, reinforcement learning data, and an inference-time memory bank. The evolution process first improves effectiveness by diagnosing recurring failure modes and augmenting, filtering, or refining data accordingly, and then improves efficiency by pruning redundant or low-utility training turns under a cost-aware objective. Experiments on ACEBench-Agent, BFCL-V4, and {\tau}^2-Bench under both labeled and wild-data settings show that CurateEvo consistently outperforms prior curation methods, improving average scores by 3.2 and 2.7 points, respectively. Further analyses demonstrate that CurateEvo is compatible with different post-training recipes and substantially reduces curation overhead.
• 中文摘要 大型语言模型（LLM）代理需要训练后方法，以提升从环境反馈中做出长期决策。然而，现有的代理式后训练流程通常将数据整理视为固定的预处理步骤，主要关注数据增强，而忽视过滤、优化和对后续故障的适应。我们提出了CurateEvo，一种基于失败驱动的动态进化框架，用于代理训练后数据管理。CurateEvo将策展策略表示为可执行代码，并利用保留的开发集中失败的轨迹迭代重写。在每个时代，演变后的策略将固定的原始语料库转化为监督微调数据、强化学习数据和推理时间记忆库。演化过程首先通过诊断反复出现的失效模式并相应地增强、过滤或优化数据来提高效率，然后在成本意识目标下通过修剪冗余或低效用的训练回合来提高效率。在ACEBench-Agent、BFCL-V4和{\tau}^2-Bench上，无论是标记数据还是野生数据设置下的实验，都显示CurateEvo始终优于以往的策展方法，平均得分分别提升了3.2分和2.7分。进一步分析显示，CurateEvo 兼容不同的培训后配方，并大幅降低了筛选开销。

Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design

通过强化学习提升LLM生成的过程模型质量：奖励函数设计的作用

• Authors: Alexander Rombach, Chantale Lauer, Nijat Mehdiyev
• Subjects: Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2607.06175
• Pdf link: https://arxiv.org/pdf/2607.06175
• Abstract Large language models (LLMs) can generate BPMN process models from natural-language descriptions, yet supervised fine-tuning (SFT) limits their output quality to the patterns present in the training data. Reinforcement learning (RL) can optimize beyond this ceiling using external quality measures, but how the reward function should be designed when quality is multi-dimensional remains unexplored. We present a systematic investigation of reward function design for RL-based process model generation, training two LLM families (Llama~3.1 8B, Qwen~2.5 14B) under 48 configurations using Group Sequence Policy Optimization with rewards derived from an automated evaluation framework comprising 38 metrics across syntactic, pragmatic, and semantic quality. Three findings emerge. First, RL significantly improves pragmatic and syntactic quality while preserving semantic fidelity, reducing output variability by more than sixfold. Second, equal reward weighting consistently outperforms targeted weighting: emphasizing a specific dimension fails to improve it and can collapse the model into a low-quality mode. Third, design choices interact with model architecture in non-trivial ways: the invalidity penalty is essential for one model but irrelevant for the other, and SFT initialization is indispensable for one architecture but counterproductive for another. These results demonstrate that reward composition is a primary determinant of optimization outcomes, with effects as large as the decision to apply RL itself. The findings generalize to any structured generation task where quality is assessed along multiple automated dimensions. We release our implementation and experimental code at this https URL.
• 中文摘要 大型语言模型（LLM）可以从自然语言描述生成BPMN过程模型，但监督微调（SFT）限制了其输出质量仅限于训练数据中的模式。强化学习（RL）可以通过外部质量衡量来优化，但当质量多维时，奖励函数应如何设计仍待探索。我们系统地探讨了基于强化学习的过程模型生成中的奖励函数设计，利用组序列策略优化在48种配置下训练了两个LLM家族（Llama~3.1 8B，Qwen~2.5 14B），奖励来源于自动评估框架，涵盖语法、语用和语义质量的38项指标。有三个发现。首先，强化学习显著提升了语用和句法质量，同时保持语义忠实度，将输出变异性降低了六倍以上。其次，等值加权始终优于定向加权：强调某一特定维度无法改善该维度，反而可能使模型陷入低质量模式。第三，设计选择与模型架构的交互方式非平凡：无效惩罚对一种模型至关重要，但对另一种模型无关紧要，SFT初始化对一种架构是必需的，但对另一种则适得其反。这些结果表明，奖励组成是优化结果的主要决定因素，其影响大小可追溯至是否应用强化学习本身。这些发现可推广到任何在多个自动化维度上评估质量的结构化生成任务。我们将实现和实验代码发布到这个 https URL。

Information Gain-based Rollout Policy Optimization: An Adaptive Tree-Structured Rollout Approach for Multi-Turn LLM Agents

基于信息获取的推出策略优化：多回合大型语言模型代理的自适应树结构推广方法

• Authors: Yijun Zhang, Fan Xu, Jiaxin Ding, Yule Xie, Shiqing Gao, Xin Ding, Haoxiang Zhang, Luoyi Fu, Xinbing Wang
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2607.06223
• Pdf link: https://arxiv.org/pdf/2607.06223
• Abstract Reinforcement learning has become a promising paradigm for improving large language model (LLM) agents on long-horizon search tasks, where the agent must make a sequence of intermediate decisions before receiving a final outcome. However, existing methods still face a key limitation: the rollout budget is often allocated without explicitly assessing the utility of intermediate states. As a result, substantial computation may be spent on low-value states, even though different branches can vary drastically in their informativeness. In this paper, we propose Information Gain-based Rollout Policy Optimization (IGRPO), a policy optimization framework that treats intermediate-state informativeness as the organizing principle of rollout collection. Specifically, IGRPO performs budget-aware tree-structured rollouts by allocating expansion budget according to node-level informativeness, so that more informative branches are expanded more frequently while unpromising branches are progressively suppressed. We further demonstrate that the information gain-based rollout induces an explicit limiting teacher distribution over trajectories, which naturally yields a clear policy optimization target, thereby unifying adaptive tree-structured exploration with principled policy learning under a single framework. Experiments on seven challenging search-augmented QA benchmarks demonstrate that IGRPO consistently outperforms strong baselines under the same rollout budget constraints, validating the effectiveness of leveraging the induced teacher distribution to guide policy optimization for long-horizon search agents.
• 中文摘要 强化学习已成为改进大型语言模型（LLM）代理在长期搜索任务中的一种有前景的范式，在长期搜索任务中，代理必须在获得最终结果前做出一系列中间决策。然而，现有方法仍面临一个关键局限：推广预算往往未明确评估中间状态的效用。因此，尽管不同分支的信息量可能差异很大，但大量计算仍可能花费在低价值状态上。本文提出基于信息增益的推出策略优化（IGRPO），这是一种策略优化框架，将中间状态信息量视为推广收集的组织原则。具体来说，IGRPO通过根据节点级信息量分配扩展预算，执行预算感知的树状结构部署，使信息丰富的分支更频繁地扩展，而无前景的分支则逐步被抑制。我们还进一步证明，基于信息获取的推广会在轨迹上实现明确的限制教师分布，这自然产生明确的策略优化目标，从而在单一框架下统一了自适应树结构探索与原则性政策学习。在七个具有挑战性的搜索增强质量保证基准测试中，实验表明IGRPO在相同的推广预算约束下持续优于强基线，验证了利用诱导教师分布指导长期搜索代理策略优化的有效性。

LAMP: Latent Motion Prior-Guided Real-World Learning for Dexterous Hand Manipulation

LAMP：潜动先验引导的真实世界学习，促进灵巧的手部操作

• Authors: Xinye Yang, Zhiyuan Ma, Hongze Yu, Yuanpei Chen, Yaodong Yang, Xiaojie Chai, Xinlei Chen, Chao Yu
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2607.06323
• Pdf link: https://arxiv.org/pdf/2607.06323
• Abstract Real-world learning for dexterous hands remains brittle because high-dimensional hand actions amplify imitation errors and make reinforcement-learning exploration prone to contact-breaking motion. While combining imitation learning (IL) with online reinforcement learning (RL) can reduce manual supervision, unconstrained exploration in raw hand-action spaces is sample-inefficient and risky for physical hardware. We introduce a latent motion prior module (\prior{}) that maps recent hand-action histories to a compact, history-conditioned latent prior and decodes continuous latent commands into executable high-dimensional hand targets. Built on this prior, \method{} is a three-stage real-world dexterous learning framework: it pretrains \prior{} from demonstrations, trains a visuomotor policy that predicts native arm commands and latent hand-action offsets, and improves the policy with online residual RL in the same latent hand-action space. This shared, decodable interface lets residual exploration make local corrections near demonstrated, contact-consistent hand motions rather than perturbing every finger joint independently. We evaluate \method{} on four real-robot dexterous manipulation tasks against raw, linear, and discrete hand-action interfaces. Starting from small task-specific demonstration sets, \method{} achieves a 56.25\% average IL success rate and raises it to 98.75\% after online RL, reaching 100\% final success on three tasks and 95\% on the remaining task.
• 中文摘要 灵巧手的真实学习仍然脆弱，因为高维手部动作会放大模仿错误，使强化学习探索容易出现接触断裂动作。虽然将模仿学习（IL）与在线强化学习（RL）结合可以减少人工监督，但在原始手部动作空间中无限制探索则对样本效率低且对物理硬件存在风险。我们引入潜动先验模块（\prior{}），将近期手部动作历史映射到紧凑的历史条件潜在先验，并将连续潜在指令解码为可执行的高维手部目标。基于此先验，\方法{}是一个三阶段的现实世界灵巧学习框架：它从演示预训练\prior{}，训练预测原生手臂指令和潜在手部动作偏移的视觉运动策略，并在同一潜在手部动作空间内通过在线残差强化学习改进策略。这种共享的、可解码的接口使残差探索能够在接近已展示的接触一致手部动作中进行局部修正，而不是单独扰动每个手指关节。我们在四种真实机器人灵巧操作任务中，针对原始、线性和离散手部操作界面进行了评估。从小型任务特定演示集开始，\method{}实现平均IL成功率为56.25%，在线强化后提升至98.75%，三个任务最终成功率达到100%，剩余任务达到95%。

Learning to Throw Objects Safely in Multi-Obstacle Environments

学习在多障碍环境中安全投掷物体

• Authors: Mohammadreza Kasaei, Klemen Voncina, Hamidreza Kasaei
• Subjects: Subjects: Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
• Arxiv link: https://arxiv.org/abs/2607.06388
• Pdf link: https://arxiv.org/pdf/2607.06388
• Abstract Robotic throwing enables fast and efficient object placement beyond the robot's immediate workspace, but reliable throwing in cluttered environments remains underexplored. Existing approaches, such as TossingBot, learn throwing strategies from visual input but assume obstacle-free settings. In this paper, we address the problem of throwing objects into a target basket while avoiding obstacles placed randomly in the scene. We introduce a potential field state representation that compactly encodes both basket attraction and obstacle repulsion on a fixed-size grid, enabling reinforcement learning (RL) policies to generalize across arbitrary numbers and configurations of obstacles. The policy is initialized from kinesthetic demonstrations and optimized in simulation using three state-of-the-art RL algorithms (SAC, DDPG, TD3). Among these, SAC achieves the most consistent performance across scenarios. We compare the potential field representation against explicit state encodings and demonstrate that it achieves higher success rates and better scalability to unseen obstacle configurations. Real-robot experiments with unseen throwable objects confirm robust sim-to-real transfer, achieving up to $90\%$ success in cluttered scenes. These results demonstrate that PFR provides a practical and robust representation for safe and efficient robotic throwing in unstructured environments. A video showcasing our experiments is available at: this https URL
• 中文摘要 机器人投掷使得快速高效的物体摆放超出机器人的直接工作空间，但在杂乱环境中的可靠投掷仍然缺乏开发。现有方法，如 TossingBot，通过视觉输入学习投掷策略，但假设无障碍物。本文探讨了在避开场景中随机设置障碍物的同时，将物体投掷进目标篮子的问题。我们引入了一种潜在场态表示，能够在固定大小网格上紧凑地编码篮子吸引和障碍物排斥，使强化学习（RL）策略能够推广到任意数量和障碍配置。该策略由动觉演示初始化，并在模拟中通过三种最先进的强化学习算法（SAC、DDPG、TD3）进行优化。在这些中，SAC在各场景中表现最为稳定。我们将潜在场表示与显式状态编码进行比较，证明其在未见障碍配置中实现了更高的成功率和更好的可扩展性。真实机器人实验中对看不见的可投掷物体验证了强大的模拟到真实传输，在杂乱场景中实现了高达90%%的成功率。这些结果表明，PFR为在无结构环境中安全高效的机器人投掷提供了实用且稳健的表示。展示我们实验的视频可在以下链接观看：此 https URL

A Definition and Roadmap for World Models

世界模型的定义与路线图

• Authors: Xinyuan Chen, Haoyu Guo, Shi Guo, Bingqi Jiang, Chunhua Shen, Xing Shen, Tianfan Xue, Yufei Xue, Mulin Yu, Weinan Zhang, Bin Zhao, Bowen Zhou, Ming Zhou
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2607.06401
• Pdf link: https://arxiv.org/pdf/2607.06401
• Abstract World models -- internal simulators that learn the structure and dynamics of an environment -- have become one of the most actively debated concepts in AI. From model-based reinforcement learning and video generation to embodied robotics and ultimately, physical AI, researchers across AI subfields are building systems that they call "world models", yet there is no consensus on what a world model fundamentally is, what it should predict, or how it should be built. This perspective article provides a scientific definition of world models, discussions of their key technical aspects, and a staged roadmap for developing effective world models.
• 中文摘要 世界模型——学习环境结构和动态的内部模拟器——已成为人工智能中最受讨论的概念之一。从基于模型的强化学习和视频生成，到具身机器人，最终到物理人工智能，人工智能各子领域的研究者都在构建他们称之为“世界模型”的系统，但对于世界模型的根本性质、应预测什么或如何构建，尚无共识。本文提供了世界模型的科学定义，讨论其关键技术方面，并提供了制定有效世界模型的分阶段路线图。

Multi-Agent Deep Reinforcement Learning for Multi Objective Battery Management in Dairy Farms

多智能体深度强化学习用于奶牛场多目标电池管理

• Authors: Marcos Eduardo Cruz Victorio, Karl Mason
• Subjects: Subjects: Artificial Intelligence (cs.AI)
• Arxiv link: https://arxiv.org/abs/2607.06489
• Pdf link: https://arxiv.org/pdf/2607.06489
• Abstract The dairy industry in Ireland has a large potential for the integration of renewable energy and the reduction of carbon emissions. However, researchers of distributed generation control are mainly focused on residential and commercial applications. To contribute to the effective integration of renewable energy in the dairy sector, this paper presents a multi-objective optimisation control system based on differential evolution and multi agent Deep Reinforcement Learning. The proposed control is organised in two layers: the upper layer uses dynamic pricing, and the lower layer is based on multi-agent reinforcement learning for battery management. This paper also simulates the electrical response of the proposed control system in a rural distribution circuit. The simulation results show that the proposed control framework can improve profits from energy arbitrage up to 18% compared to using Rule-based models, increase the use of distributed generation without significantly increasing cost, and comply with the Irish grid code in terms of voltage variation.
• 中文摘要 爱尔兰乳制品行业在整合可再生能源和减少碳排放方面具有巨大潜力。然而，分布式发电控制的研究者主要集中在住宅和商业应用领域。为促进可再生能源在乳制品行业的有效整合，本文提出了基于差分演化和多智能体深度强化学习的多目标优化控制系统。拟议的控制分为两层：上层采用动态定价，下层基于多智能体强化学习以实现电池管理。本文还模拟了农村配电电路中所提控制系统的电气响应。模拟结果显示，所提控制框架可将能源套利利润提升高达18%，在不显著增加成本的情况下增加分布式发电的使用，并在电压变化方面符合爱尔兰电网代码。

FootsiesGym: A Fighting Game Benchmark for Two-Player Zero-Sum Imperfect-Information Games

FootsiesGym：双人零和不完美信息游戏的格斗游戏标杆

• Authors: Chase McDonald, Nathan Tsang, Wesley N. Kerr
• Subjects: Subjects: Artificial Intelligence (cs.AI); Computer Science and Game Theory (cs.GT)
• Arxiv link: https://arxiv.org/abs/2607.06514
• Pdf link: https://arxiv.org/pdf/2607.06514
• Abstract We present FootsiesGym, an open-source environment for learning in a non-trivial two-player, zero-sum, imperfect-information game. Built on HiFight's minimalist 2D fighting game Footsies, it isolates the cyclic, non-transitive strategic interactions of fighting game neutral play while remaining simple enough for efficient analysis. We provide a vectorized simulator that enables high-throughput training on standard hardware, making the environment accessible and reproducible. We describe the design of the environment, benchmark several reinforcement learning algorithms, and discuss open research directions it enables. The code is available at this https URL.
• 中文摘要 我们介绍FootsiesGym，一个开源环境，用于在非平凡的双人零和、信息不完美游戏中学习。它基于HiFight极简的2D格斗游戏《Footsies》，将格斗游戏中立玩法中循环、非传递的战略互动隔离开来，同时保持足够简单以便高效分析。我们提供了一个矢量化模拟器，支持在标准硬件上进行高通量训练，使环境变得可访问且可复现。我们描述了环境设计，对多种强化学习算法进行了基准测试，并讨论了它所推动的开放研究方向。代码可在该 https URL 访问。

Embodied Human-Robot Interaction via Acoustics: A MARL Approach with AcoustoBots for Spatial Data Physicalization

通过声学实现具身人机交互：利用声学机器人进行空间数据物理化的MARL方法

• Authors: Shiqi Liu, Narsimlu Kemsaram, Prateek Mittal, Pengyuan Wei, Sriram Subramanian
• Subjects: Subjects: Robotics (cs.RO)
• Arxiv link: https://arxiv.org/abs/2607.06563
• Pdf link: https://arxiv.org/pdf/2607.06563
• Abstract Traditional data physicalization is often static and disconnected from real environments, limiting its ability to convey embodied spatial dynamics and engage users. To address this limitation, we present AcoustoBots, a mobile acoustophoretic data-physicalization platform in which TurtleBot3 robots carry upward-facing 8 x 8 ultrasonic phased arrays. Each array levitates a particle whose height (1-10 cm) encodes a local urban scalar value, such as population density, noise, or traffic. A MARL (Multi-Agent Reinforcement Learning) policy based on the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm, with centralized training and decentralized execution, selects collision-aware navigation actions, while a high-rate Gerchberg-Saxton-Phased Array of Transducers (GS-PAT) acoustic controller maintains trap stability and updates array phases to achieve the commanded height during motion. This creates a closed perception-display-action loop. We evaluate single-robot city-to-city traversal and dual-robot cooperative coverage on a 4 m x 3 m scaled UK map using PhaseSpace-based localization for repeatable multi-robot trials. Results show stable in-motion levitation and consistent, location-dependent height rendering, with task success rates of 90% and 80% for the single and dual-robot regimes, respectively, over 10 trials per regime, and low collision counts. These findings support acoustophoretic levitation as a simple, glanceable, robot-mediated communication cue for embodied human-robot interaction in spatial analytics.
• 中文摘要 传统的数据物理化通常是静态的，与真实环境脱节，限制了其传达具象空间动态和吸引用户的能力。为解决这一限制，我们介绍了 AcoustoBots，一种移动声学信息物理化平台，TurtleBot3 机器人携带朝上 8 x 8 超声波相控阵。每个阵列悬浮一个粒子，其高度（1-10厘米）编码了局部城市标量值，如人口密度、噪声或交通量。基于多智能体深度确定性策略梯度（MADDPG）算法的MARL（多智能体强化学习）策略，结合集中训练和分散执行，选择碰撞感知的导航动作，而高速率的Gerchberg-Saxton相控阵（GS-PAT）声学控制器则保持陷阱稳定性并更新阵列相位，以实现运动中的指令高度。这形成了一个封闭的感知-显示-动作循环。我们利用基于PhaseSpace的定位，评估单机器人城市到城市穿越及双机器人协作覆盖，覆盖范围为4米×3米的英国地图，实现可重复的多机器人试验。结果显示，运动中悬浮稳定，高度渲染一致且依位置变化，单机器人和双机器人模式的任务成功率分别为90%和80%，每个阶段有超过10次试验，碰撞次数也很低。这些发现支持声学漂浮作为一种简单、可一瞥的机器人介导的空间分析中具身人机交互的沟通线索。

Keyword: diffusion policy

There is no result