arXiv Weekly Digest — Week 34, 2026

Fetched: 2026-08-17 | Categories: cs.RO, cs.LG, cs.HC, cs.CV | Papers: 19

Authors: Hanfeng Lu, Tianyu Feng, Suyi Li et al. | Submitted: 2026-08-14 | arXiv: 2608.14498 Categories: cs.LG, cs.DC

Research Background: RL post-training of VLMs for embodied agents is computationally intensive; standard GPU pipelines leave significant compute idle during the generation and training phases of rollout-based RL.

Technical Approach: Rollplex introduces cross-phase GPU spatial sharing, multiplexing both the generation (inference) and training phases of RL post-training onto the same GPU simultaneously. By carefully managing memory and scheduling across phases, it avoids the idle GPU time that sequential pipelines incur, effectively treating the GPU as a single continuous compute resource across RL phases.

Key Takeaway: Cross-phase spatial sharing cuts GPU idle time in VLM RL post-training, offering a practical efficiency gain for scaling embodied-agent training without additional hardware.

Expected Free Energy-based Informative Path Planning for Robotic Mars Exploration

Authors: Ajith Anil Meera, Pablo Lanillos, Wouter Kouw | Submitted: 2026-08-14 | arXiv: 2608.14466 Categories: cs.RO, cs.IT, cs.LG

Research Background: Autonomous planetary rovers must build accurate environment maps while simultaneously seeking targets of interest under tight energy constraints — a dual objective that classical path planners handle poorly.

Technical Approach: The authors apply Active Inference’s Expected Free Energy (EFE) to informative path planning, combining epistemic value (uncertainty reduction) and pragmatic value (goal proximity) in a single objective. The robot selects paths that jointly minimize map uncertainty and maximize proximity to predicted resource locations, without requiring explicit reward engineering.

Key Takeaway: EFE-based planning provides a principled way to unify exploration and goal-directed behavior in planetary rovers, outperforming information-only baselines in sparse-target environments.

CORAL: Curriculum-Optimized Reward Adaptation for LiDAR-Based Goal-Directed Urban Driving

Authors: Anisa Saleem, Duksu Kim | Submitted: 2026-08-14 | arXiv: 2608.14332 Categories: cs.RO, cs.LG

Research Background: Reinforcement learning for urban autonomous driving requires balancing goal-reaching, collision avoidance, and traffic-rule compliance simultaneously — a multi-objective problem where reward shaping strongly influences the learned policy’s priorities.

Technical Approach: CORAL uses a curriculum that adapts the reward function over training, initially emphasizing safety and basic compliance before progressively shifting weight toward long-horizon goal achievement. LiDAR-based perception feeds an RL policy that is trained in simulation with this staged reward schedule before evaluation on urban navigation benchmarks.

Key Takeaway: Curriculum-based reward adaptation enables RL agents to reliably reach distant urban navigation goals without sacrificing collision avoidance learned in earlier training stages.

MLLM-Routed Heterogeneous Ensembles for Robust Cross-Dataset Image Classification

Authors: Daniel Perkins, John Squires, Janou Milligan et al. | Submitted: 2026-08-13 | arXiv: 2608.13463 Categories: cs.CV, cs.AI, cs.CL

Research Background: Specialist vision models trained on single datasets generalize poorly across domains; ensembling multiple models helps but requires an intelligent routing mechanism to select the best model per input.

Technical Approach: ARMDIL (Adaptive Router for Multi-Dataset Image Learning) uses a Multimodal Large Language Model as a zero-shot router that, given an image and task description, selects which specialist model in a heterogeneous ensemble is most likely to succeed. The MLLM’s semantic understanding allows routing without task-specific training data for the router itself.

Key Takeaway: MLLM-based routing over specialist ensembles achieves cross-dataset generalization that neither individual models nor naive ensembles match, with the router requiring no labeled routing examples.

Sampling Luck Masquerades as Allocation Gain: Auditing Test-Time Budget Allocation for Neural Combinatorial Optimization

Authors: Jinhyung Bae | Submitted: 2026-08-13 | arXiv: 2608.13087 Categories: cs.LG, cs.AI, math.OC

Research Background: Neural combinatorial optimization (NCO) methods report results as the best of N sampled solutions per instance; recent work claims non-uniform sample budget allocation across instances yields additional gains beyond simply using more samples.

Technical Approach: The paper audits test-time budget allocation strategies through careful controlled experiments that separate the effect of adaptive allocation from the baseline gain of increased total samples. By holding total sample budget constant and varying allocation policy, it isolates whether adaptive allocation genuinely improves over uniform sampling.

Key Takeaway: Apparent gains from adaptive test-time budget allocation in NCO largely reflect sampling luck rather than principled allocation, urging caution when interpreting benchmark improvements in this area.

HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark

Authors: Dairu Liu, Zekun Qi, Jiayu Zeng et al. | Submitted: 2026-08-13 | arXiv: 2608.13555 Categories: cs.RO, cs.AI, cs.CV

Research Background: Humanoid motion tracking is central to teleoperation and whole-body imitation learning, yet existing benchmarks use kinematic error metrics that correlate poorly with human perceptual judgments of tracking quality.

Technical Approach: HumanTracker introduces a benchmark with multi-modal motion sequences and a human-aligned evaluation protocol that combines kinematic metrics with perceptual quality scores collected from human raters. The benchmark covers diverse motion types relevant to teleoperation and tests tracking policies across hardware-realistic constraints.

Key Takeaway: Kinematic error alone systematically misranks motion tracking policies; human-aligned metrics reveal gaps that standard benchmarks hide, directly impacting how teleoperation systems should be evaluated.

DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation

Authors: DreamX Team, Rui Chen, Xiangxiang Chu et al. | Submitted: 2026-08-13 | arXiv: 2608.13489 Categories: cs.CV, cs.RO

Research Background: World models that can predict future visual observations conditioned on robot actions are valuable for policy training and planning, but generating physically plausible manipulation videos that respect action semantics remains difficult.

Technical Approach: DreamX-Phi 1.0 is a video diffusion model conditioned on observed frames, language instructions, and prescribed action sequences (joint angles or end-effector poses). It generates coherent manipulation video rollouts that track the specified action trajectory, enabling action-conditioned imagination for downstream policy use.

Key Takeaway: Action-conditioned video generation at the quality of DreamX-Phi 1.0 opens a practical path to data augmentation and model-based planning for robot manipulation without additional real-world collection.

UniTexture: Cross-Task Universal Adversarial Textures for Vision-Language-Action Models

Authors: Yukun Dai, Mingzhe Dai, Tianshi Wang et al. | Submitted: 2026-08-13 | arXiv: 2608.13453 Categories: cs.CV, cs.AI

Research Background: VLA models are increasingly deployed as generalist robot controllers, but their robustness to adversarial physical perturbations — particularly texture patches that can be placed in a scene — has not been systematically studied across diverse manipulation tasks.

Technical Approach: UniTexture optimizes a single adversarial texture patch that transfers attacks across multiple manipulation tasks without task-specific re-optimization. The texture is crafted to maximally degrade the VLA model’s action predictions by targeting shared visual representations, and is validated on physical robot setups with printable patches.

Key Takeaway: A single universal adversarial texture can reliably fool state-of-the-art VLA policies across diverse manipulation tasks, revealing a critical physical-world robustness gap that must be addressed before broad deployment.

ContactGuard: Pre-Contact Execution Monitoring with Action-Conditioned Latent World Models

Authors: Gehan Zheng, Matthew Johnson-Roberson, Weiming Zhi | Submitted: 2026-08-13 | arXiv: 2608.13438 Categories: cs.RO, cs.AI, cs.CV

Research Background: Contact-rich manipulation failures are typically detected only after damaging contact has occurred; anticipating problematic contact before it happens — especially from wrist-camera views — is an open problem for safe robot deployment.

Technical Approach: ContactGuard trains an action-conditioned latent world model on wrist-camera sequences that learns to predict the near-future visual state given the robot’s planned actions. A monitoring module flags predicted states that deviate from expected safe contact patterns, triggering intervention before physical contact is made.

Key Takeaway: Pre-contact monitoring via latent world model prediction catches impending manipulation failures substantially earlier than post-contact sensing, enabling safer and more recoverable robot behavior.

Authors: Koen P. de Vries, Xavier Alameda-Pineda, Estefanía Talavera et al. | Submitted: 2026-08-13 | arXiv: 2608.13239 Categories: cs.CV

Research Background: Embodied social agents must understand complex social interactions from audio-visual streams; multimodal LLMs with chain-of-thought reasoning are now the dominant approach, but their actual capability on social audio-visual QA is poorly characterized.

Technical Approach: The paper benchmarks current multimodal LLMs on social audio-visual QA datasets, systematically varying chain-of-thought prompting strategies and model scales. It analyzes where reasoning helps versus where models rely on spurious cues, and identifies failure modes specific to social scene understanding.

Key Takeaway: Current multimodal LLMs exhibit significant gaps in genuine social audio-visual reasoning, with CoT providing inconsistent gains — suggesting social scene understanding remains far from solved.

Semantic Radiance Fields as Simulators for Spatial Reasoning in Real-World Scenes

Authors: Nico Heider, Michał Jan Włodarczyk, Katarzyna Wasielewska-Michniewska et al. | Submitted: 2026-08-13 | arXiv: 2608.13095 Categories: cs.RO, cs.CV

Research Background: Training embodied agents for spatial reasoning requires simulators that are both geometrically faithful to real environments and semantically queryable — a combination difficult to achieve with synthetic or purely geometric reconstructions.

Technical Approach: The authors combine neural radiance fields (NeRF) with semantic feature distillation to build scene representations that support both photorealistic novel-view synthesis and language-conditioned spatial queries. Embodied agents trained in these semantic radiance field simulators can then be transferred to real scenes with reduced domain gap.

Key Takeaway: Semantic radiance fields bridge geometry and language grounding for embodied agent training, offering a route to sim-to-real spatial reasoning transfer without handcrafted semantic annotations.

Learning Unified Video and Image Representation for Video Face Forgery Detection

Authors: Haotian Liu, Yang Liu, Guoying Zhao et al. | Submitted: 2026-08-13 | arXiv: 2608.13064 Categories: cs.CV

Research Background: Detecting deepfake videos is critical for social media integrity, but models trained separately on images versus videos miss cross-modal consistency signals that could improve detection accuracy.

Technical Approach: The proposed method learns a unified representation shared between video and image forgery detection through joint training with modality-specific and shared encoder branches. Temporal inconsistency signals from video frames inform the image branch, and vice versa, via a cross-modal attention mechanism that exploits forgery artifacts that manifest differently in static versus dynamic settings.

Key Takeaway: Unified video-image representation learning enables each modality to borrow forgery-detection cues from the other, improving detection accuracy on both video and image deepfakes versus single-modal baselines.

H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models

Authors: Dingyi Rong, Yue Shi, Chaofan Ma et al. | Submitted: 2026-08-13 | arXiv: 2608.13049 Categories: cs.RO, cs.CV

Research Background: Collecting large-scale robot manipulation demonstrations is expensive; using abundant egocentric human manipulation videos as a data source requires a domain translation step — generating robot-viewpoint videos from human observations — whose quality is largely unevaluated.

Technical Approach: H2R-Bench defines standardized metrics for evaluating human-to-robot video generation quality, covering visual fidelity, action consistency, and cross-embodiment realism. World models are evaluated on their ability to synthesize robot-perspective manipulation sequences that plausibly correspond to the source human demonstrations.

Key Takeaway: H2R-Bench reveals that current world models vary substantially in their ability to produce robot-realistic manipulation videos from human demonstrations, highlighting gaps that must close before human video data can reliably augment robot training.

P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation

Authors: Yi Shi, Huichao Xie, Yuqing Wang et al. | Submitted: 2026-08-13 | arXiv: 2608.13045 Categories: cs.CV

Research Background: Fusing infrared thermal and visible-spectrum images for robust multimodal perception is important for robots and autonomous vehicles operating in low-light or occluded conditions, but reconciling the structural and thermal modality gaps remains challenging.

Technical Approach: P2Fusion introduces a prompt-based progressive fusion framework that distills dual priors — semantic structure from visible and thermal salience from infrared — into a shared representation via learnable prompts. Fusion is performed progressively across network stages, guided by modality-specific priors that are adaptively weighted by input context.

Key Takeaway: Prompt-driven dual-prior distillation enables P2Fusion to produce fused images that preserve both fine structural detail and thermal contrast, improving downstream detection performance over single-stage fusion baselines.

RGB-D Video Generation for Improving Human-to-Robot Object Handover Prediction

Authors: Tianyu Sun, Zhoujie Fu, Zihui Gao et al. | Submitted: 2026-08-13 | arXiv: 2608.13028 Categories: cs.CV, cs.RO

Research Background: Human-to-robot (H2R) object handover is a critical HRI capability, but training robust handover prediction models is limited by scarce large-scale, depth-annotated human-centric datasets.

Technical Approach: The paper uses generative RGB-D video synthesis to augment training data for handover prediction, generating diverse human approach and grasp sequences with consistent depth channels. Prediction models trained on real plus generated RGB-D data are evaluated on timing and position accuracy for robot grasp preparation before the human completes the handover motion.

Key Takeaway: Synthetic RGB-D video augmentation substantially improves handover prediction accuracy, offering a scalable data strategy for a capability that is otherwise dataset-limited.

EgoPHI: Estimating Contact and Force from Egocentric Vision

Authors: Andela Ilic, Rachel Schuchert, Yijing Jiang et al. | Submitted: 2026-08-13 | arXiv: 2608.13014 Categories: cs.CV, cs.GR, cs.HC

Research Background: Understanding hand-object physical interaction from egocentric cameras — including contact location and applied force — is essential for modeling human manipulation and enabling physically grounded HRI, but estimating forces from vision alone is underdetermined.

Technical Approach: EgoPHI trains a model to jointly estimate contact regions and interaction forces from egocentric video, using a dataset of instrumented object interactions with synchronized video and force/torque measurements. A physics-informed loss encourages force estimates to satisfy contact constraints derived from the predicted hand-object geometry.

Key Takeaway: Physics-constrained egocentric force estimation reaches practically useful accuracy on everyday manipulation tasks, opening new possibilities for teleoperation skill transfer and HRI force modeling without wrist-mounted sensors.

A Deep RL based Framework for Targeted White Matter Tractography

Authors: Ankita Joshi | Submitted: 2026-08-13 | arXiv: 2608.12960 Categories: cs.CV

Research Background: White matter tractography — reconstructing neural fiber pathways from diffusion MRI — is important for neurosurgical planning, but existing methods suffer from high false-positive rates when tracing specific target tracts.

Technical Approach: A deep RL agent learns to sequentially extend fiber streamlines by treating tractography as a navigation task, receiving rewards for approaching anatomical target regions while penalizing biologically implausible trajectories. The agent uses local diffusion orientation features as its state and is trained on annotated atlas data.

Key Takeaway: RL-based tractography improves target-tract specificity over classical deterministic and probabilistic methods, reducing false positives in clinically relevant white matter bundles.

BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving

Authors: Bing Zhan, Shuyao Shang, Jiahao Gu et al. | Submitted: 2026-08-13 | arXiv: 2608.12854 Categories: cs.RO, cs.AI, cs.CV

Research Background: End-to-end autonomous driving systems struggle to jointly satisfy semantic constraints (traffic rules, scene understanding) and predictive dynamics (trajectory forecasting), as existing architectures tend to optimize for only one of these objectives.

Technical Approach: BrainWAM introduces a world-action model that coordinates semantic priors (from a vision-language module) with learned predictive dynamics in a shared action space. The semantic module provides high-level scene interpretation while the dynamics module predicts physically plausible trajectories, with coordination occurring at the action-decision level rather than through feature fusion.

Key Takeaway: Coordinating semantic and dynamic reasoning at the action level — rather than the feature level — yields more consistent driving behavior that respects both traffic semantics and physical plausibility simultaneously.

FUSE: Active Functional Affordance Grounding through Adaptive Semantic-Geometric Evidence Acquisition

Authors: Zhou Chen, Sathyanarayanan N. Aakur | Submitted: 2026-08-13 | arXiv: 2608.12683 Categories: cs.RO, cs.CV

Research Background: Embodied agents identifying objects by function (e.g., “something to sit on”) rather than category must actively gather observations to reveal functional cues that may be occluded — a problem existing fixed-viewpoint affordance methods cannot address.

Technical Approach: FUSE combines uncertainty-driven exploration (selecting viewpoints that most reduce uncertainty about functional affordances) with a learned amortized planner that generalizes exploration strategies across objects. The framework is evaluated on a new Habitat-based benchmark for active functional grounding with diverse affordance queries.

Key Takeaway: Combining explicit uncertainty-driven exploration with an amortized planner achieves the best non-oracle grounding performance with 1.33× less computation than purely explicit search, making active affordance grounding practical for real embodied agents.

Quartz 5

Explorer

arXiv Digest — 2026-W34