本文由 AI 分析生成
建立時間： 2026-08-18 來源： https://arxiv.org/abs/2402.10340
Summary
This paper empirically demonstrates that robotic systems controlled by LLMs or VLMs are fragile to small input perturbations: rephrasing a natural-language instruction or adding minor visual noise can reduce task success rates by 14–22%. The authors conduct both white-box (gradient-based) and black-box (random search) attacks, revealing that the open-loop propagation of LLM/VLM reasoning errors directly to robot actuation creates unique attack surfaces absent in classical controllers.
논문은 LLM/VLM 기반 로봇 시스템이 명령어 재표현이나 경미한 시각적 노이즈에 취약함을 실증적으로 보여주며, 이러한 취약성이 기존 제어기에는 없는 새로운 공격 표면을 만들어냄을 주장한다.
本文實驗性地揭示 LLM/VLM 控制機器人系統對微小輸入擾動（指令重新措辭、輕微視覺雜訊）的脆弱性，任務成功率下降 14–22%，且錯誤從語言模型推理直接傳播至機器人動作執行，缺乏中間驗證層。
Prerequisites
- Adversarial robustness — the paper applies adversarial ML concepts (white-box gradient attacks, black-box perturbations) to robotics; requires understanding of adversarial examples in vision and NLP.
- LLM/VLM-based robot control — reader needs familiarity with how language and vision models are used as high-level planners or direct action predictors in robotic systems.
Core Idea
Classical robot controllers fail on out-of-distribution states but degrade gracefully because their failure modes are bounded by mathematical properties (e.g., Lyapunov stability). LLM/VLM controllers fail qualitatively differently: a single token change in the instruction or a minor RGB shift in the input image can flip the model’s output arbitrarily, and that flipped output becomes an actuation command with no intermediate sanity check. The paper argues that the high sensitivity of attention-based architectures to small input changes is fundamentally incompatible with the robustness requirements of physical robot deployment without dedicated robustification.
Results
| Attack Type | System | Success Rate Drop |
|---|---|---|
| Instruction perturbation | System A | ~22.2% |
| Perceptual perturbation | System B | ~14.6% |
| White-box vs. black-box | Both | White-box more effective |
Limitations
- Author-stated: Limited to two representative LLM/VLM robotic systems; generality to newer architectures (e.g., VLAs) is unstated.
- Unstated: No proposed defenses or robustification methods are evaluated; the paper is purely diagnostic. Also, the “safety = task completion” framing conflates robustness with safety.
Reproducibility
- Code: Not confirmed as released
- Datasets: Proprietary simulation environments
- Compute: Standard GPU inference for LLM/VLM evaluation
Insights
This paper is the adversarial complement to VLM-as-safety-filter approaches. If VLMs are used as safety reasoners (e.g., risk field papers), those VLMs are themselves vulnerable to the same perturbations studied here — a malicious actor could craft visual inputs that fool the safety VLM into rating a dangerous trajectory as safe. The field needs robustified VLMs before deploying them in either the action or safety-filter role.
Connections
- semantic-metric-bayesian-risk-fields-vlm-robot-safety — uses VLM as safety filter; this paper shows that filter is itself attackable
- safevla-safety-alignment-vla-constrained-learning — SafeVLA addresses safety alignment; vulnerability to adversarial inputs is a complementary open problem
- world-models-robot-safety — safety survey context
Raw Excerpt
“Simple input perturbations reduce task execution success rates by 22.2% and 14.6% in two representative systems.”