SafeVLA: Towards Safety Alignment of Vision-Language-Action Model via Constrained Learning

Authors: Borong Zhang, Yuhao Zhang, Jiaming Ji, Yingshan Lei, Yishuai Cai, Josef Dai, Yuanpei Chen, Yaodong Yang (PKU-Alignment)

Venue: NeurIPS 2025 Spotlight

arXiv: 2503.03480

Abstract

SafeVLA proposes an Integrated Safety Approach (ISA) for VLA models: model safety requirements systematically, elicit diverse unsafe behaviors, constrain VLA policies via constrained RL (CMDP), and rigorously evaluate safety through targeted tests. Achieves 83.58% reduction in cumulative safety violations vs. state-of-the-art, with +3.85% task success rate improvement.

Method

• CMDP formulation: separate reward and cost signals; constrain cumulative cost below threshold during training
• Unsafe behavior elicitation: actively generate diverse failure modes to train safety constraints against
• ISA pipeline: requirements modeling → elicitation → constrained training → targeted evaluation
• Benchmark: Safety-CHORES — long-horizon mobile manipulation tasks

Safety Relevance

SafeVLA applies CMDP-style safe RL (the same family as SafeDreamer) to VLA foundation models. The key difference: instead of a latent world model, the safety reasoning happens inside the VLA’s language/vision understanding. The VLA itself becomes the semantic safety reasoner.

Notes

Same PKU-Alignment group that wrote SafeDreamer. This is their follow-up applying safe RL to VLA models rather than pure RL agents.