SmolVLA: Lightweight Vision-Language-Action Model for Robotics

SmolVLA Overview

SmolVLA is Hugging Face’s lightweight foundation model for robotics. 450M parameters. Designed for easy fine-tuning on LeRobot datasets.

Inputs

• Multiple camera views
• Robot’s current sensorimotor state
• Natural language instruction

These are encoded into contextual features that condition an action expert generating an action chunk.

Fine-tuning Workflow

1. Collect ~50 episodes of your task (more = better generalization per variation)
2. Fine-tune lerobot/smolvla_base using lerobot-train
3. ~4 hours on a single A100 GPU for 20k steps
4. Evaluate and deploy using lerobot-record

Dataset Requirements

• Minimum ~50 episodes recommended; 25 was insufficient in tests
• Need enough demonstrations per variation (e.g. different cube positions)
• Key: repeat each variation multiple times

Performance (from SmolVLA paper)

Pretraining on 481 community datasets (~23,000 episodes, 10.6M frames — primarily SO-100):

• Task-specific training only: 51.7% success rate
• With SmolVLA pretraining: 78.3% success rate

Key Commands

# Fine-tune
lerobot-train --policy.path=lerobot/smolvla_base --dataset.repo_id=user/dataset --steps=20000
 
# Evaluate / real-time inference
lerobot-record --robot.type=so101_follower --policy.path=user/my_smolvla