Led a small team adapting open-source vision-language-action models to control robot arms for varied manufacturing tasks.
01 · the problem
Fixed automation can’t handle variety
Traditional robot arms repeat programmed moves, so they struggle with high-variance manufacturing work. My text-controlled arm showed that simple object detectors weren’t enough either.
02 · the hypothesis
Open VLA models can drive real arms
If open vision-language-action models, like Physical Intelligence’s openpi used through LeRobot, are retrofitted to affordable open-source arms, they can generate actions from camera images and arm positions and learn a wide variety of tasks.
03 · planning
Three arms, one pipeline
- Three open-source arms: the Parol 6 by Source Robotics, the Mikobot 1, and an SO-ARM100 leader–follower pair.
- A physical emulator, an identical arm with the same geometry (DH parameters), to make recording training episodes easy, and plans for a simulated twin to generate synthetic training data.
- A team of up to five, in practice mostly two of us, with roles, delegation and risk managed in Notion and Trello.
04 · executing
Retrofitting the models
We worked on getting the open models running on the three arms, which needed far more processing power than the text-controlled arm had used.
05 · iteration
Narrowing to what worked
The SO-ARM100 leader–follower pair proved the fastest way to record demonstrations and train policies through LeRobot.
06 · latest version
Where it led
Two years in a fast-moving field changed my direction. The SO-100 work became towel folding with ACT, my first fully autonomous task, and then the mobile robots LeKiwi and OB15.
| stack | VLA · LeRobot · Parol 6 · SO-ARM100 |
