open-source ml robotic arm

student team lead

open-source ml robotic arm

Led a small team adapting open-source vision-language-action models to control robot arms for varied manufacturing tasks.

01 · the problem

Fixed automation can’t handle variety

Traditional robot arms repeat programmed moves, so they struggle with high-variance manufacturing work. My text-controlled arm showed that simple object detectors weren’t enough either.

02 · the hypothesis

Open VLA models can drive real arms

If open vision-language-action models, like Physical Intelligence’s openpi used through LeRobot, are retrofitted to affordable open-source arms, they can generate actions from camera images and arm positions and learn a wide variety of tasks.

03 · planning

Three arms, one pipeline

  • Three open-source arms: the Parol 6 by Source Robotics, the Mikobot 1, and an SO-ARM100 leader–follower pair.
  • A physical emulator, an identical arm with the same geometry (DH parameters), to make recording training episodes easy, and plans for a simulated twin to generate synthetic training data.
  • A team of up to five, in practice mostly two of us, with roles, delegation and risk managed in Notion and Trello.

04 · executing

Retrofitting the models

We worked on getting the open models running on the three arms, which needed far more processing power than the text-controlled arm had used.

05 · iteration

Narrowing to what worked

The SO-ARM100 leader–follower pair proved the fastest way to record demonstrations and train policies through LeRobot.

06 · latest version

Where it led

Two years in a fast-moving field changed my direction. The SO-100 work became towel folding with ACT, my first fully autonomous task, and then the mobile robots LeKiwi and OB15.

stackVLA · LeRobot · Parol 6 · SO-ARM100