Skip to main content

FMCW LiDAR for dual-arm assembly

Peg-in-hole research based on GR00T. The project tests whether range, radial velocity and time data from FMCW LiDAR improve precise insertion by a vision-language-action model.

Seeking partners Seeking: VLA fine-tuning team (university lab or company research team)

Who we are looking for

A university lab or a company research team with experience in fine-tuning vision-language-action (VLA) models for embodied AI. We can agree the form of collaboration in a meeting.

Project status

This is a proposed research project. It has no completed model training or robot test results yet. All performance values below are targets or public reference values.

Research questions

The project tests whether dual-arm peg-in-hole assembly improves when FMCW LiDAR provides input to a pretrained VLA model. It asks the three questions below.

InputBenefit to test
RangeCan sparse range measurements improve alignment when stereo depth is unreliable?
Radial velocityCan directly measured velocity reduce delay, overshoot or contact force during approach?
Time historyCan separate time stamps and motion correction improve coverage without too much motion blur?

Radial velocity shows only the motion along the laser line of sight. It is not a full three-dimensional velocity, and sideways motion can give a radial velocity near zero. A nominal 2 mm range specification also does not mean submillimeter assembly accuracy. The stage inside the hole still needs force feedback.

Task and planned setup

Two fixed-base arms pick up the parts and move them into a shared field of view. The left arm holds the hole part. The right arm aligns the peg and inserts it under force control.

ComponentPlanned configuration
Robot armsTwo Unitree R1-7a arms, each with seven axes and a gripper
CameraOrbbec Gemini 335, to compare passive stereo with active stereo
FMCW LiDAR100,000 points/s across the full field of view, non-repeating Lissajous scan, 40° × 40° field of view
Contact and teachingTwo wrist six-axis force/torque sensors (to be installed), PICO 4 Ultra controllers for demonstrations

Model plan

The main model is GR00T N1.7-3B. The plan freezes the vision-language backbone and trains the full action head. The new sensor modules share a budget of 30 million parameters. Camera and LiDAR features reach the action expert through gated cross-attention. One policy controls both arms.

Experiments and targets

  • Comparison groups: RGB only, passive stereo, active stereo, FMCW range, full FMCW input (with radial velocity), and a conventional geometry method.
  • Main task: 10 mm peg diameter, 20 mm insertion depth, 0.5 mm diametral clearance. A 0.2 mm clearance tests the capability limit.
  • Research target: at least 90% first-attempt success in the main task. This is a target, not a measured result.
  • Scale: the core plan contains 3,900 robot trials. Human demonstrations give about 1,720 valid trajectories.

The project brief (PDF, in English) gives the full data plan, training plan and evaluation measures.

Join this project

Email us about your team and its relevant experience. We will reply and set a time to discuss how to work together.