I recently gave my OpenClaw is a real robotic arm to play with. The results almost blew up my own neural network.
The AI agent was able to configure the arm, operate it to see and slowly grasp objects, and even train another AI model to pick up and place specific objects. And they say that AGI is still a few years away! (kidding, probably yes).
The results confirm my belief that we may be one step away from a breakthrough in robotics. It used to be that training and controlling robots required considerable skill. Today’s AI models can make this almost basic.
“AI-based coding is incredibly exciting because it has the potential to bridge the gap between conventional engineering methods, which are reliable but do not generalize, and modern models of vision, language, and action, which generalize but are not yet foolproof,” says Ken Goldberg, a robotics scientist at the University of California, Berkeley, who studies the approach.
I bought a ready-made arm, the so-called LeRobot 101. This is part of HuggingFace’s open source project, which makes it relatively budget-friendly to start building and experimenting with robotics.
LeRobot is equipped with two arms: a steering arm that can be controlled using a handle and trigger, and a guide arm with a camera that reproduces these movements. You can train an AI model by remotely operating a controller arm and letting the model learn how to move the observer in response to what it sees in the camera.
Building with OpenClaw
Before using OpenClaw, I spent several hours trying to connect and calibrate the robot, at one point almost damaging the motors by using the wrong settings, which caused them to overheat.
Then, with the support of OpenClaw and Codex, I managed to write a uncomplicated vibration program that closed the claw gripper when it spotted a red ball. In the Codex terminal, he did the hard work of configuring connections to the robot. Then, with my support, he calibrated the position of his joints. He also wrote a Python script that used several libraries to identify and catch a given ball. Vibration encoding isn’t perfect, of course, and hallucinations can cause errors, especially when working with different hardware, but the results were impressive.
