Robots still aren’t that good at picking things up, despite billions of dollars of investment and endless videos fed into deep learning models.
That may be about to change, thanks to a manipulation architecture built by Eka Robotics, which encompasses a force-sensitive gripper and a new kind of deep-learning model. Founded by MIT professor Pulkit Agrawal and former Google DeepMind researcher Tuomas Haarnoja, the startup just closed an $150 million Series A round that values it at $1.45 billion post-money, led by venture funds a16z and F-Prime.
“They are already exceeding superhuman performance for certain tasks, and so we think over the next few months there will be superhuman performance in many dexterous manipulation tasks,” Raghu Raguram, the a16z general partner who co-led the fundraising round, told TechCrunch.
In an era of never-ending robotic demonstrations, Eka’s capabilities are shocking veteran robot builders.
Kris Hauser, a professor at the University of Illinois Urbana-Champaign, said it had a “wow factor” compared to other robot demos he saw recently. The Eka robot “was able to adapt to a large number of different objects without retraining,” Hauser said, adding that it also had an aspect of “humanness, creepy humanness that it would try over and over again if it failed the first time.”
Another roboticist, who asked for anonymity because of their work at a frontier lab, told TechCrunch that seeing Eka’s robot in action was an emotional experience, comparable to when robots first achieved reliable legged locomotion five to ten years ago. Take a look for yourself:
What’s notable here? How smoothly and rapidly the robot moves. It can pick up about 1,500 objects that weigh up to 1 kg in an hour, compared to about 600 for a human; Eka says it reliably picks up objects more than 95% of the time. And the robot arm is strong enough to hurt itself as it moves that quickly. That indicates, per the frontier lab observer, that “they have a lot of trust in their algorithm not breaking the robot. I wonder how many robots they broke along the way?”
When the frontier lab roboticist met Eka’s robot, they handed it a raspberry, the lab’s apparent favorite test item. “The experience of a big, heavy robot rushing towards my arm and then slowing down just in time and very gently touching me is a feeling that I am, as we speak, carrying this feeling in my body,” they told TechCrunch.
How not to squish the raspberry
These capabilities are a consequence of how the foundational model was built. Rather than a vision-language-action model — which is trained to understand the relationship between language, visual inputs, and actions — the schema here is a vision-force-action model, which focuses on the relationship between language, force inputs, and actions. Critics of the current direction of generative AI robotics models, like Rodney Brooks, say that they will fail without an attempt to represent force and touch; Eka’s results suggest such critics have a point.
“Vision is not the primary way in which you close this last millimeter problem,” Eka co-founder Agrawal told TechCrunch. “Vision tells you roughly where the object is, what the pose is, but the actual grabbing, the actual searching is happening through forces.”
Eka had to design a unique robotic hand that was more flexible than traditional grippers, and included more force sensors; it reacts to contact, which is why it doesn’t crush the raspberry. The founders are cagey about how they achieved that for obvious competitive reasons, but it comes down to using a physics simulator to model the multiple forces the robot will encounter as it attempts to grasp something, and using that to create a reinforcement learning environment that can be run at scale. They say they can generate thousands of hours of robot experience in minutes.
“Rather than being limited to the data that you’re collecting from, say, human demonstrations, Eka is pushing through reinforcement learning this approach where the robot can surpass any kind of demonstrations that are given to it,” Hauser said. “What’s really exciting is that they’re able to not be limited to the data that is presented to them for training.”
Agrawal and Haarnoja, who first met in graduate school at the University of California, Berkeley, are leading researchers in the field of reinforcement learning for robotics control. Eka’s key bet was that this technique could be done at scale through simulation, rather than relying on videos collected by gigworkers or robot operators performing tasks over many hours. Simulation was key to achieving their robot’s reliable manipulation, a feat that in the past that has proven more difficult for tasks that require interacting with objects, rather than avoiding them.
“What EKA is bridging is the generality-performance gap, which plagues a lot of modern robotics,” Agrawal argues.
Making a general-purpose robot useful
Right now, the most generalizable physical AI models—the ones that can attempt lots of different tasks—aren’t particularly good at them. But AI models trained to be highly capable at a single task may not close the business case for a company: The promise of generative AI is that robots can quickly learn multiple tasks, even novel ones.
The current approaches are good at producing robots that are more generally intelligent than before — Astra, ChatGPT’s latest LLM, can operate a robot in an unfamiliar setting and perform tasks with it, but slowly and at a high cost. The promise of a model like Eka’s is that it handles the physicality while a more general model makes decisions about how to solve a problem. Astra, for example, might know that plates get stacked in a cupboard when they come out of the dishwasher, but Eka’s model is capable of picking up a dish without breaking it.
“Most of the field has been focusing on the planning,” Eka co-founder Haarnoja says, “so they approach from the top: high-level intelligence, pushing that down towards the physical. Whereas we think about it from the bottom up.”
Eka still needs to prove itself in the field, though. The company is working in secretive pilots with a number of industrial customers doing pick-and-place and wire manipulation, but the leap from lab to line is almost as big as the gap between sim and real.
“What’s going to be a big challenge is when they work with integrators on real assembly lines, and it needs to run without breaking for an entire year,” Hauser said. The other open question is “‘how generalizable these techniques can be to different tasks?’ What I’ve seen is pick and place, but many industry tasks that are value added involve more than just pick-and-place.”
Agrawal is also focused on deployability; right now, he says Eka’s gripper hardware — which costs less than $1,000 — is rated for 5 million duty cycles, but the ultimate goal is to hit the 20 million cycle bar set by standard industrial arms.
“What we are building is that engineering of how to design entire workflows, but we are starting off being very pragmatic, and saying here are two verticals in pick, arrange, place, and wire handling, let’s make them be home runs,” Agrawal said. “Once we have made home runs on them, the sky is the limit in how much we can do with the approach we are developing.”