AI in Robotics

Where AI fits in a robot

A classic industrial robot needs no AI at all. It repeats a taught program in a cell where every part arrives in the same place, and its motion is computed with kinematics and controlled with feedback loops. That works as long as the world is predictable. It breaks down when parts lie in a heap, when a box is crushed, when the robot must work in a home, or when someone wants to say "put the red cup on the shelf" instead of programming coordinates.

That is where AI comes in, in roughly three places:

• Perception: recognizing objects and people in camera images and point clouds, with computer vision (see also Robot Vision). This is the most mature use and is common in industry today.
• Skills: learned policies for grasping, inserting, folding or walking, trained with reinforcement learning or imitation learning instead of written by hand.
• Task understanding: models that take an instruction in words and decide which steps to take, from language models that call a robot's skills as tools (the agent loop) to vision-language-action models that output motions directly.

Underneath, the classic layers stay: motor controllers, joint limits, collision checks and, above all, the safety functions of Robot Safety and Standards. A learned policy proposes a motion; it does not get to switch off a speed limit.

Reinforcement learning and rewards

In reinforcement learning a robot (the agent) tries actions, observes what happens and receives a number that says how good the result was. The reward function is the function the designer writes that turns each step's outcome into that number — for example, progress toward a target minus a small cost for energy. The learner maximizes the total reward and nothing else, so the reward function is in effect the robot's whole specification.

That makes rewards easy to get subtly wrong. The example scores two recorded behaviors of a gripper that should move to 0.50 m. The naive reward pays for every step that gets closer and charges nothing for moving away; the better one counts progress in both directions, costs a little per step and pays a bonus for being at the target:

# Two reward functions for "move the gripper to 0.50 m", scored on
# two recorded behaviors. A learner maximizes whatever the reward says.
TARGET = 0.50

def naive_reward(before, after):
    """Reward only for getting closer; moving away costs nothing."""
    progress = abs(before - TARGET) - abs(after - TARGET)
    return max(progress, 0.0)

def better_reward(before, after):
    """Progress counts both ways, each step costs a little,
    and being at the target pays a bonus."""
    reward = abs(before - TARGET) - abs(after - TARGET) - 0.01
    if abs(after - TARGET) < 0.005:
        reward += 1.0
    return reward

behaviors = {
    "reach and hold": [0.0, 0.2, 0.4, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5],
    "wobble forever": [0.0, 0.2, 0.4, 0.3, 0.4, 0.3, 0.4, 0.3, 0.4],
}

for name, positions in behaviors.items():
    steps = list(zip(positions, positions[1:]))
    naive = sum(naive_reward(b, a) for b, a in steps)
    better = sum(better_reward(b, a) for b, a in steps)
    print(f"{name:15} naive {naive:6.2f}   better {better:6.2f}")

reach and hold  naive   0.50   better   6.42
wobble forever  naive   0.70   better   0.32

Under the naive reward, wobbling back and forth near the target scores higher than reaching it: every wobble toward the target earns a little, and the episode never ends. A learning algorithm would find that loophole. This is called reward hacking, and in robotics it shows up as robots that vibrate, slide, flip over or exploit a bug in the simulator.

Reinforcement learning needs millions of tries, far too many — and too risky — for a real robot. It is therefore done mostly in simulation, often with randomized physics and appearance so the policy does not rely on the simulator's quirks, and then transferred to the real robot. The difference between the two worlds is the sim-to-real gap (see Simulation and Digital Twins). Controllers that let legged robots walk over rough ground are often cited as a success of this approach.

Imitation learning

Writing a reward for "fold this T-shirt" is hard; showing it is easy. Imitation learning trains a robot policy to reproduce demonstrations of a task by a person, instead of discovering the behavior through rewards. Its simplest form, behavior cloning, is supervised learning: every recorded moment is an example of "in this situation, the person did this".

Teleoperation

Demonstrations are usually recorded with teleoperation: a person controls the robot remotely and in real time — with a joystick, a VR headset and hand controllers, or a smaller "leader" arm whose movements a "follower" robot copies — while the robot records its camera images and joint positions. A well-known example is the ALOHA project ("Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware", April 2023), which collected teleoperated demonstrations for two-armed tasks with low-cost hardware. Teleoperation is also how surgical robots work (see Robots at Work) and how some "autonomous" robot demos are in fact controlled.

The weakness of copying is that the robot only knows the situations it has seen. The example clones a lane-keeping behavior with a nearest-neighbor lookup over seven demonstrations:

# Behavior cloning in its simplest form: copy the action a human chose
# in the most similar recorded situation (nearest neighbor).
# State: how far the robot is from the center of its lane (meters, + = left).
# Action: steering rate the operator gave (rad/s, + = turn left).
demonstrations = [
    (-0.10, 0.50), (-0.06, 0.30), (-0.02, 0.10), (0.00, 0.00),
    (0.03, -0.15), (0.07, -0.35), (0.10, -0.50),
]

def policy(offset):
    nearest = min(demonstrations, key=lambda demo: abs(demo[0] - offset))
    return nearest[1], abs(nearest[0] - offset)

for offset in (0.04, -0.05, 0.40):
    action, gap = policy(offset)
    note = "  <- far from every demonstration" if gap > 0.05 else ""
    print(f"offset {offset:+.2f} m -> steer {action:+.2f} rad/s "
          f"(nearest demo {gap:.2f} m away){note}")

offset +0.04 m -> steer -0.15 rad/s (nearest demo 0.01 m away)
offset -0.05 m -> steer +0.30 rad/s (nearest demo 0.01 m away)
offset +0.40 m -> steer -0.50 rad/s (nearest demo 0.30 m away)  <- far from every demonstration

Close to the demonstrations the policy steers sensibly. At 0.40 m from the center — a situation the operator never showed, because a good operator never drifts that far — it simply returns the strongest steering it knows, with the same confidence. Small errors push a cloned policy into such unfamiliar states, where it makes bigger errors: the problem of compounding errors. Real systems collect more varied demonstrations, including recoveries from mistakes, and use far richer models than a lookup table.

Diffusion policy

A demonstrated task often has several right answers: go left or right around an obstacle. A simple model that averages them goes straight into the obstacle. A diffusion policy is a robot policy that generates its next sequence of actions the way a diffusion image generator creates a picture — by starting from noise and refining it step by step, conditioned on what the robot sees — so it can represent several distinct good ways of doing a task instead of averaging them. The method was published in March 2023 ("Diffusion Policy", arXiv 2303.04137) and is widely used for learning from demonstrations; the image-generation side is explained on Vision, Speech and Multimodal AI.

Vision-language-action models

A vision-language-action model (VLA) is a neural network that takes camera images and an instruction in natural language as input and produces robot actions as output, usually built by extending a vision-language model so that knowledge learned from web images and text helps the robot. The term comes from Google DeepMind's RT-2 paper of July 2023, "Vision-Language-Action Models Transfer Web Knowledge to Robotic Control". It builds on the multimodal models explained in the AI topic.

The field moves fast. A selection of milestones, with the date of first publication (as of October 2026):

Physical Intelligence followed π0 with π0.5 (April 2025), π*0.6 (November 2025) and π0.7 (April 2026), and opened a beta of its API on 13 October 2026. Google DeepMind's Gemini Robotics 1.5 (September 2025) made its embodied-reasoning model available through the Gemini API, while the VLA itself went to selected partners. Expect this list to be out of date within months.

Robot foundation models

A robot foundation model is a large model trained on data from many robots, tasks and environments, meant to be adapted to new robots and tasks with little extra data — the robotics counterpart of the foundation models behind language assistants. Most VLAs are presented as robot foundation models. The main obstacle is data: the web offers billions of images and texts, but recordings of robots acting in the world are scarce and expensive, which is why pooled datasets, teleoperation at scale and simulation matter so much.

Humanoids: demo versus deployment

No part of robotics gets more attention, or more confident claims, than humanoid robots. A useful habit is to separate three things: a video (a demo, possibly rehearsed, sped up or teleoperated), an announcement (a plan, a factory capacity, a preorder) and a deployment (robots doing paid work for a customer, at scale). The status as of October 2026, with sources:

• Market size. The International Federation of Robotics (IFR) counted about 7,000 full-size humanoids (taller than 140 cm) sold in 2025 — next to the 603,307 industrial robots installed that year. Its position paper "Humanoid Robots – Vision and Reality" (July 2025) makes the same distinction this section does.
• Boston Dynamics presented a production version of Atlas on 5 January 2026, built in Boston, and said that all Atlas deployments for 2026 were already committed — to Hyundai and to Google DeepMind.
• Figure presented Figure 03 on 9 October 2025, with a factory (BotQ) designed for up to 12,000 humanoid robots a year; it gave no shipping date.
• Unitree sells its G1 humanoid (1,320 mm tall, 23 degrees of freedom) from $13,500 (list price on Unitree's site, as of October 2026).
• Tesla says production lines for its Optimus robot are being installed (according to reports of its second-quarter 2026 results call); there is no confirmed commercial deployment.
• 1X opened preorders for its NEO home robot on 28 October 2025 at $20,000; deliveries to customers have not been confirmed, and early units are reportedly partly teleoperated.

The developers themselves are frank about the limits: with Gemini Robotics 2, Google DeepMind wrote that "multi-finger dexterous manipulation remains challenging". Hands, battery life, safety around people and cost per useful hour are the open questions.

Limits, safety and oversight

A learned policy is a statistical model, with the weaknesses described in Limits and Myths of AI: it can fail on situations unlike its training data, it gives no reason for what it does, and its success rate in a paper is not a guarantee for your warehouse. On a robot, a wrong answer is a movement, not a sentence. Practical consequences:

• Keep the safety layer separate. Speed limits, force limits, protective zones and emergency stops must not depend on the AI model. In the EU, safety components with self-evolving behavior that use machine learning to ensure safety functions always need a third-party (notified-body) conformity assessment under the Machinery Regulation (see the law).
• Plan for human oversight. Someone must be able to see what the robot is doing, understand why it stopped and intervene — and must not simply trust it because it is usually right (see human oversight).
• Test before trusting. Run a new policy in simulation, then on the real robot at reduced speed in a guarded area, and measure its success and failure rate over many tries. The same goes for code an AI assistant writes for a robot (see Risks of Vibe Coding).
• Read robot news like AI news. Ask who made the claim, whether the video was autonomous or teleoperated, at what speed it was shown, and how many attempts failed (see reading AI news).