From a blank screen to finished code in minutes: How to use AI to program robots

Please note: English subtitles available in the video player

With an AI chatbot, most people can get started programming robots in just a few minutes. Here, the Danish Technological Institute (DTI) demonstrates how generative AI makes robot programming fast and flexible for both beginners and experts.

Imagine being able to describe a task to a robot in everyday language, and just a few minutes later watch the robot carry out the task without having written a single line of code yourself.

It may sound like science fiction, but this is exactly the reality that specialists at DTI are now demonstrating through the EU initiative AI-MATTERS.

By feeding an AI chatbot the right information, companies can reduce the time from idea to a finished robot application.

In this article, Malaika Din Hashmi, a Consultant at DTI, will guide you on how to use AI to help you program a robot.

How AI learns to "speak robot"

Let's say the task is for a robot to stack three blocks and then put them back again where it found them.

You need to get it to do this using a chatbot, entirely without writing a single line of code yourself.

However, it's not entirely without its challenges.

The large language models behind chatbots are not pre-trained in the many different "robot languages", and each robot has its own particular syntax that the AI does not necessarily know.

The solution is to provide the chatbot with the necessary knowledge so that it knows what to do and how.

Malaika Din Hashmi explains that you can prepare the chatbot for the task by following three steps, uploading specific documents:

1. A task description

A file that explains in plain language what the robot needs to do. For example, stacking three blocks, including their positions.

2. A script manual

Technical documentation of the robot's built-in functions and commands, so the AI knows what "tools" it has available.

3. A best practice file

A collection of expert knowledge and know-how.

− The last document requires a bit more than the other two, which you can usually describe yourself or find on the robot supplier's website. The best practice file tells you how to programme a robot, what you typically need to do. For instance, you usually move above an object before moving down and gripping it, instead of moving directly into it, so you avoid collisions, explains Malaika Din Hashmi.

Once the chatbot has received this information, the user writes a simple command − a prompt − and the AI generates the complete robot code.

From hours to minutes

The efficiency of this method is not just theoretical. DTI has conducted benchmarks comparing results with and without the extra information files.

In a test where a robot had to stack blocks, the results showed a marked improvement when the AI model (GPT-4) had access to both the script manual and best practice knowledge.

Benchmark: Results for the GPT-4 model

Table: Excerpt from DTI's benchmark shows that the success rate rises from 75 to 100 percent, while the number of required adjustments falls from over four to less than one, and the time consumption is more than halved.

The figures show that a correctly instructed AI not only solves the task faster, but also gets it right on the first attempt. This is crucial for making the technology practically useful in the busy world of production.

An assistant for both new and experienced users

This method opens doors for two main user groups. First, those without deep knowledge of robot programming.

− So instead of having to start from scratch every time, they have a chatbot to help them − like an assistant that helps create the programmes for them, says Malaika Din Hashmi.

Secondly, it's a gain for experienced automation houses and companies with frequent changeover needs.

When a product or process needs to be changed, you simply describe the change to the chatbot.

One thing to be aware of is that this is generative AI, so it won’t necessarily give the exact same answer every time. That’s why it’s important to test it before deploying it to the physical robot.

From prompt to production requires a trial run

But can you trust code generated by an AI?

According to Malaika Din Hashmi, it is crucial to always think about safety and quality assurance.

− One thing to be aware of is that this is generative AI, so it won't necessarily give the exact same answer every time, she says, and continues:

− That's why it's important to test it before deploying it to the physical robot.

Malaika Din Hashmi recommends always testing the code in a simulation environment first. In this way, you can ensure the robot behaves as expected, and you avoid unforeseen situations in production.

Only once the programme has been approved in simulation should it be implemented in the real world.

Would you like to know more or try it out in practice?

DTI is happy to help companies test, adapt, or further develop AI solutions for robot programming. Whether you have concrete challenges, are simply curious about the technology, or want advice on possibilities and limitations, you are welcome to get in touch.

Maybe you've already started, or maybe you're considering whether AI can help in your production. We share knowledge, give presentations, and are happy to explore new cases together with Danish companies. Do reach out if you'd like to move forward with robotics and AI in practice.