FRIDA, the Robot Taking the Art Scene by Storm

The FRIDA robot, developed by Carnegie Mellon University, uses AI and machine learning to create paintings in response to user input. It is capable of generating new images based on audio, text, or image inputs, and its output is expressionistic and rough, with human-like imperfections. The robot uses machine learning models to match image and text, and to understand cues from the user. The team developing the robot added news headlines to the input while training the model to avoid biases.

The FRIDA robot (Frameworks and Robotics Initiative for Developing Arts), named after Mexican artist Frida Kahlo, is an initiative developed by The Robotics Institute of Carnegie Mellon University that is making waves in the art world. It’s a cutting-edge technology that is designed to produce intricate works of art with its arm using brush strokes, changing how people view the dynamics between art and artificial intelligence.

The machine is capable of creating precise paintings, using a style that is similar to other impressionistic works of art. The innovative painting robot was developed by the Robotics Institute faculty members Jean Oh and Jim McCann. It’s able to recreate famous masterpiece paintings but also creates whole new pieces of its own. The robot taps into machine learning and AI to generate different images. Not only does FRIDA generate paintings with pictures or written text as input, the system is also able to take instructions stemming from music or spoken language and create a whole representation. The team has already experimented with audio inputs, such as playing ABBA’s ‘’Dancing Queen’’ and asking FRIDA to paint a visual representation of the song.

So how does it work? FRIDA works by combining engineering and creativity. The robot uses AI models similar to OpenAI’s newest ChatGPT, using instructions to respond with generating a text or image. The user can submit any text or image and ask FRIDA to paint it. After that, the system simulates an image and starts painting it with brush strokes. One of the developers, McCann stated that ‘’FRIDA is a project exploring the intersection of human and robotic creativity.” He continued: “FRIDA is using the kind of AI models that have been developed to do things like caption images and understand scene content and applying it to this artistic generative problem.” The robot uses AI and machine learning in several phases of the process. Before starting the actual painting, FRIDA spends at least an hour learning how to use the paintbrush. After that, it uses vision-language models, such as OpenAI’s Contrastive Language-Image Pre-training (CLIP) trained to match image and text to understand the cues that the user has given.

But in contrast to the other known vision-language models, FRIDA takes it to the next level by using its robotic arm to actually physically create paintings instead of digital images only. Another difference with other robotic models is that FRIDA’s output is expressionistic, whimsical, almost rough. It lacks the precision that other robotic models so often seek. But for FRIDA this is what makes the paintings so realistic. If FRIDA makes a mistake while painting, the robotic arm uses this error in the end result, creating a human-like touch to the painting. The team developing the robot also added headlines of news articles to the input while training the model, to give FRIDA some sense of what’s currently happening in the world. This way, the developers tried to avoid biases that have caused other large vision-language models a lot of controversy

Have we sparked your interest to learn more about all the possibilities in the world of AI? Contact us via our Baise website and let us know what you're looking for. We offer our personal advice and expertise on AI for your company. Our aim is to connect brains with business by providing you with talented and ambitious students looking to gain real experience in the field.

Published on
June 14, 2023
Philip Gast

Unlock to full potential of AI.

Generative AI

Discover how AI can transform your business! Our workshops unveil powerful tools for innovation and productivity. Start today

Contact us