AI Robotics Coding Teaching Robots to Map Large Environments: A BreakthroughNortheastern Team Creates New AI tool that can Detect Melanoma with 99% AccuracyNortheastern Team Creates New AI tool that can Detect Melanoma with 99% AccuracyCMU Team Develops Sotopia, an AI Tool Designed to Evaluate the Social Intelligence of AI AgentsPhysical AI: CMU Leads the Next Wave of Intelligent MachinesThe Future of Smart RobotsBuilding Biological Robots from Human Cells.The Hunger GamesBrave New WorldUCLA Scientists Build Light-Powered Robot That Flaps Like a Bee1984CMU Trains Navy Robotics Specialists for the Age of AIHarvard Researchers Advanced Quantum InternetMIT’s New Bionic Knee Feels Like Part of the Body, Not Just a ToolCaltech Researchers Create a New, Noninvasive Way to Measure Blood PressureAI Takes Flight: Boston University’s AIR Initiative Leads Cross-Disciplinary InnovationResearchers at Carnegie Mellon University are Developing a New Cancer Implant TechAutonomous Insect Robotics LaboratoryNortheastern University Research Could Bring Virtual Reality Closer to RealityCaltech Creating AI Program to Solve Decades Old Math ProblemsAI/Biomedicine lab at the University of WashingtonFuture of Data Science EducationCMU – AI Driven RobotsFahrenheit 451The Assistive Robotics and ManipulationAI Predicting Health OutcomesHow Robots Are Helping PeopleGTGraffiti A Robot That Paints Like A HumanThe MartianFour-Legged Robotic System From MITAI Making Streets SaferHUBO RobotsAI Simulation To See Your Future SelfRobot Muscles At MIT

AI Robotics Coding Teaching Robots to Map Large Environments: A BreakthroughNortheastern Team Creates New AI tool that can Detect Melanoma with 99% AccuracyNortheastern Team Creates New AI tool that can Detect Melanoma with 99% AccuracyCMU Team Develops Sotopia, an AI Tool Designed to Evaluate the Social Intelligence of AI AgentsPhysical AI: CMU Leads the Next Wave of Intelligent MachinesThe Future of Smart RobotsBuilding Biological Robots from Human Cells.The Hunger GamesBrave New WorldUCLA Scientists Build Light-Powered Robot That Flaps Like a Bee1984CMU Trains Navy Robotics Specialists for the Age of AIHarvard Researchers Advanced Quantum InternetMIT’s New Bionic Knee Feels Like Part of the Body, Not Just a ToolCaltech Researchers Create a New, Noninvasive Way to Measure Blood PressureAI Takes Flight: Boston University’s AIR Initiative Leads Cross-Disciplinary InnovationResearchers at Carnegie Mellon University are Developing a New Cancer Implant TechAutonomous Insect Robotics LaboratoryNortheastern University Research Could Bring Virtual Reality Closer to RealityCaltech Creating AI Program to Solve Decades Old Math ProblemsAI/Biomedicine lab at the University of WashingtonFuture of Data Science EducationCMU – AI Driven RobotsFahrenheit 451The Assistive Robotics and ManipulationAI Predicting Health OutcomesHow Robots Are Helping PeopleGTGraffiti A Robot That Paints Like A HumanThe MartianFour-Legged Robotic System From MITAI Making Streets SaferHUBO RobotsAI Simulation To See Your Future SelfRobot Muscles At MIT

CMU Team Develops Sotopia, an AI Tool Designed to Evaluate the Social Intelligence of AI Agents

November 7, 2025

Introduction

Sotopia is an AI tool whose main purpose is to conclude the intelligence of other AI tools and agents. It was created by a team at CMU led by Xuhui Zhou, a PhD student in the Language Technologies Institute and Hao Zhu, a former PhD student. Some other contributors to the project are Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis‑Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, and Maarten Sap.

What is this AI tool?

The AI tool evaluates the social intelligence of other AI agents. The team believes that AI tools can do tremendous things but they are likely socially unaware or even biased and display harmful behaviors. Sotopia is an agent which is supposed to connect human and AI interaction stronger than before and situations in Sotopia typically public and secret information to determine how the AI agent will react.

How does this AI tool work?

Sotopia decides the AI agent’s social intelligence by simulating complex social interactions between the large language model’s artificial agents. The artificial agents and characters in Sotopia have names, a gender, a personality, job, public information and secret information. This allows for a lot of different scenarios and many different outcomes while testing AI tools and how they react.

CMU’s Impact

The development of Sotopia at CMU has made a huge impact because it highlights the fact that even extremely advanced models of today do worse at social interactions than humans. Sotopia also evaluates other variables like believability, relationship maintenance, sharing information and keeping secrets. In summary, the work of the team at CMU in developing Sotopia has made a major impact because of the ability to evaluate not only effectiveness of AI agents but also social variables.

By:
Aarush