Simple Interactive Bot (Inspired by Social Robots)

Social robots/Assistive robots/Personal companions are getting quite popular nowadays. As a robotics enthusiast, I have been following various products on the market and prior research that involve making robots behave like friends who can express emotions as well as communicate their emotions via gestures and speech. Social robots such as Jibo, Buddy, Kismet, Cozmo and Pepper that intrigue me usually have cameras for perception, actuators for movement, speakers and microphones for communication, and LCD display to show interesting things (e.g. emotions, facial expression, visualization of user requested information). I really admire the people who equip these robots with high intelligence and social skills and would love to see how they implement the software platform and algorithms. However, most source code of these robots are not open-source so over winter break, I decided to do a simple project using existing open-source technologies to build an interactive bot with basic intelligence and social skills.

I know the advanced robots have numerous sensors such as inbuilt cameras, tactile sensors, actuators to perceive the world through complex data analysis and algorithms. My simplified project focuses on speech recognition and basic AI to communicate with human user via text and speech. In order to make the bot’s responses to be like human’s, I followed several online tutorials and used the Artificial Intelligence Markup Language (AIML). This XML-based language stores knowledge inside XML tags.

Basic tags inside AIML files. Fun fact: my friends sometimes call me “Megatran” which is a wordplay of my last name and my favorite robot franchise Transformers.

Based on this basic AIML knowledge file, when a user asks the robot “What is your name?”, the robot replies, “My name is Megatran!” I first tinkered around with AIML knowledge files using the PyAIML interpreter in Python. It uses backtracking DFS algorithm to search for the response and give the proper reply.

If you’re using Linux, you can quickly install the package by running:

sudo apt-get install python-aiml

After the installation, you can run this quick test:

import aiml
robot = aiml.Kernel()
robot.setBotPredice("name", "Megatran")
robot.learn("sample.aiml") # load an aiml file by following the basic syntax above or download any online.
print(robot.respond("What is your name?"))

After playing with the AIML files, I created an interactive AIML based bot using the Robot Operating System (ROS). Below is the block diagram of the interactive bot.

The system can accept user input as text or converts the user’s speech into text using the speech recognition in ROS. The input is then passed to the AIML engine to yield intelligent reply from the database. The output of the AIML interpreter is then converted to speech using the text-to-speech module. The yellow block in the diagram shows the component that I plan on implementing in the future, which is an animated display of the bot’s virtual face that syncs with the speech.

To install all necessary components and packages, please run these linux commands:

list=$(apt-cache --names-only search ^gstreamer1.0-* | awk '{ print $1 }' | grep -v gstreamer1.0-hybris)
sudo apt-get install $list
sudo apt-get install festival

git clone

cd to the ros workspace and run catkin_make to compile the packages.

After installing all dependencies, I created a new ros package called “ros_simple_aimlbot”

catkin_create_pkg ros_simple_aimlbot rospy std_msgs sound_play


The ros package has 4 main nodes: client_aiml, server_aiml, aiml_tts, and speech_recognition_client. As shown in the visualization of ROS’ rqt_graph plugin, the client_aiml waits for user input and when a user enters/says something, it will publish the text to the /chatter (std_msgs/String) topic. Secondly, server_aiml node loads AIML files from the databases I found online and saves them into the brain files. Additionally, it subscribes to /chatter topic to feed the user input into the AIML interpreter then publishes the interpreter’s response through the /response (std_msgs/String) topic. Third, the text_to_speech client (tts) subscribes to the /response topic and convert the response to speech. Lastly, the speech_recognition_client publishes text to /chatter topic by listening to speech recognition node.

Visualization of ROS nodes and topics of the interactive bot


Snippet of the client code portion in Rospy

After implementing the ros nodes, I created the ros launch files to launch these nodes and start playing with the interactive bot! Before running the launch files, all python and launch files must have executable permission (chmod +x) or else there will be weird errors. Big thanks to the ROS community and Lentin Joseph for all the helpful knowledge and support! Below is a sample screenshot of the interactive bot’s responses.

Testing the launch file and asking the bot some questions

Moreover, in December 2017, I was among a few developers invited to participate in Misty Robotics‘s first ever Robothon (similar to hackathon) and behind-the-scenes peek of their upcoming social robot which will likely launch in 2018! During the event at this Boulder, Colorado based startup, I got to “hack” the robots with with the engineers and hear the missions from the leadership team of Misty as well as the co-founder Ian Bernstein (who also started Sphero)!

Sphero co-founder Ian Bernstein starts the Misty Robotics’s Hackathon aka Robothon

In conclusion, I learned a lot about knowledge representation, social intelligence, and ultimately the application of AI in robotics. I enjoyed getting hands-on with this ROS+AIML project and attempting to to make the interactive robot answer to human queries similar to human-human interaction. I look forward to getting one social robot and tinkering with it! I recently applied to a couple graduate programs that might have some research opportunities in social robots and interactive systems; being able to work along with the experts in the field would be a dream comes true. If you might have suggestion or feedback regarding the project or with what robot I should tinker, feel free to contact me!



Posted on

January 3, 2018