Research Intern, Multimodal Perception for Human-Robot Interaction

Research Context

This internship is embedded within an active PhD research project aimed at building a robust multimodal perception module for Human-Robot Interaction (HRI).

In HRI, it is critical for a social robot to understand not just who is in front of it, but how they are engaging. This research focuses on estimating high-level user states (such as attention, engagement, or confusion) by processing real-time, non-verbal cues like gaze direction or head pose.

Internship Topic

The exact topic is not set in stone; we will define it together.

We are looking for a student to take ownership of a specific part of the perception pipeline. Depending on your interests and strengths, your internship could focus on one or a combination of the following:

• Data Annotation & Ground Truth Creation: You could take the lead on analyzing video data of human-robot interactions and annotating high-level user states (e.g., labeling when a user is actively paying attention vs. distracted).

• Model Training & Evaluation: You could use the annotated data to train, fine-tune, and evaluate machine learning models (e.g., classifiers or deep learning architectures) to predict user states from low-level cues. Your model would directly feed into the broader user state estimation module.

• Data Collection: You could design and execute new experiments to capture naturalistic human-robot interactions, expanding the diversity of our dataset for gaze and head pose variations.

Regardless of the specific direction, the internship will involve close collaboration with the PhD supervisor to ensure the work contributes directly to an upcoming research paper.

Skill and Requirements

• Education: Master's student in Computer Science, Machine Learning, Robotics, or related fields.
• Technical Skills:
• Strong proficiency in Python.
• Ideally, experience with Annotation Software (e.g., CVAT).
• Basic understanding of Machine Learning and Computer Vision concepts (prior exposure to gaze tracking, head pose estimation, or facial landmark detection is a strong plus).

Please note that our internships are not fully paid. We invest in you by providing mentorship, learning opportunities, and hands-on experience in pushing the boundaries of what’s possible. However, we do offer some support for living expenses, relocation, etc., based on individual circumstances.

Important: We can only accept candidates who are eligible to work in Sweden with an active residency.