Lakomkin Egor

Prosodic feature recognition for threat detection

Principal Supervisor:

Prof. Dr. Stefan Wermter
Universität Hamburg

Collaboration partners:

  • Fondazione Istituto Italiano Di Tecnologia
  • Honda Research Institute Europe GmbH

Competence Area: Situation


In an interaction, cues for detecting threatening situations could be changes in the intonation or loudness indicating dangerous situations. Robots should be able to perceive and react to such signals and change their behaviour to avoid the threatening situation or, in case the threat cannot yet be identified, be alert and act more carefully. In order to achieve this goal, this project aims at identifying robust representation of an acoustic signal using neural networks. As a result a robot could have an ability to detect if there is a change in an affective state of a speaker and signal this information to the downstream modules.

My research interests are:

  • Learning from a limited number of training data (weakly labelled data)
  • Multi-modal sentiment and emotion recognition
  • Learning to make fast and accurate predictions at the same time using reinforcement learning

Expected Results

The outcome of this project will be a new learning approach integrating perception of prosodic features to identify and avoid possibly threatening situations by subsequently changing the robot’s behaviour or trigger the inclusion of other sensory pathways to identify the threat.


Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2019, May)

Incorporating End-to-End Speech Recognition Models for Sentiment Analysis

To appear in Proceedings of the International Conference on Robotics and Automation 2019


Lakomkin, E., Weber C., Magg S., Wermter, S. (2018, November)

KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos

Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 90--95 - 2018


Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2018, October)

On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks 

Proceedings of the International Conference on Intelligent Robots, pages 854--860 - 2018


Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2018, January)

EmoRL: Real-time Acoustic Emotion Classification using Deep Reinforcement Learning 

Proceedings of the International Conference on Robotics and Automation (ICRA), pages 4445--4450 - May 2018


Lakomkin, E., Weber C., Magg S., Wermter, S. (2017, November)

Reusing neural speech representations for auditory emotion recognition.
Proceedings of the Eighth International Joint Conference on Natural Language Processing, Volume 1, pages 423--430 - Nov 2017

Lakomkin, E., Bothe, C., and Wermter, S. (2017, September). 
GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection
Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis at EMNLP-2017, pages 169--174 - Sep 2017

Lakomkin, E., Weber, C., Wermter, S. (2017, April). 
Automatically augmenting an emotion dataset improves classification using audio 
15th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Valencia, Spain.

Short Curriculum Vitae

  • Since February 2016: Research Associate (PhD Student: SECURE Project) at Knowledge Technology Research Group, Department of Computer Science, University of Hamburg, Germany
  • M.Sc. in Computer Science, Moscow State Technical University n.a. Bauman , Moscow, Russia. Faculty – Informatics and control systems, department - Automatic Information Processing and Control Systems, diploma in computer engineering, class of 2011, GPA 4,4 (of 5)
  • Nanyang Technological University, researcher and developer, Summer Research Internship, School of Computer Engineering
  • Nanyang Technological University, Research associate, Computer Linguistics and Bioinformatics




Lakomkin Egor
Research Associate SECURE Project
Knowledge Technology Group (WTM)
Department of Informatics
University of Hamburg
Vogt-Koelln-Str. 30
22527 Hamburg, Germany
Phone: +49 40 42883 2318
Fax: +49 40 42883 2515
lakomkin at