Lakomkin Egor

Prosodic feature recognition for threat detection

Principal Supervisor:

Prof. Dr. Stefan Wermter
Universität Hamburg

Collaboration partners:

  • Fondazione Istituto Italiano Di Tecnologia
  • Honda Research Institute Europe GmbH

Competence Area: Situation


Objectives

In an interaction, cues for detecting threatening situations could be changes in the intonation or loudness indicating dangerous situations. Robots should be able to perceive and react to such signals and change their behaviour to avoid the threatening situation or, in case the threat cannot yet be identified, be alert and act more carefully. In order to achieve this goal, this project aims at identifying robust representation of an acoustic signal using neural networks. As a result a robot could have an ability to detect if there is a change in an affective state of a speaker and signal this information to the downstream modules.

My research interests are:

  • Learning from a limited number of training data (weakly labelled data)
  • Multi-modal sentiment and emotion recognition
  • Learning to make fast and accurate predictions at the same time using reinforcement learning

Expected Results

The outcome of this project will be a new learning approach integrating perception of prosodic features to identify and avoid possibly threatening situations by subsequently changing the robot’s behaviour or trigger the inclusion of other sensory pathways to identify the threat.

Publications

 

Lakomkin, E., Weber C., Magg S., Wermter, S. (2017, November)
Reusing neural speech representations for auditory emotion recognition.
To be appear in IJCNLP-2017 (oral presentation).


Lakomkin, E., Bothe, C., and Wermter, S. (2017, September). 
GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection
To be appear in Workshop (WASSA) at EMNLP-2017.


Lakomkin, E., Weber, C., Wermter, S. (2017, April). 
Automatically augmenting an emotion dataset improves classification using audio.
15th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Valencia, Spain.

Short Curriculum Vitae

  • Since February 2016: Research Associate (PhD Student: SECURE Project) at Knowledge Technology Research Group, Department of Computer Science, University of Hamburg, Germany
  • M.Sc. in Computer Science, Moscow State Technical University n.a. Bauman , Moscow, Russia. Faculty – Informatics and control systems, department - Automatic Information Processing and Control Systems, diploma in computer engineering, class of 2011, GPA 4,4 (of 5)
  • Nanyang Technological University, researcher and developer, Summer Research Internship, School of Computer Engineering
  • Nanyang Technological University, Research associate, Computer Linguistics and Bioinformatics


Awards

Contact

Lakomkin Egor
Research Associate SECURE Project
Knowledge Technology Group (WTM)
Department of Informatics
University of Hamburg
Vogt-Koelln-Str. 30
22527 Hamburg, Germany
Phone: +49 40 42883 2318
Fax: +49 40 42883 2515
lakomkin at informatik.uni-hamburg.de