In an interaction, cues for detecting threatening situations could be changes in the intonation or loudness indicating dangerous situations. Robots should be able to perceive and react to such signals and change their behaviour to avoid the threatening situation or, in case the threat cannot yet be identified, be alert and act more carefully. In order to achieve this goal, this project aims at identifying robust representation of an acoustic signal using neural networks. As a result a robot could have an ability to detect if there is a change in an affective state of a speaker and signal this information to the downstream modules.
My research interests are:
- Learning from a limited number of training data (weakly labelled data)
- Multi-modal sentiment and emotion recognition
- Learning to make fast and accurate predictions at the same time using reinforcement learning
The outcome of this project will be a new learning approach integrating perception of prosodic features to identify and avoid possibly threatening situations by subsequently changing the robot’s behaviour or trigger the inclusion of other sensory pathways to identify the threat.
Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2019, May)
Incorporating End-to-End Speech Recognition Models for Sentiment Analysis
To appear in Proceedings of the International Conference on Robotics and Automation 2019
Lakomkin, E., Weber C., Magg S., Wermter, S. (2018, November)
KT-Speech-Crawler: Automatic Dataset Construction for Speech Recognition from YouTube Videos
Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 90--95 - 2018
Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2018, October)
On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks
Proceedings of the International Conference on Intelligent Robots, pages 854--860 - 2018
Lakomkin, E., Zamani M., Weber C., Magg S., Wermter, S. (2018, January)
EmoRL: Real-time Acoustic Emotion Classification using Deep Reinforcement Learning
Proceedings of the International Conference on Robotics and Automation (ICRA), pages 4445--4450 - May 2018
Lakomkin, E., Weber C., Magg S., Wermter, S. (2017, November)
Reusing neural speech representations for auditory emotion recognition.
Proceedings of the Eighth International Joint Conference on Natural Language Processing, Volume 1, pages 423--430 - Nov 2017
Lakomkin, E., Bothe, C., and Wermter, S. (2017, September).
GradAscent at EmoInt-2017: Character- and Word-Level Recurrent Neural Network Models for Tweet Emotion Intensity Detection.
Proceedings of the 8th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis at EMNLP-2017, pages 169--174 - Sep 2017
Lakomkin, E., Weber, C., Wermter, S. (2017, April).
Automatically augmenting an emotion dataset improves classification using audio
15th Conference of the European Chapter of the Association for Computational Linguistics (EACL). Valencia, Spain.
Short Curriculum Vitae
- Since February 2016: Research Associate (PhD Student: SECURE Project) at Knowledge Technology Research Group, Department of Computer Science, University of Hamburg, Germany
- M.Sc. in Computer Science, Moscow State Technical University n.a. Bauman , Moscow, Russia. Faculty – Informatics and control systems, department - Automatic Information Processing and Control Systems, diploma in computer engineering, class of 2011, GPA 4,4 (of 5)
- Nanyang Technological University, researcher and developer, Summer Research Internship, School of Computer Engineering
- Nanyang Technological University, Research associate, Computer Linguistics and Bioinformatics
Research Associate SECURE Project
Knowledge Technology Group (WTM)
Department of Informatics
University of Hamburg
22527 Hamburg, Germany
Phone: +49 40 42883 2318
Fax: +49 40 42883 2515
lakomkin at informatik.uni-hamburg.de