This is an old revision of the document!


Gesture Based Human Computer Interaction in Emergency Management Systems (GEMS)

Emergency Management relies on timely and sufficient information. During an emergency, the efficient display of large amounts of information in a central command room is a vital requirement The purpose of this project was to develop human-computer interaction methods for such environments. We have concentrated on three subjects in detail: * Face analysis, * Body analysis, * Hand gesture analysis.

For face analysis, we have worked on both face recognition and facial expression analysis. For body gesture analysis, one needs to track and analyse body movements. For hand gesture recognition, we have used color and depth sensors to track human bodies in general environments. Hand gestures consist of two main components: the hand shape and its trajectory. Therefore, the signal is inherently a multimodal time series signal and techniques used to represent time series are suited for this application. We have worked on shape classification and time series analysis for gesture classification. We have used probabilistic methods to a fit a 3D skeletal model of the body and hand. We have generated systems to recognize gestures in real time.

As a kick-off meeting of GEMS, a 2-day workshop was organized in Istanbul, Turkey on 26-27 March 2009. The workshop involved experts from disaster management centers, who were invited to give talks and participate in scenario building. Several centers were visited and example interaction scenarios were solidified.

Some sample figures and screenshots from the software developed

Publications related to the project

Theses

  • Yunus Emre Kara, Computer Vision-Based Human Action Recognition Via Keypoint Tracking, Boğaziçi Üniversitesi.

Proceedings in International Conferences

  • C. Keskin, İ. Arı, T. Eren, F. Kıraç, L. Rybok, H. Ekenel, R. Stiefelhagen, ve L. Akarun, “Vision Based Hand Puppet,” Proceedings of eNTERFACE 2010, The Summer Workshop on Multimodal Interfaces, 2010, sf. 10-17
  • Cem Keskin, Furkan Kıraç, Yunus Emre Kara, Lale Akarun, “Real Time Hand Pose Estimation using Depth Sensors”, IEEE ICCV 2011. 1)
  • B. E. Demiröz, İ. Arı, O. Eroğlu, A. A. Salah, and L. Akarun, “Feature based Tracking on a Multi-Omnidirectional Camera Dataset, International Symposium on Communications, Control and Signal Processing (ISCCSP), Rome, 2012.
  • Neşe Alyüz, Berk Gökberk, Luuk Spreeuwers, Raymond Veldhuis, Lale Akarun, “Robust 3D Face Recognition in the Presence of Realistic Occlusions”, International Conference on Biometrics (ICB), New Delhi, Hindistan, Nisan 2012.
  • Keskin C., Cemgil A. T., and Akarun, L. “DTW Based Clustering to Improve Hand Gesture Recognition”, HBU 2011

Proceedings in Local Conferences

  • İ. Arı, H. Gao, H. K. Ekenel ve L. Akarun, " Yüz nirengi noktalarının zamansal öz-benzerliğine ve kelime çantasına dayalı yüz ifadesi ve kafa hareketi tanıma”. IEEE Sinyal İşleme ve İletişim Uygulamaları Konferansı, Diyarbakır, 2010.
  • Neşe Alyüz, Berk Gökberk, Luuk Spreeuwers, Raymond Veldhuis, Lale Akarun, “Geri Çatma ve Bölgesel Sınıflandırıcılar ile Örtmeye Dayanıklı 3B Yüz Tanıma”, 19. Sinyal İşleme ve İletişim Uygulamaları Kurultayı, SIU 2011, Antalya, Nisan 2011.
  • Yunus Emre Kara, Lale Akarun (2011). Human action recognition in videos using keypoint tracking. Signal Processing and Communications Applications (SIU), 2011 IEEE 19th Conference on, 1129-1132.
  • Furkan Kıraç, Lale Akarun, “Real-time Pose Tracking Based on a 3D Skeletal Model Using Multiple Cameras”, Signal Processing and Communications Applications (SIU), 2011 IEEE 19th Conference on.
  • İ. Arı, F. O. Alsaran, ve L. Akarun, “Görü-tabanlı Gerçek-zamanlı Duygu Tanıma,” IEEE 19. Sinyal İşleme ve Uygulamaları Konferansı, Antalya, 2011.
  • İ. Arı, Y. Açıköz, “Pinotator ile Hızlı İmge İşaretleme,” IEEE 19. Sinyal İşleme ve Uygulamaları Konferansı, Antalya, 2011
  • Keskin, C.; Cemgil, A.T.; Akarun, L.; , “Explicit duration models for isolated hand gesture recognition,” Signal Processing and Communications Applications (SIU), 2011 IEEE 19th Conference on , vol., no., pp.1169-1172, 20-22 April 2011
  • İ. Arı, U. Şimşekli, A. T. Cemgil, and L. Akarun, “SVD-based Polyphonic Music Transcription,” IEEE 20th Signal Processing and Communications Applications Conference, Fethiye, 2012.
  • Cem Keskin, Furkan Kıraç, Yunus Emre Kara, Lale Akarun, " 3D Hand Pose Estimation and Classification using Depth Sensors,” IEEE 20th Signal Processing and Communications Applications Conference, Fethiye, 2012 2).
  • Furkan Kıraç, Yunus Emre Kara, Cem Keskin, Lale Akarun, “Depth Image based 3D Hand Pose Estimation Framework,” IEEE 20th Signal Processing and Communications Applications Conference, Fethiye, 2012.
1) An extended version of this paper is invited to be published as a book chapter in the book titled “Consumer Depth Cameras for Computer Vision: Research Topics and Applications” by Springer.
2) Nominated for Best Paper Award