Information Terminal For the Disabled (Engelliler İçin Bilgi Terminali)

In this project, we have conducted research on gesture and speech recognition methodologies to aid the communication hearing impaired and visually impaired individuals, with the support of the Scientific and Technological Research Council of Turkey. The main objective of this project is to design and implement an information terminal system that can translate fingerspelling to speech and vice versa, by using recognition and synthesis techniques for each modality.Through the use of multimodal input and output methods, the information terminal acts as a communication medium between deaf and blind people. The system converts fingerspelled words to speech and vice versa using fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis in Czech, Russian and Turkish Languages. Throughout this project, we have worked in the following areas:

  • Development of Multi-Lingual Fingerspelling databases for Czech, Russian and Turkish Languages
  • Research on the analysis of hand gestures from videos and the development of a prototype of a fingerspelling recognizer for Czech, Russian and Turkish Languages
  • Research on the development of a fingerspelling sytnhesis avatar through using motion capture data
  • The development of a multimodal information kiosk and the integration of fingerspelling recognition, fingerspelling synthesis, speech recognition and speech synthesis modules.

The output of the project is in the form of theses, reports, journal papers and conference papers. In addition, we have produced databases and demonstrator programs.

This work is supported by RFBR and TÜBİTAK foundations in the framework of the bilateral Russian-Turkish project (# 09-07-91220 / 108E113), and by the Grant of the President of Russia (# MK-64898.2010.8).

Some sample figures and screenshots from the software developed

Publications related to the project

Theses

  • Turan Can Gürel ...

Journal Articles

Proceedings in International Conferences

  • Oya Aran, Cem Keskin, Lale Akarun, “Sign Language Tutoring Tool”, EUSIPCO’05, Antalya, September 2005.
  • Cem Keskin, Oya Aran, Lale Akarun, “Real Time Gestural Interface For Generic Applications”, EUSIPCO’05, Antalya, September 2005.

Proceedings in Local Conferences

Database