MAMEM – Multimedia Authoring and Management using your Eyes and Mind

Loss of voluntary muscular control while preserving cognitive functions is a common symptom of neuromuscular diseases, leading to a variety of functional deficits including the ability to operate software tools that require the use of such conventional interfaces as a mouse, keyboards or touch-screens. As a result, the affected individuals are marginalized and unable to keep up with the rest of society in a digitized world. MAMEM’s goal is to integrate people back into society by increasing their potential for communication and exchange in and out of the workplace.

In this direction, MAMEM delivers the technology to enable interface channels that can be controlled through eye movements and mental commands. This is accomplished by extending the core API (application programming interface) of current operating systems with advanced function calls, appropriate for accessing the signals captured by an eye-tracker, an EEG recorder and bio-measurement sensors (e.g. heart rate, galvanic skin response). Then, pattern recognition and tracking algorithms are employed to jointly translate these signals into meaningful control and enable a set of novel paradigms for multimodal interaction (move a mouse, tick a box) and high-level control of interface applications through eyes and mind.

MAMEM will engage three different cohorts of disabilities (i.e. Parkinson’s disease, neuromuscular disease and tetraplegia) that will be asked to test a set of prototype applications dealing with multimedia authoring and management. MAMEM’s overarching goal is to integrate people with disabilities back into society by endowing them with the critical skill of managing and authoring multimedia content using novel and more natural interface channels. These channels will be controlled by eye movements and mental commands, significantly increasing the potential for communication and exchange in casual settings (e.g. social networks) and in a more professional context (e.g. workplace).

MAMEM’s final objective is to assess the impact of this technology in achieving social integration by, for instance, becoming more active in sharing content through social networks and communicating with friends and family.

OBJECTIVES
• Capture, record and make real-time and accurate information about eye movements, brain electric signals and bio-measurements available on the necessary scale
• Develop the necessary algorithms for translating this information into meaningful control that will take the form of semantic widgets
• Implement a middleware sitting on top of current operating systems so as to make these semantic widgets available as elementary building blocks for implementing multimedia-related interfaces
• Model users based on their (dis-)abilities, interaction behaviour, emotions and intentions so as to follow a set of persuasive design principles and develop interfaces that will effectively stimulate users to utilize them and encourage their behavioural change
• Design, implement and evaluate a set of prototype interface applications that rely on MAMEM’s middleware to execute the multimedia-related usage scenarios through the user’s eyes and mind
• Enable the use of prototype interface applications in non-controlled environments, such as the user’s home environment
• Assess the degree of success in bringing disabled people back into society as a result of their newly acquired ability to manage and author multimedia content

JOINT VENTURE
1. National Centre for Research and Technology – Information Technologies Institute (CERTH–ITIPE) – www.iti.gr
2. University of Koblenz–Landau (UKob), WeST Institute – www.west.uni-koblenz.de
3. SensoMotoric Instruments GmbH (SMI) – www.smivision.com
4. The Medical Research Infrastructure Development & Health Services Fund by the Sheba Medical Center (SMC – SHEBA) – www.sheba.co.il
5. EB Neuro S.p.A. (EBN) – www.ebneuro.biz
6. C Department of Neurology Clinic – Aristotle University of Thessaloniki (AUTh) – www.med.auth.gr
7. Eindoven University of Technology (TUe) – www.tue.nl
8. MDA Hellas – www.mdahellas.gr