The Synergia Project

An interdisciplinary initiative connecting intelligent medical systems with human multisensory perception.

Why Synergia

A demanding information environment

Modern operating rooms generate information from imaging, tracking, monitoring, and AI-based analysis. Clinicians must interpret these streams under time pressure, with limited visual bandwidth and high cognitive demands.

Synergia studies how sound, spatial audio, and other computationally mediated signals can complement visual guidance in realistic ophthalmic and microsurgical scenarios.

Project Vision

Connecting computation and perception

The project aims to establish scientifically grounded ways for intelligent medical systems to communicate with clinicians through perceptually meaningful audiovisual information, from controlled laboratory studies to realistic clinical validation.

Scientific Objectives

Six interconnected objectives define the project’s scientific framework, guiding the development, evaluation, and clinical translation of multisensory interaction for computer-assisted medicine.

Objective 1

Establish Experimental Platforms for Multisensory Research

Establish realistic experimental environments that enable the systematic investigation of multisensory perception and AI-assisted guidance in complex surgical tasks.

Objective 2

Characterize Operating Room Soundscapes

Model and analyze realistic auditory operating-room environments to establish ecologically valid foundations for multisensory interaction research.

Objective 3

Advance Intelligent Sonification

Investigate perceptually meaningful sonification strategies that communicate clinically relevant information beyond conventional visual interfaces.

Objective 4

Understand Multisensory Human Performance

Investigate how audiovisual information influences perception, spatial navigation, cognitive load, and human performance in high-intensity environments.

Objective 5

Understand Expertise and Learning

Determine how expertise and training shape multisensory perception, interaction, and performance during complex surgical tasks.

Objective 6

Validate Clinical Translation

Evaluate optimized multisensory guidance in realistic ophthalmic surgical settings to establish its effectiveness and translational potential.

Project Workflow

The experimental programme advances from passive perception studies to active navigation and, finally, realistic ophthalmic validation.

Synergia project workflow showing three experimental stages: passive laboratory perception studies, active XR-based surgical navigation studies, and clinical pig-eye ophthalmic validation, together with common surgical tasks and evaluation dimensions.
Experimental progression from controlled perception studies through active navigation to realistic ophthalmic validation.

Work Package Flow

WP1 establishes the experimental foundation. WP2 and WP3 develop auditory environments and sonification methods; WP4 and WP5 evaluate human performance, expertise, and learning; WP6 brings the optimized approach into clinical validation.

  1. WP1

    Experimental Platforms

    Foundation
  2. WP2

    OR Soundscapes

    Technology development
  3. WP3

    Intelligent Sonification

    Technology development
  4. WP4

    Human Evaluation

    Human evaluation
  5. WP5

    Expertise & Training

    Human evaluation
  6. WP6

    Clinical Validation

    Clinical validation

Project Roadmap

A three-year progression from experimental platforms and models to human studies, clinical validation, and reusable research outputs.

Year 1

Build and model

  • Experimental Platforms
  • OR Soundscapes
  • Sonification Framework

Year 2

Study and optimize

  • Perception Studies
  • Navigation Studies
  • Optimization

Year 3

Translate and release

  • Expertise
  • Training
  • Clinical Validation
  • Open-source Software

Expected Impact

Synergia aims to advance understanding of multisensory human performance while developing methods for more interpretable and human-centred computer-assisted interventions.

Funding and Institutional Partnership

Synergia is funded by the Deutsche Forschungsgemeinschaft and conducted jointly by the Technical University of Munich and TU Dresden, together with clinical and technology collaborators.