Development of a Holographically Augmented Medical Simulator

Project Outline:

Simulation has been an important educational tool for physicians for over 15 years. It is an excellent method of developing the basic skills and habits without subjecting patients to novice errors.

Fidelity refers to the degree of exactness with which something is copied. In simulation, it refers to how well the simulated environment reproduces the real clinical situation. Fidelity is relatively unimportant for junior medical learners who are learning fundaments, but becomes increasingly important as the level of expertise develops.

Holographic augmented reality headsets, such as Microsoft HoloLens, have potential for significantly improving simulation fidelity by providing positional and physiologic responses of the virtual patient. Sensor arrays built into the headset could also allow acquiring more information about trainee performance

The students selected for this project will work together with engineers at PerkLab in School of Computing ( and physicians at Queen’s Faculty of Health Sciences Simulation Lab to develop and evaluate holographic augmentation of traditional patient simulators. Experience in computer graphics and familiarity with C# and C++ is recommended.


Develop an animated patient holograph suitable to superimpose over the provided simulation manikin using Microsoft HoloLens. Virtual patient may be developed using Unity engine ( and communicate with the physical patient simulator and 3D Slicer (


  1. Develop holographic objects that can be scaled/warped and displayed virtually on the physical patient simulator.
  2. Implement spatial registration between physical patient simulator and the augmented reality headset.
  3. Implement communication link to physical patient simulator.
  4. Tentative: Evaluate feasibility of computing performance metrics based on trainee body, head, and hand motion

Project supervisors:




Project type: 

Undergraduate project
Masters project