Masters Thesis Defense “Model Preparation and User Interface Aspects for Microsoft Hololens Medical Tutorial Applications” By Andrew J. McNutt

Committee:  Drs. Yong Pei, Advisor, Thomas Wischgoll, and Mateen Rizki

ABSTRACT:

Augmented Reality is the combination of computer vision and graphics to allow a device context about the environment around it to then display relevant information to the user. The Microsoft Hololens is the latest wearable AR device that follows this by having a series of infrared sensors perform Simultaneous Localization and Mapping and hand tracking, a wave guide optics system that does not obscure the user’s vision while adding holograms, and an operating system that is able to perform applications independently and untethered from any server. The goal of this thesis is to explore the hardware and software capabilities of the device, prepare 3D graphical models for use with specific applications, and understand what aspects of the user interface would best address specific needs of an application. Particularly, in our research, the target application is a tutorial on the human body by displaying and controlling a 3D virtual human arm and using that to develop knowledge of manipulating a human joint system through a HoloLens platform, with the model in question displaying the skin, muscle, bone, nervous, endocrine, and cardiovascular system of a generic arm. The thesis will also address additional features that could be added to this basic model in order to develop an immersive learning tool that can be used before, and potentially in the middle of, a surgical operation.

Keywords:  Augmented Reality, User Interface, Three Dimensional Models, Microsoft Hololens, Human Anatomy