|
Keywords: Hololens - Mixed Reality - Hand gesture - Voice command - Optimization - Big data - Visualization - Interaction - Map - ESRI - City Engine - Prototype - Shader- Unity
Overview
|
|
Hololens tourist app was a project that was given to us as a group of SCAD students by ESRI in 2017. The mission of the Hololens app was to showcase how maps are going to look in the future as hologram and how can we interact with them. The data was given to us through ESRI's software City Engine. Which were 3d model of buildings from downtown of Savannah city. We only had 5 weeks to come up with a scenario that we could built an app around it and develop the app and then present it at ESRI Fed GIS conference. My role as the main developer was to working with the artists to make sure their output is optimized enough for Hololens and also to work with the interaction designers to make sure if their gestures and voice commands are doable in limited time we had.
Final version of the app 2017
Development and Challenges
We came up with tourist app as our concept. Optimizing hundreds of buildings in a way that Hololens could handle in short time was something that I needed to work with artists about. Vertices, mesh combination, UV seems and shaders were some of the areas that I educated the art team to optimize. My background in 3D art came very handy to communicate with our artists and also let them know some tricks for how to optimize faster.
Programing was another challenging area for me. because there were very few tutorials and resources to learn about Hololens and its challenges. Even the documentation wasn't completed at the time. Holotoolkit was very basic and I had to make my own library in Unity. Due to Hololens hardware limitation, I had to make a balance between CPU and GPU to render and interact with the building to have a robust demo that could run stable when we were streaming the project to a 3rd screen. I presented this project in many places for SCAD, such as Siggraph, Fed GIS in DC, ESRI UC conference in San Diego and etc. Although our hand gestures were supper basic, I could easily see that users were struggling sometimes to remember the gestures or doing it in the right way. Sometimes Hololens couldn't recognize the gestures, even the tap gestures, it was very interesting how people were reacting in those moments. Fatigue was another problem that I saw very often. All of those problems made me to start thinking about interactions in Mixed reality and I ended up doing my thesis in the same subject. |
|
|
Some of our concepts and early prototype in Hololens 2017
