Keywords: HoloLens 2 - MRTK - Hand gesture - Voice command - Optimization  - Visualization - Interaction - Prototype - Shader- Unity - Interviews - Research - Design process - Problem solving - Branding - Wire Frames

Overview
         This prototype was the very first Hololens project created at 3M. Many design and technical challenges were learned and documented for future projects. Post-it is one of the most iconic products from 3M, and we wanted to use AR to make creating Post-it murals easier and faster. I helped the team in research, design, and development.
Post-it Mural AR prototype June-July 2021

The Process

We reviewed existing company research and the current app to gather insights. With this data and our observations, we pinpointed the challenges users encounter and the features our app requires. Here are some identified challenges:
. Visualizing the proposed mural on a wall for design decision-making is challenging.
. Determining the appropriate mural size for a wall is time-consuming.
. Continually referring back to mural references while installing the post-its is both tiring and time-intensive.
. Ensuring the post-its are leveled presents a significant challenge.
. Having a tally of the total number of Post-its for each color would be extremely beneficial.
Mural Journey map
With the use of HoloLens, users can effortlessly preview their design, adjusting the mural size on the wall in real-time. Thanks to HoloLens' built-in sensors, we can ensure that the post-its are always level. We've implemented a grid system as a guide for the placement of post-its. Each cell within the grid matches the size of a Post-it and disappears when the user steps away. This system aids in achieving a polished final result.

Example of the grid system helping user
Post-it has strong branding; we wanted to carry its energy and creative vibe into our design to match its identity.
A Post-it mural
The brand influenced our decisions on color palettes, sound selections, icon designs, and motion graphics.
Changing MRTK button to a new design that matches Post-it branding
 Our final design was shaped after many sketches, creating wire frames in Figma, and tests.
Some screenshots from our Figma page.
On the left, the MRTK's default buttons and menu. On the right is our design.

The outcome

The Post-it prototype was the first project for our team. It helped our team get to know each other and set up a pipeline to work together. We got familiar with MRTK and modified its classes and shaders when needed. We wrote modular code that would be easy to use for someone else on the team without dependency. We also documented our learnings from the hardware, interaction design, user testing, and development challenges. The project got much attention from the higher managers and other departments at 3M. Sparking conversations about AR's potential applications in their projects. The cherry on top? Our project was showcased at 3M's CES 2023 booth, showing off how we're keeping up with the tech times.
Showcasing our project in 3M's booth at CES 2023
And here is a final demo of making a Post-it mural. The user on the left is using her phone to see a reference of the design, and the user on the right uses our app on the Hololens 2.
The user who is using the prototype is finishing its mural faster.