CRAFT

To help us understand how we should develop our idea, we needed to get insights from experts in the field and potential users. Since we were foraying into an unexplored territory, we required new perspectives to better understand the interactions and complications involved. Some of the interviews we did were with:

• Todd Bryant, NYU Professor for VR who helped us understand VR technicalities and also pointed out there was no accessibility currently in VR
• Serap Yigit, a User Experience Researcher at Google who helped us learn about user research techniques and usability functionalities
• Claire Kearney-Volpe, NYU Professor at the Ability Lab who guided us to focus on multi-modal interactions and put us in touch with potential users at Adapt Community Network
• Erica Wayne, Account Manager at Tobii Pro who helped us understand the eye tracking mechanisms
• Peter Cobb, Director of Adapt Community Network who explained how users with limited physical mobility currently create art
• Adapt Community members who suffered from cerebral palsy and were quite excited about our multi-modal tool to create digital art

These interviews gave us a real world experience of how users reacted to our idea. Some of the key insights were as follows:

Based on our interview insights, we identified four unique persona to focus on for our project. Each persona had their own goals, motivations and frustrations and different mobility levels as well. Hence it helped us make our design more universal in nature.

After understanding our persona spectrum and user flow, we proceeded to create basic prototypes to demonstrate and test the concept of our idea. To make our tool accessible, we had to empathize for our users and identify pain points and intuitive ways of interaction. We decided to go for multi modal interaction, since the user has limited hand mobility, we wanted to make use of voice and eye gestures to perform tasks. We wanted to experiment the entire user flow with both eye interactions and voice, to see which is more intuitive. We started with creating a video to demonstrate how voice commands and eye gaze can be used to operate the tool without hands. This prototype helped us demonstrate our idea and make it understandable to our users as it was quite a new and unfamiliar concept to many people.

After working on the video prototype, we compiled our product features and interactions. Once we got insights from professionals, we decided to build a paper prototype. The idea was to build something simple which we could take to our users and test it out. We were looking to add common and easy to comprehend tools. We came up with a list of tools which included drawing tools, system tools and functionality tools. After this we printed out each tool icon on a A4 sheet as demonstrated in the pictures below.

After our prototype was ready, we wanted to come up with a cheap and easy solution to demonstrate eye tracking. We concluded that a laser pen would be the most optimal way to do that. Our plan was to attach the laser to pen on a hat and once the person moved their head, the beam of the laser would move. This would give an idea to the user where they were pointing on the paper prototype. Once the user gazes at a particular tool, the selected tool is highlighted using the blue-violet light. To draw, the user would move the laser point across the canvas and a student would trace the trajectory using a marker. Some of the tools such as scale and color palette were made expandable and we used different sheets of paper to make pop ups. We decided to test it with designers and artists with experience in experience in VR as well as 2D and 3D software drawing tools and normal hand movements to get quick feedback on usability. We then planned to test it with the members of the ADAPT community who had limited hand movement, but were interested in art and the concept of drawing using eye movements.

After understanding our user journey and building our information architecture by reorganizing the tool structure and finalizing on our interactions, we proceeded to create the final hi fidelity prototype using the Unreal Engine. We also did a mockup of an on-boarding AI assistant called Crafty to help new users get familiar with the interface. The assistant guided the user with the features and functionalities of each tool and was available if the user got stuck at any point. The final prototype process included Movement tracking, Interactable user interface, Time-based gaze selection, Teleport function, Painting function and changing the Environment. These tasks helped us accomplish a working prototype which users could interact with in the immersive environment.

We have received good feedback for our prototype but we aim to work towards improving it even more especially in the following areas:

• Use eye movements to draw
• Incorporate voice functionality by using machine learning and artifical intelligence
• Use natural language processing for building an effective AI assistant
• Test with more users and refine the user interface
• Brainstorm on how to transform the drawing to 3D coordinates
• Release it on the Oculus Store and Vive Store

We hope that this tool would be extremely useful for people with limited mobility to not only experience VR but also to create interesting content and share it with the world.

We analyzed our project, the basic concept and our vision for this in future and started creating the business model canvas. This was filled in and refined every week as we spoke to more people and tested out hypotheses. Our final version of the business model canvas was as shown below:

During the course of the entire bootcamp, we listed out hypotheses to test on a weekly basis to see which of our assumptions were true after talking to customers. Some of our key hypotheses to test were as follows:

• People with both permanent and temporary limited limb mobility will be comfortable drawing art in VR
• The transition from using hand controllers to eye gestures and voice commands will be a seamless experience
• Our tool would be needed by all our targeted customer segments and they will find the tool easy and useful
• Therapy Institutes, Disability Centers and Hospitals will be our B2B clients and they will be fine with the pricing model
• HMD Manufacturers will be willing to partner with us and Influencers will be willing to promote our product
• Drawing with other people in VR would be engaging for our users and children with disabilities will also be comfortable using our tool
• Product needs to be customized for specific customer segments and In-App Store will create employment opportunities

We visited the Adapt Community Network which is a powerful community of people with Cerebral Palsy and spoke to them about our project. We encouraged them to try out our application with the VR headset and they simply loved the experience.

We also visited the Axis Project which is a multidisciplinary center committed to providing high quality services for those with physical disabilities. We asked people to interact with our application and recorded their feedback about the experience which was extremely positive and encouraging and they were excited about purchasing this application when launched.