Project Website

Role: Unity Developer

Responsibilities:

• – Built a .NET application to send speech recognition data using Microsoft’s Speech Recognition API to a Unity Game Client
• – Built a system to handle verbal interactions between the player and an NPC, including: interruptions, positive reinforcement, negative reinforcement, and answering/refusing to answer questions directed at the player.
• – Built a system to specify the NPC Comedian’s script along with accepted player responses by through modification of an XML file.
• – Imported and synced assets into the Unity Project (textures, models, animations, audio).

Length: 15 weeks

Speak with Purpose was an ETC faculty pitched project that centered around using speech recognition technology to create voice driven gameplay. The goal of the project was not to improve speech recognition technology but to discover ways to use the current limitations in the tech to our advantage to create an immersive and interactive experience. Since this project was more research oriented, our objective was to deliver a proof of concept prototype as opposed to a polished product. Our final deliverable was a single player experience that used the Oculus Rift in conjunction with Microsoft’s Speech API for voice recognition. The Oculus Rift was used to make players feel more immersed in the world, which was a comedy club.

Players were able to interact with the comedian by heckling, praising, and answering questions that the comedian fielded to the audience. The comedian not only reacted to each interaction but would change his performance based on them. For example, heckling and praising directly affected the comedian’s confidence index. If the comedian had a very low confidence index, what he said and his body language would change to reflect that. The comedian was also able to recognize specific answers to the questions he asked. If an answer was recognized, he would give an appropriate response back. If an answer was not recognized, he would give a generic response back. We found that as long as there was a higher ratio of appropriate to generic responses, players still felt fully immersed in the experience. You can read a full summary of our research and what we discovered throughout the development process here.