Behavioral Prototype: Real-time Conversation Assistant

For our Behavioral Prototype project, my group chose to conduct a user test on an innovative real-time conversational assistant we developed. This assistant is designed to not only translate but also correct dialogue in real-time between two participants.

Prototype

For our behavioral prototype project, we designed a system consisting of two primary components: the microphone into which users would speak, and a PA speaker linked to another microphone operated by me, acting as the “wizard” to translate conversations in real-time. The concept was for users to communicate in their native languages using these microphones, while I, fluent in both languages and out of sight, translated and corrected any grammatical errors in “real-time”. During our testing phase, Jessica took on the role of facilitator, and Lance served as the Scribe and Documentarian, with me playing the role of the Wizard. To enhance the authenticity of the assistant’s voice, I manipulated the pitch using a formant shifter in my Digital Audio Workstation. The system’s functionality was quite simple: I just needed to ensure all microphones were connected to the speaker system.

Our Setup with the Microphone and PA Speaker

Analysis

To conduct our behavioral prototype test, I enlisted the help of a friend from South Korea and another from Michigan, encouraging them to converse in their native languages. Initially, Jessica briefed them on how the “magical” microphones they were holding would work. After her explanation, we prompted them to start a conversation in Korean and English, respectively. From my hidden position, I translated their dialogue in real time using my own microphone. The results were a success — the translations facilitated their communication effectively, allowing them to understand each other with ease.

Furthermore, when our friend from Michigan tried speaking Korean, I was able to immediately correct any grammatical errors he made, clarifying his sentences so that our Korean friend could comprehend them better. Peer feedback highlighted that our project brought out an emotional response from the users, evident from the joy they expressed in our video once they could successfully communicate.

However, there were some areas for improvement noted in our project review. One classmate suggested it would have been useful to include translated captions in the video when the Korean speaker was talking, to aid viewers’ understanding. Another point of critique was the impractical size of the PA speaker used as our “device.” If we were to make this a real thing in the future I plan on creating develop a more portable version of our system to enhance its feasibility for everyday use. Despite these areas for improvement, my peers were particularly impressed by the voice alterations provided by the formant shifter, and they rated the usability and desirability of our system quite high.

[embed]Real-time Conversation Assistant Video