Imagine going into virtual reality (VR) and navigating your way not with controllers, but with your voice. This ultimate mode of immersive commands has been looked into by Unity, including speech recognition and analysis tools that could make this a reality, and there are even a couple of speech-to-text solutions and sample scenes to put theory into action.
In the blog post, which gives a hefty load of in depth information, there is a fair amount of theory that goes into how speech recognition could benefit VR. The basics of how speech can be recognized is detailed greatly, but what this would all mean for VR can be summed up in one sentence: “So in order for a VR experience with simulated people to feel real, its AI needs to be skilled at analyzing your words and giving an appropriate response.” Basically, it is one thing for an application to recognise speech, but another for it to react in the way you wish it to.
For there to be understood contextual analysis, the AI must pick up patterns in speech over time with users, to come to understand how they behave and react accordingly. “If an application is designed to be used by a single person for an extended period of time, a good AI will pick up on their speech patterns and not only tailor its responses to this person, but also figure out what to expect them to say in specific situations.”
What Unity Labs has done with this research is developed a package for the Asset Store that takes several solutions for this and put it into Unity C# scripts. This includes a scene that compares text transcriptions and allows the user to select and speak a phrase which can then be judged by its accuracy.
The speech-to-text package on the Asset Store provided by Unity is currently available for free. “Our Asset Store package currently integrates a few speech-to-text solutions – but these are enough to easily compare some of the biggest solutions out there and to see what general strengths and weaknesses exist among today’s speech recognition tools.”
For more on the latest developments in the world of VR, as well as all the news, updates, and features, make sure to check back with VRFocus.
