How do Voice Assistants Work? A Comprehensive Guide

Voice assistants are certainly not something unheard of. You must have used Siri if you have an iPhone or Google Assistant if you’re using Android. Google’s voice search is also a familiar example of voice assistant software. But if you’re wondering how they work, and what technologies are working behind the scenes to make it possible, then this article is for you.

In this article, we will be talking about what a voice assistant is, a brief history of its evolution, how it works, pros and cons, and much more…

What are Voice Assistants?

Voice assistants can be grouped under a wider category of digital assistants which include all software’s capable of performing simple tasks like answering questions, scheduling events, setting up reminders, etc. However, they can even include AI softwares that work exactly like voice assistants but uses textual data instead of audio. That said, voice assistants specifically use voice-activated commands with speech-to-text or text-to-speech capability.

In technical terms, a voice assistants can be defined as “…a digital assistant that uses voice recognition, language processing algorithms, and voice synthesis to listen to specific voice commands and return relevant information or perform specific functions as requested by the user.” (Alan AI) Apple’s Siri, Google’s Google Assistant, Amazon’s Alexa, and Microsoft’s Cortana are popular examples of voice assistants. Out of these, Siri was the first voice assistant to be publicly available with its launch in 2010.

Surprisingly, voice assistance technology has existed since the 1960s with a few traces dating to the 1920s. That said, now let’s get a brief overview of the evolution of voice assistance technology…

Evolution of Voice Assistants

Speech recognition technology can be traced back to the 1920s when a voice-activated product called ‘Radio Rex’ was invented in 1922. It looked like a dog house with a toy dog (named ‘Rex’) attached to a spring inside the house. Whenever you called its name, by which I mean shouted ‘Rex’, the toy dog would spring out of its house. This was however a crude technology that mostly recognized only adult male voices. Thus, women and children had to either shout out loud or pronounce it differently for the device to sense their voice.

 

This was followed by Audrey, the ‘automatic digital recognizer’, invented by New York’s Bell Laboratories in 1952. It could recognize the 10 numbers, from ‘0’ to ‘9’, for which it required a 6-foot tall casing to house all of its circuitry system.

IBM Shoebox was launched in 1962 and could perform simple mathematical operations like addition, subtraction, multiplication, etc on numbers from 0-9. It could recognize 16 spoken words in total – including the numbers (zero, one, two, etc.) and operations (plus, minus, etc). It was named so because of its size which was similar to that of a standard American shoebox.

Next in the line was the Dragon Dictate which was invented by Dr. James Baker in 1977. It was the first speech recognition software that was commercially available at a startling price of $9000! Designed for DOS-based computers, Dragon Dictate required the user to dictate one word at a time perfectly and pause for the computer to process it before moving on to the next one. That said, it was frustrating to use, unlike today’s natural voice typing programs.

Then in 2010 came our familiar Siri, developed by SRI International as a dedicated app on the iOS app store. It was acquired by Apple Inc. in April in the same year. In 2011, a beta version of Siri was introduced as an integrated program in iPhone 4S. Now, Siri has advanced to all Apple products including iPhones, iPads, Apple TV, Mac, etc.

Soon other famous voice assistant models began coming up – like Google Voice Search in 2011 and Google Assistant in 2016. Amazon’s Alexa was announced in 2015 which popularized smart devices with integrated speech recognition technology.

Here’s a comprehensive timeline showing the evolution of voice assistance technology…

(WARNING: requires a lot of scrolling)

How do voice assistants work?

Voice assistants work using a combination of various technologies – like speech recognition, STT (speech-to-text), machine learning, etc. Let’s understand each one of them and their role one by one…

Speech recognition

Speech recognition, also known as automatic speech recognition (ASR), helps a computer to interpret and process spoken words. It may involve steps like preprocessing, feature extraction, pattern matching, etc. Speech-to-text (STT) involves the conversion of spoken words (audio) into written words (text) to make it readable to the computer. Speech recognition utilizes 2 models to work:

• The acoustic model matches audio signals to its corresponding phonemes (basic sound units).
• The language model ensures the recognized words form sensible and meaningful sentences.

Natural Language Processing (NLP)

Natural Language Processing or NLP is the technology that helps computers to interpret and generate data in natural human language. It involves Natural Language Understanding (which is the comprehension aspect) and Natural Language Generation (which is the generative aspect).

In voice assistant technology, NLP ensures that the computer understands the intended meaning of the user’s input and responds in a humanistic way.

Artificial Intelligence (AI) and Machine Learning (ML)

AI and machine learning play a crucial role in the working of voice assistants. It enables features like:

• Identifying the user’s voice in particular
• Providing personalized responses and recommendations
• Tone modulation
• Making natural human-like conversations (NLP), etc.

Text-to-speech (TTS)

Opposite to Speech-to-text, which converts spoken words to text format, Text-to-speech (TTS) converts written text into audio. It is through TTS technology that the voice assistant conveys the output or response to the user.

All the steps involved in the working of voice assistants can be summarized in the following illustration…

Benefits

• Work efficiency: Voice assistants can take up various tasks in the work environment – like setting reminders, scheduling calls, writing emails, etc. They are much quicker than manually performing these tasks. Moreover, writing using voice assistance technology is more natural along with fewer spelling errors.
• Easy to learn: Voice assistants are simple and quick to use. You only need to speak a command and they will perform the tasks on their own.
• Device integration: Smart home devices are the best examples of device integration using voice assistants. House lighting and music player systems are some devices that can be integrated with voice assistance technology.
• Personalization: AI and machine learning enable voice assistants to suit a specific individual’s needs and behavior based on past interactions.
• Accessibility: Voice assistants are a blessing for people with disabilities. It can help visually impaired people to use computers through voice navigation and screen reading.

Challenges & Limitations

• Accuracy: Although they are continuously improving, voice assistants may still struggle with understanding accents, dialects, slang, and incorrect pronunciation. Their accuracy can also be affected by background noise.
• Privacy and security: Voice assistants continuously collect data from their environment to detect wake words. This can cause concerns related to data breaches and constant recording.
• Bias: A major issue in AI/ML-supported technology like voice assistants is dependence on training or past data. That said, any bias or inaccuracy in the training data may be reflected in the voice assistant’s output.

Conclusion