Smart assistants are everywhere. Besides nestling in our smartphones, we also have access to an ever-expanding catalog of speakers that support these AI-powered devices. But which one is the best? If you've ever wondered, one company has taken it upon themselves to test and compare them . Here's what they found out.
Google Assistant, Alexa, Siri or Cortana, which is better?
The team at Loup Ventures was responsible for testing the assistants. This firm, dedicated to conducting studies and research in the technology sector, pitted four smart speakers against each other , each with a different assistant: Google Assistant , Alexa , Siri , or Cortana . The devices used for the tests were (all second-generation) the Google Home, an Amazon Echo , the HomePod, and a Harman Kardon Invoke, respectively.
The procedure involved asking each person 800 questions (the same ones, of course) and verifying the effectiveness of their answers as well as the participants' comprehension. For this purpose, two aspects were assessed for each question:
- 1º) Did you understand what was said?
- 2º) Did you give a correct answer?
As for the questions themselves, they can be grouped into five different categories:
- Local – Example: where is the nearest coffee shop?
- Business – Example: can you ask me for more toilet paper?
- Navigation – Example: how do I get to the bus stop?
- Information – Example: what is the Real Madrid line-up for today?
- Commands – Example: remind me to call Maria at 2 PM
After passing all the tests, it was found that Google Home with the Google Assistant was the most effective , being able to answer correctly 88% of the time ("Answered Correctly") and understand 100% of the questions asked ("Understood Query"). It was followed in the ranking by Apple's HomePod with Siri , which answered correctly 75% of the time and was only unable to understand 3 questions.
Amazon's Echo with Alexa answered 73% of the 800 questions correctly and failed to respond to 8, while the Invoke with Cortana answered correctly 64% of the time and failed to understand 8 questions. The study indicates that almost all of the misunderstood questions involved a proper noun, often the name of a city, restaurant, or other establishment.
It's curious that Alexa has been the one that has failed the most at understanding questions. At El Output, we've tested Echo speakers , and their understanding capabilities are truly excellent—we'd even venture to say that the subjective feeling is that they're superior to some of their competitors. In the graph below, you can see the accuracy rate depending on the question category—the Business section shows the biggest differences, perhaps due to the support of having their own maps in the case of Google and Apple.
To better see what differences exist, Loup Ventures gives an example of one of the questions (from Business) and the answers obtained by Alexa and Google Assistant:
- Interrogatory: How much would a manicure cost me?
- Alexa: The best search result for manicures is Beurer Electric Manicure & Pedicure Kit. It's $59 on Amazon. Do you want to buy it?
- Google Assistant: On average, a basic manicure will cost you about 20 dollars. However, special types of manicures like acrylic, gel, shellac, or others range from $20 to $50 in price, depending on the salon.
The study also highlights the HomePod's disadvantage of having limited Siri compared to its effectiveness on the iPhone . In fact, several of the questions it didn't answer correctly were due to Apple's speaker responding that it couldn't provide that information "on the HomePod."
The study concludes with a graphical comparison showing how these assistants have improved over time —from last year to this year. Their helpfulness and usefulness have increased enough to warrant praise for the companies behind them. This is certainly more than evident and points to a promising future for this segment.
Do you have a smart speaker or do you usually use one of these assistants?