Apple’s Siri (barely) outsmarted Alexa in 5,000-question IQ test

Clobbered by Google Assistant and Microsoft’s Cortana, led in percentage of answers that were “simply wrong.”

Posted Tuesday by Statista data journalist Felix Richter:

According to research conducted by digital agency Stone Temple “smart assistants” may not be quite as smart as they are made out to be. Take Amazon’s Alexa for example: the assistant powering the company’s popular line of voice-enabled speakers was able to answer just 20.7 percent of the 5,000 questions fired at it as part of the experiment. Notably, Google Assistant and Microsoft’s Cortana were much more knowledgeable when it came to these factual questions while Apple’s Siri performed similar to Alexa.

Statista Siri IQ Cortana Google Alexa Amazon Echo

Lots more detail at Stone Temple, including this damning bar chart:

Siri simply wrong

8 Comments

  1. Fred Stein said:

    Ah. Another of my pet paraphrases. “First mover disadvantage”
    Second movers start with better base technology, be it chipsets, software tools, or best practices. First movers struggle with the challenge of when to rip and replace, a painful and essential decision. Plus second movers take advantage of the market education, deeper insight into the UX.
    Ironically, Apple usually exemplifies the second mover advantage – Watch, Pay, RFID, LTE, MP3 players.
    Long term, it’s about AI that enables NLP, NLU, and more. AI is the future – way more important than supercycle or supply chain rumor du jour. June 5 – I’m all ears.

    0
    May 30, 2017
  2. Gianfranco Pedron said:

    The way Google delves into data gathering of every form, some of it overt and some succinct (as in people are not really aware of how much info is gathered about them by Google) it is not surprising that it should be able to serve up a fair number of correct answers.

    I’d be almost willing to bet that Google knows how much milk is present in each of the world’s “smart” refrigerators on a daily basis.

    0
    May 30, 2017
  3. Apple must know that voice is at the forefront of all future interfaces (CarPlay, HomeKit, Goggles/AR?) hence it was a bit of a head-scratcher when they didn’t make a move for Viv. To their credit, Siri works an awful lot better (for unfathomable reasons) on the Apple Watch and AirPods, but this doesn’t begin to explain errors – perhaps it’s the reliance on Bing search? Whatever it is, Siri’s unreliability is one of those things that can severely damage Apple’s reputation and Tim Cook should devote all the required resources to solving this problem. this may well be a watershed moment in Tim cook’s reign at Apple.

    on a separate, but related note, can Apple possibly release a smart speaker at WWDC (à la Echo) with Siri’s current reliability issues?

    0
    May 30, 2017
  4. Gregg Thurman said:

    “AI is the future – way more important than supercycle or supply chain rumor du jour. June 5 – I’m all ears.”

    Absolutely. Apple takes the long view on everything. Being first to market is not near as important as being best when you do enter a market.

    Given Apple’s parade of AI acquisitions and a suspected Neural Engine chipset, which will be highly optimized for each other, I can see Apple becoming at least #2 (same test) 2 years from now.

    I do not see Google and Intel working together to achieve peak integration. Qualcomm is out of the equation.

    No matter which “Neural Engine” the Android community uses, because of the inability to upgrade existing devices, it will take a long time to generate a viable Android user base. Oh, did I say Android user base? Apple Watch is outselling the entire “smart assistants” industry by ~3X. This is important because the media continues to refer to the Apple Watch as a failure. What does that make Alexa, Cortana and Assistant.

    The market for personal assistants is way to young to be implying winners.

    0
    May 30, 2017

Leave a Reply