Hey Tim Cook, you’ve got a Siri problem

why siri so bad

I’m not the only one asking why Siri is so bad.

I thought Siri would be better on the HomePod, with AC power and all those microphones. It isn’t. And I want to know why.

I get that my $349 HomePod makes a better music speaker than my $99.99 Amazon Echo or my $129 Google Home. What I don’t get is why Apple hasn’t caught up to Amazon or Google in terms of answering simple, direct questions. See John Gruber’s much re-tweeted How many quarts in a gallon?

Apple had a head start. It has more cash than Scrooge McDuck. It’s dedicated, as Tim Cook never tires of reminding us, to making great products that improve people’s lives. Where did it go wrong with Siri?

Here are the theories I’ve seen so far. The first came from clicking “I’m feeling lucky” on the Google Search above.

  • Sean O’Kane, The Verge: Apple has lagged behind in building out its artificial intelligence efforts. Apple’s tendency for secrecy has reportedly scared away some of the best minds in AI out of fear that there’s no chance for [voice] recognition in Cupertino.
  • Tripp Mickle, Wall Street Journal: Progress has been slowed by a failure to set ambitious goals, shifting strategies and a culture that prioritizes user privacy—making it difficult to personalize and improve the product. The project also has suffered from the departures of key team members, some of whom went to competitors.
  • Walt Mossberg, The Verge: There are three reasons for my doubts: first, Apple’s history with cloud-based services in general has been weak and inconsistent. Second, Apple has done shockingly little to capitalize on its lead with Siri. And third, Apple’s steadfast devotion to privacy and lack of a search service or social network means it doesn’t have the range and volume of data its competitors hope to use to power personalized, actionable AI capabilities.
  • Eddy Cue, Apple SVP whose portfolio until last Fall includes Siri: “I don’t think anyone does an A+ on conversation. It’s a challenging problem and there’s a lot of work to be done in that area. It will get a lot better and needs to get a lot better. [From WSJ interview.]

My take: The stories of leadership dysfunction in Mickle’s piece may be reason enough, but I wonder if there might be a deeper, structural problem, a legacy issue, maybe, dating back to the original SRI International implementation? I’m open to ideas.

UPDATE:  Speaking of deep legacy issues, a friend-of-the-blog at iPad Insight forwarded this link:

  • Andrew Tarantola, Engadget: Fast forward to 2014. Apple is at the end of its rope with Siri’s listening and comprehension issues. The company realizes that minor tweaks to Siri’s processes can’t fix its underlying problems and a full reboot is required. So that’s exactly what they did. The original Siri relied on hidden Markov models — a statistical tool used to model time series data (essentially reconstructing the sequence of states in a system based only on the output data) — to recognize temporal patterns in handwriting and speech recognition. The company replaced and supplemented these models with a variety of machine learning techniques including Deep Neural Networks and “long short-term memory networks” (LSTMNs). These neural networks are effectively more generalized versions of the Markov model. However, because they posses memory and can track context — as opposed to simply learning patterns as Markov models do — they’re better equipped to understand nuances like grammar and punctuation to return a result closer to what the user really intended.

My question: What models are Amazon and Google using?

19 Comments

  1. David Emery said:

    It’s not clear to me if the problem is with Speech Recognition, or if it’s with Natural Language Understanding.

    0
    February 23, 2018
  2. Ken Cheng said:

    If you looking for trivia arcana, then most smart assistants won’t do very well, but Siri is likely to do worse, because it’s all about the databases you’ve indexed, and Google has indexed the most, given its position as a search engine. As far as the 90% of questions being asked, like what time is it, how’s the weather, do I have any appts scheduled, when’s the basketball game tonite, then Siri does fine for me. Not to mention, Siri has been really good with my music library, even understanding how I pronounce some Chinese musician’s names.

    1
    February 23, 2018
  3. Richard Wanderman said:

    I use Siri daily to good effect. I don’t have or use a HomePod or competing device. I use Siri on my iPhone mostly, my Mac and my iPad. In a sense, I’ve been trained by Siri to side step its inadequacies and limitations: I know how to phrase my interactions and I don’t expect too much.

    Where Apple blew it is by not limiting the domain that the initial implementation of Siri worked in. Amazon did this with the Echo and added more breadth as it came online. Apple left things wide open and people got frustrated with Siri’s limitations and many gave up on it. I know I did. However, it’s extremely useful in limited domains and once you know what those are and use it with a decent connection (Joe, it’s cloud based just like the others) it works just fine.

    I use it for cooking timers (one at a time).
    I use it in the car hands-off to look things up, get directions, make calls, send texts.
    I use it to look things up that I know it won’t give me a web page for (if I wanted a web page I’d run Safari and look myself).

    If you have a decent connection to the internet and limit its domain it’s fine. Can it be better? Of course, and no doubt it will be. But, part of using it is learning how so I think it’s best to continue to struggle with it and make it work as one can rather than simply give up on it altogether.

    This is very much like Apple maps: Apple has stumbled here but I don’t think it’s too late to catch up. After all, Siri is software so it can improve without folks buying new iPhones and other Apple hardware and it will.

    0
    February 23, 2018
  4. Grady Campbell said:

    I wonder if this is not a side-effect of Apple’s greatest strength: focus. Each product seems to go in and out of favor on some multi-year schedule. They come out with a good product, incrementally refine it for a couple of years, and then largely neglect it. Some examples: iWork apps, Macs (they seem able to focus on only one line at a time: MacBook, then MacBook Pro, then Mac Pro and Mac Mini (?)), iPads, iCloud, Safari, Airport/TimeCapsule, Maps (Portland, ME is now on a small Island in the bay, unreachable by car). At some point for each of these, they decide they’ve neglected it too long and refocus on it, usually to good effect. Maybe it’s coming to be Siri’s turn.

    0
    February 23, 2018
  5. Jonathan Mackenzie said:

    I tried very hard to get Siri to fail on the how many quarts to the gallon question and I could not. I used accents, spoke fast. I even asked “how many coos to the gallon?” I got the same answer every time. Siri doesn’t do lots of things, but I’ve rarely had problems with it understanding me. When it has screwed up it was because of ambiguous words.

    If I have an interesting question I start with “do a google search for.” I agree with those who say it’s limited and behind the competition, but it understands what I’m saying better than most of my friends. And it knows what I mean to say way better than, for example, auto correct, which has frustrated me multiple times in the course of just this one post.

    0
    February 23, 2018
    • Jonathan Mackenzie said:

      I just tried the exact same experiment on the exact same phone (but in a new location). I had a comical string of failures. It kept asking me what I wanted to covert into quartz. When I finally gave up and asked slowly and clearly… it still asked me what I wanted to convert into quarts. I literally could not stump it only a few hours ago and now it won’t even recognize careful speech. I haven’t been drinking, I promise. I have no idea why it would fail so hard at the same task it was just flawless at.

      0
      February 23, 2018
  6. Gregg Thurman said:

    All voice assistants are only as good as the data bases they are linked to. SIRI is not linked to as many as Google, Facebook or Amazons assistants. That’s because of Apple’s privacy strategy vs everyone else mining your information.

    Apple has chosen to link data bases to SIRI that can be done on your device, do not mine user data and not require an online experience.

    Additionally, SIRI is primarily designed to do something (take a positive action), whereas all others are primarily designed to answer knowledge questions (internet searches). It may be a subtle difference, but not knowing that subtlety can and does lead to frustrations. If you want the answer to a question use Google’s AI, if you want your assistant to do something use SIRI. There will be some overlap in capabilities, but that is the difference in the two.

    1
    February 23, 2018
  7. Sean Callahan said:

    Have we forgotten that, from its introduction, Apple has been stressing HomePod was a revolutionary (even for the $350 price tag) music player? And most audiophiles agree that it is. With an Apple Music subscription, it’s even better. (APL hasn’t always played nice with competitors like Spotify.)

    Yes, refinements to Siri have been slow coming and that’s inexcusable given the lead APL had on this technology. But Walt Mossberg makes a good point when he says, “Apple’s steadfast devotion to privacy and lack of a search service or social network means it doesn’t have the range and volume of data its competitors hope to use to power personalized, actionable AI capabilities.” Other commenters here are picking up on this.

    That’s the conundrum APL faces. The high bar they’ve set for encryption is slowing their ability to make Siri as smart as Alexa or Google.

    I’m confident that Siri will get there and HomePod will become an even more indispensable home appliance once their new AirPlay update is released allowing it to be integrated into other smart home devices.

    This is a new product category for APL and this is only version 1.0. We know that they have a history of introducing a product that might underwhelm some segments of the market (and their doomsday pundits) but has just enough innovation to keep the fanboys happy while they innovate themselves to an even more successful product.

    I’m not buying a HomePod. Yet. But I trust that it won’t be long before Siri becomes the smartest kid in the room and there will be a lot of third party devices that want to talk to her.

    2
    February 23, 2018
  8. Gregg Thurman said:

    [[ But Walt Mossberg makes a good point when he says, “Apple’s steadfast devotion to privacy and lack of a search service or social network means it doesn’t have the range and volume of data its competitors hope to use to power personalized, actionable AI capabilities.” Other commenters here are picking up on this. ]]

    Nailed it. It isn’t that Apple can’t make SIRI better. The issue is that, at this point, making SIRI better violates Apple’s privacy policy.

    2
    February 24, 2018

Leave a Reply