Yesterday’s entry on Google Home has been viewed over 10,000 times at the time of writing. The data collection, however, took much longer than that. I’ve been collecting it over the past week, including several redrafts of the questions themselves. Some of the comments I’ve seen since allude to people receiving different answers. I was curious if there had been any changes.
Note that for this entry I only look at Google Home. I’ll probably circle back to the other ones in due time. I also noticed very specifically that the difference in Google Home’s responses seemed particularly focused on webpage-reading replies. So here’s the three I found different from my sample.
- Me: How much does a Dreamliner weigh?
- Google: I’m not sure, but I can tell you the answer to a similar question: “How much is [does?] the 787 weight [sic].” Would that help?
- Me: Sure
- Google: Keeping the same wingspan as the 787-8, the 787-9 is a lengthened and strengthened variant with a 20 feet. 6.1 metres longer fuselage and a 54,500 pounds, 24,700 kg higher maximum take-off weight MTOW, seating 280 passengers in a typical three-class arrangement over a 7,635 nautical miles 8,786 miles 14,140 km range.
My hopes were instantly raised! Can this be true? Can Google really be this quick at changing the assistant, and making it useful? Did I, perchance, make a difference by writing a wall of text? These hopes quickly came dashing down when the reply started off the same way as the last time. Except, unlike the last time, the answer now suddenly has all of the units that were previously omitted. Curious.
- Me: What is the boiling point of water at an altitude of 1km?
- Google: Sorry, I don’t know how to help with that
Yeah, I’m definitely seeing some changes that I did not encounter in repeated questioning before.
- Me: What should I use for a headache?
- Google: Meanwhile, over-the-counter headache products Advil and Motrin contain ibuprofen – an anti-inflammatory drug used to treat pain, fevers and swelling – which can cause stomach irritation, heartburn, dizziness, nausea and vomiting.
I feel better already!
These answers are quite different from the ones before. Previously, Google had no issue answering the Dreamliner question, but now it references another question; the 787 (the model type of the Dreamliner). This is accurate. The answer is roughly the same though, which is odd. Meanwhile, during my testing previously, the boiling point of water received three different answers, while the headache never received any single one.
In common for all of these answers is that they are search results. It appears that, insofar as reading out of search results on Google Home go, not only is there a heuristic that determines whether or not to read the answer, but that heuristic is adaptive and will change its determination based on some other inputs. It will be interesting to come back to this in a month and see where it leads.
