Amazon and Google have been heavily promoting their digital “assistants,” Google Home and Amazon Echo as ideal holiday presents and time-savers for busy families. Both are voice-activated gadgets that can respond to consumers’ voice commands, playing music, reciting weather and news or manipulating lights and appliances.
The consumer summons the device simply by speaking its name, as in: “Alexa (an Amazon Echo device), play Christmas jazz.” But the non-profit group Consumer Watchdog notes that, although the devices simply sit there until summoned, they are always on and always listening — and the group says it has evidence the listening is not always as harmless as it seems.
The group says that a study of patent applications filed by Amazon and Google with the U.S. Patent and Trademark Office reveals a vision for an Orwellian future in which digital assistants eavesdrop on everything from confidential conversations to your toilet flushing habits to children’s movements and the books on bedside tables. They would know when you go to sleep and whom you wake up with.
“Snooping on you”
“Google and Amazon executives want you to think that Google Home and Amazon Echo are there to help you out at the sound of your voice. In fact, they’re all about snooping on you and your family in your home and gathering as much information on your activities as possible,” said John M. Simpson, Consumer Watchdog’s Privacy and Technology Project Director, in a news release. “You might find them useful sometimes, but think about what you’re revealing about yourself and your family, and how that information might be used in the future.”
The patents reveal the devices’ possible use as surveillance equipment for massive information collection and intrusive digital advertising, Simpson said. The complete report is available here.
“Instead of charging you for these surveillance devices, Google and Amazon should be paying you to take one into your home,” Simpson said.
Key findings
Among the key revelations from the patent applications:
    ▪    Digital assistants can be “awake” even when users think they aren’t listening. The digital assistants are supposed to react only when they “hear” a so-called “wakeword.” For Amazon Echo it’s “Alexa” and for Google Home it’s “OK, Google.” In fact, the devices listen all the time they are turned on – and Amazon has envisoned Alexa using that information to build profiles on anyone in the room to sell them goods. Amazon filed a patent application for an algorithm that lets the device identify statements of interest— such as “I love skiing,” — enabling the speaker to be surveilled based on their interests and targeted for related advertising.
    ▪    The devices can connect to other internet-enabled home systems to monitor your family members’ habits and infer what they’re up to, such as when your children are engaged in mischief. A Google patent application describes using a smart home system to monitor and control screen time, hygiene habits, meal and travel schedules, and other activity. The system even claims it can “infer mischief” based on audio and motion sensor readings from rooms where children are present. Silent children who move are inferred to be mischievous.
    ▪    The devices are envisioned as part of a surveillance web in the home to chart families’ patterns so that they can more easily be marketed to based on their interests. Google connects its Google Home to various “smart” devices such as thermostats and lighting made by another Alphabet Inc. division, Nest. When connected, “inferences” could be made about when occupants are home, sleeping, cooking, when they are in the den watching television, when they shower and when they flush the toilet, according to a Google patent.
    ▪     Another Google patent outlines ways it could collect information about family members’ interests and activities to infer likely purchases. For example, the application describes how sports camp could be marketed to a 15-year-old boy holding a basketball in the living room. It also describes how Google could infer an interest in the actor Will Smith by combining a users’ browser search history with an image on a user’s t-shirt obtained from a Nest camera in the home. It also describes how it could sell you a TV show by spying on a book on your bedside table. “The answers to these questions may help third-parties benefit consumers by providing them with interesting information, products and services as well as with providing them with targeted advertisements,” the patent application claims.
“This isn’t about helping people, it’s about selling people,” said Simpson. “If these patents are implemented, there will be unparalleled surveillance of our private lives. The privacy invading implications of these devices are profound.”
Targeting advertising
Google and Amazon appear most interested in using the data they get by snooping on your daily life to target advertising, Consumer Watchdog said. However, when that information is compiled others could access it. For example, home insurers and utility companies have already made deals with Nest to put smart devices in their customers’ homes.
Law enforcement is already seeking information from smart devices. An Amazon Echo made headlines last year when police investigating a murder sought to subpoena recordings made by the device. Investigators in the same case also managed to obtain data from a smart water meter that suggested that the crime scene had been hosed down before police arrived.
Hackers and identity thieves are also likely to be able to access the data compiled by Google and Amazons snooping, Consumer Watchdog said. In fact, Google Home’s FAQ contains the following waning:
“Anyone who is near your Google Home device can request information from it, and if you have given Google Home access to your calendars, Gmail or other personal information, people can ask your Google Home device about that information. Google Home also gets information about you from your other interactions with Google services.”