Google And Amazon Patent Creepy SPY Systems That Use Cameras And Sensors In Your Home To Know Everything From Your Mood To Your Medical Conditions
By Ellie Zolfagharifard, THE DAILY MAIL UK
April 2, 2018
- Series of patents filed by the firms ‘outline an array of possibilities’ for spying
- Google plans to use audio and visual signals to better understand a user’s mood
- Cameras could spot objects in the home and combine that with search history
- Google could target adverts at users based on this combined data, patent says
- Amazon has also filed a patent that describes how a ‘voice sniffer’ algorithm could be used on various devices to analyse audio in real-time
Amazon and Google really do want to watch your every move.
A series of patents filed by the firms ‘outline an array of possibilities’ for how their smart speakers could be used to better listen in on users.
They suggest their always-on Google Home and Amazon Echo devices could know everything from a user’s mood to their medical condition, and target advertising based on this data.
The development comes amid a growing scandal involving the handling of the private data of 50 million users by Facebook and Cambridge Analytica.
Google’s patent application outlines how audio and visual signals could be used to better understand a speaker’s mood or medical condition, according to the New York Times.
The devices could listen to the ‘volume of the user’s voice, detected breathing rate, crying’. They could also detect a user’s coughing and sneezing.
The same patent applications reveals a device that could ‘recognize a T-shirt on a floor of the user’s closet’ with Will Smith’s face.
It could then combine this with a browser history that shows searches for Mr Smith.
This would allow Google to ‘provide a movie recommendation that displays, ‘You seem to like Will Smith. His new movie is playing in a theatre near you.’
In a separate patent application, Google describes a device that would give advice to parents for ‘areas of improvement’ such as spending more time with their children at supper.
‘When children are near a drinks cabinet or are in their parents’ bedroom alone, the system may infer that mischief is likely to be occurring’, the patent read.
On detecting mischievous behaviour, the smart device could even hand out punishments such as restricting mobile phone use.
In response to the Times article Google said it does not ‘use raw audio to extrapolate moods, medical conditions or demographic information.’
The company added: ‘All devices that come with the Google Assistant, including Google Home, are designed with user privacy in mind.’
But not everyone is convinced.
‘When you read parts of the applications, it’s really clear that this is spyware and a surveillance system meant to serve you up to advertisers,’ Consumer Watchdog President Jamie Court told The New York Times.
He added that the companies are ‘basically going to be finding out what our home life is like in qualitative ways.’
Amazon has also filed a patent that describes how a ‘voice sniffer’ algorithm could be used on various devices to analyse audio in real-time.
This means that when it hears words like ‘hate’, ‘love’ or ‘want’, it could bring up ads based on the user’s thoughts and interests.
The patent includes a diagram thatshows how a call between two friends could result in one being offered a wine club membership while the other could receive a deal for San Diego Zoo.
DO SMART SPEAKERS LIKE APPLE’S HOMEPOD LISTEN IN ON YOUR CONVERSATIONS?
Like every home speaker, the HomePod has an ‘always-on’ mic that has the potential to hear private conversations.
But it may come with a way to turn it off. Experts recently discovered icons in the latest version of iOS that suggest always-listening microphones on Apple’s Homepod speaker can be disabled.
The delayed £350 ($349) ‘Echo-killer’, which actives when it hears ‘Hey Siri’, released February 9 2018.
The microphone mute feature assuages privacy concerns raised over smart speakers eavesdropping on conversations.
The find was reported by Apple expert Filipe Espósito, based in Londrina, Brazil, who discovered Homepod icons with microphone off symbols in the developer build of iOS 11.2.5.
The icons were found buried in the assets for Apple’s Home app.
Some of the microphone mute icons also contain the text ‘Siri-off’ in their filenames, lending further support to his conclusion.
As with other smart speakers, like Amazon’s Echo and Google’s Home ranges, command phrases are used to trigger voice activated services.
But some users are uncomfortable with the idea that the devices are always listening out for these vocal prompts.
Users previously uncovered a quirk in some models of the Google HomeMini which meant that the gadget was constantly monitoring and recording their speech.
Due to what Google has called ‘phantom touches’, the Home Mini was in a constant state of anticipation, listening for and awaiting verbal instructions, even before the ‘Ok Google’ vocal prompt was given.
While this in itself is a fairly minor inconvenience, it has major implications for the privacy of users and could, in theory, let Google monitor everything that is said in your home.
This information is also transmitted to the Mountain View firm’s servers, raising the possibility of the information being recorded.
The ability to switch off the microphones until they’re needed may go some way to addressing similar privacy concerns in the Homepod.
In a statement, Amazon said the company took ‘privacy seriously’ and did ‘not use customers’ voice recordings for targeted advertising.’
Amazon said that it filed ‘a number of forward-looking patent applications that explore the full possibilities of new technology,’ and that they ‘take multiple years to receive and do not necessarily reflect current developments to products and services.’
At the end of last year patent applications by Amazon and Google revealed ways their Voice Assistant powered smart speakers could already be spying on people.
The findings were published in a report created by Santa Monica, California based advocacy group Consumer Watchdog.
It says patents reveal the devices’ possible use as surveillance equipment for massive information collection and intrusive digital advertising.
The study found that digital assistants can be ‘awake’ even when users think they aren’t listening.
The digital assistants are supposed to react only when they hear a so-called ‘wakeword.’
John Simpson, Consumer Watchdog’s privacy and technology project director, said at the time: ‘Google and Amazon executives want you to think that Google Home and Amazon Echo are there to help you out at the sound of your voice.
‘In fact, they’re all about snooping on you and your family in your home and gathering as much information on your activities as possible.
‘You might find them useful sometimes, but think about what you’re revealing about yourself and your family, and how that information might be used in the future.
‘Instead of charging you for these surveillance devices, Google and Amazon should be paying you to take one into your home.’
