SALT LAKE CITY — If you ever worried that your Amazon device was listening to you even when you weren't talking to it, you now have a new reason to fear for your privacy. A recent patent application by Amazon describes plans that would allow Alexa, Amazon's virtual assistant, to listen and record users at all times.
While the patent application explains devices would record audio in 10- to 30-second increments and automatically delete unneeded clips, privacy experts say it is cause for concern because it demonstrates tech companies' growing ability to surveil customers at all times and potentially misuse collected information.
"The patent application is just a part of Amazon’s attempt to make voice computing more natural and more human. But in order for Amazon to do so, it appears that we’ll have to let Alexa listen in on more of our conversations," Nicole Nguyen wrote for Buzzfeed News.
"If you care about privacy, you should throw your Alexa devices into the sea," Jennings Brown wrote for Gizmodo.
Alexa runs on Echo smart speakers and other devices and currently works by listening for a 'wakeword' like 'Alexa,' 'Echo,' or 'Computer.' After it hears the wakeword, the device records the spoken command that follows. Like, 'Hey Alexa, play The Beatles.' At that point, the audio is sent to Amazon servers that use speech recognition to deduce the meaning and relay a resonse back to the user. Wakewords are similarly used to control Apple and Google devices, i.e. 'Hey Siri,' 'Hey Google.'
But Amazon's patent, filed with the U.S. Patent and Trademark Office back in January and made public last week, outlines a “system for capturing and processing portions of a spoken utterance command that may occur before a wakeword." In other words, a user could say, 'Play The Beatles, Alexa,' and the device would be able to rewind the audio to identify and execute the command that was given before the wakeword was ever spoken. Buzzfeed pointed out that the idea is similar to Apple's live photos, where video is recorded before and after a user takes a picture.
Amazon provided a statement to several press outlets, including Buzzfeed, that said, “The technology in this patent is not in use, and referring to the potential use of patents is highly speculative.” The statement added that patent applications do not necessarily reflect Amazon's future plans and often explore technology that never makes it into products or services.
If Amazon does decide to use the tech in its products, it’s unclear whether customers would be able to opt out of the "always-on" recording.
"The debate over whether or not Alexa or any voice assistant is spying on us is years old at this point, and it’s not going away," Adam Clark Estes wrote for Gizmodo.
The patent application comes amidst reports that potentially thousands of Amazon employees are listening to and annotating audio clips Alexa records in order to improve accuracy and that Amazon devices have inadvertently sent recordings to strangers or acquaintances. In May of 2018, an Amazon smart speaker sent portions of a woman’s private conversation to one of her husband’s employees.
"While Amazon’s speech recognition software is constantly being improved, the Echo and third-party Alexa devices still accidentally record plenty of audio snippets triggered by false wakewords," Nguyen wrote, adding that constantly recording users as the patent application suggests, could expose more private conversations to Amazon workers tasked with reviewing recordings.
In addition, recent reports have revealed Amazon's efforts to develop a wearable device that can recognize human emotions. The wrist-worn gadget was described as a health and wellness product in internal documents reviewed by Bloomberg. The project is a collaboration between Lab126, the hardware development group behind the Echo smart speaker, and the Alexa voice software team, Bloomberg reported.
An Amazon patent filed in 2017 describes voice-analyzing software that can determine whether a user is feeling “joy, anger, sorrow, sadness, fear, disgust, boredom, stress, or other emotional states.” The patent suggests the company could use the information to recommend products or personalize responses, Bloomberg reported.
"The concept is likely to add fuel to the debate about the amount and type of personal data scooped up by technology giants, which already collect reams of information about their customers," Matt Day wrote for Bloomberg.
But Heidi Messer wrote for The New York Times that consumers should not be so paranoid about privacy. The right to absolute privacy no longer exists, she wrote, and excessive regulation of tech companies will only stifle innovation and prevent job creation.
"Privacy evangelists have made villains of the very companies the world emulates. Rather than debate how to expand this economic opportunity, they call for fettering it," Messer wrote.
The concept is likely to add fuel to the debate about the amount and type of personal data scooped up by technology giants, which already collect reams of information about their customers. – Heidi Messer wrote for The New York Times
January's patent was careful to address “privacy concerns associated with an ‘always-on’ speech processing system” and clarified that all recorded speech will not be sent to Amazon’s servers. Devices may store just 10 to 30 seconds of audio at a time using local memory and will delete what isn't needed “to avoid capturing too much speech and causing privacy concerns.”
Authors of the patent application, Kurt Wesley Piersol and Gabriel Beddingfield, explained the goal is to help Alexa respond to voice commands that are more natural and conversational.
“Current speech processing systems are not configured to handle commands that are not preceded by a wakeword,” they wrote. “Offered is a system to correct this problem.”
But commentators like Anton Lucian, who writes for BeInCrypto, think it's a "major privacy risk" because even though Amazon says it won't access the recordings that aren't commands for Alexa, the company still has the ability to do so.
"If Alexa was unable to be accessed by Amazon’s centralized servers, then its recordings would not be a privacy concern. However, because Alexa does communicate with Amazon as an entity, we have every reason to be suspicious. It is disappointing that such great ideas such as natural voice commands or socially-useful AI are overshadowed by very real privacy concerns," Lucian wrote. "This is because all of these breakthroughs are occurring under the auspices of data-mining tech giants."
"In short, Alexa’s technology is, indeed, a breakthrough. However, without decentralized systems that make it impossible for our private information to be accessed, we should be especially wary of these tech developments," Lucian added.
"This week, I read through a history of everything I’ve said to Alexa, and it felt a little bit like reading an old diary," Estes wrote for Gizmodo. "Until I remembered that the things I’ve told Alexa in private are stored on an Amazon server and have possibly been read by an Amazon employee."
You can opt out of letting Amazon use your recordings to develop new features and improve transcriptions in the Alexa privacy settings. However, you can't opt out of allowing Amazon to retain your recordings for other purposes, Estes noted.
A MailOnline investigation revealed audio archives of users' friends, families and children being recorded without knowing it, and without using a wakeword. Eerily, recordings were often taken when there wasn't even a human in the house who could have activated the device. In one household, Alexa woke up 13 times to record a dog barking.
You can review and delete voice recordings captured by the Echo and third-party Alexa devices by accessing Settings in the Alexa app and selecting Alexa Account and Alexa Privacy.