Smart Speaker Privacy Concerns - How to Protect Yourself

  AUTHOR: Carlos Paras
 UPDATED: March 23, 2024

Smart speakers are becoming increasingly popular among smart home enthusiasts. They have many practical uses that offer convenience, cost savings, and can help improve the quality of our lives.

Whether it be entertainment, managing day to day tasks, or controlling home automation systems, smart speakers can accomplish all of these functions with directions from simple voice commands.

Despite their popularity, there are growing concerns about how smart speakers impact users’ privacy and security of their personal data.

There have been accounts of users lowering their voices during conversation in an attempt to keep their smart speaker from listening in. Some even go to the extreme of moving their smart speaker into another room or completely turning it off to avoid being “spied on”.

In this article, we’ll explain how smart speakers work, why smart speaker companies collect user data, and what they do with the information they collect. At the end of the article, I’ll provide some simple steps you can take to help protect yourself and your personal data.

Smart Home Deals @ Amazon

How Do Smart Speakers Work?

To understand why there are concerns with smart speakers, it’s important to know the basics of how a smart speaker works.

Smart speakers come equipped with a voice assistant. This is what allows the smart speaker to perform tasks such as playing music, giving news updates, making appointments, and controlling other smart devices.

Voice assistants use a technology called voice recognition to translate and interpret human speech. A user can simply speak a voice command and the voice assistant will carry out the task.

The smart speaker is activated when a wake word like “Alexa”, “Ok Google”, or “Hey Siri” is detected. Once the wake word is detected, the smart assistant is ready to listen for a command. Which brings us to the first privacy concern…

Are Smart Speakers Listening All the Time?

In order to be ready for the wake word, a smart speaker has to be “listening” to its surroundings at all times. A smart speaker continuously analyzes short snippets of audio to check for the wake word.

The process of listening for the wake word happens internally on the device itself. If the wake word is not heard, the audio snippets are deleted. No real privacy invasion there.

However, if the wake word is detected, the smart speaker will start recording what is said after the wake word. It will send this recording to a cloud server to be processed by a voice recognition service.

The way the three major manufacturers of smart speakers (Google, Amazon, and Apple) use these recordings has recently brought them under more scrutiny and attention.

Before we go any further (and without getting too technical), let’s briefly go over a general description of how voice recognition works.

How Does Voice Recognition Work?

Voice recognition software takes spoken words and sentences and breaks them up into little chunks of sound. It “digitizes” these sounds by turning the analog sound waves into a digital format.

The software takes the digitized sounds and compares them to saved or known patterns that it is already familiar with. This is how voice recognition software is able to “understand” and recognize human speech.

Over time, as the software’s “library” of known speech patterns grows, it gets better at understanding human speech. The process by which the voice recognition software of the major smart speaker companies learns to improve brings up other privacy concerns.

Who Listens to Smart Assistant Recordings?

The voice recognition technology behind Alexa, Google Assistant, and Siri is far more complex than the brief overview given in this article. There are many different nuances to language and speech such as slang, accents, and dialects which the software needs to account for and understand.

Voice recognition software must constantly be updated in order to handle these subtleties and the software itself isn’t fully capable of learning to improve on its own.

It has recently come to light that Amazon, Google, and Apple have employees that listen to smart speaker audio samples to help improve their voice recognition technology.

Amazon

In April 2019, seven Amazon employees spoke to Bloomberg and described their work with Amazon’s human review of Alexa recordings. The workers in this program transcribe and annotate recordings from Alexa enabled devices. This information is entered into the voice recognition software to help improve its accuracy.

Two of the seven Amazon employees who spoke to reporters described spending 9 hours per day listening to up to 1,000 recordings per work shift.

Workers reported hearing recordings that were intimate and private in nature. Other recordings were somewhat amusing. Some heard audio snippets that caused concern and involved what could be considered criminal activity.

Google

In July 2019, over 1,000 recordings from Google Assistant were leaked to a Belgium news website. Reporters spoke with subcontractors who reviewed Google Assistant recordings.

Some of the Google recordings that were leaked contained confidential and private information, including medical and business information. Additionally, it was reported that 153 of the leaked audio samples were recorded without the wake word being detected.

Apple

Following the Google leak, an Apple employee came forward with reports of a similar experience working in Siri’s human review program. Workers in this program rate the voice assistant’s response to a user’s request.

Recordings are graded on factors including whether or not Siri was activated intentionally and if the voice assistant’s response was appropriate. The Apple employee gave an account of contracted workers hearing confidential information, sexual acts, and drug deals.

Apple has said that less than 1% of their recordings are sent to review and that the recordings are linked to an anonymous ID with no personal information attached.

Amazon claims it only reviews a small sample of recordings and that employees don’t have access to any personally identifiable information (PII).

Google claims that they only review 0.2% of all recordings, that the recordings have no PII, and that the audio is distorted to disguise the user’s voice.

Amazon, Google, and Apple Halt Human Review

As a result of growing concerns and pressure from both consumers and governments, the three major stakeholders in the smart speaker market have implemented changes to how they handle smart speaker recordings.

In August 2019, Apple said they would suspend the practice of having employees listen to audio recordings while it conducts a review of their program. Additionally, they announced a future software update which would allow users to opt in to participate in the program.

Google has also acted by stopping human review of Google Assistant recordings until they complete an investigation into their leaked recordings. While they didn’t give a time frame of when reviews would resume, Google was ordered by the Hamburg Commissioner for Data Protection and Freedom of Information to stop its human review for at least 3 months.

Amazon has also made changes, but instead of discontinuing its human review of recordings, it has made updates to its privacy policies and has added an opt out setting in its Alexa options. Recordings will still be uploaded to Amazon servers, however. It’s up to users to delete the recordings on their own.

Amazon Smart Speaker Patent - New Technology, New Concerns

In January 2019, Amazon filed for a patent that would allow their smart speaker to start recording before the wake word is spoken. While a patent application doesn’t necessarily mean the new technology will be rolled out to consumers, the nature of this particular innovation has created a new cause for concern.

The patent application describes a process where the smart speaker would record audio and detect pauses in between speech. The location of the pauses would be notated with a marker and the audio would also be scanned for the wake word.

If the wake word is detected, the system will check for a marker that precedes the wake word. It will take the recording from the marker before the wake word until the marker after the wake word and will send it to the cloud for processing.

With this process, the wake word wouldn’t necessarily have to begin each command. For example, instead of saying, “Alexa, play some music.”, the user could say, “Play some music, Alexa.”, resulting in the same outcome.

This flexibility could lead to a more seamless experience and a more natural style of communication with the smart speaker. That being said, it is a slippery slope.

Many feel that if smart speakers are allowed to record in this way, it could lead to a greater invasion of privacy as a larger portion of audio could be saved for processing.

How to Turn Off Smart Speaker Active Listening

There are a few simple steps you can take to protect your privacy and prevent smart speaker companies from listening to your audio recordings. Turning off the active listening of smart speakers is an easy way to make sure conversations don’t accidently get recorded.

If you’re currently using Alexa, Google Assistant, or Siri as your smart assistant, keep reading for instructions on how to turn off their listening capabilities.

Amazon

To disable active listening on the Amazon line of smart speakers, press the microphone mute button. It is normally located on top of the device and looks like a microphone with a slash through it. Once the light turns red, the microphone for the device is no longer active and can no longer listen for voice commands.

There is also a way to prevent Amazon from manually reviewing recordings and using them to improve Alexa. This is done through the Alexa app.

Open the Alexa app and look for the menu located in the top left corner of the screen. Select the Settings option from the menu. In Settings, select “Alexa Privacy”, then “Manage How Your Data Improves Alexa”.

Now, switch off the toggle next to “Help Improve Amazon Services and Develop New Features”. Then, switch off the toggle next to “Use Messages to Improve Transcriptions”.

Doing this will prevent Amazon from listening to your smart speaker voice recordings. However, this does come with a trade-off. Amazon claims that by disabling these 2 options, the device may become less responsive to voice commands and new features may not work properly.

Google

To disable active listening on Google Home smart speakers, press the mute microphone button on the back of the speaker. This will prevent the microphone from hearing voice commands.

The Google Home Mini has a mute switch on the side of the device that you can toggle on or off. And the Google Home Max has the mic on/off switch on the back of the device.

Google also makes it possible to prevent any of your Google Assistant recordings from being saved on their server.

Go to the Google Home app and open the Accounts tab. From there, select “Manage your Google Account”. Then choose “Manage your data & personalization”. Find and click on “Voice and Audio Activity”, then toggle its switch off.

Apple

Apple’s HomePod doesn’t have a physical button to mute the microphone, but you can still turn off active listening.

To do so, use the voice command: “Hey, Siri, stop listening”. The speaker will then be unresponsive to voice commands, but will still listen for the command to resume its service. You can turn active listening back on with the command: “Hey, Siri, start listening”.

You can also disable active listening through the Apple Home app.

Open the Home app, then tap on the Rooms tab at the bottom of the screen. Swipe to the room that contains the HomePod. Press and hold on the HomePod icon. In the details section, toggle the switch for the ‘Listen for Hey Siri’ option to off.

How to Delete Existing Smart Speaker Recordings

Stopping a smart speaker from listening does not delete recordings that have already been saved. There are a few more steps to take in order to remove those recordings. Again, the process is different for different speakers.

Amazon

Alexa users can review their smart speaker recordings and delete them as well. There are a few different methods to achieve this.

The first one involves using the app. In the Alexa app, open the menu and go to Settings. Then, go to “Alexa Privacy” and select “Review Voice History”. You can listen to and delete saved recordings there.

There is an option to “Delete All Recordings for Today”. The drop down menu for Date Range allows you to delete all recordings or recordings from a particular date range.

You can also delete recordings by voice command. Currently, two voice commands are available to perform this operation. “Alexa, delete what I just said.” will delete the last recorded voice command. And “Alexa, delete everything I said today.” will delete everything between 12am and right before the command was spoken.

However, you must opt in before you can use these voice commands to delete recordings. To do this, open up the Alexa app. In the menu, go to Settings. Then, go to “Alexa privacy” and select “Review Voice History”. You will see the option to “Enable deletion by voice”, click the toggle switch on.

Recordings can also be scheduled to be automatically deleted. In the Alexa app, open the menu and go to Settings. Then, go to “Alexa Privacy” and choose “Manage Your Alexa Data”.

You will see an option to “Automatically delete recordings”. Tap the “Off >” button. A menu will pop up where you can choose to manually delete recordings, keep the recordings for 18 months and delete older ones automatically, or keep the recordings for 3 months and delete older ones automatically. Choose one of these options, then click Confirm.

Google

You can delete Google Assistant voice recordings using a phone or a computer by going to My Google Activity. There you will find all of your Google activity, things such as what you have searched for, read, or watched.

To view your voice recordings, click on “Filter by Date” under the search box. Uncheck All Products, select Assistant and Voice & Audio, then click Apply. From there you can listen to and delete individual recordings.

To delete an individual recording click on the three-dot menu. This will show you details about each recording and an option to delete it. You can also delete a whole day’s worth of recordings by clicking the three-dot menu next to the date.

Apple

The process to delete Siri voice recordings is simple, but the instructions on how to do so are not completely spelled out and not so easily found.

On Apple’s “Our Approach to Privacy” page, in the Siri and Dictation section, you will find a sentence that describes how to delete user data: “When you turn Siri and Dictation off, Apple will delete the User Data associated with your Siri identifier, and the learning process will start all over again.”

So, to delete Siri voice recordings, you will have to repeat the following steps on every Apple device you own. Go to Settings, then Siri and Search. There, you should turn off “Listen for Hey Siri” and if you’re on an iPhone or iPad, turn off “Press for Siri”. Some devices will also show a single toggle for “Siri”.

Once you turn off all settings that activate Siri, a prompt will pop up with a disclaimer that reads, “The information Siri uses to respond to your requests will be removed from Apple servers. If you want to use Siri later, it will take some time to re-send this information.” Click on “Turn Siri Off” to complete the process.

To turn off Dictation on an iPad or iPhone, go to settings, then General, then Keyboards. Look for “Enable Dictation” and turn the toggle switch off. A similar message will appear to inform you that recordings will be deleted from the server: “The information Dictation uses to respond to your requests will be removed from Apple servers. If you want to use Dictation later, it will take time to re-send this information.”

Recommended Products