According to the contractors Motherboard spoke to, this has been going on since before Cortana arrived on the scene. Motherboard's sources said they listened to audio when Xbox could be controlled via voice commands with the Kinect system. Microsoft removed Cortana from the Xbox in July, but the voice assistant can control the console via the Cortana Android and iOS apps.
Like Apple, Amazon, Facebook and Google -- which have all been charged with listening to voice assistant queries -- Microsoft listened in on audio in an attempt to improve products and services. The company admits that it uses voice data to improve voice-enabled services In a statement (which can be read below), a Microsoft spokesperson told Engadget, "We've long been clear that we collect voice data to improve voice-enabled services and that this data is sometimes reviewed by vendors."
After recent news coverage and public outcry, Apple, Facebook and Google temporarily halted those practices. Amazon will let users opt out of having Alexa conversations reviewed by humans. Microsoft previously said it would continue listening to users but amended its privacy policies. In an updated statement, Microsoft said it stopped listening to voice content collected through Xbox for product improvement purposes, but it will occasionally listen in if it thinks there is a terms of service violation.
The full statement from Microsoft is below:
"We've long been clear that we collect voice data to improve voice-enabled services and that this data is sometimes reviewed by vendors. We've recently updated our privacy statement to add greater clarity that people sometimes review this data as part of the product improvement process. We always get customer permission before collecting voice data, we take steps to de-identify voice snippets being reviewed to protect people's privacy, and we require that handling of this data be held to the highest privacy standards in the law. At the same time, we're actively working on additional steps we can take to give customers more transparency and more control over how their data is used to improve products." – a Microsoft spokesperson
Update 8/22/2019 9:10AM ET: Microsoft provided an updated comment stating:
"We stopped reviewing any voice content taken through Xbox for product improvement purposes a number of months ago, as we no longer felt it was necessary, and we have no plans to re-start those reviews. We occasionally review a low volume of voice recordings sent from one Xbox user to another when there are reports that a recording violated our terms of service and we need to investigate. This is done to keep the Xbox community safe and is clearly stated in our Xbox terms of service." – a Microsoft spokesperson