Professionalism/Thomas le Bonniec and Siri

Siri

Siri is a voice controlled personal assistant developed by Apple originally released in 2011. It is available on all Apple devices such as the iPhone, iPad, and MacOS systems. Siri uses natural learning processing and machine learning to understand and respond to user prompts.

Siri works by converting user voice commands into text using speech recognition. That text is then analyzed through natural language processing methods to understand user intent. That result is used with machine learning algorithms to generate a response from Siri.

In order to improve accuracy and performance, Apple must collect a large amount of user interactions with Siri to analyze. Each interaction must be human reviewed to determine if the correct response was given and what corrections need to be made. This large amount of work requires a large workforce which led to Apple originally outsourcing the work to contractors.

Thomas le Bonniec Background

Thomas le Bonniec completed a master's degree in Sociology from Paris-Sorbonne University in 2018. Post-graduation, Le Bonniec struggled to find a job, going eight months unemployed. Le Bonniec eventually received an offer through a jobs board he had subscribed to for a tech consulting GlobeTech.

During the recruitment process, Le Bonniec received very little information regarding the role. He knew that the salary was decent and the job required proficiency in French and willingness to do repetitive tasks.

Le Bonniec signed an non-disclosure agreement and accepted the offer. Two weeks later, he moved to Cork, Ireland to begin his training. Le Bonniec's first day on the job was May 13, 2019, and it was only then that he found out what his role would entail.

Le Bonniec's role was to review Siri audio recordings and check them against Siri's transcript. Le Bonniec would correct and mistakes and was expected to review 1,300 recordings a day. The job felt wrong to Le Bonniec as he listened in on recordings of "conversations people were having about their cancer treatments, intimate conversations with family and others about sex and relationships."

The Recording

Around three months on the job, Thomas Le Bonniec encountered a recording describing a sexual fantasy involving children. The recording disgusted Le Bonniec, and he flagged it to discuss with his manager. Management thanked him and directed another employee to listen to the audio in order to verify what Le Bonniec had said. Le Bonniec was upset by the recording and appalled that another employee would be subjected to hearing it.

Le Bonniec was directed to a helpline due to the distressing nature of the recording but received no real help. The helpline was primarily concerned with whether he was suicidal and could not offer expert assistance.

Reaction

Once Le Bonniec saw that his superiors were going to do nothing about the pedophilia recording, he started to take multiple screenshots of his work every day. At first, he was scared of taking screenshots since he thought Apple may be monitoring him after his report of the recording. However, he eventually realized how bad the security was at his current company. Additionally, he started to look at the company's Confluence pages to see what the developers thought about Siri and what the developers thought were sensitive topics.

A week after the recording, he decided to quit his job by going back to France permanently. The company tried to reach out about missing work multiple times through emails and calls. However, he never responded to any of these attempts. After four days, the company fired him for not showing up to work.

Whistleblowing

Reasons

At first, Le Bonniec was not going to come forward with his experience at Apple since his family and friends convinced him that Apply may sue him for breaching the NDA he signed. However, he finally decided to come forward for four different reasons:

• His disgust with the lack of action from superiors at Apple and his former company from stopping the disturbing content in the recordings
• His disgust with the invasion of privacy that Apple created with outsourcing the Siri recordings to contractors
• The possibility that Apple may be selling the data it gets from the Siri recordings to other third-party companies
• Gaining the courage to come forward due to other contractors coming forward earlier anonymously in the Guardian newspaper about their experience with Apple

Media

He decided to send an email to Mediapart, which is a French investigative online newspaper, and met with a journalist from Mediapart to discuss the information he had on Apple. On August 30, 2019, the journalist would publish the story with the information he provided anonymously.

Around November 2019, the same journalist from Mediapart would introduce Le Bonniec with a documentary filmmaker who wanted to interview him on camera. He was unsure of being interviewed but was reassured when suggested that they would film him in two different ways, one done remaining anonymous and the other showing his face. They gave him the choice to decide on which part would be released.

Around January 2020, he would decide to go public with his face since he decided that this was going to be something that he would dedicate himself to trying to get addressed. On February 2020, the interview would be released with his face publicized.

Government

On May 28 2020, he sent letters to the European National Data Protection Authorities, the European Data Protection Board, and the European Data Protection Supervisor. In addition, he sent multiple letters to European nations that are not a part of the European Union, such as Liechtenstein, Switzerland, Norway, and Iceland. In the letters, he stated that he was concerned with Apple “wiretapping entire populations despite European citizens being told the EU has one of the strongest data protection laws in the world”.

Apple Response

As the story caught the public eye, Apple was quick to suspend their review program to afford them time to consider next steps. Apple decided on the following policy changes before resuming their review program:

• By default, audio recordings will no longer be retained. However, transcripts will continue to be used in order to improve Siri.
• Users will be able to opt-in to allow for the use of audio recordings of their exchanges with Siri.
• Only Apple employees will review the audio recordings. Contractors will no longer be used for this purpose.
• Inadvertent triggers of Siri will be deleted upon recognition.

Additionally, users were now aware of the review program, whereas the program had been kept undisclosed prior.

As of September, 2022, users are also able to delete recordings previously made of them. This is done through random identifiers associated with individual devices not user accounts.

With the release of iOS 13 in September 2019, Apple introduced on-device processing which allows Apple to do some of the processing, such as speech recognition and natural language processing, on the user’s device, reducing the need for data to be transferred off the device. In iOS 14, updates to Siri allowed it to process certain requests completely on the device, improving response time and user data privacy at the same time.

Data Privacy in Big Tech

Cambridge Analytica

In 2018, it was revealed that political consulting firm Cambridge Analytica had obtained the data of nearly 30 million users in 2014. The data was used to create targeted political ads during the United States 2016 presidential election. The scandal resulted in increased criticism of Meta’s data privacy practices and has been a major talking point in the conversation of the need for better data privacy regulations.

Amazon

In 2019, a report by Bloomberg revealed that human review was being done on Amazon Alexa recordings similar to what Apple was doing with Siri. The recordings contained personal and sensitive information. This raised serious concerns regarding privacy as it was not made clear to users that their conversations were being recorded. As a result, Amazon updated its privacy policies to inform users that human reviewer’s may listen to user recordings and added the option to opt out of the program.

Google

In 2019, the National Commission on Informatics and Liberty fined Google 50 million Euros for violating the European Union’s General Data Protection Regulation for not obtaining informed consent from users before collecting their personal data. It was found that Google’s policy did not provide users with sufficient information about what the data would be used for. In response, Google modified their policy to make it more clear how they would use the data and provided users with a clearer way to access and manage their data.

TikTok

Leaked audio from 2022 has shown repeated cases of ByteDance employees in China accessing private US-based TikTok data which was claimed to have been kept separate from the rest of the data. There have been many attempts to restrict the use of TikTok in the United States such as the attempt by the Trump administration in 2020 and in February 2023, the White House declared that all United States federal agencies had 30 days to delete TikTok from all government-issued devices. As a result, TikTok CEO Shou Chew appeared before Congress to discuss United States data privacy and TikTok’s ties to China in 2023.

Relation to GDPR

The EU's GDPR provides a useful reference when considering the ethics of potential privacy violations. GDPR is primarily concerned with data that can identify the source user. Apple emphasizes that the data is anonymized through association with random tokens rather than user accounts. However, Apple does not address the possibility of identifiable information being in the audio. This fact, combined with the lack of public awareness, suggests an ethics problem.