Lentis/Algorithmic bias by gender

Algorithmic bias is any program that is designed to learn from large amounts of data to produce predictions on new data. Examples include facial recognition, self-driving cars, and resume analyzers.

Gender Bias in AI

Algorithmic bias occurs when an algorithm produces “systematically unfair” outcomes due to biases encoded in data . Gender bias often appears in male-dominated fields. Medical data often lacks female participants, leading to potentially inaccurate diagnoses and treatment suggestions. Hiring algorithms for computer science tend to favor males due to the imbalance of males in the field. These algorithms become self-reinforcing. Decisions due to the algorithm’s bias create data that enforces the bias for future generations of the algorithm.

Sources of Bias

AI training data can include biased human decisions or reflect historical and social inequalities. These human biases often form the basis of all algorithmic biases. This bias can result in incorrect decision-making. For instance, researchers at MIT showed that facial analysis technologies have higher error rates for minorities and women. This was due to unrepresentative training data: most of the training images were white males.

Another form of bias comes from intentionally encoded bias. Natural Language Processing models “word embeddings” are used to process text. A word embedding is a mechanism of identifying hidden patterns in the words of a given block of text, considering language statistics, grammatical and semantic information, and human-like biases. This intentional accounting of human-like bias can increase the accuracy of these models, as human writing often includes human biases, but it can have negative consequences in applications that make use of the models.

Bias in Generative AI

Most generative AI models are trained on data scraped from the internet. But the information available on the internet is not truly representative and is full of biases. Unless companies take active measures to curate their training data, the AI models they create will perpetuate and exacerbate biases. Machine learning is a field dominated by white men who frequently do not see the importance of ensuring their models are representative. Women only make up 20% of technical employees in machine learning.

Unless purposefully removed, there is a lot of sexual and pornographic material of women included in these training sets. One reporter tested a model that creates avatars based on headshots. She received hypersexual images while her male colleagues’ avatars were of astronauts or explorers. Of the 100 generated avatars, 16 were fully nude and 14 were scantily clad or posed suggestively. Additionally, “of 15,000 deepfake videos it found online, an astonishing 96 percent were non-consensual pornographic content featuring the swapped-in faces of women”.

Models trained on biased data will perpetuate the stereotypes it learns. Bloomberg used Stable Diffusion to generate images for 7 ‘high paying’ jobs, 7 ‘low paying’ jobs, and three types of criminals. They found that the model made gender and race disparities worse. For example, 34% of judges in the United States are women, but only about 3% of the images generated with the key word “judge” were of women.

Generative AI companies are aware of this disparity and some companies have made attempts to mitigate it. DALL-E 2 and DALL-E 3 adds words to user’s prompts to add diversity. For example, the model would add words like ‘woman’ or ‘black’ to user prompts (without the user knowing) in order to produce more diverse images. This solution does not attempt to fix the underlying biased training data and instead focuses on debiasing the output.

Case study

In 2018, Amazon stopped using a hiring algorithm after discovering that it was biased towards selecting male applicants. This came from a bias in NLP; however, it was not due to intentionally encoded biases. Rather, most of the resumes the model trained on came from males, as tech is a male-dominated field. This led the algorithm to prefer words that males typically used. By not ensuring equal gender representation in the data, Amazon created an AI that further reinforced the gender imbalance in their field.

Social Perspectives

The effort to eradicate gender bias in AI algorithms is fought by many different social groups, all with their own approaches and efforts.

Corporations

Tech companies such as Google, Apple, and Facebook make most of the progress on the latest machine learning models which are then open-sourced for the world to use. When these models have bias, that bias propagates to many applications. For example, Google’s Switch Transformer and T5 models, which are used by other businesses for NLP applications, were shown to have “been extensively filtered” to remove black and Hispanic authors, as well as materials related to gay, lesbian, and other minority identities. This affected countless NLP applications without the authors of the applications being aware. Activists argue that a few engineers at the largest tech firms bear the responsibility for building unbiased AI, not the broader public. Corporations often publicly commit to addressing algorithmic bias. Algorithmic bias negatively affects AI performance, and these companies recognize the necessity of mitigating this impact.

Feminist Data Manifest-No

The Feminist Data Manifest-No is a manifesto committed to the cause of “Data Feminism”. This movement aims to change how data is collected, used, and understood to better line up with the ideals of gender equality espoused in modern day feminism.

Manifest-No consists of a series of points broken into a refusal and a commitment. The refusal consists of a rejection of a currently established idea about data and algorithms prefaced with the words “we refuse.” The commitment then offers a rebuttal to that idea in the form of a new standard embraced by the authors of the paper, prefaced with the words “we commit”. The language creates a sense of community within the manifesto and enables the authors to clearly lay out both their understanding of the current world and their vision for a better one. This Manifest-No, while not always explicitly used or cited, forms a basis for many of the modern approaches to combating gender bias in algorithms.

Berkeley Haas Center for Equity, Gender, and Leadership

The Berkeley Haas Center for Equity, Gender, and Leadership has written a large playbook for business leaders to mitigate algorithmic bias within their companies. This playbook consists of overviews and deep dives into the issue of bias in AI algorithms and how businesses can address the issues. The existence of this playbook indicates the belief of its authors that effective mitigation of this issue must come from the top down. However, the authors understand that effective incentives for change can be provided by grassroots organizers, even when these organizers often lack the power necessary to actually implement the change.

To this end, the Berkeley Haas Center for Equity, Gender, and Leadership maintains a list of gender-biased algorithms that have been used by different companies over the years. Recognizing the negative press associated with using biased algorithms, the use of this list aims to encourage the correction and avoidance of bias by making it public when it occurs.

Conclusion and further steps

The tech companies and organizations should make sure that researchers and engineers who collect data and architect AI algorithms should be aware of potential bias against minorities and underrepresented groups. Some companies seem to be taking steps in this direction. For example, Google has published a set of guidelines for AI use both internally and for businesses that use its AI infrastructure.

It is imperative to study the anticipated and unforeseen consequences of using artificial intelligence algorithms in a timely manner, especially as current government policies may not be sufficient to identify, mitigate and eliminate the consequences of such subtle bias in legal relations. Solving algorithmic bias problems solely by technical means will not lead to the desired results. The world community thought about the introduction of standardization and the development of ethical principles to establish a framework for the equitable use of artificial intelligence in decision-making. It is necessary to create special rules that set limits for algorithmic bias in all sectors.