The perceptron
Welcome to the perceptron, an early version of artificial neural networks, which are essential building blocks of machine learning and “artificial intelligence”.
In a documentary from the 1950s and 60s, we see scientists in a high-tech environment working on a perceptron. It is being trained to distinguish “males from females”.
We are told that this task is something all of us can do easily.
We then see the perceptron struggling to decide whether one of the members of the Beatles is a man or woman. His hair is long, by 60s standards. We see a similar struggle with a British barrister wearing the wig that is so iconic of his trade and culture.
However, despite these struggles, we are promised that through lots and lots of examples, the machine is capable of learning.
IMDB-WIKI Dataset
Welcome to the IMDB-WIKI Dataset from 2015, a large publicly available dataset of face images with gender and age labels for training.
In total, the dataset contains 460,723 face images from 20,284 celebrities from IMDb. Additionally, it contains 62,328 images from Wikipedia, making a total of 523,051 images.
5% of the celebrities have more than 100 photos. On average each celebrity has around 23 photos.
The dataset is accompanied by a deep learning solution called DEX, which is trained on the age labels in IMDB-WIKI.
We are told that age estimation from a single face image is an important task in human and computer vision which has many applications such as in forensics or social media.
We are told that age estimation is closely related to the prediction of other biometrics and facial attributes tasks such as gender, ethnicity, hair color and expressions.
DEX can also be used for gender estimation.
UTKFace
Welcome to the UTKFace dataset from 2017, a large-scale face dataset with long age span in a range from 0 to 116 years old.
The dataset consists of over 20,000 face images with annotations of age, gender, and ethnicity. The words ethnicity and race are used interchangeably.
We are told that we could use this dataset on a variety of tasks. For instance face detection, age estimation, etc. But we could probably also use it for race and gender estimation.
The age, race and gender labels for this dataset comes from the DEX algorithm, itself a result of training on the IMDB-WIKI dataset which we just became familiar with.
We are told that age is an integer from 0 to 116.
We are told that gender is either 0 for male or 1 for female.
We are told that race is an integer from 0 to 4, denoting White, Black, Asian, Indian and Others.
We are told that the race label “Others” might include Hispanic, Latino or Middle Eastern.
11k hands
Welcome to the 11k hands dataset, a collection of 11,076 hand images of varying ages between 18 - 75 years old. However, more than 90% of the hands are in their 20s.
There is a record of metadata associated with each image which includes: the subject ID, gender, age, skin color, and logical indicators referring to whether the hand image contains accessories, nail polish, or irregularities.
It is not mentioned on the website what constitutes an irregularity.
None of the images show male hands with nail polish. And less than 10% of the hands with accessories belong to men.
7% of the hands have dark skin. Less than 3% have very fair skin. 59% have what is called medium skin.
11k hands has images of 190 people in total. That is approximately 58 images per person.
Amazon Rekognition
Welcome to Amazon Rekognition, an API that can detect faces in images and videos. Amazon Rekognition also returns predictions for emotion, gender, age, face occlusion, eye gaze direction, and other attributes for each face.
A gender binary (male/female) prediction is based on the physical appearance of a face in a particular image. It doesn't indicate a person’s gender identity, and we are told we shouldn't use Amazon Rekognition to make such a determination.
We are told that Amazon doesn’t recommend using gender binary predictions to make decisions that impact an individual's rights, privacy, or access to services. We are not told how we are supposed to identify those decisions.
Similarly, we are told that a prediction of an emotional expression is based on the physical appearance of a person's face in an image. It doesn't indicate a person’s actual internal emotional state, and we are told we shouldn't use Amazon Rekognition to make such a determination.
A person pretending to have a happy face in a picture might look happy, but might not be experiencing happiness.
It is recommended that we use a threshold of 99% or more for use cases where the accuracy of classification could have any negative impact on the subjects of the images. We are not told how to identify those negative impacts.
face-api.js
Welcome to face-api.js, an open source JavaScript face recognition API for the browser.
face-api.js detects, recognizes and analyzes faces. It is also capable of estimating age, facial expressions and gender.
The age and gender recognition model has been trained and tested on the following databases: UTK, FGNET, Chalearn, Wiki, IMDB, CACD, MegaAge and MegaAge-Asian.
We are told that face-api.js has a total gender accuracy of 95%.
face-api.js has been favourited on Github more than 15,000 times and forked more than 3,500 times. It has 420 reported issues.
Gender shades
Welcome to the Gender Shades research project of 2017, which evaluates the accuracy of “AI” powered gender classification products.
Three companies - IBM, Microsoft, and Face++ - that offer gender classification products were chosen for the evaluation. The companies appear to have relatively high accuracy overall, though there are notable differences in the error rates between different groups.
All companies perform better on lighter subjects as a whole than on darker subjects.
When analyzing the results by intersectional subgroups - darker males, darker females, lighter males, lighter females - we see that all companies perform worst on darker females.
Of the three companies, Microsoft achieved the best score for darker females with 79.2% accuracy. Yet error analysis reveals that 93.6% of faces misgendered by Microsoft were those of darker subjects.
5 years later on June 21st 2022, Microsoft said that it will stop offering gender recognition features to new customers, while existing customers will have their access revoked on June 30th, 2023. As of August 24th, 2023, existing customers still have access.
The Misgendering Machine
And finally, welcome to The Misgendering Machine, a gender expression exploration experience. Running on face-api.js, the machine tries and fails to guess our gender. By following the instructions contained in the booklet, we are taken through a journey of exploring our own gender expression.
We learn how to use our hair, eyes, nose, head and facial expression to trick the algorithm. We find ourselves able to be placed in a different gender category than what the machine initially sees.
We are prompted to reflect on the misogynist, racist, transphobic and ableist dimensions inherent in these tricks. Why does smiling make you more female? Why does it make you more male to squint your eyes? Why does having long hair make you a woman?
The Misgendering Machine aims to question, rethink and criticize gender recognition technologies. By going through the experience we hopefully learn that gender recognition is not something all of us can do easily.