How is data bias a tool of oppression?

by Madeleine Berenyi

We are all familiar with algorithmic profiling – the use of personal data to advertise to us. How many times have you searched for something, only for an ad for that same thing to pop up on your social media minutes later? What you might be less familiar with is how big data strategies have consistently been used to rob people of political, financial and social power. 

For Black communities, data founded in implicit bias has historically been used to reinforce inequality, and in 2020 it has become a ‘high tech enterprise’. Data for Black Lives is an organization committed to using data science to create positive, measurable changes that dismantle long-held data-driven policies. Below are three examples of data used to systemically discriminate.

Redlining & Predatory Lending 

Redlining is the process by which financial institutions modify offerings based on where an applicant lives. Following segregation and since the creation of postal codes in the early 1900s, where an applicant lives has served as proxy for race and often impetus to offer bad rates or deny mortgages. FICO, a for-profit entity with no transparency into their algorithm, uses this data too. If you are a ’high risk candidate’ that lives in a ‘high risk area’ what effect does that have on your life trajectory? Can you qualify for a car loan or to rent an apartment? Building wealth becomes incredibly difficult. 

Algorithms cannot remove human bias if the data used to fuel them is made by humans.

The School-to-Prison Pipeline and ‘At-Risk’ Youth

A theory about a new wave of juvenile super predators (mostly young Black men from ‘bad’ neighborhoods) was popularized by John J. Delilio Jr. in the early 1990s. The juvenile super predator theory used a lot of emotional language and anecdotal evidence and failed to clearly cite data sources for its condemning conclusions. It stated that there was to be a huge wave of crime committed by these young people, and this theory ultimately lead to juvenile cases being treated with adult sentencing standards. This theory turned out to be wholly false, and yet its impacts are still influencing the justice system.

As researchers, it is our responsibility to be cognizant of bias in our research and our tools – and those that exist within our industry.

Predictive Policing & Risk-Based Sentencing 

These algorithms are based a combination of data points including geographic location and arrest data. If police focus attention on certain groups and certain neighborhoods, it is likely that police records will over‐represent those groups and neighborhoods, which will then be amplified by the algorithm. Police‐recorded data sets are rife with systematic bias, and algorithms cannot remove human bias if the data used to fuel them is made by humans. The New York Times recently reported an example of wrongful arrest because of an algorithm (https://www.nytimes.com/2020/06/24/technology/facial-recognition-arrest.html)

As researchers, it is our responsibility to be cognizant of bias in our research and our tools – and those that exist within our industry. Assumptions and biases about how people behave is at best, bad data, and at worst, life threatening.

Links

Data for Black Lives is a movement committed to the mission of using data science to create concrete and measurable change in the lives of Black people.

http://d4bl.org/about.html


Founder & Executive Director of Data for Black Lives, Yeshimabeit Milner, speaks about Abolishing Big Data:

https://www.youtube.com/watch?v=wR5i6qXJH4o