Casual Physics Enjoyer

I'd like to thank Hana Chockler for some clarifying points on her paper.

In the previous post, I discussed an estimate for the dollar cost of discrimination from CV selection bias in Malaysia. In that post, I built an economic cost for current practices that racially discriminate in the Malaysian CV selection process. However, it is clear that this bias is not restricted to just human selection.

Since I like to write with the future in mind, it may not surprise my readers to know that there is a risk of discrimination from machine learning models in a wide range of contexts. These contexts include interviewing for jobs [2], and in ad selection. Currently, clear evidence is limited - other than just anecdotal examples of querying GPT3, and this is part of the problem. According to Forbes, “99% of Fortune 500 companies rely on the aid of talent-sifting software, and 55% of human resource leaders in the U.S. use predictive algorithms to support hiring.”, but there is nothing more granular than this article.

In this blog post, I wanted to discuss some points on an interesting paper by Hana Chockler and Joseph Halpern on testing for discrimination in these sorts of systems. I will then test to see if a simple toy decision tree can be deemed unfair on some loan data from the Deloitte loan dataset.

I read the paper "On Testing for Discrimination Using Causal Models", by Hana Chockler and Joseph Y. Halpern. In the paper, they outline definition for a "fair" piece of software that performs selection based on "sensitive" and "allowed" inputs, and then analyse a possible regulation scheme along with the computational complexity of certifying algorithms as fair.

" Given the sensitive and allowed variables, our notion of fairness then says, roughly speaking, that the bank’s software
is fair (i.e., acceptable to the regulator) provided that, if
we fix the values of the allowed variables, changing the values
of the sensitive variables has no impact on the outcome "

To make sense of this definition, let's consider a population of 10 people, each applying for a loan through a piece of AI software. The purpose of the AI would be to either grant or deny a loan. Let's suppose that the only inputs to the loan decision model are each applicant's gender, and income. Finally, assume for simplicity that the outcome of this application is either a success or fail.

For a single participant, the model might look like the mapping below. The model is the function f. The inputs would be the gender, encoded with either a 0 or 1, and the salary consists of the positive real numbers. The outcome would be represented by a 0 or a 1 for success or failure.

$$ f : \{ 0 , 1\} \times \mathbb R ^ + \to \{ 0 , 1 \} $$

In this case, a regulator might wish to control "sensitive variables" to ensure that they play no part in the decision-making process. On the other hand, they might have "allowed variables" that they let entities using in their decision process. Obvious sensitive values would be race, religion and gender, to name a few. On the other hand, something like salary would probably considered an "allowed variable" since there is a business justification.

So, suppose that we had values of (gender, salary) as inputs, with gender being sensitive and salary being allowed, both used to determine whether a loan application was a failure or was a success. A model would be unfair if changes in the sensitive variable lead to different outcomes, keeping all other variables fixed. In other words, in this base case, we would require

$$ f(0, x) = f(1, x) \text{ for all } x \in \mathbb R ^ + $$

Creating a toy example of fairness in a random forest

In the below, I'll test to see if simple machine learning models can be deemed "fair" or "unfair", by a similar metric suggested in the paper above. I will use the dataset issued publicly from Deloitte in their loan forecasting competition. To illustrate the concept of an unfair AI system, I will use a random forest model on this loan dataset, and then assess its fairness based on a variable that encodes the number of convictions of a loan applicant shown in the "Public Record" variable. The screenshot below is a sample of this dataset.

I will train this model on half the data naively using the just a few features to illustrate an example. On the other section of data, I will then use this model to make predictions.

Below are the details of my implementation:

• I've taken a publicly available loan dataset from Deloitte, and then trained a mildly above average loan default classifier. In this case, I had just above a 50% success rate of predicting correctly a loan. The accuracy of these models is for a different blog post. For now, I just wanted to illustrate an example of a metric of fairness.

• One of the input variables is instances of prior convictions in the "public record" column. In my opinion, this should be a sensitive variable, but even if people don't agree with this it feels like good example anyways.

• I then scramble this "Public Record" variable in a second pass and check how many predictions are different. This metric is what I call "fairness".

The result in this case, is that 98% of the time, do we see a model outputting the same result even if we change the "Public Record" variable. This is shown in the bottom line of output. This level of fairness is not perfect.

Update - 21st May 2023 - A regulatory framework?  

We still do not care enough about AI discrimination in models used to scan CVs in the job selection process, nor do we have rigorous methods of tracking how prevalent this practice is in developing countries. Whilst the rest of the world is still concerned about long-term existential risk from AI, there doesn't seem to be a prudent enough focus on the current risk of discrimination in selection algorithms.

A simple Google trends search yields the following plots for AI fairness and AI bias

There is currently some legislation below:

• In the state of Illinois, Artificial Intelligence Video Interview Act (HB 2557)

References

On Testing for Discrimination Using Causal Models. / Chockler, Hana; Halpern, Joseph Y. Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22). 2022.