Decisions based on machine learning (ML) are potentially advantageous over human decisions, as they do not suffer from the same subjectivity, and can be more accurate and easier to analyse. At the same time, data used to train ML systems often contain human and societal biases that can lead to harmful decisions: extensive evidence in areas such as hiring, criminal justice, surveillance, and healthcare suggests that ML decision systems can treat individuals unfavorably (unfairly) on the basis of characteristics such as race, gender, disabilities, and sexual orientation – referred to as sensitive attributes.
Currently, most fairness criteria used for evaluating and designing ML decision systems focus on the relationships between the sensitive attribute and the system output. However, the training data can display different patterns of unfairness depending on how and why the sensitive attribute influences other variables. Using such criteria without fully accounting for this could be problematic: it could, for example, lead to the erroneous conclusion that a model exhibiting harmful biases is fair and, vice-versa, that a model exhibiting harmless biases is unfair. The development of technical solutions to fairness also requires considering the different, potentially intricate, ways in which unfairness can appear in the data.
Understanding how and why a sensitive attribute influences other variables in a dataset can be a challenging task, requiring both a technical and sociological analysis. The visual, yet mathematically precise, framework of Causal Bayesian networks (CBNs) represents a flexible useful tool in this respect as it can be used to formalize, measure, and deal with different unfairness scenarios underlying a dataset. A CBN (Figure 1) is a graph formed by nodes representing random variables, connected by links denoting causal influence. By defining unfairness as the presence of a harmful influence from the sensitive attribute in the graph, CBNs provide us with a simple and intuitive visual tool for describing different possible unfairness scenarios underlying a dataset. In addition, CBNs provide us with a powerful quantitative tool to measure unfairness in a dataset and to help researchers develop techniques for addressing it.
Causal Bayesian Networks as a Visual Tool
Characterising patterns of unfairness underlying a dataset
Consider a hypothetical college admission example (inspired by the Berkeley case) in which applicants are admitted based on qualifications Q, choice of department D, and gender G; and in which female applicants apply more often to certain departments (for simplicity’s sake, we consider gender as binary, but this is not a necessary restriction imposed by the framework).
Source: https://deepmind.com/blog/article/Causal_Bayesian_Networks
