publications

2025

1. 

   Beyond Consistency: Nuanced Metrics for Individual Fairness

   Madeleine Waller, Odinaldo Rodrigues, and Oana Cocarascu

   Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency, FAccT 2025, Athens, Greece, June 23-26, 2025, 2025

   Abs DOI Bib PDF

   Individual fairness is the principle aiming for equitable treatment for each individual affected by decisions. Despite its intuitive appeal, the practical applications of individual fairness for algorithmic decision-making systems remain relatively unexplored. In this paper, we investigate the consistency score metric and demonstrate how it fails to adequately capture fairness at the individual level, underscoring the need for a more fine-grained approach. We show that (1) the consistency score obscures instances where individuals are treated significantly differently to the individuals most similar to them and (2) the perceived fairness of individual decisions can be affected by several factors, including the similarity notion itself. To address these issues, we propose four new metrics that measure different aspects of the treatment of individuals with respect to similar individuals, under varying similarity definitions. Our comprehensive evaluation of the new metrics shows that they offer a more nuanced approach to assessing individual fairness, enabling decision-makers to focus on individuals most adversely affected by controversial decisions.

   @article{wallerfacct,
     author = {Waller, Madeleine and Rodrigues, Odinaldo and Cocarascu, Oana},
     title = {Beyond Consistency: Nuanced Metrics for Individual Fairness},
     journal = {Proceedings of the 2025 {ACM} Conference on Fairness, Accountability,
                       and Transparency, FAccT 2025, Athens, Greece, June 23-26, 2025},
     pages = {2087--2097},
     publisher = {{ACM}},
     year = {2025},
     doi = {10.1145/3715275.3732141},
   }

2024

1. 

   Explaining the algorithm does not explain the decision: unpacking accountabilities in organisational decision making

   Paul Waller, Karen Yeung, and Madeleine Waller

   Available at: SSRN 4961460, 2024

   Abs Bib PDF

   An organisational decision-making process has many component parts (with or without the involvement of a computer-based algorithm). Many technical discussions such as on transparency and explainability of algorithm-supported decisions omit many of them and thus address issues about accountability, liability and explanations for decisions in too narrow a sense. There are many choices made in the construction and operation of an organisational decision-making process, particularly if an algorithm-based model is used as part of it. This may lead to a chain of accountabilities of persons who may be required to explain or justify choices made at any point. This paper unpacks the general architecture of organisational decision making and examines the location and role of one or more algorithmic components that may feature within it. It will identify the design choices involved in constructing a decision-making process and the corresponding responsibilities and accountabilities. Within those accountabilities, the differences between functional reasons, explanations and justifications will be explored together with the actors who may be responsible for providing them. A case study of a public sector algorithmic decision-making system illustrates how the architecture helps unpack the key issues to interrogate. Crucially, the architecture makes a clear distinction between the generation of a prediction by an algorithmic process and the execution of an organisation’s decision-making policy. In “automated decision-making”, it is the execution of an organisation’s decision-making policy that is automated. A computerised algorithm may or may not provide input to it.

   @article{waller2024explaining,
     title = {Explaining the algorithm does not explain the decision: unpacking accountabilities in organisational decision making},
     author = {Waller, Paul and Yeung, Karen and Waller, Madeleine},
     journal = {Available at: SSRN 4961460},
     howpublished = {Available at SSRN 4961460},
     year = {2024},
   }

2. 

   Bias Mitigation Methods: Applicability, Legality, and Recommendations for Development

   Madeleine Waller, Odinaldo Rodrigues, Michelle Seng Ah Lee, and 1 more author

   Journal of Artificial Intelligence Research, 2024

   Abs Bib PDF

   As algorithmic decision-making systems (ADMS) are increasingly deployed across various sectors, the importance of research on fairness in Artificial Intelligence (AI) continues to grow. In this paper we highlight a number of significant practical limitations and regulatory compliance issues associated with the application of existing bias mitigation methods to ADMS. We present an example of an algorithmic system used in recruitment to illustrate these limitations. Our analysis of existing methods indicates a pressing need for a change in the approach to the development of new methods. In order to address the limitations, we provide recommendations for key factors to consider in the development of new bias mitigation methods that aim to be effective in real-world scenarios and comply with legal requirements in the European Union, United Kingdom and United States, such as non-discrimination, data protection and sector-specific regulations. Further, we suggest a checklist relating to these recommendations that should be included with the development of new bias mitigation methods.

   @article{waller2024bias,
     title = {Bias Mitigation Methods: Applicability, Legality, and Recommendations for Development},
     author = {Waller, Madeleine and Rodrigues, Odinaldo and Lee, Michelle Seng Ah and Cocarascu, Oana},
     journal = {Journal of Artificial Intelligence Research},
     volume = {81},
     pages = {1043--1078},
     year = {2024},
   }

3. 

   Identifying Reasons for Bias: An Argumentation-Based Approach

   Madeleine Waller, Odinaldo Rodrigues, and Oana Cocarascu

   Proceedings of the AAAI Conference on Artificial Intelligence, 2024

   Abs DOI Bib PDF Video

   Test abstract

   @article{walleraaai,
     title = {Identifying Reasons for Bias: An Argumentation-Based Approach},
     volume = {38},
     doi = {https://doi.org/10.1609/aaai.v38i19.30165},
     number = {19},
     journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
     author = {Waller, Madeleine and Rodrigues, Odinaldo and Cocarascu, Oana},
     year = {2024},
     pages = {21664-21672},
   }

2023

1. 

   Bias Mitigation Methods for Binary Classification Decision-Making Systems: Survey and Recommendations

   Madeleine Waller, Odinaldo Rodrigues, and Oana Cocarascu

   CoRR, 2023

   Abs DOI Bib PDF

   Bias mitigation methods for binary classification decision-making systems have been widely researched due to the ever-growing importance of designing fair machine learning processes that are impartial and do not discriminate against individuals or groups based on protected personal characteristics. In this paper, we present a structured overview of the research landscape for bias mitigation methods, report on their benefits and limitations, and provide recommendations for the development of future bias mitigation methods for binary classification.

   @article{waller203bias,
     author = {Waller, Madeleine and Rodrigues, Odinaldo and Cocarascu, Oana},
     title = {Bias Mitigation Methods for Binary Classification Decision-Making
                       Systems: Survey and Recommendations},
     journal = {CoRR},
     volume = {abs/2305.20020},
     year = {2023},
     url = {https://doi.org/10.48550/arXiv.2305.20020},
     doi = {10.48550/arXiv.2305.20020},
     eprinttype = {arXiv},
     archiveprefix = {arXiv},
     timestamp = {Wed, 07 Jun 2023 17:14:28 +0200},
     biburl = {https://dblp.org/rec/journals/corr/abs-2305-20020.bib},
     bibsource = {dblp computer science bibliography, https://dblp.org},
   }

2. 

   Investigating the Legality of Bias Mitigation Methods in the United Kingdom

   Mackenzie Jorgensen, Madeleine Waller, Oana Cocarascu, and 4 more authors

   IEEE Technology and Society Magazine, 2023

   DOI Bib PDF

   @article{wallerjorgensen,
     author = {Jorgensen, Mackenzie and Waller, Madeleine and Cocarascu, Oana and Criado, Natalia and Rodrigues, Odinaldo and Such, Jose and Black, Elizabeth},
     journal = {IEEE Technology and Society Magazine},
     title = {Investigating the Legality of Bias Mitigation Methods in the {United Kingdom}},
     year = {2023},
     volume = {42},
     number = {4},
     pages = {87-94},
     doi = {10.1109/MTS.2023.3341465},
   }

2020

1. 

   Why Predictive Algorithms are So Risky for Public Sector Bodies

   Madeleine Waller and Paul Waller

   Available at SSRN 3716166, 2020

   Abs Bib PDF

   This paper collates multidisciplinary perspectives on the use of predictive analytics in government services. It moves away from the hyped narratives of “AI” or “digital”, and the broad usage of the notion of “ethics”, to focus on highlighting the possible risks of the use of prediction algorithms in public administration. Guidelines for AI use in public bodies are currently available, however there is little evidence these are being followed or that they are being written into new mandatory regulations. The use of algorithms is not just an issue of whether they are fair and safe to use, but whether they abide with the law and whether they actually work. Particularly in public services, there are many things to consider before implementing predictive analytics algorithms, as flawed use in this context can lead to harmful consequences for citizens, individually and collectively, and public sector workers. All stages of the implementation process of algorithms are discussed, from the specification of the problem and model design through to the context of their use and the outcomes. Evidence is drawn from case studies of use in child welfare services, the US Justice System and UK public examination grading in 2020. The paper argues that the risks and drawbacks of such technological approaches need to be more comprehensively understood, and testing done in the operational setting, before implementing them. The paper concludes that while algorithms may be useful in some contexts and help to solve problems, it seems those relating to predicting real life have a long way to go to being safe and trusted for use. As “ethics” are located in time, place and social norms, the authors suggest that in the context of public administration, laws on human rights, statutory administrative functions, and data protection — all within the principles of the rule of law — provide the basis for appraising the use of algorithms, with maladministration being the primary concern rather than a breach of “ethics”.

   @article{wallerWhyPredictiveAlgorithms2020,
     title = {Why Predictive Algorithms are So Risky for Public Sector Bodies},
     author = {Waller, Madeleine and Waller, Paul},
     url = {http://dx.doi.org/10.2139/ssrn.3716166},
     journal = {Available at SSRN 3716166},
     year = {2020},
   }