We are happy to announce that the following paper has been accepted at the IEEE ICDE 2025 conference, one of the premier venues for research in data management.
Experimental analysis of multi-step pipelines for fair classifications – More than the sum of their parts?
Nico Lässig and Melanie Herschel
This paper presents a deep experimental study of over 40 fair classification algorithms in settings where they are integrated into a data processing pipeline rather than being considered in isolation. Some interesting insights include: (1) Choosing a bias reducing algorithm greatly simplifies when implementing suited data preparation or parameter optimization, as the difference in performance between methods shrinks, making almost any choice a good one. (2) Several component or pipeline implementations often assumed to have positive or negative effects on performance prove to have little or even contrary effects to the expectations. (3) While many approaches have been published for fair classification in the last decade and shown to improve on previous solutions in specific settings, our broad analysis reveals a stagnating performance trend.
Our analysis shows that synergetic effects between pipeline components need to be carefully taken into account for further research on fair end-to-end data processing. It further raises the more fundamental question of how the study of the problem evolves, both in terms of proposed solutions and benchmarking.