Monitoring algorithmic fairness under partial observations
Henzinger TA, Kueffner K, Mallik K. 2023. Monitoring algorithmic fairness under partial observations. 23rd International Conference on Runtime Verification. RV: Conference on Runtime Verification, LNCS, vol. 14245, 291–311.
Download (ext.)
https://doi.org/10.48550/arXiv.2308.00341
[Preprint]
Conference Paper
| Published
| English
Scopus indexed
Department
Series Title
LNCS
Abstract
As AI and machine-learned software are used increasingly for making decisions that affect humans, it is imperative that they remain fair and unbiased in their decisions. To complement design-time bias mitigation measures, runtime verification techniques have been introduced recently to monitor the algorithmic fairness of deployed systems. Previous monitoring techniques assume full observability of the states of the (unknown) monitored system. Moreover, they can monitor only fairness properties that are specified as arithmetic expressions over the probabilities of different events. In this work, we extend fairness monitoring to systems modeled as partially observed Markov chains (POMC), and to specifications containing arithmetic expressions over the expected values of numerical functions on event sequences. The only assumptions we make are that the underlying POMC is aperiodic and starts in the stationary distribution, with a bound on its mixing time being known. These assumptions enable us to estimate a given property for the entire distribution of possible executions of the monitored POMC, by observing only a single execution. Our monitors observe a long run of the system and, after each new observation, output updated PAC-estimates of how fair or biased the system is. The monitors are computationally lightweight and, using a prototype implementation, we demonstrate their effectiveness on several real-world examples.
Publishing Year
Date Published
2023-10-01
Proceedings Title
23rd International Conference on Runtime Verification
Publisher
Springer Nature
Acknowledgement
This work is supported by the European Research Council under Grant No.: ERC-2020-AdG 101020093.
Volume
14245
Page
291-311
Conference
RV: Conference on Runtime Verification
Conference Location
Thessaloniki, Greece
Conference Date
2023-10-03 – 2023-10-06
ISBN
ISSN
eISSN
IST-REx-ID
Cite this
Henzinger TA, Kueffner K, Mallik K. Monitoring algorithmic fairness under partial observations. In: 23rd International Conference on Runtime Verification. Vol 14245. Springer Nature; 2023:291-311. doi:10.1007/978-3-031-44267-4_15
Henzinger, T. A., Kueffner, K., & Mallik, K. (2023). Monitoring algorithmic fairness under partial observations. In 23rd International Conference on Runtime Verification (Vol. 14245, pp. 291–311). Thessaloniki, Greece: Springer Nature. https://doi.org/10.1007/978-3-031-44267-4_15
Henzinger, Thomas A, Konstantin Kueffner, and Kaushik Mallik. “Monitoring Algorithmic Fairness under Partial Observations.” In 23rd International Conference on Runtime Verification, 14245:291–311. Springer Nature, 2023. https://doi.org/10.1007/978-3-031-44267-4_15.
T. A. Henzinger, K. Kueffner, and K. Mallik, “Monitoring algorithmic fairness under partial observations,” in 23rd International Conference on Runtime Verification, Thessaloniki, Greece, 2023, vol. 14245, pp. 291–311.
Henzinger TA, Kueffner K, Mallik K. 2023. Monitoring algorithmic fairness under partial observations. 23rd International Conference on Runtime Verification. RV: Conference on Runtime Verification, LNCS, vol. 14245, 291–311.
Henzinger, Thomas A., et al. “Monitoring Algorithmic Fairness under Partial Observations.” 23rd International Conference on Runtime Verification, vol. 14245, Springer Nature, 2023, pp. 291–311, doi:10.1007/978-3-031-44267-4_15.
All files available under the following license(s):
Copyright Statement:
This Item is protected by copyright and/or related rights. [...]
Link(s) to Main File(s)
Access Level
Open Access
Export
Marked PublicationsOpen Data ISTA Research Explorer
Sources
arXiv 2308.00341