Question 14/27 v4 lecture 5

What is evaluation bias?


When the common datasets used for evaluating research / tools themselves exhibit representation bias. When the benchmark that is widely used has bias, it is replicated at scale.

Relevant part of lecture

supplementary material

21 fairness definitions and their politics - a summary / talk by Arvind Narayanan