Create Targeted Minimal Pair "Stress-Tests" for Sensitivity to Social Groups

[epavlick]

Coordinate with Meg Mitchell about this

[XudongOliverShen]

Hi,

I had recently created gender sensitivity tests that cover binary and non-binary genders (in English and Chinese), and name sensitivity test that is geographically diverse.

Seems very related and would love to join efforts! I guess the contrast sets collected in above pointers can be combined with context sentences from established datasets such as CrowS-Pairs and StereoSet, to create stress-tests?

[mmitchellai]

This is great! We are working on exactly this within the Evaluation Bias Fairness Social Impact subgroup. Come join the channel and you can start iterating with us. =)