Racist tech and the AI Act discussed during Parliamentary round table
After a few delays, Parliament finally got around to organizing a round table on the opportunities and risks of AI. Two contributions stood out. Quirine Eijkman of the Netherlands Institute for Human Rights spoke about Robin Pocornie’s case against the discriminatory use of proctoring software by VU University Amsterdam. During Covid, the university made students use Proctorio when sitting for exams. One of those students, Pocornie, noticed the software often failed to properly "process" her face. As a result, Pocornie struggled to access her exams, and even resorted to sitting through them with a bright light shining in her face. Supported by our friends at the Racism and Technology Center"Dutch Institute for Human Rights: Use of anti-cheating software can be algorithmic discrimination (i.e. racist)", the VU-student filed a complaint with the Netherlands Institute for Human Rights. At the end of 2022, the Institute ruled she had presented sufficient facts for a presumption of discrimination, and it was now on VU University to prove otherwise. As public institutions are increasingly making use of algorithms, and governments around the world are deciding on the rules to govern them, it’s hard to overstate the importance of cases like this.
The second contribution came from our own Nadia Benaissa, and focused in particular on the opportunities for raising protections in the AI ActThe Dutch Senate shares the concerns we have about the Artificial Intelligence Act.. Benaissa relayed our position on the right to transparency, explainability and verifiability. She also pointed to how the current text falls short in safeguarding people’s right to effective legal protection. Finally, she reiterated the importance of a broad ban on biometric identification"A limited ban on biometric surveillance undermines its own potential": this was our contribution at the European Parliament., which includes not just identification taking place in "real time" and applies to private actors as well as enforcement agencies.