Bias in Artificial Intelligence Effort

Video tutorials and Talks

• “Tutorial on Fairness Accountability Transparency and Ethics in Computer Vision at CVPR 2020” by Timnit Gebru & Emily Denton. Link
  • Goal: “From law enforcement, to border control, to employment, healthcare diagnostics, and assigning trust scores, computer vision systems have started to be used in all aspects of society. Seminal works showed that commercial gender classification systems have high disparities in error rates by skin-type and gender. We believe the vision community is well positioned to foster serious conversations about the ethical considerations of some of the current use cases of computer vision technology. Our workshop also seeks to highlight research on uncovering and mitigating issues of unfair bias and historical discrimination that trained machine learning models learn to mimic and propagate.”
  • Content:
    • Part 1: “Computer vision in practice: who is benefiting and who is being harmed?”
    • Part 2: “Data ethics”
    • Part 3: “Towards more socially responsible and ethics-informed research practices”
•  “2020 Vision: Reimagining the Default Settings of Technology & Society” by Ruha Benjamin. Link.
  • Goal: “I explore a range of discriminatory designs that encode inequity: by explicitly amplifying racial hierarchies, by ignoring but thereby replicating social divisions, or by aiming to fix racial bias but ultimately doing quite the opposite. This presentation takes us into the world of biased bots, altruistic algorithms, and their many entanglements, and provides conceptual tools to decode tech promises with sociologically informed skepticism. In doing so, it challenges us to question not only the technologies we are sold, but also the ones we manufacture ourselves.”

Research articles

• “Datasheets for Datasets” by Timnit Gebru et al. Link.
  • Goal: “The machine learning community currently has no standardized process for documenting datasets, which can lead to severe consequences in high-stakes domains. We propose that every dataset be accompanied with a datasheet that documents its motivation, composition, collection process, recommended uses, and so on.”
• “Model Cards for Model Reporting” by Margaret Mitchell et al. Link.
  • Goal: “Trained machine learning models are increasingly used to perform high-impact tasks in areas such as law enforcement, medicine, education, and employment. In order to clarify the intended use cases of machine learning models and minimize their usage in contexts for which they are not well suited, we recommend that released models be accompanied by documentation detailing their performance characteristics.”
  • Content: “We propose a framework that we call model cards, to encourage such transparent model reporting. To solidify the concept, we provide cards for two supervised models: One trained to detect smiling faces in images, and one trained to detect toxic comments in text.”
• “Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification” by Joy Buolamwini, Timnit Gebru. Link.
  • Goal: “Recent studies demonstrate that machine learning algorithms can discriminate based on classes like race and gender. In this work, we present an approach to evaluate bias present in automated facial analysis algorithms and datasets with respect to phenotypic subgroups.”
• “Eye in the Sky: Real-time Drone Surveillance System (DSS) for Violent Individuals Identification using ScatterNet Hybrid Deep Learning Network” by Singh, Amarjot at al. Link.
  • Goal: “Drone systems have been deployed by various law enforcement agencies to monitor hostiles, spy on foreign drug cartels, conduct border control operations, etc. This paper introduces a real-time drone surveillance system to identify violent individuals in public areas.”

News articles

• “IBM walked away from facial recognition. What about Amazon and Microsoft?” by Khari Johnson. Source: venturebeat.com
• “AI Weekly: A deep learning pioneer’s teachable moment on AI bias.” by Khari Johnson. Source: venturebeat.com
• “Inside China’s Dystopian Dreams: A.I., Shame and Lots of Cameras.” by Paul Mozur. Source: New York Times
• “The perpetual line-up unregulated police face recognition in america” by Clare Garvie et al. Source: Georgetown Law
• “Ibm Used NYPD Surveillance Footage to Develop Technology that Lets Police Search by Skin Color” by George Joseph and Kenneth Lipp. Source: The Intercept

Talks

• “Machine Learning, Ethics, and Fairness” by Solon Barocas, Foster Provost.  Link.
  • Content: Machine learning techniques have been embraced in regulated domains such as employment, credit, and insurance precisely because they promise to improve the consistency and quality of decision-making. Yet there is growing recognition that learning models from historical data can end up replicating the human biases they promised to stamp out. Less well understood, however, are the many ways that machine learning figures into the far more quotidian business decisions that do not fall under any regulation, but nevertheless raise concerns with fairness, ranging from marketing and advertising to information retrieval and personalization. In this talk, Solon will offer a survey of the wide range of fairness concerns prompted by businesses’ embrace of machine learning.
• “Are robots racists?” by Ruha Benjamin.  Link.
  • Content:
    • Racism is productive: it constructs.
    • Race and technology are coproduced.
    • Imagination is a battlefield.

Slides from presentations

• “Fairness in Machine Learning” by Delip Rao.  Link.
  • Content: How could an app that helps you report potholes cause harm? And when you identify some kind of unfair bias, how do you fix it? Slides from a talk describing how discrimination of a protected class can easily arise from unintended ways of performing classification tasks or gathering informations. Introduces the concept of “Fairness constraints” for classifiers. Great for teaching in class!

Tutorials

• “Responsible AI Practices” by Google.  Link.
  • Goal: “Reliable, effective user-centered AI systems should be designed following general best practices for software systems, together with practices that address considerations unique to machine learning. Our top recommendations are outlined below, with additional resources for further reading.”

Books

• “Fairness in machine learning” by Solon Barocas, Moritz Hardt, Arvind Narayanan.  Link.
  • Content: “This book gives a perspective on machine learning that treats fairness as a central concern rather than an afterthought. We’ll review the practice of machine learning in a way that highlights ethical challenges. We’ll then discuss approaches to mitigate these problems. We’ve aimed to make the book as broadly accessible as we could, while preserving technical rigor and confronting difficult moral questions that arise in algorithmic decision making.”
• “Ethics and Data Science” by Mike Loukides, Hilary Mason, DJ Patil.  Link.
  • Content: 40 page short book by the former Obama era US Chief Data Scientist urges putting ethics issues to practical work in projects, prefers short “checklists” to generic oaths, and lays out a brief road map of levels of concern.

Research articles 

• “Race and Gender” by Timnit Gebru.  Link.
  • Goal: “While many technical solutions have been proposed to alleviate bias in machine learning systems, we have to take a holistic and multifaceted approach. This includes standardization bodies determining what types of systems can be used in which scenarios, making sure that automated decision tools are created by people from diverse backgrounds, and understanding the historical and political factors that disadvantage certain groups who are subjected to these tools.”
  • Content: Extensive overview of the origin of racial and gender bias in computer vision AI and ways to fix it going forward.
• “Combating Anti-Blackness in the AI Community” by Devin Guillory. Link.
  • Goal: “In response to a national and international awakening on the issues of anti-Blackness and systemic discrimination, we have penned this piece to serve as a resource for allies in the AI community who are wondering howthey can more effectively engage with dismantling racist systems. This work aims to help elucidate areas where the AI community actively and passively contributes to anti-Blackness and offers actionable items on ways to reduce harm.”
• “Co-Designing Checklists to Understand Organizational Challenges and Opportunities around Fairness in AI” by Michael A. Madaio, L. Stark, H. Wallach. Link.
  •  Goal: “To understand the role of checklists in AI ethics, we conducted an iterative co-design process with 48 practitioners, focusing on fairness. We co-designed an AI fairness checklist and identified desiderata and concerns for AI fairness checklists in general. We found that AI fairness checklists could provide organizational infrastructure for formalizing ad-hoc processes and empowering individual advocates. We highlight aspects of organizational culture that may impact the efficacy of AI fairness checklists, and suggest future design directions.”
• “The Measure and Mismeasure of Fairness:A Critical Review of Fair Machine Learning” by Sam Corbett-Davies and Sharad Goel. Link.
  • Goal: “All existing fairness definitions suffer from significant statistical limitations.  In contrast to these formal fairness criteria, we argue that it is often preferableto treat similarly risky people similarly,  based on the most statistically accurate estimates ofrisk that one can produce. By highlighting these chal-lenges in the foundation of fair machine learning, we hope to help researchers and practitioners productively advance the area.”
• “50 years of test (un) fairness: Lessons for machine learning” by Hutchinson, B., & Mitchell, M. Link.
• “ Fairness in machine learning: Lessons from political philosophy” by Binns, R. Link.

News articles

• “Google Will Not Renew Pentagon Contract That Upset Employees” by Daisuke Wakabayashi and Scott Shane. Source: New York Times
• “Microsoft Employees Question C.E.O. Over Company’s Contract With ICE” by Sheera Frenkel. Source: New York Times
• “Microsoft researchers create AI ethics checklist with ML practitioners from a dozen tech companies” by Khari Johnson. Source: Venturebeat

Research Articles

• “Debiasing Embeddings for Reduced Gender Bias in Text Classification” by Flavien Prost et al. Link.
• • Goal: “Trained machine learning models are increasingly used to perform high-impact tasks in areas such as law enforcement, medicine, education, and employment. In order to clarify the intended use cases of machine learning models and minimize their usage in contexts for which they are not well suited, we recommend that released models be accompanied by documentation detailing their performance characteristics.”
  • Content: “Pretrained word embeddings can inherit gender bias from the data they were trained on. We investigate how this bias affects downstream classification tasks, using the case study of occupation classification. We show how these same techniques can be used to simultaneously reduce bias and maintain high classification accuracy.”

News article

• “Bias in Word Embeddings” by Dharti Dhami. Source: medium.com
• “An A.I. Training Tool Has Been Passing Its Bias to Algorithms for Almost Two Decades” by Hayden Field.” Source: medium.com

• Resistance AI workshop @ NeurIPS: Link.
  • Goal: “The goal of the Resistance AI Workshop is to examine how AI shifts power and how we can build human/AI systems that shift power to the people.
  • Content: “It has become increasingly clear in recent years that AI research, far from producing neutral tools, has been concentrating power in the hands of governments and companies and away from marginalized communities. Unfortunately, the NeurIPS Conference — one of the largest and most esteemed machine learning conferences in the world —  has until now lacked a venue explicitly dedicated to understanding and addressing this concerning reality. As Black feminist scholar Angela Davis famously said, “Radical simply means grasping things at the root.” Resistance AI exposes the root of the current reality: technology rearranges power. We believe that when we are engaged in Resistance AI, we can both resist AI that centralizes power in the hands of the few and we can dream up and build human/AI systems that put power in the hands of the people. This workshop will be a space for AI researchers and marginalized communities to discuss and reflect on AI-fueled inequity and co-create our dreams and tactics of how to work toward Resistance AI.”

The material includes articles accepted to the workshop, talks and panel discussions, from leading AI researchers.

Seminar Series

• “Data ethics conversation series 2020”  Link to the Program Schedule.
• • Content: Monthly webinar-style conversation series on Data Ethics.
  • Organizer: Ramon Alvarado, Department of Philosophy and Data Science Initiative @ UO.

Courses

• PHIL123 Internet, Society, & Philosophy Link.
  • Content: Introduction to philosophical problems of the Internet. Primary focus on social, political, and ethical issues with discussion of epistemological and metaphysical topics.
  • Instructor: Ramon Alvarado, Department of Philosophy and Data Science Initiative @ UO.
• PHIL223 Internet, Society, & Philosophy Link.
  • Content:
  • Instructor: Ramon Alvarado, Department of Philosophy and Data Science Initiative @ UO.

• “Virtual Event Awards,” Codes for Science & Society. Link.
  • Aim: Grants for virtual events focused on improving or connecting research-driven data science tools, practices, and the communities themselves.