Artikel

Why Organization Matters in “Algorithmic Discrimination”

Research into “algorithmic discrimination” has largely dismissed the fact that algorithms are often developed and used by organizations. In this article, we show that organizational sociology can contribute to a more nuanced perspective on “algorithmic decision-making.” Drawing on the concept of decision premises, we differentiate between various formal structures, particularly between different decision programs (conditional and purposive). This allows us to challenge two key assumptions, namely that human decision-makers rely heavily on algorithmically generated recommendations and that discrimination against protected groups needs to be solved mainly at the level of code. We identify the usefulness of distinguishing between conditional and purposive decision programs via a case study centered on the legal context: the risk assessment software “Correctional Offender Management Profiling for Alternative Sanctions” (COMPAS) that is employed in the US criminal justice system to inform judicial personnel about the recidivism risk of defendants. By analyzing the organizational structures, according to which the COMPAS score is formally and informally embedded in courts, we point out that the score represents an ambiguous and redundant information source for judges. The practice of minimizing the relevance of the score and decoupling it from the legal reasoning backstage particularly reflects the professional decision autonomy of judges, which is inherent in the legal system. The core finding of our approach is that strategies to reduce discrimination should not only scrutinize data quality or the statistical model but also consider the specific forms, functions, and consequences of the organizational structures that condition the ways in which discriminatory differences may or may not be (re)produced.

Weitere Titel
Warum Organisationen einen Unterschied bei „algorithmischer Diskriminierung“ machen
Sprache
Englisch

Erschienen in
Journal: KZfSS - Kölner Zeitschrift für Soziologie und Sozialpsychologie ; ISSN: 1861-891X ; Volume: 74 ; Year: 2022 ; Issue: S1 ; Pages: 307-330 ; Berlin: Springer

Klassifikation
Sozialwissenschaften, Soziologie, Anthropologie
Thema
organization
discrimination
automated decision making (ADM)
legal system
COMPAS
court
predictive analytics
risk assessment
Organisation
Diskriminierung
Rechtssystem
Gericht
prädiktive Analytik
Risikobewertung

Ereignis
Geistige Schöpfung
(wer)
Schwarting, Rena
Ulbricht, Lena
Ereignis
Veröffentlichung
(wer)
Springer
(wo)
Berlin
(wann)
2022

DOI
doi:10.1007/s11577-022-00838-3
Handle
Letzte Aktualisierung
10.03.2025, 11:42 MEZ

Datenpartner

Dieses Objekt wird bereitgestellt von:
ZBW - Deutsche Zentralbibliothek für Wirtschaftswissenschaften - Leibniz-Informationszentrum Wirtschaft. Bei Fragen zum Objekt wenden Sie sich bitte an den Datenpartner.

Objekttyp

  • Artikel

Beteiligte

  • Schwarting, Rena
  • Ulbricht, Lena
  • Springer

Entstanden

  • 2022

Ähnliche Objekte (12)