Scaling up search engine audits: Practical insights for algorithm auditing
Abstract: Algorithm audits have increased in recent years due to a growing need to independently assess the performance of automatically curated services that process, filter and rank the large and dynamic amount of information available on the Internet. Among several methodologies to perform such audits, virtual agents stand out because they offer the ability to perform systematic experiments, simulating human behaviour without the associated costs of recruiting participants. Motivated by the importance of research transparency and replicability of results, this article focuses on the challenges of such an approach. It provides methodological details, recommendations, lessons learned and limitations based on our experience of setting up experiments for eight search engines (including main, news, image and video sections) with hundreds of virtual agents placed in different regions. We demonstrate the successful performance of our research infrastructure across multiple data collections, with
- Standort
-
Deutsche Nationalbibliothek Frankfurt am Main
- Umfang
-
Online-Ressource
- Sprache
-
Englisch
- Anmerkungen
-
Veröffentlichungsversion
begutachtet (peer reviewed)
In: Journal of Information Science (2022) ; 1-16
- Ereignis
-
Veröffentlichung
- (wo)
-
Mannheim
- (wer)
-
SSOAR, GESIS – Leibniz-Institut für Sozialwissenschaften e.V.
- (wann)
-
2022
- Urheber
- DOI
-
10.1177/01655515221093029
- URN
-
urn:nbn:de:0168-ssoar-81272-3
- Rechteinformation
-
Open Access; Der Zugriff auf das Objekt ist unbeschränkt möglich.
- Letzte Aktualisierung
-
25.03.2025, 13:52 MEZ
Datenpartner
Deutsche Nationalbibliothek. Bei Fragen zum Objekt wenden Sie sich bitte an den Datenpartner.
Beteiligte
- Ulloa, Roberto
- Makhortykh, Mykola
- Urman, Aleksandra
- SSOAR, GESIS – Leibniz-Institut für Sozialwissenschaften e.V.
Entstanden
- 2022
Ähnliche Objekte (12)
Scaling up search engine audits : Practical insights for algorithm auditing
Erweiterung einer Workflow-Engine um Auditing
Computer auditing : auditing EDP systems, using the computer as an audit tool
Machine Learning for Health: Algorithm Auditing & Quality Control
Scaling a Cloud Based Analytics Engine
ICT in auditing: Impact of audit quality norms on interpersonal interactions
Alternate Scaling Algorithm for Biproportional Divisor Methods
Internal auditing and audit committees: Four essays on issues in research and practice
How do capital providers perceive current developments in accounting and auditing? : evidence from lease accounting, audit lags, and audit exclusions
Skalierbares Auditing
Umwelt-Auditing
Essays on auditing and accounting in the light of crises, scandals, and audit quality
Scaling up search engine audits : Practical insights for algorithm auditing
Erweiterung einer Workflow-Engine um Auditing
Computer auditing : auditing EDP systems, using the computer as an audit tool
Machine Learning for Health: Algorithm Auditing & Quality Control
Scaling a Cloud Based Analytics Engine
ICT in auditing: Impact of audit quality norms on interpersonal interactions
Alternate Scaling Algorithm for Biproportional Divisor Methods
Internal auditing and audit committees: Four essays on issues in research and practice
How do capital providers perceive current developments in accounting and auditing? : evidence from lease accounting, audit lags, and audit exclusions
Skalierbares Auditing
Umwelt-Auditing
Essays on auditing and accounting in the light of crises, scandals, and audit quality
Scaling up search engine audits : Practical insights for algorithm auditing
Erweiterung einer Workflow-Engine um Auditing
Computer auditing : auditing EDP systems, using the computer as an audit tool
Machine Learning for Health: Algorithm Auditing & Quality Control
Scaling a Cloud Based Analytics Engine
ICT in auditing: Impact of audit quality norms on interpersonal interactions
Alternate Scaling Algorithm for Biproportional Divisor Methods
Internal auditing and audit committees: Four essays on issues in research and practice
How do capital providers perceive current developments in accounting and auditing? : evidence from lease accounting, audit lags, and audit exclusions
Skalierbares Auditing
Umwelt-Auditing