Arbeitspapier

Content analysis with language models: Exploring a zero-shot learning approach

In this paper, we discuss whether and how Natural Language Processing (NLP) can be integrated into the workflows of conventional qualitative research to support the researcher. We focus on textual data which the researcher wants to analyse using predefined categories. We explore the possibility of car- rying out the coding step using NLP instead of assigning codes manually. Integrating such models into the qualitative research process makes the research more accessible and comprehensible for third parties and can contribute to moving research more into the direction of open science. Our study in- dicates that the procedure is potentially able to identify the core results. However, the findings also indicate that there are weaknesses which strongly depend on the specific text and the research ques- tion under investigation. We find that off-the-shelf language models are not able to distinguish be- tween related topics as clearly as humans can. Moreover, humans are able to understand and classify passages which only implicitly refer to a topic. Findings suggest that off-the-shelf language models frequently fail to identify such passages. We conclude that an importantstep to reliably apply language models in qualitative research consists of improving the models. Nevertheless, in their current state, off-the shelf language models can be used to validate the results obtained by manual coding. This could make qualitative research more traceable and fully reproducible.

Sprache
Englisch

Erschienen in
Series: IAW Diskussionspapiere ; No. 143

Klassifikation
Wirtschaft
Thema
Inhaltsanalyse
Text

Ereignis
Geistige Schöpfung
(wer)
Kugler, Philipp
Bodry, Yvette
Kalweit, René
Koch, Andreas
Reiner, Marcel
Scheu, Tobias
Ereignis
Veröffentlichung
(wer)
Institut für Angewandte Wirtschaftsforschung (IAW)
(wo)
Tübingen
(wann)
2023

Handle
Letzte Aktualisierung
10.03.2025, 11:44 MEZ

Datenpartner

Dieses Objekt wird bereitgestellt von:
ZBW - Deutsche Zentralbibliothek für Wirtschaftswissenschaften - Leibniz-Informationszentrum Wirtschaft. Bei Fragen zum Objekt wenden Sie sich bitte an den Datenpartner.

Objekttyp

  • Arbeitspapier

Beteiligte

  • Kugler, Philipp
  • Bodry, Yvette
  • Kalweit, René
  • Koch, Andreas
  • Reiner, Marcel
  • Scheu, Tobias
  • Institut für Angewandte Wirtschaftsforschung (IAW)

Entstanden

  • 2023

Ähnliche Objekte (12)