LLMs for Psychological Digital Content Analysis (SoSe 2026)

Opal: Link here.

Time Thursdays, 5. DS

Location APB/E010

Department Computer Science

Modules INF-BAS3 (Software- und Web-Engineering)

Language English

Assessment oral exam

Description:

Digital communication produces vast amounts of content that reflect how people think, feel, and interact. Social media posts, online discussions, reviews, images, and other forms of digital media contain traces of psychological phenomena such as persuasion, identity expression, moral judgment, and emotions. Recent advances in large language models (LLMs) and multimodal models have opened new possibilities for analyzing such content at scale. However, meaningful analysis requires more than technical tools: psychological constructs must be clearly defined, operationalized, and measured. This seminar explores how psychological concepts can be systematically studied in digital content using LLM-based annotation workflows.

Throughout the course, students will learn how to translate abstract psychological constructs into measurable indicators in digital data, including both textual and visual content. The seminar introduces a full research pipeline for this task: identifying relevant constructs, operationalizing them as annotation tasks, implementing LLM-based annotation through prompt engineering, evaluating annotation quality using human feedback, and analyzing the resulting data statistically.

While LLMs will play a central role in the workflow, the course is not intended as a purely technical training in model usage. Instead, it focuses on a careful and scientifically grounded reflection on what it means to interpret digital content through psychological lenses and what methodological challenges arise when doing so.

The seminar is organized around an individual mini-project developed over the course of the semester. Each student will select a psychological construct and a dataset of digital content, design an LLM-based annotation approach, evaluate and refine the annotation process, and conduct a basic analysis of the resulting data. The final deliverables are a short project notebook documenting the annotation pipeline and analysis, and a brief presentation of the results during the final sessions.

The seminar assumes basic familiarity with Python. Example scripts and templates will be provided, but students are expected to write and adapt their own code. Guidance and support for setting up these environments will be provided during the course.

Schedule:

Week 1 Seminar Introduction: Capturing Psychological Phenomena in Digital Content (Lecture)

Week 2 Psychological Constructs and Digital Behavior (Lecture & Exercises)

Week 3 Operationalization and Measurement of Psychological Constructs (Lecture & Exercises)

Week 4 Project Design and Setup (Workshop)

Week 5 LLMs for Psychological Content Annotation (Lecture & Exercises)

Week 6 LLM Prompt Engineering I: Basic Techniques (Lecture & Exercises)

Week 7 LLM Prompt Engineering II: Advanced Prompt Design (Workshop)

Week 8 LLM Prompt Engineering III: Selected Constructs Implementation (Workshop)

Week 9 Human-in-the-Loop: Evaluating LLM Annotations (Lecture & Exercises)

Week 10 Refining Prompts After Human Evaluation (Workshop)

Week 11 Statistical Analysis of Annotation Data (Lecture & Exercises)

Week 12 Analyzing and Presenting LLM Annotation Results (Workshop)

Week 13 Project Presentations (Presentations)

Week 14 Final Discussion: Reflections, Feedback, and Q&A (Discussion)