Large language models (LLMs) are rapidly changing behavioral and social science: as tools for annotating and analyzing data at scale, as instruments for simulating human respondents, as sources of semantic representation and measurement, and as objects of scientific study in their own right. This course gives doctoral students hands-on experience in taking an LLM-based research idea from proposal to working prototype in an intensive three-day hackathon format.

Before the event, each participant develops a one-page project proposal that clearly specifies (1) a research question, ideally connected to the participant's own dissertation work, (2) the dataset(s) to be used, and (3) a motivation for why LLMs are the appropriate methodological tool for the problem, that is, what LLMs add over conventional approaches and what risks and limitations (e.g., validity, bias, contamination, reproducibility) need to be addressed.

The hackathon itself begins with a half-day introduction to the conceptual and technical foundations of working with LLMs in behavioral and social science: accessing models via APIs and open-weights frameworks, prompt design and structured output, embeddings, evaluation strategies, and methodological pitfalls. Participants then pitch their projects in short initial presentations. The remaining two and a half days are dedicated to intensive project work under the supervision of the instructor and with peer support and exchange among participants. The event concludes with final presentations in which participants report their results, the obstacles they encountered, and next steps toward turning the prototype into a research output.

After completing this course, students will be able to:

• identify use cases for LLMs in behavioral and social science and critically evaluate when LLM-based approaches are (and are not) methodologically warranted,
• formulate a well-scoped research proposal that connects a research question, a concrete dataset, and a justified analytical approach,
• implement an LLM-based research pipeline, including model access, prompt design, structured data extraction, and basic evaluation,
• communicate research plans and results effectively under time constraints, and give and incorporate constructive peer feedback in a fast-paced collaborative setting.

Attendance is mandatory on all three days of the block event, including:

• participation in the introductory sessions,
• presence and active engagement during the supervised project work,
• attendance of and participation in the discussion of other participants' presentations.

Absences are possible only in justified, documented exceptional cases and must be communicated to the instructor in advance where possible.

The course combines a half-day interactive lecture on the foundations of LLM-based research methods with project-based learning in a hackathon format. Participants work on their own individual project under the supervision of the instructor and benefit from informal exchange and peer feedback throughout the event. Group work (2–3 participants) is possible in justified cases upon prior approval by the instructor. Two rounds of presentations, an initial pitch and a final report, frame the project work. Participants receive brief written feedback on their proposal before the event.

• Project proposal (40%): A one-page proposal, submitted two weeks before the event, clearly outlining the research question, the dataset(s) to be used, and a motivation for the AI/LLM-based approach, including its anticipated advantages and limitations. The proposal must be the participant's own work; the use of AI writing tools is not permitted for this deliverable.
• Initial presentation (30%): A short pitch (approx. 3 minutes) on Day 1 presenting the research question, data, planned approach, and success criteria.
• Final presentation (30%): A presentation (approx. 7 minutes) on Day 3 reporting results, challenges encountered, deviations from the plan, and next steps.

All partial assessments must be completed for a positive grade.

Enrollment in a doctoral/PhD program. Basic programming skills (Python or R) are strongly recommended; no prior experience with LLMs is required. Participants must bring their own laptop; API access will be provided or arranged before the event.

Please log in with your WU account to use all functionalities of read!t. For off-campus access to our licensed electronic resources, remember to activate your VPN connection connection. In case you encounter any technical problems or have questions regarding read!t, please feel free to contact the library at readinglists@wu.ac.at.

During the event: before and after sessions and throughout the supervised work phases. Otherwise by email and by appointment.

Recommended preparatory reading:

• Hussain, Z., Binz, M., Mata, R., & Wulff, D. U. (2024). A tutorial on open-source large language models for behavioral science. Behavior Research Methods. https://doi.org/10.3758/s13428-024-02455-8 
• Wulff, D. U., & Mata, R. (in press). Addressing longstanding challenges in cognitive science with language models. Trends in Cognitive Sciences. https://doi.org/10.48550/arXiv.2511.00206 

Additional technical materials will be provided via the course platform before the event.