Deep Learning with Humans-In-The-Loop: Active Learning for NLP (2025)

With Lukas Rauch
Kassel, June 26th 2025

Deadline to apply: May 14, 2025
Please note the information on the application process below before applying.

The abundance of text data and the advent of powerful deep learning models has led to rapid advancements in Natural Language Processing (NLP). However, adapting models to specialized tasks still requires a nuanced understanding of the data.

This adaption often requires human-annotated data that are considered to be time-consuming and labor-intensive. Active Learning addresses this by strategically involving humans in the learning loop, selecting instances for annotation that are most likely to maximize performance gains.

This approach not only optimizes human effort but also enhances the model’s adaptability to specific tasks with fewer annotated instances, making the training process more efficient and effective.

In this full day workshop in Kassel, you will get introduced to the concepts of Human-In-The-Loop Learning and (Deep) Active Learning. In a hands-on practical session, you will design a (Deep) Active Learning Cycle using Python.

Application

The deadline to apply for a seat in this free workshop is May 14, 2025.

The workshop is completely free of charge; however, as part of this opportunity, we kindly ask all participants to actively contribute to our evaluation process. This helps us to continuously improve the workshop.

As the number of participants is limited, and to ensure the best fit between the workshop’s content and its participants, we will ask you to specify in the application form why you would like to participate in this workshop. We will review all applications after the deadline and notify you of the outcome by the end of May at the latest.

The workshop is open to all researchers; however, as part of our role within the consortium for business, economics, and related data, priority will be given to researchers working in these fields.

Program

Part 1: Introduction to Human-in-the-loop Learning and Deep Active Learning

09:00 – 09:30 Introduction to Human-in-the-loop Learning and NLP 
  • Motivation & Basic Principles
  • Workshop Goals
09:30 – 10:30 Foundations of Active Learning
  • Getting Started with Active Learning
  • Code Tutorial: Actively Querying Instances in AL
  • What is Deep Active Learning?
10:45 – 12:00 Deep Active Learning in NLP
  • Challenges & Motivation
  • Deep Active Learning Cycle
    • Model & Data Selection
    • Cycle Initialization
    • Query Strategies
      • Uncertainty Sampling
      • Diversity Sampling
      • Hybrid Sampling
    • Budget Determination
    • Model Training
  • Working with Humans as Annotators
    • Practical Deep Active Learning
    • Principles of Annotations
    • Uncertain Annotators

Part 2: Practical Session: Designing a Deep Active Learning Cycle

13:00 – 14:30 Setting Up An Deep Active Learning Cycle
  • Tools & Libraries,
  • Common Pitfalls,
  • Adjusting the Deep Active Learning Settings & Modules
14:30 – 15:45 Implementing An Deep Active Learning Cycle
  • Implementing Query Strategies
  • Annotating Data
  • Running a Cycle

15:45 – 16:00 Wrap-up

Requirements

  • Basic Python Skills
  • Basic experience in Machine Learning/Deep Learning
  • Bringing your own laptop to the workshop

About the Instructor

Lukas Rauch is a researcher in the AI for Computationally Intelligent Systems (AI4CIS) team at the University of Kassel, Germany. After completing his master’s degree, he began his PhD in Natural Language Processing, with a focus on Transformer models and Deep Active Learning. Currently, he is working in the field of Avian Bioacoustics to apply these research methods to