AI as a Teachers Assistant

Introduction:

Introduction: The digital transformation of higher
education has led to a significant shift in teaching
methods, with blended learning emerging as a
favored approach. In this paradigm, traditional faceto-face teaching is combined with online learning,
resulting in the well-known inverted classrooms. In
this setting, students begin with self-study, supported
by multimedia materials, followed by interactive faceto-face sessions.
However, experience shows that students may
struggle with self-study, highlighting the need for AIsupported learning assistants. These assistants
should provide students with personalized guidance,
feedback, and assessment, adapting to their
individual needs and learning styles. By leveraging AI,
primarily large language models (LLMs), these
assistants can help students progress towards
competency in an efficient and effective manner.

Approach: Three state-of-the-art LLMs (GPT-3.5,
GPT-4 and Mixtral-8x7B) are evaluated on three
subtasks based on lecture notes about political rights
in Switzerland:

- Generating questions
- Evaluating answers
- Providing feedback on the current study level

The models are fine-tuned and the quality of their
outputs were compared to their non-fine-tuned
equivalents.
A prototype application for a chat bot that supports
multiple languages, model selection from a graphical
user interface and an approach that combines chat
history and RAG is built. Model access is wrapped
under an abstracted class, allowing extensibility and
enabling rapid integration of new models.
Administrators assign documents and system
prompts to individual chat bots, giving them granular
control over their behavior and available information.
The documents get embedded with a locally hosted
Multilingual-E5-base instance.

Result: Our work has shown that while there is
potential in using LLMs as assistants for pre-study. It
was shown that existing chat models can work well on
text-based lecture scripts out of the box. However,
they could not interpret lecture scripts using images
and text. The vast range of possible user inputs in this
case makes finetuning challenging, as it would
require an enormous amount of specific training data.
Given that the non-finetuned versions already perform
well, finetuning with few datapoints actually worsened
the overall quality of the output.
The prototype serves as a blueprint for history-aware
and RAG-enabled chat bot applications. When
provided with a locally hosted Mixtral-8x7B instance,
the entire RAG process can be done locally. This
gives its users control over the use of their data and
ensures that classified information can be used to
enhance the chat bots.
In its current form, the prototype is primarily limited by
its resource consumption and therefore scalability
concerns.