Visual–Temporal Retrieval-Augmented Generation for Lecture Question Answering

Meenakshi Mittal, Micah Mok, Zhuojie Kuang

Answers to lecture questions live in speech, slides, charts, and gestures. We bundle OCR, ASR, and VLM frame descriptions into time-aligned chunks and add a carry-forward rule for slides shown just before a window. On a new 156-question benchmark, lecture-filtered retrieval reaches 82% accuracy on DATA 100, up from 20% with no context, in about 3 s per question.