We propose a multimodal pipeline that integrates Large Language Model (LLM) re-scoring to enhance classroom speaker diarization. Our approach combines Adaptive Centroid Enrollment (ACE) for audio embedding restructuring and GPT-based refinement of clustering predictions, achieving relative improvements of 8.2% in word-level accuracy and 16.5% in F1-score on classroom datasets.

We introduce novel approaches for automatic paraphasia detection, leveraging a generative pretrained transformer (GPT) and end-to-end models that jointly handle ASR and classification. Our results show that a single-sequence model outperforms GPT baselines for identifying multiple types of paraphasias in speech.

We present PronScribe, a novel method for phonemic transcription from speech and text input based on careful finetuning and adaptation of a massive, multilingual, multimodal speech-text pretrained model. We show that our model is capable of phonemically transcribing pronunciations of full utterances with accurate word boundaries in a variety of languages covering diverse phonological phenomena, achieving phoneme error rates in the vicinity of 1-2% which is comparable to human transcribers.