Jiao-Liao Mandarin, a prominent dialect in China, embodies the distinctive linguistic features of the region while preserving its rich cultural traditions. However, the manual transcription of speech data is both labor-intensive and costly, significantly limiting the scale of the Jiao-Liao Mandarin transcribed corpus, thereby presenting substantial challenges for speech recognition models specific to the dialect. In this paper, we present the creation of a transcribed corpus for Jiao-Liao Mandarin and introduce a novel model architecture specifically tailored for low-resource speech recognition in this dialect, termed multi-dialect knowledge transfer. This approach strategically leverages phonetic and linguistic knowledge from neighboring dialects to enhance the performance of speech recognition systems for resource-constrained Jiao-Liao Mandarin. We propose two specialized modules, WFAdapter (adapter with weight factorization) and AttAdapter (adapter with attention), designed to improve the model’s adaptability across different dialects and mitigate overfitting risks. Experimental results demonstrate that our proposed method significantly enhances model performance on the low-resource Jiao-Liao Mandarin speech dataset compared to the baseline of full-parameter fine-tuning, achieving reductions of 7.7% in Character Error Rate (CER) and 10.8% in Word Error Rate (WER). Furthermore, compared to conventional adapters, our approach yields reductions of 5.4% in CER and 7.9% in WER, respectively.
You can send an email to 202100800625@mail.sdu.edu.cn to get the code.