Fourth Workshop on Child, Computer and Interaction (WOCCI 2014)

September 19, 2014

Automatic Assessment of Language Background in Toddlers through Phonotactic and Pitch Pattern Modeling of Short Vocalizations

Hynek Bořil (1), Qian Zhang (1), Ali Ziaei (1), John H. L. Hansen (1), Dongxin Xu (2), Jill Gilkerson (2), Jeffrey A. Richards (2), Yiwen Zhang (3), Xiaojuan Xu (3), Hongmei Mao (3), Lei Xiao (3), Fan Jiang (3)

(1) Center for Robust Speech Systems (CRSS), University of Texas at Dallas, U.S.A.
(2) LENA Foundation, Boulder, Colorado, USA
(3) Shanghai Children's Medical Center, Shanghai Jiao Tong University School of Medicine, Shanghai, China

This study utilizes phonotactic and pitch pattern modeling for automatic assessment of toddlers' language background from short vocalization segments. The experiments are conducted on audio recordings of twelve 25.31 months old USborn and Shanghainese toddlers. Each recording captures a whole-day sound track of an ordinary day in the toddlers' life spent in their natural environment. In a preliminary study, we observed that in spite of the limited presence of linguistic content in the early age child vocalizations, certain phonotactic and prosodic patterns were correlated with the child's language background. In the current effort, we analyze to what extent these language-salient cues can be leveraged in the context of automatic language background classification. Besides a traditional parallel phone recognition with statistical language modeling (PPRLM) and phone recognition with support vector machines (PRSVM), a novel scheme that utilizes pitch patterns (PPSVM) is proposed. The classification results on very short vocalizations (on average less than 3 seconds long) confirm that both phonotactic and prosodic features capture a languagespecific content, reaching equal error rates (EER) of 32.45% for PRSVM, 31.33% for PPSVM, and 29.97% in a fusion of PRSVM and PPSVM systems. The competitive performance of PPSVM suggests that pitch contours carry a significant portion of the language-specific information in toddlers' vocalizations.

Index Terms: language background assessment, toddlers, child vocalization, phonotactic modeling, pitch patterns, PPRLM, PRSVM, PPSVM.

Full Paper

Bibliographic reference.  Bořil, Hynek / Zhang, Qian / Ziaei, Ali / Hansen, John H. L. / Xu, Dongxin / Gilkerson, Jill / Richards, Jeffrey A. / Zhang, Yiwen / Xu, Xiaojuan / Mao, Hongmei / Xiao, Lei / Jiang, Fan (2014): "Automatic assessment of language background in toddlers through phonotactic and pitch pattern modeling of short vocalizations", In WOCCI-2014, 39-43.