Keelan's Home Page

This is me rounding Devil's Elbow in the 2003 Mt. Diablo Challenge. It is a bicycle race near the Bay Area, and I was one of 9 unicyclists in the pack that year. The total distance was 10.8 miles and the gain in elevation was 3,249 feet (my time 2:10).

About Me
I'm a Principal NLU Engineer at Kasisto, Inc where I work on developing conversational AI technology in the banking domain. Previously, I worked for 11 years in the NLP & Speech group at Educational Testing Service, where I conducted research on automated scoring of non-native spoken responses to assessments such as the TOEFL iBT and TOEIC. I received my PhD in 2009 from the University of Pennsylvania in the Linguistics Department. My thesis advisor was Bill Labov.

I have two blogs:

• The Evanini Family: A blog mostly dedicated to providing updates about our family to out-of-town family members.
• SpeechTechie: A repository for posting some coding / application details that I have found useful for my research.

Publications

• Qin, Y., Qian, Y., Loukina, A., Lange, P., Misra, A., Evanini, K., & Lee, T. (2021). Automatic detection of word-level reading errors in non-native English speech based on ASR output. In Proceedings of the International Conference on Chinese Spoken Language Processing (ISCSLP). [pdf]
• Wang, X., Evainini, K., Qian, Y., & Mulholland, M. (2021). Automated scoring of spontaneous speech from young learners of English using transformers. In Proceedings of IEEE SLT 2021. [pdf]
• Gretter, R., Matassoni, M., Falavigna, D., Evanini, K., & Leong, C.W. (2020). Overview of the Interspeech TLT2020 shared task on ASR for non-native children’s speech. In Proceedings of Interspeech, pp. 245-249. [pdf]
• Loukina, A., Evanini, K., Mulholland, M., Blood, I., & Zechner, K. (2020). Do face masks introduce bias in speech technologies? The case of automated scoring of speaking proficiency. In Proceedings of Interspeech, pp. 1942-1946. [pdf]
• Hirschi, K., Kang, O., Cucchiarini, C., Hansen, J., Evanini, K., & Strik, H. (2020). Mobile-assisted prosody training for limited English proficiency: Learner background and speech learning pattern. In Proceedings of Interspeech, pp. 4452-4456. [pdf]
• Evanini, K., Futagi, Y., & Hauck, M.C. (2020). Using Automated Scoring in K-12 English Language Proficiency Assessments. In Assessing English Language Proficiency in U.S. K–12 Schools, Wolf (ed.), pp. 207-225. New York, NY: Routledge.
• Cahill, A. & Evanini, K. (2020). Natural Language Processing for Writing and Speaking. In Handbook of Automated Scoring: Theory into Practice, Yan, Rupp, & Foltz (eds)., pp. 69-92. Boca Raton, FL: CRC Press.
• Zechner, K. & Evanini, K. (2019). Automated Speaking Assessment: Using Language Technologies to Score Spontaneous Speech. New York, NY: Routledge.
• Qian, Y., Ubale, R., Lange, P., Evanini, K., Ramanarayanan, V., & Soong, F.K. (2019). Spoken language understanding of human-machine conversations for language learning applications. Journal of Signal Processing Systems.
• Ubale, R., Ramanarayanan, V., Qian, Y., Evanini, K., Leong, C.W., & Lee, C.M. (2019). Native language identification from raw waveforms using deep convolutional neural networks with attentive pooling. In Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 403-410. [pdf]
• Wang, X., Evanini, K., Qian, Y., & Zechner, K. (2019). Using very deep convolutional neural networks to automatically detect plagiarized spoken responses. In Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 764-771. [pdf]
• Yoon, S.-Y., Lee, C.M., Zechner, K., Evanini, K. (2019). Development of robust automated scoring models using adversarial input for oral proficiency assessment. In Proceedings of Interspeech 2019, pp. 1871-1875. [pdf]
• Wang, X., Yoon, S.-Y., Evanini, K., Zechner, K., Qian, Y. (2019). Automatic detection of off-topic spoken responses using very deep convolutional neural networks. In Proceedings of Interspeech 2019, pp. 4200-4204. [pdf]
• Qian, Y., Lange, P., Evanini, K., Pugh, R., Ubale, R., Mulholland, M., & Wang, X. (2019). Neural approaches to automated speech scoring of monologue and dialogue responses. In Proceedings of the 41st IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pp. 8112-8116. Brighton, UK: IEEE. [pdf]
• Chen, L., Zechner, K., Yoon, S.-Y., Evanini, K., Wang, X., Loukina, A., Tao, J., Davis, L., Lee, C.-M., Ma, M., Mundkowsky, R., Lu, Chi, Leong, C.-W., &. Gyawali, B. (2018). Automated scoring of non-native speech using the SpeechRaterSM v. 5.0 engine. (Research Report No. RR-18-10). Princeton, NJ: Educational Testing Service. [pdf]
• Forsyth, C.M., Luce, C., Zapata-Rivera, D., Jackson, G.T., Evanini, K., & So, Y. (2018). Evaluating English language learners' conversations: Man vs. Machine. Computer Assisted Language Learning, v.32, issue 4, pp. 398-417. [pdf]
• Qian, Y., Ubable, R., Lange, P., Evanini, K., & Soong, F. (2018). From speech signals to semantics -- Tagging performance at acoustic, phonetic, and word levels. In Proceedings of the 11th International Symposium on Chinese Spoken Language Processing (ISCSLP 2018). Taipei, Taiwan, November, 2018. [pdf]
• Qian, Y., Ubale, R., Mulholland, M., Evanini, K., & Wang, X. (2018). A prompt-aware neural network approach to content-based scoring of non-native spontaneous speech. In Proceedings of the IEEE Workshop on Spoken Language Technology (SLT 2018). Athens, Greece, December, 2018. [pdf]
• Ubale, R., Qian, Y., & Evanini, K. (2018). Exploring end-to-end attention-based neural networks for native language identification. In Proceedings of the IEEE Workshop on Spoken Language Technology (SLT 2018). Athens, Greece, December, 2018. [pdf]
• Wang, X., Gyawali, B., Bruno, J.V., Molloy, H.R., Evanini, K., & Zechner, K. (2018). Discourse modeling of non-native spontaneous speech using the Rhetorical Structure Theory framework. In Proceedings of the IEEE Workshop on Spoken Language Technology (SLT 2018). Athens, Greece, December, 2018. [pdf]
• Evanini, K., Mulholland, M., Ubale, R., Qian, Y., Pugh, R., Ramanarayanan, V., & Cahill, A.. (2018). Improvements to an automated content scoring system for Spoken CALL responses: The ETS submission to the Second Spoken CALL Shared Task. In Proceedings of Interspeech, pp. 2379-2383. Hyderabad, India, September, 2018. [pdf]
• Evanini, K., Timpe-Laughlin, V., Tsuprun, E., Blood, I., Lee, J., Bruno, J., Ramanarayanan, V., Lange, P., & Suendermann-Oeft, D. (2018). Game-based spoken dialog language learning applications for young students. In Proceedings of Interspeech, pp. 548-549. Hyderabad, India, September, 2018. [pdf]
• Ramanarayanan, V., Pautler, D., Lange, P., Tsuprun, E., Ubale, R., Evanini, K., & Suendermann-Oeft, D. (2018). Toward scalable dialog technology for conversational language learning: Case Study of the TOEFL® MOOC. In Proceedings of Interspeech, pp. 1960-1961. Hyderabad, India, September, 2018. [pdf]
• Sydorenko, T., Smits, T., Evanini, K. & Ramanarayanan, V. (2018). Simulated speaking environments for language learning: insights from three cases. Computer Assisted Language Learning, vol. 32, issue 1-2, pp. 17-48. [pdf]
• Qian, Y., Evanini, K., Lange, P., Pugh, R.A., Ubale, R., & Soong, F.K. (2017). Improving native language (L1) identification with better VAD and TDNN trained separately on native and non-native English corpora. In Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 606-613. Okinawa, Japan, December, 2017. [pdf]
• Qian, Y., Ubale, R., Ramanarayanan, V., Lange, P., Suendermann-Oeft, D., Evanini, K., & Tsuprun, E. (2017). Exploring ASR-free end-to-end modeling to improve spoken language understanding in a cloud-based dialog system. (2017). In Proceedings of the IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), pp. 569-576. Okinawa, Japan, December, 2017. [pdf]
• Cofino, K., Ramanarayanan, V., Lange, P., Pautler, D., Suendermann-Oeft, D., & Evanini, K. (2017). A modular, multimodal open-source virtual interviewer dialog agent. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (ICMI), pp. 520-521. Glasgow, Scotland, November 13, 2017. [pdf]
• Evanini, K., Hauck, M. C., & Hakuta, K. (2017). Approaches to automated scoring of speaking for K–12 English language proficiency assessments (Research Report No. RR-17-18). Princeton, NJ: Educational Testing Service. [pdf]
• Evanini, K., Mulholland, M., Tsuprun, E., & Qian, Y. (2017). Using an automated content scoring system for spoken CALL responses: The ETS submission for the Spoken CALL Challenge. In Proceedings of the Interspeech Workshop on Speech and Language Technology for Education (SLaTE 2017), pp. 105-110. Stockholm, Sweden, September 2017. [pdf]
• Evanini, K., Najafian, M., Safavi, S., & Berkling, K. (2017). WOCCI 2017: 6th International Workshop on Child Computer Interaction (Workshop Summary). In Proceedings of the 19th ACM International Conference on Multimodal Interaction (ICMI), pp. 667-669. Glasgow, Scotland, November 13, 2017. [pdf]
• Evanini, K., Tsuprun, E., Laughlin, V., Ramanarayanan, V., Lange, P., & Suendermann-Oeft, D. (2017). Evaluating the impact of local context on CALL applications using spoken dialog systems. In Proceedings of the XVIIIth International CALL Research Conference (CALL 2017), pp. 217-226, Berkeley, USA, July 2017.
• Lee, C.M., Yoon, S.-Y., Wang, X., Mulholland, M., Choi, I., & Evanini, K. (2017). Off-topic spoken response detection using Siamese convolutional neural networks. In Proceedings of Interspeech 2017, pp. 1427-1431. [pdf]
• Malmasi, S., Evanini, K., Cahill, A., Tetreault, J., Pugh, R., Hamill, C., Napolitano, D., & Qian, Y. (2017). A report on the 2017 Native Language Identification shared task. In Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 62-75. Copenhagen, Denmark: ACL. [pdf]
• Qian, Y., Evanini, K., Wang, X., Lee, C.M., & Mulholland, M. (2017). Bidirectional LSTM-RNN for improving automated assessment of non-native children's speech. In Proceedings of Interspeech 2017, pp. 1417-1421. [pdf]
• Qian, Y., Evanini, K., Wang, X., Suendermann-Oeft, D., Pugh, R.A., Lange, P.L., Molloy, H.R., & Soong, F.K. (2017). Improving sub-phone modeling for better native language identification with non-native English speech. In Proceedings of Interspeech 2017, pp. 2586-2590. [pdf]
• Qian, Y., Ubale, R., Ramanarayanan, V., Suendermann-Oeft, D., Evanini, K., Lange, P., & Tsuprun, E. (2017). Towards end-to-end modelling of spoken language understanding in a cloud-based spoken dialog system. In Proceedings of the 21st Workshop on the Semantics and Pragmatics of Dialogue (SemDial 2017), pp. 143-152. Saarbrücken, Germany, August, 2017. [pdf]
• Ramanarayanan, V., Lange, P., Evanini, K., Molloy, H., Tsuprun, E., & Suendermann-Oeft, D. (2017). Crowdsourcing multimodal dialog interactions: Lessons learned from the HALEF case. In Proceedings of the AAAI 2017 Workshop on Crowdsourcing, Deep Learning and Artificial Intelligent Agents, pp. 423-431. San Francisco, CA, February 2017. [pdf]
• Ramanarayanan, V., Lange, P.L., Evanini, K., Molloy, H.R., Suendermann-Oeft, D. (2017). Human and automated scoring of fluency, pronunciation and intonation during human-machine spoken dialog interactions. In Proceedings of Interspeech 2017, pp. 1711-1715. [pdf]
• Ramanarayanan, V., Lange, P., Evanini, K., Molloy, H., Tsuprun, E., Qian, Y., & Suendermann-Oeft, D. (2017). Using vision and speech features for automated prediction of performance metrics in multimodal dialogs. ETS Research Report Series, RR-17-20. Princeton, NJ: Educational Testing Service. [pdf]
• Ramanarayanan, V., Leong, C.W., Suendermann-Oeft, D., & Evanini, K. (2017). Crowdsourcing ratings of caller engagement in thin-slice videos of human-machine dialog: Benefits and pitfalls. In Proceedings of the 19th ACM International Conference on Multimodal Interaction (ICMI), pp. 281-287. Glasgow, Scotland, November 13, 2017. [pdf]
• Ramanarayanan, V., Suendermann-Oeft, D., Lange, P., Mundkowsky, R., Ivanov, A.V., Yu, Z., Qian, Y., & Evanini, K. (2017). How multiple open standards are synergistically combined in the HALEF multimodal dialog system. In D.A. Dahl (ed.), Multimodal interaction with W3C standards: Towards natural user interfaces to everything, pp. 295-310. Springer.
• Timpe-Laughlin, V., Evanini, K., Green, A., Blood, I., Dombi, J., & Ramanarayanan, V. (2017). Designing interactive, automated dialogues for L2 pragmatics learning. In Proceedings of the 21st Workshop on the Semantics and Pragmatics of Dialogue (SemDial 2017), Saarbrücken, Germany, August 15-17, 2017, pp. 143-152. [pdf]
• Timpe-Laughlin, V., Lee, J., Evanini, K., Bruno, J., & Blood, I. (2017). Can you guess who I am?: An interactive task for young learners to practice yes/no question formation in English. In Proceedings of the Interspeech Workshop on Child, Computer, and Interaction (WOCCI 2017), pp. 62-67. Glasgow, Scotland. [pdf]
• Wang, X. & Evanini. (2017). Empirical evaluation of the communicative effectiveness of an automatic speech-to-speech translation system. In Proceedings of the Interspeech Workshop on Speech and Language Technology for Education (SLaTE 2017), pp. 158-163. Stockholm, Sweden, September 2017. [pdf]
• Wang, X., Evanini, K., Zechner, K., & Mulholland, M. (2017). Modeling discourse coherence for the automated scoring of spontaneous spoken responses. In Proceedings of the Interspeech Workshop on Speech and Language Technology for Education (SLaTE 2017), pp. 140-145. Stockholm, Sweden, September 2017. [pdf]
• Yoon, S.-Y., Lee, C.M., Choi, I., Wang, X., Mulholland, M., & Evanini, K. (2017). Off-topic spoken response detection with word embeddings. In Proceedings of Interspeech 2017, pp. 2754-2758. [pdf]
• Wang, X., Evanini, K., Bruno, J., & Mulholland, M. (2016). Automatic plagiarism detection for spoken responses in an assessment of English language proficiency. In Proceedings of the IEEE Workshop on Spoken Language Technology (SLT 2016), San Diego, USA, December 2016. [pdf]
• Qian, Y., Tao, J., Suendermann-Oeft, D., Evanini, K., Ivanov, A., & Ramanarayanan, V. (2016). Noise and metadata sensitive bottleneck features for improving speaker recognition with non-native speech input. In Proceedings of Interspeech, pp. 3648-3652. [pdf]
• Qian, Y., Wang, X., Evanini, K., & Suendermann-Oeft, D. (2016). Self-Adaptive DNN for improving spoken language proficiency assessment. In Proceedings of Interspeech, pp. 3122-3126. [pdf]
• Schuller, B., Steidl, S., Batliner, A., Hirschberg, J., Burgook, J.K., Baird, A., Elkins, A., Zhang, Y., Coutinho, E., & Evanini, K. (2016). The INTERSPEECH 2016 Computational Paralinguistics Challenge: Deception, Sincerity & Native Language. In Proceedings of Interspeech, pp. 2001-2005. [pdf]
• Huang, B. & Evanini, K. (2016). Think, sink, and beyond: Phonetic variants and factors contributing to th pronunciation among Chinese speakers. Journal of Second Language Pronunciation, 2(2), pp. 253-275.
• Mulholland, M., Lopez, M., Evanini, K., Loukina, A., Qian, Y. (2016). A comparison of ASR and human errors for transcription of non-native spontaneous speech. In Proceedings of the 41st IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Shanghai, China, March 2016. [pdf]
• Evanini, K., Singh, S., Loukina, A., Wang, X., & Lee, C.M. (2015). Content-based automated assessment of non-native spoken language proficiency in a simulated conversation. In Proceedings of the Machine Learning for SLU & Interaction NeurIPS 2015 Workshop, Montréal, Canada, December 2015. [pdf]
• Ivanov, A., Ramanarayanan, V., Suendermann-Oeft, D., Lopez, M., Evanini, K., & Tao, J. (2015). Automated speech recognition technology for dialogue interaction with non-native interlocutors. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2015), pp. 134-138, Prague, Czech Republic, September 2015. [pdf]
• Loukina, A., Lopez, M., Evanini, K., Suendermann-Oeft, D., Ivanov, A., and Zechner, K. (2015). Pronunciation accuracy and intelligibility of non-native speech. In Proceedings of Interspeech, pp. 1917-1921, Dresden, Germany, September 2015. [pdf]
• Loukina, A., Lopez, M., Evanini, K., Suendermann-Oeft, D., & Zechner, K. (2015). Expert and crowdsourced annotation of pronunciation errors for automatic scoring systems. In Proceedings of Interspeech, pp. 2809-2813, Dresden, Germany, September 2015. [pdf]
• Ma, M., Evanini, K., Loukina, A., Wang, X., & Zechner, K. (2015). Using F0 contours to assess nativeness in a sentence repeat task. In Proceedings of Interspeech, pp. 653-657, Dresden, Germany, September 2015. [pdf]
• Ramanarayanan, V., Suendermann-Oeft, V., Ivanov, A., & Evanini, K. (2015). A distributed cloud-based dialog system for conversational application development. In Proceedings of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2015), pp. 432-434, Prague, Czech Republic, September 2015. [pdf]
• Wang, X., Evanini, K., & Yoon, S.-Y. (2015). Word-level F0 modeling for the automated assessment of read aloud speech. In Proceedings of the Interspeech Workshop on Speech and Language Technology for Education (SLaTE 2015), pp. 23-27, Leipzig, Germany, September 2015. [pdf]
• Evanini, K., Heilman, M., Wang, X., & Blanchard, D. (2015). Automated scoring for the TOEFL Junior® Comprehensive Writing and Speaking Test. ETS Research Report Series, RR-15-09. Princeton, NJ: Educational Testing Service. [pdf]
• Yang, X., Loukina, A., & Evanini, K. (2014). Machine learning approaches to improving pronunciation error detection on an imbalanced corpus. In Proceedings of the IEEE Workshop on Spoken Language Technology (SLT 2014), pp. 300-305. South Lake Tahoe, NV: IEEE. [pdf]
• Tao, J., Evanini, K., & Wang, X. (2014). The influence of automatic speech recognition accuracy on the performance of an automated speech assessment system. In Proceedings of the IEEE Workshop on Spoken Language Technology (SLT 2014), pp. 294-299. South Lake Tahoe, NV: IEEE. [pdf]
• Keelan Evanini, Youngsoon So, Jidong Tao, Diego Zapata, Christine Luce, Laura Battistini, and Xinhao Wang. 2014. Performance of a trialogue-based prototype system for English language assessment for young learners. Proceedings of the Interspeech Workshop on Child Computer Interaction (WOCCI 2014), Singapore, September 19, 2014. [pdf]
• Evanini, K. & Wang, X. (2014). Automatic detection of plagiarized spoken responses. Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications (BEA-9), pp. 22-27, Baltimore, MD, June 26, 2014. [pdf]
• Klaus Zechner, Keelan Evanini, Su-Youn Yoon, Lawrence Davis, Xinhao Wang, Lei Chen, Chong Min Lee and Chee Wee Leong. 2014. Automated scoring of speaking items in an assessment for teachers of English as a Foreign Language. Proceedings of the Ninth Workshop on Innovative Use of NLP for Building Educational Applications (BEA-9), Baltimore, MD, June 26, 2014. [pdf]
• Christopher Mitchell, Keelan Evanini, and Klaus Zechner. 2014. A trialogue-based spoken dialogue system for assessment of English language learners. Proceedings of the 5th International Workshop on Spoken Dialog Systems. Napa, January 17-20, 2014. [pdf]
• Lai, C., Evanini, K, & Zechner, K. (2013). Applying rhythm metrics to non-native spontaneous speech. Proceedings of the Workshop on Speech and Language Technology in Education, pp. 159-162, Grenoble, August 30 - September 1, 2013. [pdf]
• Xiong, W., Evanini, K., Zechner, K., & Chen, L. (2013). Automated content scoring of spoken responses containing multiple parts with factual information. Proceedings of the Workshop on Speech and Language Technology in Education, pp. 137-142, Grenoble, August 30 - September 1, 2013. [pdf]
• Evanini, K. & Xinhao, W. (2013). Automated speech scoring for non-native middle school students with multiple task types. In Proceedings of Interspeech, 2435-2439. [pdf]
• Evanini, K., Xie, S., & Zechner, K. (2013). Prompt-based Content Scoring for Automated Spoken Language Assessment. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 157-162, Atlanta, Georgia, June 13 2013. [pdf]
• Wang, X., Evanini, K., & Zechner, K. (2013). Coherence Modeling for the Automated Assessment of Spontaneous Spoken Responses. Proceedings of NAACL-HLT, pp. 814-819, Atlanta, Georgia, 9-14 June 2013. [pdf]
• Keelan Evanini and Becky Huang. 2013. Production of English vowels by speakers of Mandarin Chinese with prolonged exposure to English. Proceedings of Meetings on Acoustics, volume 18. [pdf]
• Klaus Zechner, Keelan Evanini, and Cara Laitusis. 2012. Using Automatic Speech Recognition to Assess the Reading Proficiency of a Diverse Sample of Middle School Students. Proceedings of the Interspeech Workshop on Child, Computer, and Interaction. [pdf]
• Keelan Evanini and Becky Huang. 2012. Automatic detection of [th] pronunciation errors for Chinese learners of English. Proceedings of the International Symposium on Automatic Detection of Errors in Pronunciation Training, Stockholm. [pdf]
• Xie, S., Evanini, K., & Zechner, K. (2012). Exploring content features for automated speech scoring. Proceedings of the 2012 Meeting of the North American Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 103-111. [pdf]
• Keelan Evanini and Klaus Zechner. 2011. Using crowdsourcing to provide prosodic annotations for non-native speech. Proceedings of Interspeech 2011. [pdf]
• Su-Youn Yoon, Keelan Evanini, and Klaus Zechner. 2011. Non-scorable response detection for automated speaking proficiency assessment. Proceedings of The 6th Workshop on Innovative Use of NLP for Building Educational Applications. [pdf]
• Xie Sun and Keelan Evanini. 2011. Gaussian mixture modeling of vowel durations for automated assessment of non-native speech. Proceedings of ICASSP 2011. [pdf]
• Keelan Evanini, Catherine Lai, and Klaus Zechner. 2011. The importance of optimal parameter setting for pitch extraction. Proceedings of Meetings on Acoustics. [pdf]
• Lei Chen, Keelan Evanini, and Xie Sun. 2010. Assessment of non-native speech using vowel space characteristics. Proceedings of the IEEE Workshop on Spoken Language Technology (SLT 2010), pp. 139-144, Berkeley, CA, USA: IEEE. [pdf]
• Aaron Dinkin and Keelan Evanini. 2010. An elementary linguistic definition of Upstate New York. Selected Papers from NWAV 38, University of Pennsylvania Working Papers in Linguistics, v. 16.2. [pdf]
• David Suendermann, Jackson Liscombe, Roberto Pieraccini, and Keelan Evanini. 2010 "How am I Doing?": A new framework to effectively measure the performance of automated customer care contact centers. In Advances in Speech Recognition: Mobile Environments, Call Centers, and Clinics, ed. A. Neustein., p. 155 - 180. Springer.
• Keelan Evanini, Derrick Higgins, and Klaus Zechner. 2010. Using Amazon Mechanical Turk for transcription of non-native speech. Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon's Mechanical Turk, June 2010, pp. 53–56. [pdf]
• Josh Tauberer and Keelan Evanini. 2009 Intrinsic vowel duration and the post-vocalic voicing effect: Some evidence from dialects of North American English. Proceedings of Interspeech 2009. [pdf]
• Keelan Evanini, Stephen Isard, and Mark Liberman. 2009. Automatic formant extraction for sociolinguistic analysis of large corpora. Proceedings of Interspeech 2009. [pdf]
• David Suendermann, Keelan Evanini, Jackson Liscombe, Phillip Hunter, Krishna Dayanidhi, and Roberto Pieraccini. 2009. From rule-based to statistical grammars: Continuous improvement of large-scale spoken dialog systems. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Taipei, Taiwan, 2009, pp. 4713-4716. [pdf]
• David Suendermann, Jackson Liscombe, Keelan Evanini, Krishna Dayanidhi, Roberto Pieraccini. 2008. C^5. Proceedings of the IEEE Workshop on Spoken Language Technology (SLT), Goa, India, December 2008, pp. 125-128. [pdf]
• Keelan Evanini, Phillip Hunter, Jackson Liscombe, David Suendermann, Krishna Dayanidhi, Roberto Pieraccini. 2008. Caller Experience: A method for evaluating dialog systems and its automated prediction. Proceedings of the IEEE Workshop on Spoken Language Technology (SLT), Goa, India, December 2008, pp. 129-132. [pdf]
• Keelan Evanini. 2008. A shift of allegiance: The case of Erie and the North / Midland dialect boundary. Selected Papers from NWAV 36, University of Pennsylvania Working Papers in Linguistics, v. 14.2. [pdf]
• Keelan Evanini. 2008. Classifying and clustering dialects of North American English. Proceedings of NESCAI08: The Third North East Student Colloquium on Artificial Intelligence, Ithaca, NY, May 2008. [pdf]
• Keelan Evanini, David Suendermann, Roberto Pieraccini. 2007. Call classification for automated troubleshooting on large corpora. Proceedings of the IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU), Kyoto, Japan, 2007, pp. 207-212. [pdf]
• Keelan Evanini. 2007. How to formalize variation: Stochastic OT models and /s/ deletion in Spanish. Proceedings of LACUS 33, Linguistic Association of Canada and the US. [pdf]
• Keelan Evanini. 2007. Book note for American English: dialects and variation. Language in Society 36(1):7-8.
• Keelan Evanini. 2007. The phonetic realization of pitch accent in Huave. Proceedings of the 33rd Annual Berkeley Linguistics Society., pp. 53-64. [pdf]
• Tonya Wolford and Keelan Evanini. 2006. Features of AAVE as Features of PRE: A Study of Adolescents in Philadelphia. Selected Papers from NWAV 34, University of Pennsylvania Working Papers in Linguistics, v. 12.2. [pdf]
• Keelan Evanini and Giang Nguyen. 2005. Papers from NWAVE 32, University of Pennsylvania Working Papers in Linguistics, v. 10.2.

Dissertation
The title of my dissertation is The permeability of dialect boundaries: A case study of the region surrounding Erie, Pennsylvania. The entire dissertation can be downloaded from Penn's electronic disseration repository. Individual chapters are available here:

• Front Matter: title page, acknowledgements, abstract, table of contents, list of figures, list of tables
• Chapter 1: introduction to the empirical, methodological, and theoretical aims of the dissertation
• Chapter 2: a review of the evidence from earlier dialect geography studies (LAMSAS, DARE) that show Erie's phonology and lexicon to be Northern
• Chapter 3: a description of my data collection procedure, including how I contacted speakers for inclusion in my corpus, a description of the interview materials, and how the interviews were processed
• Chapter 4: a description of the procedure I used for automatic vowel analysis including forced alignment, measurement point selection, and formant prediction
• Chapter 5: natural break maps for 17 vowel classes
• Chapter 6: a study of the merger of /o/ and /oh/ in the Erie region
• Chapter 7: a study of other vowels that pattern differently in the Midland and the North (/ow/, /uw/, /ae/, /o/)
• Chapter 8: an investigation into the use of Midland lexical and morphosyntactic features in Erie
• Chapter 9: the settlement history of Erie and how it helps explain the course of linguistic change in the region
• Chapter 10: some brief concluding remarks
• References
• Appendices: materials used in the interview, including the word list, minimal pairs, sentences for acceptability judgments, and DARE's reading passage

Software
These are programs that I have written and found useful for my research. The quality of documentation, portability, usability, etc. varies widely among them. Please feel free to contact me if you have any questions about them, or if you'd like to suggest improvements.

• quickTrans: a collection of shell, python, and Praat scripts that I use for transcribing interviews [download | README]
• extractFormants: a program for automatically obtaining vowel formant measurements; takes as input a speech audio file and a word- and phone-level transcription produced by forced alignment, outputs a set of formant measurements for each vowel in the audio file [download | README]
• vowel.r: some R functions I use for plotting vowel systems [download | example data file]