James E. Purpura is a Professor of linguistics and education in the Applied Linguistics and TESOL Program at Teachers College, Columbia University. Jim’s research has focused on grammar assessment, meaning assessment, and the cognitive underpinnings of assessment. He is also pursuing research in learning-oriented assessment (LOA) as a means of understanding how learning can be incorporated into scenario-based assessments, and how L2 assessments function when embedded in teaching and learning. He is currently Co-editor of Language Assessment Quarterly, and series co-editor of New Perspectives on Language Assessment and Innovations in Language and Learning Assessment at ETS. He served as President of ILTA in 2007 and 2008. He was a Fulbright Scholar in Italy in 2017.
Abstract: Refocusing the Language Assessment Paradigm: Assessing meaning
Non-native speakers use second or foreign languages (L2) to get/give information at school, to create and maintain relationships online, to get a glimpse into other cultures, or more subtly, to decipher intentions in political discourse. In other words, they use their L2 to express a wide range of meanings within social-interpersonal contexts (e.g., a friend recounting a commuting story), social-transactional contexts (e.g., a client resolving a problem with a bill), academic contexts (e.g., a student writing a term paper), professional contexts (e.g., a scientist giving a talk), and literary or imaginative contexts (e.g., a poet writing/reciting a poem at a poetry slam). Since the ability to effectively express, understand, co-construct, negotiate, and repair meanings is the quintessential quality of communicative success, it stands to reason then that meaning and meaning conveyance should play more than a tangential role in L2 assessment (Purpura, 2004, 2016, 2017).
Notwithstanding, since the 1980s, language testers have focused almost exclusively on functional proficiency (the conveyance of functional meaning—e.g., can-do statements), to the exclusion of the conveyance of propositional meanings (i.e., content) or other implied pragmatic meanings. While the ability to use language to get things done is important, excluding propositional content from the assessment process is like having language ability with nothing to say, and excluding pragmatic meanings guts the heart and soul out of communication.
In this talk, I review how L2 testers have theoretically conceptualized “meaning” in models of L2 proficiency throughout the years, and argue for a reprioritization of meaningfulness over wellformedness in L2 assessment. This logically leads to a discussion of the use of language to encode a range of meanings, deriving not only from an examinee’s topical knowledge, but also from an understanding of the contextual factors of language use. Throughout the discussion, I highlight how the expression and comprehension of meaning have been operationalized in L2 assessments. Finally, I maintain that despite the complexities of defining and operationalizing meaning, testers need to seriously think about what meanings they want to test, and what meanings they are already assessing implicitly.