English Proficiency Assessment

How Chatterize Measures English Proficiency

When it comes to learning English, incorporating regular assessment of progress is essential. Assessment, or evaluation, helps us to understand what a learner already knows and what they still need to learn. And when we have this information, we can determine what needs to happen in order for a learner to move forward in their language learning process. As English as a Second Language (ESL) learning shifts from traditional classrooms to online digital spaces, the ways we measure English proficiency become more sophisticated. Thanks to technology, new tools allow us to analyze language in ways that we previously couldn’t with human raters.  

In brick and mortar early learning ESL classrooms in China, the accuracy of a student’s pronunciation, grammar, and vocabulary are the typical measurements of English proficiency.  While accuracy is certainly an important part of speaking English effectively, true English proficiency is really about how well a speaker can communicate a message. Applied linguists often measure this using three dimensions: complexity, accuracy, and fluency (CAF) (Skehan 1998; Ellis 2003, 2008; Ellis and Barkhuizen 2005).  In an earlier Chatterize blog post, we explored each of these dimensions and how they work together to provide a holistic framework for assessing performance. These dimensions are at the heart of Chatterize’s learning design and of its assessment system.

The Chatterize Way

On the Chatterize platform, students learn English in a pressure-free, gamified environment from the comfort of home. They build English proficiency with conversational practice centered around true-to-life experiences. The learning objectives on the Chatterize platform align with the well-known language learning framework, The Common European Framework of Reference for Languages (CEFR).  But students aren’t generally thinking about the CEFR and English proficiency levels while they are navigating Chatterize’s virtual environment.  Because instead, they are busy having fun!

However, behind the scenes, Chatterize’s intelligent system monitors students’ English proficiency to understand what learners already know. And to understand what they still need to learn.  Chatterize carefully measures each learner’s progress towards CEFR benchmarks and Chatterize’s associated learning objectives. Then, Chatterize uses these English proficiency measurements to inform an effective personalized learning experience on the platform. The results determine which tasks will be most supportive in helping students to improve their English proficiency. Students immerse in a virtual environment that provides exactly what they need to confidently accelerate their speaking. Furthermore, as progress is assessed, the results are compiled into regular reports that are shared with students and their parents.  This reporting brings transparency to the process.

To measure progress towards text-based learning objectives and English proficiency levels, Chatterize uses three key metrics that align with the CAF dimensions of complexity. These are lexical depth, lexical breadth, and fluency. Let’s take a look at how Chatterize defines each of these.

Lexical Depth & Lexical Breadth

When it comes to English proficiency, most second language researchers agree that what learners know about language can’t be thought of as all or nothing.  Rather, this knowledge consists of several degrees and dimensions. For example, how is a word pronounced? What does it mean? How is it used in a grammatically ‘correct’ sentence? Across various frameworks for lexical knowledge, there is clear agreement that lexical knowledge has at least two dimensions: depth (quality) and breadth (size).  In other words, lexical knowledge can be thought of in terms of quantity and quality.

Lexical depth is learner’s practical understanding of various aspects of a linguistic form, or how well the learner ‘knows’ what it is and what it does. In other words, how closely a learner’s speech align with targeted forms. When measuring lexical depth, we are looking at how accurately a learner uses language.  To determine lexical depth, we track errors. For example, we consider pronunciation errors, grammatical errors within sentences, and word choice errors. In the end, we are measuring how effectively a learner is using English to convey meaning to understand more about their English proficiency.

Meanwhile, lexical breadth refers to how broad the range of language a learner uses is. This metric doesn’t take into account how well the learner knows the language. Instead, this measurement English proficiency relates to the number of forms of which a learner has at least some superficial knowledge.  In other words, lexical breadth is about how much language a learner can produce regardless of how well they understand it. Furthermore, there is a link between lexical breadth and complexity. More variety means broader breadth and more complexity. 

Fluency

The word fluency is often used interchangeably with proficiency.  We’ve all heard people describe a high level of English proficiency as someone being ‘fluent in English’.  But, it’s important to note that fluency has a number of different definitions. Within the CAF framework of complexity, accuracy, and fluency, fluency refers to a language user’s control and speed.  Fluency is about how quickly a learner can recall and produce linguistic forms and structures automatically, smoothly, and without awkward stops and starts. On the Chatterize platform, the fluency metric allows us to understand how quickly and cohesively a learner uses language. We measure how much spoken language a learner produces within a given period of time and how connected and automatic that speech is.  


Chatterize exposes learners to native-sounding pronunciation support and conversation structures based on ongoing measurements. The use of regular, sophisticated assessment accelerates spoken English proficiency. And in addition, Chatterize’s assessment system provides useful information about English proficiency to students and their parents. 

Chatterize measures English proficiency so that each learner’s experience can be a personalized one. Check out our SpeakOut English Speaking Assessment below!

About the Author

Jessica Madsen is the linguistics advisor at Chatterize. 

Jessica is a versatile education professional with a master’s degree in linguistics and over fifteen years of experience in the field of English language teaching. She began her TESOL career teaching in intensive English programs at universities in the United States, including the University of Wisconsin Madison and Purdue University Calumet. She now works as a full-time consultant on national and international education projects in curriculum and assessment design. 

To contact her, find her on LinkedIn at https://www.linkedin.com/in/jessicamadsen813.

Share this: