A study on content validity of TEM4 listening comprehension (2016-2023)

Abstract

Test for English Majors-Band 4 (TEM4) is a large-scale nationwide criterion-referenced test for English majors in Chinese colleges and universities. Since its implementation in 1990, there have been continuous studies on its validity. However, most focused on its reading comprehension section and were based on the traditional classification validity view. Moreover, TEM4 was officially reformed in 2016, especially its listening comprehension section, but relevant validity study is still limited. Therefore, this study aims to assess the content validity of TEM4 listening comprehension after the reform, attempting to enrich the relevant validity study, helping test developers improve question types and promoting the pedagogical teaching to strengthen college students’ listening comprehension ability. Based on Bachman and Palmer’s framework of task characteristics and reference to the interlocutor characteristics proposed by Weir, this study established its framework and analyzed quantitative statistics on the linguistic characteristics of TEM4 listening comprehension. Three dimensions are included in the verification framework: language input characteristics, expected response characteristics, and the relationship between input and response. The study found that the selected TEM4 listening comprehension tests meet the requirements of the teaching and testing syllabuses in terms of length, vocabulary, topic, speech speed, and accent; the question types and expected skills are varied with prominent emphasis; the range and scope of relationship between input and response align with the test proposition principle. All the research results demonstrated a high content validity of TEM4 listening comprehension since its reform in 2016.