刊讯｜SSCI 期刊《语言测试》2022年第3-4期

LANGUAGE TESTING

Volume 39, Issue 3, July 2021

LANGUAGE TESTING (SSCI一区，2021 IF：2.400）2022年第3-4期共刊文18篇。2022年第3期共发文10篇，其中'Editorial'1篇，研究性论文6篇，'Test Review'1篇，书评2篇。研究性论文主题涉及分级测试、英语听力测试、EFL听力测试、中国本土英语测试、瑞典外语评估等。第4期共发文8篇，其中研究性论文6篇，'Test Review'1篇，书评1篇。研究论文涉及英语阅读能力分测试、二语听力、二语教师的批判性语言评估素养量表等。（2022已更完）

往期推荐：

第3期目录

EDITORIAL

■ Introduction to the Special Issue, by Slobodanka Dimova, Xun Yan, April Ginther, Pages 341–354.

ARTICLES

■ Using instructor judgment, learner corpora, and DIF to develop a placement test for Spanish L2 and heritage learners, by Melissa A. Bowles, Pages 355-376.

■ Local placement test retrofit and building language assessment literacy with teacher stakeholders: A case study from Colombia, by Gerriet Janssen, Pages 377-400.

■ Developing a local academic English listening test using authentic unscripted audio-visual texts, by Yena Park, Senyung Lee, Sun-Young Shin, Pages 401-424.

■Bridging local needs and national standards: Use of standards-based individualized feedback of an in-house EFL listening test in China, by Shangchao Min, Juan Zhang, Yue Li, Lianzhen He, Pages 425-452.

■ Local English testing in China’s tertiary education: Contexts, policies, and practices, by Jason Fan, Kellie Frost, Yan Jin, Pages 453–473.

■National assessment of foreign languages in Sweden: A multifaceted and collaborative venture, by Gudrun Erickson, Linda Borger, Eva Olsson, Pages 474–493.

TEST REVIEW

■Review of the Japanese-Language Proficiency Test, by Hitoshi Nishizawa , Daniel R. Isbell , Yuichi Suzuki, Pages 494–503.

BOOK REVIEWS

■Book Review: Assessing Speaking in Context—Expanding the Construct and its Applications, by Lynda Taylor, Pages 504-507.

■Book Review: Scoring Second Language Spoken and Written Performance: Issues, Options and Directions, by Santi Budi Lestari, Kathrin Eberharter, Pages 507-510.

第3期摘要

Using instructor judgment, learner corpora, and DIF to develop a placement test for Spanish L2 and heritage learners

Melissa A. Bowles, Department of Spanish and Portuguese, University of Illinois at Urbana-Champaign, USA

Abstract This study details the development of a local test designed to place university Spanish students (n = 719) into one of the four different course levels and to distinguish between traditional L2 learners and early bilinguals on the basis of their linguistic knowledge, regardless of the variety of Spanish they were exposed to. Early bilinguals include two groups—heritage learners (HLs), who were exposed to Spanish in their homes and communities growing up, and early L2 learners with extensive Spanish exposure, often through dual immersion education, who are increasingly enrolling in university Spanish courses and tend to pattern with HLs. Expert instructor judgment and learner corpora contributed to item development, and 12 of 15 written multiple-choice test items targeting early-acquired vocabulary had differential item functioning (DIF) according to the Mantel–Haenszel procedure, favoring HLs. Recursive partitioning revealed that vocabulary score correctly identified 597/603 (99%) of L2 learners as such, and the six HLs whose vocabulary scores incorrectly identified them as L2 learners were in the lowest placement groups. Vocabulary scores also correctly identified 100% of the early L2 learners in the sample (n = 7) as having a heritage profile. Implications for the local context and for placement testing in general are provided.

Key words DIF, dual immersion, heritage learners, placement testing, Spanish

Local placement test retrofit and building language assessment literacy with teacher stakeholders: A case study from Colombia

Gerriet Janssen, Educational Testing Service, USA

Abstract This article provides a single, common-case study of a test retrofit project at one Colombian university. It reports on how the test retrofit project was carried out and describes the different areas of language assessment literacy the project afforded local teacher stakeholders. This project was successful in that it modified the test constructs and item types, while drawing stronger connections between the curriculum and the placement instrument. It also established a conceptual framework for the test and produced a more robust test form, psychometrically. The project intersected with different social forces, which impacted the project’s outcome in various ways. The project also illustrates how test retrofit provided local teachers with opportunities for language assessment literacy and with evidence-based knowledge about their students’ language proficiency. The study concludes that local assessment projects have the capacity to benefit local teachers, especially in terms of increased language assessment literacy. Intrinsic to a project’s sustainability are long-term financial commitment and institutionally established dedicated time, assigned to teacher participants. The study also concludes that project leadership requires both assessment and political skill sets, to conduct defensible research while compelling institutions to see the potential benefits of an ongoing test development or retrofit project.

Key words Item analysis, language assessment literacy, Latin America, placement test, teachers, test retrofit

Developing a local academic English listening test using authentic unscripted audio-visual texts

Yena Park, Indiana University, USA

Senyung Lee, Northeastern Illinois University, USA

Sun-Young Shin, Indiana University, USA

Abstract Despite consistent calls for authentic stimuli in listening tests for better construct representation, unscripted texts have been rarely adopted in high-stakes listening tests due to perceived inefficiency. This study details how a local academic listening test was developed using authentic unscripted audio-visual texts from the local target language use (TLU) domain without compromising the reliability of the test results and validity of the score interpretations. The purpose of the listening test was to identify international students who need additional language support at a U.S. university. We show that efficiency persists when using authentic unscripted texts that are representative of the local context both at the test development phase and at the classification phase where placement decisions are made in a dependable manner. Expert judgments highlighted the improved correspondence of the listening test using locally sourced audio-visual texts to the local TLU domain, providing additional support for using the listening test for local placement purposes. Additionally, dimensionality assessments demonstrated that test design decisions inevitably entailed with using authentic unscripted texts did not threaten the internal structure of the test. We argue that local resources are indispensable in developing authentic test stimuli and in supporting the validity of local test interpretation and use.

Key words Authentic listening test, L2 listening, testing listening, unscripted spoken texts, video listening test

Bridging local needs and national standards: Use of standards-based individualized feedback of an in-house EFL listening test in China

Shangchao Min, Juan Zhang, Yue Li, Lianzhen He, Zhejiang University, China

Abstract Local language tests are an arena where national language standards can be operationalized to create a hub for integrating assessment results and language support. Few studies, however, have examined the operationalization of national standards in local language assessment contexts. In this study, we proposed a model to present the integration of national standards in local contexts to guide assessment, teaching, and learning. Using this model, we conducted a 5-month longitudinal study involving 689 college students to examine (1) the consistency between their internal and external assessment feedback (i.e., standards-based self-assessment ratings and standards-based individualized English as a Foreign Language [EFL] listening test feedback) and (2) the effectiveness of standards-based teaching intervention in enhancing their perceived and actual language development. The results showed that the test feedback generally aligned well with students’ self-assessment and perceptions at the overall listening skill and subskill levels, yet student perceptions outlined needs for feedback refinements. In addition, the use of the standards-based individualized feedback, in conjunction with language support courses and practice materials, facilitated students’ perceived and actual listening achievement. This study makes an important contribution to local language testing by demonstrating the potential of a local instrument to provide a bridge between local instructional goals and national standards.

Key words Individualized feedback, local language tests, national standards, self-assessment, tailored instruction

Local English testing in China’s tertiary education: Contexts, policies, and practices

Jason Fan, The University of Melbourne, Australia

Kellie Frost, The University of Melbourne, Australia

Yan Jin, Shanghai Jiao Tong University, China

Abstract Accompanying the still pervasive influence of the College English Test (CET), some universities in China, especially the higher-ranking ones, have in recent years devised and implemented their own local English testing regimes. This significant shift, however, has captured limited research attention. Building on considerations in local language testing and adopting a broad view of language tests as embedded within policy spaces, we examined the policy and curricular contexts surrounding the introduction of local testing regimes, the local testing practices that emerged from these contexts, and stakeholders’ reactions to changes in local testing practices at China’s universities. Data were collected through in-depth, semi-structured interviews with local testing project coordinators from five prominent universities in China. Our findings revealed two contrasting approaches to local testing, each implemented to promote productive English skills, widely associated with university-level internationalisation efforts, and to better align assessment practices with university-specific student cohorts and College English (CE) teaching programmes. Each local testing regime was characterised by different processes of test development and implementation, and its tensions and challenges. This study provides important insights into the situated nature of local testing policies and practices in Chinese universities, highlighting the affordances and constraints offered by different approaches to local testing.

Key words Language testing in China, local language testing, local testing policy, test use, testing practices

National assessment of foreign languages in Sweden: A multifaceted and collaborative venture

Gudrun Erickson, Linda Borger, Eva Olsson, University of Gothenburg, Sweden

Abstract The article addresses the local system of national assessment of foreign languages in Sweden, a contextually specific, large-scale system with a summative aim, but also a system aimed to support teachers in their continuous assessment and grading of their students’ competences. In the text, the educational context and the multifaceted nature of national assessment are described and discussed. Furthermore, based on a broad view of validity with use and consequences in focus, different, and partly interwoven, aspects of collaboration in test development are exemplified and discussed, including policy, stakeholders, and research. Special attention is given to contributions by stakeholders, in particular students and teachers. Their involvement is regarded as a central component in the test development process, not only because it widens, deepens, and further develops the competences needed, but also because it increases the possibility to affect and enhance the use of the materials for the justice and beneficence of test-takers and society at large—aspects at the heart of validity. It is emphasized that collaboration requires sensitivity and sensibility from those involved to optimize overall quality and generate reciprocal benefits for all parties.

Key words Collaboration, foreign languages, national assessment in Sweden, policy, reciprocity, research, stakeholders, test development, validity

第4期目录

ARTICLES

■ Assessing Rasch measurement estimation methods across R packages with yes/no vocabulary test data, by Christopher Nicklin, Joseph P. Vitta, Pages 513-540.

■ IRT-based classification analysis of an English language reading proficiency subtest, by Elif Kaya, Stefan O’Grady, İlker Kalender, Pages 541–566.

■Development and initial validation of productive vocabulary tests for isiZulu, Siswati and English in South Africa, by Carien Wilsenach, Maxine Schaefer, Pages 567–592.

■ Roles of working memory, syllogistic inferencing ability, and linguistic knowledge on second language listening comprehension for passages of different lengths, by Minkyung Kim, Yunjung Nam, Scott A. Crossley, Pages 593–617.

■ Register variation in spoken and written language use across technology-mediated and non-technology-mediated learning environments, by Kristopher Kyle, Masaki Eguchi, Ann Tai Choe, Geoff LaFlair, Pages 618–648.

■Critical language assessment literacy of EFL teachers: Scale construction and validation, by Zia Tajeddin, Mohammad Khatib, Mohsen Mahdavi, Pages 649–678.

TEST REVIEW

■ Test Review: The International English Language Testing System (IELTS), by John Read, Pages 679–694.

BOOK REVIEW

■ Book review: Another Generation of Fundamental Considerations in Language Assessment: A Festschrift in Honor of Lyle F. Bachman, by Ying Xu, Xiaodong Li, Pages 695–698.

第4期摘要

Assessing Rasch measurement estimation methods across R packages with yes/no vocabulary test data

Christopher Nicklin, Rikkyo University, Japan

Joseph P. Vitta, Kyushu University, Japan

Abstract Instrument measurement conducted with Rasch analysis is a common process in language assessment research. A recent systematic review of 215 studies involving Rasch analysis in language testing and applied linguistics research reported that 23 different software packages had been utilized. However, none of the analyses were conducted with one of the numerous R-based Rasch analysis software packages, which generally employ one of the three estimation methods: conditional maximum likelihood estimation (CMLE), joint maximum likelihood estimation (JMLE), or marginal maximum likelihood estimation (MMLE). For this study, eRm, a CMLE-based R package, was utilized to conduct a dichotomous Rasch analysis of a Yes/No vocabulary test based on the academic word list. The resulting parameters and diagnostic statistics were compared with the equivalent results from four other R-based Rasch measurement software packages and Winsteps. Finally, all of the packages were utilized in the analysis of 1000 simulated datasets to investigate the extent to which results generated from the contrasting estimation methods converged or diverged. Overall, the differences between the results produced with the three estimation methods were negligible, and the discrepancies observed between datasets were attributable to the software choice as opposed to the estimation method.

Key words R, Rasch analysis, simulation, vocabulary, yes/no test

IRT-based classification analysis of an English language reading proficiency subtest

Elif Kaya, Stefan O’Grady, İlker Kalender, Bilkent University, Turkey

Abstract Language proficiency testing serves an important function of classifying examinees into different categories of ability. However, misclassification is to some extent inevitable and may have important consequences for stakeholders. Recent research suggests that classification efficacy may be enhanced substantially using computerized adaptive testing (CAT). Using real data simulations, the current study investigated the classification performance of CAT on the reading section of an English language proficiency test and made comparisons with the paper-based version of the same test. Classification analysis was carried out to estimate classification accuracy (CA) and classification consistency (CC) by applying different locations and numbers of cutoff points. The results showed that classification was suitable when a single cutoff score was used, particularly for high- and low-ability test takers. Classification performance declined significantly when multiple cutoff points were simultaneously employed. Content analysis also raised important questions about construct coverage in CAT. The results highlight the potential for CAT to serve classification purposes and outline avenues for further research.

Key words Classification accuracy, classification consistency, computerized adaptive testing, language proficiency, rudner approach

Development and initial validation of productive vocabulary tests for isiZulu, Siswati and English in South Africa

Jean-Marc Dewaele, Department of Applied Linguistics and Communication, Birkbeck, University of London, London, UK

Abstract Multilingualism in education is encouraged in South Africa, and children are expected to become bilingual and biliterate during the early primary grades. Much focus has been placed on measuring literacy in children’s first language, often the medium of instruction (MOI), and English, the language typically used as MOI from fourth grade. However, vocabulary development in African contexts is underexplored, owing to the cost of existing English standardized tests, and the comparatively fewer linguistically and contextually appropriate vocabulary assessments in African languages. To address this gap, we document the development of corpus-informed contextually appropriate tests of productive vocabulary in isiZulu, Siswati, and English, which were used for a project evaluation. The initial validation phase included 412 children. Both tests were reliable and were concurrently validated with reading comprehension tests in each language, and oral language skills in English. This study contributes to our understanding of the factors that affect the variation in vocabulary knowledge in an African context, including age, grade repetition, and vocabulary in the other language. Only English vocabulary was affected by the remote rural location of the school. We recommend some modifications to the tests before they are validated further in other populations.

Key words Emergent bilingual, English second language, isiZulu, productive vocabulary, reading comprehension, Siswati, South Africa

Roles of working memory, syllogistic inferencing ability, and linguistic knowledge on second language listening comprehension for passages of different lengths

Minkyung Kim, Nagoya University of Commerce and Business, Japan

Yunjung Nam, Georgia State University, USA

Scott A. Crossley, Georgia State University, USA

Abstract This study investigated the effects of working memory capacity (WMC), first language (L1) syllogistic inferencing ability, and second-language (L2) linguistic knowledge on L2 listening comprehension for passages of different lengths. Participants were 193 Korean ninth-grade learners of English. A path analysis was used to examine multivariate relationships among variables. Findings indicated that L2 linguistic knowledge was pivotal in explaining L2 listening comprehension for passages of different lengths. Findings also indicated that over and above L2 linguistic knowledge, greater WMC facilitated comprehending longer L2 listening passages, while better L1 syllogistic inferencing ability facilitated comprehending shorter L2 listening passages. WMC may help form more information-dense representations of longer passages, while L1 syllogistic inferencing ability may help build integrated propositional representations of shorter passages. In addition, greater WMC had indirect impacts on L2 listening comprehension through L1 syllogistic inferencing ability and L2 linguistic knowledge, which suggests that WMC may lead to better L2 listening comprehension when learners have greater L2 knowledge or better L1 syllogistic inferencing ability. Overall, first, this study suggests a pivotal role for L2 linguistic knowledge in L2 listening comprehension and, second, roles for WMC and L1 syllogistic inferencing, which function differently depending on passage length.

Key words Linguistic knowledge, listening comprehension, passage length, syllogistic inferencing, working memory

Register variation in spoken and written language use across technology-mediated and non-technology-mediated learning environments

Kristopher Kyle, University of Oregon, USA

Masaki Eguchi, University of Oregon, USA

Ann Tai Choe, University of Hawaiʻi at Mānoa, USA

Geoff LaFlair, University of Hawaiʻi at Mānoa, USA

Abstract In the realm of language proficiency assessments, the domain description inference and the extrapolation inference are key components of a validity argument. Biber et al.’s description of the lexicogrammatical features of the spoken and written registers in the T2K-SWAL corpus has served as support for the TOEFL iBT test’s domain description and extrapolation inferences. In the time since the T2K-SWAL corpus was collected, however, university learning environments have increasingly become technology-mediated. Accordingly, any description of the linguistic features of university language should account for the language produced in technology-mediated learning environments (TMLEs) in addition to non-technology-mediated learning environments (non-TMLEs). Kyle et al. recently began to address this issue by collecting a corpus of TMLE language use, which they then compared to language use in non-TMLEs using multidimensional analysis (MDA). The results indicated both similarities and substantive differences across the learning environments, but the study did not investigate the effects of particular registers on these results. In this study, we build on previous research by investigating lexicogrammatical features of specific spoken and written registers across technology-mediated and non-technology-mediated learning environments.

Key words Academic English proficiency, corpus linguistics, domain description, technology-mediated learning environments, validity argument

Critical language assessment literacy of EFL teachers: Scale construction and validation

Zia Tajeddin, Tarbiat Modares University, Iran

Mohammad Khatib, Allameh Tabataba’i University, Iran

Mohsen Mahdavi, Allameh Tabataba’i University, Iran

Abstract Critical language assessment (CLA) has been addressed in numerous studies. However, the majority of the studies have overlooked the need for a practical framework to measure the CLA dimension of teachers’ language assessment literacy (LAL). This gap prompted us to develop and validate a critical language assessment literacy (CLAL) scale to further underscore the role of CLA principles and their practice as an essential part of teachers’ LAL. In the first phase, a pool of items was generated through a comprehensive review of the related studies. In the quantitative phase, the developed scale was administered to 255 English as a foreign language teachers selected through convenience and snowball sampling. The data were analyzed through exploratory factor analysis for construct validity and Cronbach’s alpha for estimating internal consistency. The results showed that the items loaded on five factors: (a) teachers’ knowledge of assessment objectives, scopes, and types; (b) assessment use consequences; (c) fairness; (d) assessment policies; and (e) national policy and ideology. It was found that the scale had a high level of internal consistency and construct validity, which suggests that this scale has the potential to be useful in assessing language teachers’ CLAL and to raise language teachers’ awareness of CLAL constructs.

Key words Assessment culture, critical language assessment, critical language assessment literacy, language assessment, language assessment literacy, teacher assessment literacy

期刊简介

Language Testing is an international peer reviewed journal that publishes original research on foreign, second, additional, and bi-/multi-/trans-lingual (henceforth collectively called L2) language testing, assessment, and evaluation. Since 1984 it has featured high impact L2 testing papers covering theoretical issues, empirical studies, and reviews. The journal's scope encompasses the testing, assessment, and evaluation of spoken and signed languages being learned as L2s by children and adults, and the use of tests as research and evaluation tools that are used to provide information on the language knowledge and language performance abilities of L2 learners. Many articles also contribute to methodological innovation and the practical improvement of L2 testing internationally. In addition, the journal publishes submissions that deal with L2 testing policy issues, including the use of tests for making high-stakes decisions about L2 learners in fields as diverse as education, employment, and international mobility.

《语言测试》是一份国际同行评审期刊，发表关于外国、第二、辅助和双/多/跨语言（以下统称为L2）语言测试、评估和评估的原创研究。自1984年以来，它以高影响力的L2测试论文为特色，涵盖理论问题、实证研究和评论。该期刊的范围包括对儿童和成人作为L2学习的口语和手语的测试和评估，以及使用测试作为研究和评估工具，用于提供有关语言知识和语言表现的信息L2学习者的能力。许多文章还为国际上二语测试的方法创新和实际改进做出了贡献。此外，该期刊还发表处理L2测试政策问题的论文，包括使用测试对L2学习者在教育、就业和国际流动等不同领域做出高风险决策。
The journal welcomes the submission of papers that deal with ethical and philosophical issues in L2 testing, as well as issues centering on L2 test design, validation, and technical matters. Also of concern is research into the washback and impact of L2 language test use, the consequences of testing on L2 learner groups, and ground-breaking uses of assessments for L2 learning. Additionally, the journal wishes to publish replication studies that help to embed and extend knowledge of generalisable findings in the field. Language Testing is committed to encouraging interdisciplinary research, and is keen to receive submissions which draw on current theory and methodology from different areas within second language acquisition, applied linguistics, educational measurement, psycholinguistics, general education, psychology, cognitive science, language policy, and other relevant subdisciplines that interface with language testing and assessment. Authors are encouraged to adhere to Open Science Initiatives.该期刊欢迎提交涉及 L2 测试中的伦理和哲学问题的论文，以及以 L2 测试设计、验证和技术问题为中心的问题。同样值得关注的是对 L2 语言测试使用的反作用和影响、测试对 L2 学习者群体的影响以及 L2 学习评估的开创性使用的研究。此外，该杂志希望发表有助于贡献和扩展该领域可推广发现的知识的重复研究。《语言测试》 致力于鼓励跨学科研究，并接收来自二语习得、应用语言学、教育测量、心理语言学、通识教育、心理学、认知科学、语言政策和与语言测试和评估相关的其他相关子学科研究。鼓励作者遵守开放科学倡议。

官网地址：

https://journals.sagepub.com/home/LTJ

本文来源：LANGUAGE TESTING官网

点击文末“阅读原文”可跳转下载

课程推荐

欢迎加入

“语言学心得交流分享群”“语言学考博/考研/保研交流群”
请添加“心得君”入群请务必备注“学校+研究方向/专业”

今日小编：丸子

审    核：心得小蔓

转载&合作请联系

"心得君"

微信：xindejun_yyxxd

点击“阅读原文”可跳转下载