A comparative study of analytical assessment and holistic assessment in English writing tests
Keywords:analytical assessment, holistic assessment, English writing, writing assessment
The choice of assessment method is particularly important in English writing testing and assessment. Analytical assessment and holistic assessment are two common methods in English writing assessment. The choice of assessment method usually depends on the content and focus of the test. This paper provides a review of previous empirical studies and compares the two assessment methods in terms of reliability, construct validity, practicability, and impact, followed by suggestions in the selection, design, and operation of the assessment methods.
Bacha, N. (2001). Writing evaluation: What can analytic versus holistic essay scoring tell us? System, 29(3), 371-383. https://doi.org/10.1016/S0346-251X(01)00025-2
Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.
Bauer, B. A. (1981). A study of the reliabilities and the cost-efficiencies of three methods of assessment for writing ability.
Beeth, M. E., Cross, L., Pearl, C., Pirro, J., Yagnesak, K., & Kennedy, J. (1999). A continuum for assessing science process knowledge in Grades K-6.
Campbell, D. M., Melenyzer, B. J., Nettles, D. H., & Wyman Jr, R. M. (1999). Portfolio and performance assessment in teacher education.
Cohen, Y. (2017). Estimating the intra-rater reliability of essay raters. Frontiers in Education, 2(49). https://doi.org/10.3389/feduc.2017.00049
Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T. (1999). Dictionary of language testing (Vol. 7). Cambridge: Cambridge University Press.
Denner, P. R., Salzman, S. A., & Harris, L. B. (2002). Teacher work Sample assessment: An accountability method that moves beyond teacher testing to the impact of teacher performance on student learning.
East, M., & Young, D. (2007). Scoring L2 writing samples: Exploring the relative effectiveness of two different diagnostic methods (Vol. 13): Applied Linguistics Association of New Zealand.
ETS. (2019, April 5, 2021). TOEFL iBT Test Writing Rubrics. Retrieved from https://www.ets.org/s/toefl/pdf/toefl_writing_rubrics.pdf
Ghalib, T. K., & Al-Hattami, A. A. (2015). Holistic versus analytic evaluation of EFL writing: A case study. English Language Teaching, 8(7), 225-236.
Hamp-Lyons, L. (1991). Pre-text: Task-related influences on the writer. In L. Hamp-Lyons (Ed.), Assessing second language writing in academic contexts. Norwood, NJ: Ablex.
Hunter, D. M., Jones, R. M., & Randhawa, B. S. (1996). The use of holistic versus analytic scoring for large-scale assessment of writing. The Canadian Journal of Program Evaluation, 11(2), 61.
Huot, B. (1996). Toward a new theory of writing assessment. College Composition and Communication, 47, 549-566. https://doi.org/10.2307/358601
Jacobs, H., Zinkgraf, S., Wormuth, D., Hearfiel, V., & Hughey, J. (1981). Testing ESL composition: A practical approach.
Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144. https://doi.org/10.1016/j.edurev.2007.05.002
Knoch, U. (2009). Diagnostic assessment of writing: A comparison of two rating scales. Language Testing, 26(2), 275-304. https://doi.org/10.1177/0265532208101008
Li, H., & He, L. (2015). A comparison of EFL raters’ essay-rating processes across two types of rating scales. Language Assessment Quarterly, 12(2), 178-212. https://doi.org/10.1080/15434303.2015.1011738
Luft, J. A. (1999). Rubrics: Design and use in science teacher education. Journal of Science Teacher Education, 10(2), 107-121. https://doi.org/10.1023/A:1009471931127
Lumley, T. (2005). Assessing second language writing: The rater's perspective: P. Lang.
Nakamura, Y. (2004). A comparison of holistic and analytic scoring methods in the assessment of writing. Paper presented at the 3rd annual JALT Pan-SIG Conference.
Popp, S. E. O., Ryan, J. M., Thompson, M. S., & Behrens, J. T. (2003). Operationalizing the rubric: The effect of benchmark selection on the assessed quality of writing.
Rezaei, A. R., & Lovorn, M. (2010). Reliability and validity of rubrics for assessment through writing. Assessing Writing, 15(1), 18-39. https://doi.org/10.1016/j.asw.2010.01.003
Schamber, J. F., & Mahoney, S. L. (2006). Assessing and improving the quality of group critical thinking exhibited in the final projects of collaborative learning groups. The Journal of General Education, 103-137. https://doi.org/10.2307/jgeneeduc.55.2.0103
Stuhlmann, J., Daniel, C., Dellinger, A., Kenton, R., & Powers, T. (1999). A generalizability study of the effects of training on teachers’ abilities to rate children’s writing using a rubric. Reading Psychology, 20(2), 107-127. https://doi.org/10.1080/027027199278439
Waltman, K., Kahn, A., & Koency, G. (1998). Alternative approaches to scoring: The effects of using different scoring methods on the validity of scores from a performance assessment.
Weigle, S. C. (1999). Investigating rater/prompt interactions in writing assessment: Quantitative and qualitative approaches. Assessing Writing, 6(2), 145-178. https://doi.org/10.1016/S1075-2935(00)00010-6
Weigle, S. C. (2002). Assessing Writing. Cambridge: Cambridge University Press.
Weir, C. J. (1990). Communicative Language Testing. London: Prentice Hall.
Zhang, B., Xiao, Y., & Luo, J. (2015). Rater reliability and score discrepancy under holistic and analytic scoring of second language writing. Language Testing in Asia, 5(1), 1-9. https://doi.org/10.1186/s40468-015-0014-4
How to Cite
Copyright (c) 2021 Journal of Language Teaching
This work is licensed under a Creative Commons Attribution 4.0 International License.