A comparative study of analytical assessment and holistic assessment in English writing tests

Hao Wu

doi:10.54475/jlt.2021.001

Authors

Hao Wu The University of Oxford https://orcid.org/0000-0003-2383-1133

DOI:

https://doi.org/10.54475/jlt.2021.001

Keywords:

analytical assessment, holistic assessment, English writing, writing assessment

Abstract

The choice of assessment method is particularly important in English writing testing and assessment. Analytical assessment and holistic assessment are two common methods in English writing assessment. The choice of assessment method usually depends on the content and focus of the test. This paper provides a review of previous empirical studies and compares the two assessment methods in terms of reliability, construct validity, practicability, and impact, followed by suggestions in the selection, design, and operation of the assessment methods.

References

Bacha, N. (2001). Writing evaluation: What can analytic versus holistic essay scoring tell us? System, 29(3), 371-383. https://doi.org/10.1016/S0346-251X(01)00025-2

Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University Press.

Bauer, B. A. (1981). A study of the reliabilities and the cost-efficiencies of three methods of assessment for writing ability.

Beeth, M. E., Cross, L., Pearl, C., Pirro, J., Yagnesak, K., & Kennedy, J. (1999). A continuum for assessing science process knowledge in Grades K-6.

Campbell, D. M., Melenyzer, B. J., Nettles, D. H., & Wyman Jr, R. M. (1999). Portfolio and performance assessment in teacher education.

Cohen, Y. (2017). Estimating the intra-rater reliability of essay raters. Frontiers in Education, 2(49). https://doi.org/10.3389/feduc.2017.00049

Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T. (1999). Dictionary of language testing (Vol. 7). Cambridge: Cambridge University Press.

Denner, P. R., Salzman, S. A., & Harris, L. B. (2002). Teacher work Sample assessment: An accountability method that moves beyond teacher testing to the impact of teacher performance on student learning.

East, M., & Young, D. (2007). Scoring L2 writing samples: Exploring the relative effectiveness of two different diagnostic methods (Vol. 13): Applied Linguistics Association of New Zealand.

ETS. (2019, April 5, 2021). TOEFL iBT Test Writing Rubrics. Retrieved from https://www.ets.org/s/toefl/pdf/toefl_writing_rubrics.pdf

Ghalib, T. K., & Al-Hattami, A. A. (2015). Holistic versus analytic evaluation of EFL writing: A case study. English Language Teaching, 8(7), 225-236.

Hamp-Lyons, L. (1991). Pre-text: Task-related influences on the writer. In L. Hamp-Lyons (Ed.), Assessing second language writing in academic contexts. Norwood, NJ: Ablex.

Hunter, D. M., Jones, R. M., & Randhawa, B. S. (1996). The use of holistic versus analytic scoring for large-scale assessment of writing. The Canadian Journal of Program Evaluation, 11(2), 61.

Huot, B. (1996). Toward a new theory of writing assessment. College Composition and Communication, 47, 549-566. https://doi.org/10.2307/358601

Jacobs, H., Zinkgraf, S., Wormuth, D., Hearfiel, V., & Hughey, J. (1981). Testing ESL composition: A practical approach.

Jonsson, A., & Svingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130-144. https://doi.org/10.1016/j.edurev.2007.05.002

Knoch, U. (2009). Diagnostic assessment of writing: A comparison of two rating scales. Language Testing, 26(2), 275-304. https://doi.org/10.1177/0265532208101008

Li, H., & He, L. (2015). A comparison of EFL raters’ essay-rating processes across two types of rating scales. Language Assessment Quarterly, 12(2), 178-212. https://doi.org/10.1080/15434303.2015.1011738

Luft, J. A. (1999). Rubrics: Design and use in science teacher education. Journal of Science Teacher Education, 10(2), 107-121. https://doi.org/10.1023/A:1009471931127

Lumley, T. (2005). Assessing second language writing: The rater's perspective: P. Lang.

Nakamura, Y. (2004). A comparison of holistic and analytic scoring methods in the assessment of writing. Paper presented at the 3rd annual JALT Pan-SIG Conference.

Popp, S. E. O., Ryan, J. M., Thompson, M. S., & Behrens, J. T. (2003). Operationalizing the rubric: The effect of benchmark selection on the assessed quality of writing.

Rezaei, A. R., & Lovorn, M. (2010). Reliability and validity of rubrics for assessment through writing. Assessing Writing, 15(1), 18-39. https://doi.org/10.1016/j.asw.2010.01.003

Schamber, J. F., & Mahoney, S. L. (2006). Assessing and improving the quality of group critical thinking exhibited in the final projects of collaborative learning groups. The Journal of General Education, 103-137. https://doi.org/10.2307/jgeneeduc.55.2.0103

Stuhlmann, J., Daniel, C., Dellinger, A., Kenton, R., & Powers, T. (1999). A generalizability study of the effects of training on teachers’ abilities to rate children’s writing using a rubric. Reading Psychology, 20(2), 107-127. https://doi.org/10.1080/027027199278439

Waltman, K., Kahn, A., & Koency, G. (1998). Alternative approaches to scoring: The effects of using different scoring methods on the validity of scores from a performance assessment.

Weigle, S. C. (1999). Investigating rater/prompt interactions in writing assessment: Quantitative and qualitative approaches. Assessing Writing, 6(2), 145-178. https://doi.org/10.1016/S1075-2935(00)00010-6

Weigle, S. C. (2002). Assessing Writing. Cambridge: Cambridge University Press.

Weir, C. J. (1990). Communicative Language Testing. London: Prentice Hall.

Zhang, B., Xiao, Y., & Luo, J. (2015). Rater reliability and score discrepancy under holistic and analytic scoring of second language writing. Language Testing in Asia, 5(1), 1-9. https://doi.org/10.1186/s40468-015-0014-4