Exploring the shift to an improvement-oriented approach to external evaluation in Aotearoa New Zealand
The case of the Education Review Office
DOI:
https://doi.org/10.26686/nzaroe.v28.8359Abstract
External evaluation in Aotearoa New Zealand is an important accountability mechanism in education. In 2019 the Tomorrow’s Schools Independent Taskforce recommended that the Education Review Office (ERO) develop and implement an improvement-oriented approach to external evaluation in schools. This approach requires fundamental shifts in evaluation practice. In implementing an improvement-oriented approach, while maintaining accountability functions in a public sector context, evaluators need to balance key tensions: relational, epistemological, pedagogical, contextual, political, methodological, and organisational. The role of the evaluator in implementing ERO’s new approach, and managing the shifts required, is key to the approach’s success. Building evaluation capability and capacity and strengthening the evaluation evidence base are critical areas for further development.
Downloads
References
Adams, A., Nnawulezi, N., & Vandenberg, L. (2014). “Expectations to Change” (E2C): A participatory method for facilitating stakeholder engagement with evaluation findings. American Journal of Evaluation, 36(2), 243-255. https://doi.org/10.1177/1098214014553787
Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16-25.
https://doi.org/10.3102/0013189X11428813
Austin, M., Parata-Blane, A., & Edwards, W. (1997). Achieving excellence: A review of the education external evaluation services. Ministry of Education.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school. National Academy Press.
Bryk, A. S. (2015). Accelerating how we learn to improve. Educational Researcher, 44(9), 467-477. https://doi.org/10.3102/0013189X15621543
Chelimsky, E. (1997). The coming transformations in evaluation. In E. Chelimsky & W. Shadish (Eds.), Evaluation for the 21st century: A handbook (pp. 1-26). Sage.
Chouinard, J. (2013). The case for participatory evaluation in an era of accountability. American Journal of Evaluation 34(2), 237-253. https://doi.org/10.1177/1098214013478142
Cousins, B., Goh, S., Elliott, C., & Bourgeois, I. (2014). Framing the capacity to do and use evaluation. In J. B. Cousins & I. Bourgeois (Eds.), Special issue: Organizational capacity to do and use evaluation. New Directions for Evaluation, 141, 7-23. https://doi.org/10.1002/ev.20076
Cram, F., Kennedy, V., Paipa, K., Pipi, K., & Wehipeihana, N. (2015). Being culturally responsive through kaupapa Māori evaluation. In S. Hood, R. Hopson, & H. Frierson (Eds.), Continuing the journey to reposition culture and cultural context in evaluation theory and practice, (pp. 289-311). University of Auckland.
Department of Education. (1988). Tomorrow’s Schools: The Reform of Education Administration in New Zealand. Government Printer.
Earl, L. (2014). Effective school review: Considerations in the framing, definition, identification and selection of indicators of education quality and their potential use in evaluation in the school setting. Background paper prepared for the review of the Education Review Office’s Evaluation Indicators for School Reviews.
Education Review Office. (1996). Annual Report: 1 July 1996 to 30 June 1997. Education Review Office.
Education Review Office. (2003). Manual of standard procedures for education reviews. Education Review Office.
Education Review Office. (2016). School evaluation indicators: Effective practice for improvement and learner success. Education Review Office.
https://ero.govt.nz/how-ero-reviews/schoolskura-english-medium/school-evaluation-indicators
Education Review Office. (2021). Principles of practice: Education evaluation for improvement in schools and early childhood centres. Education Review Office. https://ero.govt.nz/sites/default/files/2021-11/Principles%20of%20Practicev8.pdf
Education Review Office. (2022). Te ara huarau: Your go-to-guide. Education Review Office. https://ero.govt.nz/sites/default/files/2022-04/Te%20Ara%20Huarau%20Go%20To%20Guide%20March%202022.pdf
Education Review Office. (2023). Strategic intentions 2023-2026. Education Review Office. https://ero.govt.nz/our-strategies/strategic-intentions-2023-2026
Fierro, L., & Christie, C. (2017). Evaluator and programme manage perceptions of evaluation capacity and evaluation practice. American Journal of Evaluation, 38(3), 376-392. https://doi.org/10.1177/1098214016667581
French, A. (2000). The heart of the matter: How the Education Review Office evaluates pre-tertiary education. Cases in public sector innovation; No. 9. Victoria Link Ltd. Victoria University of Wellington.
Fullan, M., Rincon-Gallardo, S., & Hargreaves, A. (2015). Professional capital as accountability. Education Policy Analysis Archives, 23(15), 1-22. https://doi.org/10.14507/epaa.v23.1998
Galport, N., & Azzam, T. (2017). Evaluator training needs and competencies: A gap analysis. American Journal of Evaluation, 38(1), 80-100. https://doi.org/10.1177/1098214016643183
Garcia, G., & Stevahn, L. (2020). Situational awareness and interpersonal competence as evaluator competencies. American Journal of Evaluation, 41(1), 107-124.
https://doi.org/10.1177/1098214018814941
Goodrick, D. (2022). Schools evaluation for improvement approach: Implementation case studies.
Haynes, M., & Johnson, A. (2017). Training needs of site visitors. In R. K. Nelson, & D. L. Roseland (Eds.), Conducting and using evaluative site visits, 156 (pp. 75-82). Jossey-Bass and American Evaluation Association. https://doi.org/10.1002/ev.20268
Hofstetter, C., & Alkin, M. (2003). Evaluation use revisited. In T. Kellaghan, & D. Stufflebeam (Eds.), International handbook of educational evaluation (pp. 197-222). Springer.
Labin, S., Duffy, J., Meyers. D., Wandersman, A., & Lesesne, C. (2012). A research synthesis of the evaluation capacity building literature. American Journal of Evaluation, 33(3), 307-338. https://doi.org/10.1177/1098214011434608
LaFrance, J., Nichols, R., & Kirkhart, K. (2012). Culture writes the script: On the centrality of context in indigenous evaluation. In D. J. Rog, J. L. Fitzpatrick, & R. F. Conner (Eds.), Context: A framework for its influence on evaluation practice, (pp. 59-74). https://doi.org/10.1002/ev.20027
Lai, M. K., & McNaughton, S. (2016). The impact of data use professional development on student achievement. Teaching and Teacher Education, 60, 434-443. https://doi.org/10.1016/j.tate.2016.07.005
MacBeath, J. (2012). Future of teaching profession. University of Cambridge.
Mark, M., Henry, G., & Julnes, G. (2000). Evaluation: An integrated framework for understanding, guiding, and improving policies and programmes. Jossey-Bass.
Ministry of Education. (1990). Today’s Schools: A review of the education reform implementation process. Report prepared for the Minister of Education, April, 1990. Ministry of Education.
Ministry of Education. (2019). Supporting all schools to succeed. Reform of the Tomorrow’s Schools system. Ministry of Education. https://conversation.education.govt.nz/assets/TSR/November-2019/TSR-Government-Response-WEB.pdf
Mutch, C. (2012). Complementary Evaluation: The development of a conceptual framework to integrate external and internal evaluation in the New Zealand school context. Policy Futures in Education, 10(5), 569-586. https://doi.org/10.2304/pfie.2012.10.5.569
Mylopoulos, M., & Regehr, G. (2009). How student models of expertise and innovation impact the development of adaptive expertise in medicine. Medical Education, 43(2), 12-132. https://doi.org/10.1111/j.1365-2923.2008.03254.x
Nusche, D., Laveault, D., MacBeath, J., & Santiago, P. (2012). OECD reviews of evaluation and assessment in education: New Zealand 2011. OECD Publishing.
Parsons, R. (2006). External evaluation in New Zealand schooling (Unpublished doctoral thesis). Massey University.
Patton, M. Q. (2008). Utilization-focused evaluation (4th ed.). Sage.
Penninckx, M. (2017). Effects and side effects of school inspections: A general framework. Studies in Educational Evaluation, 52, 1-11. https://doi.org/10.1016/j.stueduc.2016.06.006
Preskill, H. (2014). Now for the hard stuff: Next steps in ECB research and practice. American Journal of Evaluation, 35(1), 116-119. https://doi.org/10.1177/1098214013499439
Preskill, H., & Torres, R. (2000). The learning dimension of evaluation use. In V. Caracelli, & H. Preskill (Eds.), The expanding scope of evaluation use. New Directions for Evaluation, 88, (pp. 25-37). https://doi.org/10.1002/ev.1189
Rodger, S., Holden, J., Meade, A., Millar, A., & Smith, B. (2000). Report to the Minister of Education: A review of the roles and responsibilities of the Education Review Office. Ministry of Education.
Schmidt, W., Burroughs, N., Ziodo, P., & Houang, R. (2015). The role of schooling in perpetuating educational inequality: An international perspective. Educational Researcher, 44(7), 371-386. https://doi.org/10.3102/0013189X15603982
Schulha, L., Whitmore, E., Cousins, J. B., Gilbert, N., & al Hudib, H. (2016). Introducing evidence-based principles to guide collaborative approaches to evaluation: Results of an empirical process. American Journal of Evaluation, 37(2), 93-215. https://doi.org/10.1177/1098214015615230
Schwandt, T. (2015) Evaluation Foundations Revisited : Cultivating a Life of the Mind for Practice, Stanford University Press, 2015. ProQuest Ebook Central, https://ebookcentral.proquest.com/lib/vuw/detail.action?docID=2038803.
Taskforce to Review Education Administration. (1988). Administering for Excellence: Effective Administration in Education: Report of the Taskforce to Review Education Administration. The Taskforce.
Thrupp, M., & Smith, R. (1999). A decade of ERO. In M. Thrupp (Ed.), A Decade of Reform in New Zealand Education: Where to Now? Special Issue of The New Zealand Journal of Educational Studies, 186-198.
Timperley, H. (2013). Learning to Practise: A paper for discussion. Ministry of Education.
http://hdl.voced.edu.au/10707/257067
Tomorrow’s Schools Independent Taskforce. (2019). Our schooling futures: Stronger together: Whiria ngā kura tuātinitini. Ministry of Education. https://conversation.education.govt.nz/assets/TSR/Tomorrows-Schools-Review-Report-13Dec2018.PDF
Downloads
Published
Issue
Section
License
The Author(s) retain ownership of the copyright in the Article but hereby grant the Publisher an exclusive license to publish the article.
NZAROE gives authors full permission to deposit their articles in publicly accessible institutional repositories, providing that:
- Articles are placed in repositories after publication.
- Metadata about articles include the DOI and journal issue information.
