A Randomized Control Trial Study: Evaluating Pedagogical Practices in Science Classrooms
Keywords:
United States, classroom observation, evaluation, randomized control trial, science, pedagogical practices, virtual professional development, virtual mentoring and coachingAbstract
In this randomized control study, we evaluated science teachers’ pedagogical practices via classroom observations following ongoing, intensive, and structured instructional support sessions. These sessions included virtual professional development (VPD) and virtual mentoring and coaching (VMC) that accompanied a literacy-infused science curriculum. Using a low-inference observational instrument, we explored the direct impact of VPD and VMC on fifth-grade science teachers’ observed time allocation in a randomized controlled trial (RCT) validation study. The observations were collected three times during science instruction from 121 teachers in 68 schools from 35 public school districts in the U.S. state of Texas, during the 2017-2018 school year. Preliminary findings revealed pedagogical differences in time allocation among teachers between treatment and control classrooms. We identified improved instructional practices within treatment classrooms, which suggests the intervention had a positive effect by enhancing the quality of pedagogy as well as the content-area instruction in science.
Downloads
References
Author. (1994).
Author. (2009).
Author. (2013).
Author. (2015a).
Author. (2015b)
Author. (2016).
Author. (2018a).
Author. (2018b).
Author. (2018c).
Author. (2019a).
Author. (2019b).
Author. (2020).
Author. (2021a).
Author. (2021b).
Author. (2021c).
Author. (2022a).
Author. (2022b).
Bang, E. (2013). Hybrid-mentoring programs for beginning elementary science teachers. International Journal of Education in Mathematics, 1(1), 1-15. https://ijemst.net/index.php/ijemst/article/view/31
Bang, E., & Luft, J. A. (2014). Exploring the written dialogues of two first-year secondary science teachers in an online mentoring program. Journal of Science Teacher Education, 25, 25-51. https://doi.org/10.1007/s10972-013-9362-z
Binmohsen, S. A., & Abrahams, I. (2022). Science teachers continuing professional development: Online vs. face-to-face. Research in Science & Technological Education, 40(3), 291-319. https://doi.org/10.1080/02635143.2020.1785857
Bragg, L. A., Walsh, C., & Heyeres, M. (2021). Successful design and delivery of online professional development for teachers: A systematic review of the literature. Computers & Education, 166, 104158. https://doi.org/10.1016/j.compedu.2021.104158
Brophy, J. E., & Good, T. L. (1974). Teacher-student relationships: Causes and consequences. Holt, Rinehart & Winston.
Buyers, A., Koba, S., Sherman, G., Scheppke, J., & Bolus, R. (2011). Developing a web-based mechanism for assessing teacher science knowledge. Journal of Science Teacher Education, 22, 273-289. https://doi.org/10.1007/s10972-011-9227-2
Bybee, R., & Landes, N. M. (1990). Science for life and living: An elementary school science program from Biological Sciences Improvement Study (BSCS). The American Biology Teacher, 52(2), 92-98. https://doi.org/10.2307/4449042
Cady, J. A., Aydeniz, M., & Rearden, K. T. (2011). E-learning environments for math and science teachers. Journal of Curriculum and Instruction, 5(1), 17-33. https://doi.org/10.3776/joci.2011.v5n1p17-33
Calderón, M., Slavin, R., & Sánchez, M. (2011). Effective instruction for English learners. The Future of Children, 21(1), 103-127. https://psycnet.apa.org/doi/10.1353/foc.2011.0007
Cardichon, J., Darling-Hammond, L., Yang, M., Scott, C., Shields, P. M., & Burns, D. (2020). Inequitable opportunity to learn: Student access to certified and experienced teachers. Learning Policy Institute. https://learningpolicyinstitute.org/media/392/download?inline&file=CRDC_Teacher_Access_REPORT.pdf
Carson, C. D., Callard, C., Gillespie, R., Choppin, J., & Amador, J. M. (2019). Bridging the distance: One-on-one coaching supports rural teachers. The Learning Professional, 40(6), 66-70. https://learningforward.org/journal/coaching/30872-2/
Carson, C., & Choppin, J. (2021). Coaching from a distance: Exploring video-based online coaching. Online Learning, 25(4), 104-124. https://doi.org/10.24059/olj.v25i4.2881
Cavanaugh, C., & Dawson, K. (2010). Design of online professional development in science content and pedagogy. A pilot study. Journal of Science Education and Technology, 19, 438-446. https://doi.org/10.1007/s10956-010-9210-2
Coley, J. D., & Hoffman, D. M. (1990). Overcoming learned helplessness in at risk readers. Journal of Reading, 33, 497-502. http://www.jstor.org/stable/40032010
Community for Advancing Discovery Research in Education (CADRE). (2017). Emerging design principles for online and blended teacher professional development in K-12 STEM Education. Education Development Center, Inc. http://cadrek12.org/resources/emerging-design-principles-online-and-blended- teacher-professional-development-k-12-stem
Cummins, J. (1986). Empowering minority students: A framework for intervention. Harvard Educational Review, 56, 18-36. https://psycnet.apa.org/doi/10.17763/haer.56.1.b327234461607787
Darling-Hammond, L., Hyler, M. E., & Gardner, M. (2017). Effective teacher professional development. Learning Policy Institute. https://learningpolicyinstitute.org/sites/default/files/product-files/Effective_Teacher_Professional_Development_REPORT.pdf
Dede, C., Eisenkraft, A., Frumin, K., & Hartley, A. (2016). Teacher learning in the digital age: Online professional development in STEM education. Harvard Education Press.
Dede, C., Ketelhut, D. J., Whitehouse, P., Breit, L., & McCloskey, E. M. (2009). A research agenda for online teacher professional development. Journal of Teacher Education, 60(1), 8-19. https://doi.org/10.11772F0022487108327554
Diaz, S., Moll, L. C., & Mehan, H. (1970). Sociocultural resources in instruction: A context-specific approach. In Beyond language: Social and cultural factors in schooling language minority students (pp.187-230). Bilingual Education Office, California State Department of Education.
Fairman, J. C., Mills, T., & Lech, P. L. (2019). Challenges with teacher retention and staffing and shortages in Maine school districts. https://digitalcommons.usm.maine.edu/cgi/viewcontent.cgi?article=1041&context=cepare_improvement#:~:text=Maine%20schools%20experiencing%20the%20greatest,significantly%20lower%20than%20in%20other
Fischer, C., Fishman, B., Dede, C., Eisenkraft, A., Frumin, K., Foster, B., Lawrenz, F., Jurist Levy, A., & McCoy, A. (2018). Investigating relationships between school context, teacher professional development, teaching practices, and student achievement in response to a nationwide science reform. Teaching and Teacher Education, 72, 107-121. https://doi.org/10.1016/j.tate.2018.02.011
Fishman, B., Konstantopoulos, S., Kubitskey, B. W., Vath, R., Park, G., Johnson, H., & Edelson, D. C. (2013). Comparing the impact of online and face-to-face professional development in the context of curriculum implementation. Journal of Teacher Education, 64(5), 426-438. https://doi.org/10.1177/0022487113494413
Fishman, B., Marx, R., Best, S., & Tal, R. (2003). Linking teacher and student learning to improve professional development in systemic reform. Teaching and Teacher Education, 19(6), 643-658. https://doi.org/10.1016/S0742-051X(03)00059-3
Friedrichsen, P., Chval, K. B., & Teuscher, D. (2007). Strategies and sources of support for beginning teachers of science and mathematics. School Science and Mathematics, 107(5), 169-181. https://doi.org/10.1111/j.1949-8594.2007.tb17781.x
Gamez, R., & Parker, C. A. (2018). Becoming science learners: A study of newcomers’ identity work in elementary school science. Science Education, 102(2), 377-413. https://doi.org/10.1002/sce.21323
Gaudin, C., & Chaliès, S. (2015). Video viewing in teacher education and professional development: A literature review. Educational Research Review, 16, 41-67. https://doi.org/10.1016/j.edurev.2015.06.001
Germuth, A. A. (2018). Professional development that changes teaching and improves learning. Journal of Interdisciplinary Teacher Leadership, 2(1), 77-90. https://doi.org/10.46767/kfp.2016-0025
Gosselin, D. C., Thomas, J., Redmond, A., Larson-Miller, C., Yendra, S., Bonnstetter, R. J., & Slater, T. (2010). Laboratory Earth: A model of online K-12 teacher coursework. Journal of Geoscience Education, 58(4), 203-213. https://doi.org/10.5408/1.3534859
Gupta, A., & Lee, G. (2020). The effects of a site-based teacher professional development program on student learning. International Electronic Journal of Elementary Education, 12(5), 417-428. 10.26822/iejee.2020562132
Gwet, K. L. (2008). Computing inter‐rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology, 61(1), 29-48. https://doi.org/10.1348/000711006x126600
Hathaway, D., & Norton, P. (2012). An exploratory study comparing two modes of preparation for online teaching. Journal of Digital Learning in Teacher Education, 28(4), 146-152. https://doi.org/10.1080/21532974.2012.10784695
Jaber, L. Z., Dini, V., Hammer, D., & Danahy, E. (2018). Targeting disciplinary practices in an online learning environment. Science Education, 102(4), 668–692. https://doi.org/10.1002/sce.21340
Jackson, J. K., Huerta, M., Garza, T., & Narvaez, R. (2019). Examining the effects of a professional development initiative on English learning and economically disadvantaged adolescents’ scores on a high-stakes science test. Journal of Science Teacher Education, 30(2), 122-143. https://doi.org/10.1080/1046560X.2018.1538299
Joyce, B., & Calhoun, E. (2010). Models of professional development: A celebration of educators. Corwin Press.
Kim, S., Raza, M., & Seidman, E. (2019). Improving 21st-century teaching skills: The key to effective 21st-century learners. Research in Comparative and International Education, 14(1), 99-117. https://doi.org/10.1177/1745499919829214
Krashen, S. (1981). Second language acquisition and second language learning. Pergamon Press.
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159-174. https://doi.org/10.2307/2529310
Learning Forward. (2011). Standards for professional learning: Quick reference guide. Journal of Staff Development, 32(4), 41-44.
Lee, O., Maerten-Rivera, J., Penfield, R. D., LeRoy, K., & Secada, W. G. (2008). Science achievement of English language learners in urban elementary schools: Results of a first-year professional development intervention. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 45(1), 31-52. https://doi.org/10.1002/tea.20209
Lee, O., Grapin, S., & Haas, A. (2023). Teacher professional development programs integrating science and language with multilingual learners: A conceptual framework. Science Education, 107, 1302–1323. https://doi-org.srv-proxy2.library.tamu.edu/10.1002/sce.21807
Lee, S. C., Nugent, G., Kunz, G. M., Houston, J., & DeChenne-Peters, S. (2018). Case study: Value-Added benefit of distance-based instructional coaching on science teachers’ inquiry instruction in rural schools. Journal of Science Teacher Education, 29(3), 179–199. https://doi.org/10.1080/1046560x.2018.1432226
Llosa, L., Lee, O., Jiang, F., Haas, A., O’Connor, C., Van Booven, C. D., & Kieffer, M. J. (2016). Impact of a large-scale science intervention focused on English language learners. American Educational Research Journal, 53(2), 395-424. https://doi.org/10.3102/0002831216637348
Luz, M. R. M. P., Rolando, L. G. R., Salvador, D. F., & Souza, A. H. S. (2018). Characterization of the reasons why Brazilian science teachers drop out of online professional development courses. International Review of Research in Open and Distributed Learning, 19(5), 145-164. https://www.irrodl.org/index.php/irrodl/article/view/3642
McChesney, K., & Aldridge, J. M. (2021). What gets in the way? A new conceptual model for the trajectory from teacher professional development to impact. Professional Development in Education, 47(5), 834-852. https://doi.org/10.1080/19415257.2019.1667412
Melton, J., Miller, M., & Brobst, J. (2019). Mentoring the mentors: Hybridizing professional development to support cooperating teachers’ mentoring practice in science. Contemporary Issues in Technology and Teacher Education, 19(1), 23-44. https://citejournal.org/volume-19/issue-1-19/science/mentoring-the-mentors-hybridizing-professional-development-to-support-cooperating-teachers-mentoring-practice-in-science/
Meskill, C., & Oliveira, A. W. (2019). Meeting the challenges of English learners by pairing science and language educators. Research in Science Education, 49, 1025-1040. https://doi.org/10.1007/s11165-019-9837-9
Monk, D. H. (2007). Recruiting and retaining high-quality teachers in rural areas. The Future of Children, 155-174. http://doi.org/10.1353/foc.2007.0009
Moon, J., Passmore, C., Reiser, B. J., & Michaels, S. (2014). Beyond comparisons of online versus face-to-face PD: Commentary in response to Fishman et al., “Comparing the impact of online and face-to-face professional development in the context of curriculum implementation.” Journal of Teacher Education, 65(2), 172-176. https://doi.org/10.1177/0022487113511497
National Research Council. (2010). Preparing teachers: Building evidence for sound policy. https://doi.org/10.17226/12882
Nugent, G., Kunz, G., Houston, J., Kalutskaya, I., Wu, C., Pedersen, J., Lee, S., DeChenne, S., Luo, L., & Berry, B. (2016). The effectiveness of technology-delivered science instructional coaching in middle and high school. Working paper. National Center for Research on Rural Education. https://r2ed.unl.edu/workingpapers/2016/R2Ed-WorkingPaper-2016-1.pdf
O’Leary, M., Cui, V., Kiem, M. T., Dang, D. T., Nguyen, G. T. H., & Hoang, K. H. T. (2023). The role of classroom observation in the development and assessment of schoolteachers in Vietnam: A review of national policy and research. Asia-Pacific Journal of Teacher Education, 51(4), 395-407. https://doi.org/10.1080/1359866X.2023.2191307
Ossola, A. (2014). Why kids won’t quit technology. The Atlantic. http://www.ikzadvisors.com/wp-content/uploads/Why-Kids-Wont-Quit-Tech_-Atlantic.pdf
Padrón, Y. N., & Waxman, H. C. (1993). Teaching and learning risks associated with limited cognitive mastery in science and mathematics for limited-English proficient students. In Office of Bilingual Education and Minority Language Affairs (Eds.), Proceedings of the Third National Research Symposium on Limited English Proficient Students. Focus on middle and high school issues (Vol. 2, pp. 511-547). National Clearinghouse for Bilingual Education.
Quinn, F., Charteris, J., Adlington, R., Rizk, N., Fletcher, P., & Parkes, M. (2022). The potential of online technologies in meeting PLD needs of rural teachers. Asia-Pacific Journal of Teacher Education, 50(1), 69-83. https://doi.org/10.1080/1359866X.2020.1849538
Rodgers, W. J., Kennedy, M. J., VanUitert, V. J., & Myers, A. M. (2019). Delivering performance feedback to teachers using technology-based observation and coaching tools. Intervention in School and Clinic, 55(2), 103–112. https://doi.org/10.1177/1053451219837640
Sancar, R., Atal, D., & Deryakulu, D. (2021). A new framework for teachers’ professional development. Teaching and Teacher Education, 101, 103305. https://doi.org/10.1016/j.tate.2021.103305
Schaefer, J. M., & Ottley, J. R. (2018). Evaluating immediate feedback via bug-in-ear as an evidence-based practice for professional development. Journal of Special Education Technology, 33(4), 247–258. https://doi.org/10.1177/0162643418766870
Sexton, D. M. (2018). Becoming a math teacher: Recruitment, preparation, and practice. (ERIC No. ED587885) [Doctoral dissertation, University of California, Santa Cruz]. ERIC.
Simonsen, L., Luebeck, J., & Bice, L. (2009). The effectiveness of online paired mentoring for beginning science and mathematics teachers. Journal of Distance Education, 23(2), 51-68. http://www.ijede.ca/index.php/jde/article/view/592/893
Sims, S., & Fletcher-Wood, H. (2021). Identifying the characteristics of effective teacher professional development: a critical review. School Effectiveness and School Improvement, 32(1), 47-63. https://doi.org/10.1080/09243453.2020.1772841
Sutcher, L., Darling-Hammond, L., & Carver-Thomas, D. (2016). A coming crisis in teaching? Teacher supply, demand, and shortages in the US. Learning Policy Institute. https://learningpolicyinstitute.org/media/179/download?inline&file=A_Coming_Crisis_in_Teaching_REPORT.pdf
Sweigart, C., Collins, L., Evanovich, L., & Cook, S. (2016). An evaluation of the evidence base for performance feedback to improve teacher praise using CEC’s quality indicators. Education and Treatment of Children, 39, 419-444. https://doi.org/10.1080/1046560X.2020.1727622
Tang, A., Li, W., & Liu, D. (2022). The impact of teachers’ professional development in science pedagogy on students’ achievement: Evidence from TIMSS 2019. Journal of Baltic Science Education, 21(2), 258-274.
Tudge, J. (1992). Vygotski, the zone of proximal development, and peer collaboration: Implications for classroom practice. In L.C. Moll (Ed.), Vygotski and education: Instructional implications and applications of sociohistorical psychology (pp.155-172). Cambridge University Press.
Unver, A. O., Okulu, H. Z., Muslu, N., Yilmaz, Y. O., Senler, B., Arabacioglu, S., & Bektaş, O. (2023). The readiness of stakeholders in the scientific inquiry-supported mentoring project. International Journal of Education in Mathematics, Science and Technology, 11(1), 37-55. https://doi.org/10.46328/ijemst.2651
Valiandes, S., & Neophytou, L. (2018). Teachers’ professional development for differentiated instruction in mixed-ability classrooms: investigating the impact of a development program on teachers’ professional learning and on students’ achievement. Teacher Development, 22(1), 123-138. https://doi.org/10.1080/13664530.2017.1338196
Vera, E. M., Heineke, A., Israel, M., Hill, M., Goldberger, N., Hook, K., & Anderson, B. (2022). Learning about English learners: teachers’ and leaders’ perceptions of effective professional development. International Multilingual Research Journal, 16(2), 93-112. https://doi.org/10.1080/19313152.2021.1971474
Vygotsky, L. S. (1978). Mind and society: The development of higher psychological processes. Harvard University Press.
