The Effects of STEM Training on the Academic Achievement of 4 th Graders in Science and Mathematics and their Views on STEM Training Teachers

* This study is part of the corresponding author’s doctoral dissertation. a,** Corresponding Author: Gazi University Faculty Department of Elementary Education, Ankara/Turkey. E-mail: dilber.kaptan@gmail.com b Neşe Tertemiz, Gazi University Education Faculty Department of Elementary Education, Ankara/ Turkey. E-mail: tertemiz@gazi. edu.tr c Adem Taşdemir, Ahi Evran University Education Faculty Department of Elementary Education, Kırşehir/Turkey. Email:ademtasdemir@ahievran.edu.tr Abstract


Introduction
The rapid advances in knowledge and technology in the 21st century require societies to keep themselves up-to-date.Any society that wishes to improve must be productive, and in the 21st century, this happens mostly in the field of technology.It is natural that productive societies will progress and lead others.Therefore, 21st century schools must not only induce academic success, but also equip students with upper-level skills such as critical thinking, problem solving, cooperation, analytical thinking and creativity.As the Turkish educational system is based on exams, students who can solve the highest number of questions in the shortest time can graduate successfully from high schools and gain admission to top universities, but have difficulty finding jobs afterwards.One reason for this is the expectation among employers that their employees will not only possess theoretical information but also know how to reach information, use it in solving problems, and possess other 21st century skills.
In recent years, a new trend known as STEM (Science, Technology, Engineering and Mathematics) in the USA and translated as FeTeMM (Fen, Teknoloji, Mühendislik ve Matematik) into Turkish (Çorlu, Adıgüzel, Ayar, Çorlu & Özel, 2012) emerged in order to prepare students for the business life of the future and increase their interest in careers in the fields of science, technology, engineering and mathematics.Owing to thecontributionof STEM training to both scientific and technological progress, as well as sustainable growth, many developed and developing countries have been creating national policies based on STEM training and investing seriously in this field (Aydagül & Terzioğlu, 2014).A review of Turkey's standing in international science and mathematics tests TIMMS (Trends in International Mathematics and Science Study) and PISA (Programme for International Student Assessment) leads to concerns about the place of STEM education in the country.In TIMMS 2015,Turkish 4th graders ranked 36 among 49 countries.Only 4% of these Turkish students performed at a high competence level, while 43% and 24% performed at a low level in mathematics and science, respectively.The results of PISA 2015 which evaluated science, mathematics and reading abilities of 15-year-olds showed that Turkey scored 425, well below the average score of participant countries (465) in science literacy, which is defined as the ability to engage in scientific thought and phenomena as active citizens.Similarly, Turkish students scored 420 and remained below the average score (461) in mathematics literacy (Taş, Arıcı, Ozarkan & Özgürlük, 2016).For better TIMSS and PISA results, it has been recommended that STEM training should be prioritized (Ministry of National Education (MoNE), 2016).

What is STEM training?
STEM training, which focuses on science and mathematics and combines these two disciplines with technology and engineering, has been defined in various ways.Vasquez, Sneider and Comer (2013) define it as a learning approach which eliminates the usual barriers between science, technology, engineering and mathematics and combines them with real life learning experiences, while Gomez and Albrecth (2014) define the term as a cooperative study philosophy based on practical knowledge which offers students comprehensive and meaningful real life experiences.STEM is based on the belief that science and mathematics achievement is increasingly important and that technology and engineering should be properly integrated (Jorgenson, Vanosdall, Massey & Cleveland, 2014).
When each discipline within STEM is separately examined, it becomes obvious that, on the whole, technology and engineering are not adequately understood.This may be because school curricula are based on science and mathematics, and most people perceive technology solely as digital tools and they perceive engineering as a mere profession (Vasquez, et al., 2013).However, science refers to natural sciences such as physics, chemistry and biology (Jorgenson, et al., 2014); while mathematics is not limited to numbers and operations alone, but is a discipline to be used in daily life; and technology refers to everything created by humans to ease human life (Jorgenson,et al., 2014); and finally engineering refers to a design process.As STEM training is implemented, disciplines should be brought together and integrated to enable students to see the relations between concepts and principles; students should be able to understand how to use their knowledge and skills in daily life; 21st century skills should be emphasized in this process; the tasks given to students should not be easy enough to bore them or too hard to continue; and students should be allowed at the end of the process to explain why they did things in a certain way (Bybee, 2013;Sanders, 2009;Vasquez, et al., 2013).
STEM courses are based on real life problems.In STEM courses, students look for solutions to social, economic and environmental problems.As real life problems are more personal and more important for students' daily lives, they attract students into the topic (Bryan, Moore, Johnson & Roehrig, 2016).Students first define a problem within its limitations, then search it and identify potential solutions via brainstorming, followed by interaction and sharing with each other (Bender, 2017;Jolly, 2017).In this process, students can develop different approaches in order to improve their problem solution skills, creativity, and higher-level thinking skills (Bender, 2017;Moore, Johnson, Peters-Burton & Guzey, 2016).This nature of STEM courses requires problem solution and project-based learning in the instructional process.
Becoming increasingly more important, STEM training is starting to be recognized in Turkey as well.The dissapointing results obtained in international studies such as TIMSS and PISA and the alarmingly small number of students in the advanced bands reveal the need for emphasizing higher level thinking skills such as critical thinking, cooperative learning, problem solution in the country.In addition to advancing their knowledge in each field of STEM, it is crucial for students to also see the relations between them and thus become creative, analytical and critical thinkers and problem solvers.Improving these skills, especially in primary and pre-school, increasing the interest in future STEM careers (Dejonckheere, Wit, Keere & Vervaet, 2016).It is worth noting that STEM training is particularly influential in elementary school (Becker & Park, 2011;Murphy & Mancini-Samuelson, 2012;Lamb, et al., 2015), and the lack of studies in Turkey at this level means that the results of the present study will contribute significantly to the literature.It is also expected that the findings will benefit the field of STEM, which is new in the country, and encourage teachers to get to know and implement it more commonly.
The study aimed to identify the effects of STEM practices on 4th graders' science and mathematics achievement and their views about STEM training.Therefore, the following questions were probed in the study: 1. Is there a significant difference between the science achievement pretest-posttest mean scores of experimental groups 1 and 2 (STEM activity groups) and the control group (regular curriculum group)?
2. Is there a significant difference between the mathematic achievement pretest-posttest mean scores of experimental groups 1 and 2 (STEM activity groups) and the control group (regular curriculum group)?
3. What are the views of students experimental group regarding STEM training?

Methodology
This study is a quasi experimental design.The research design, study group, data collection tools, experiment process and data analysis are as follows:

Research design
This quasi experimental pretest-posttest control group study examined the effects of STEM training on elementary pupils' academic achievement.Quasi experimental design is preferred when the level of control required by real experimental design cannot be achieved (Karasar, 2012, p.99).Qualitative data were also gathered by focus group interview technique to support the quantitative data.

Study group
The study group includes 4th graders from two different state elementary schools with a middle socioeconomic profile located in Niğde.The middle socioeconomic profile was preferred in order to exclude the extremes which might have influenced the results and to investigate the data in average schools.
The study group was identified by using the group matching method.The method involves selecting groups with equal and/or close mean scores (cited from Eckhardt & Ermann by Büyüköztürk, 2014, p. 22).For group matching in the study, 4th graders from 8 different classes in two elementary schools were given the "Science Achievement Test" and the "Mathematics Achievement Test".Based on the data obtained, three equivalent groups were selected.The criterion sampling method was used when the students to be interviewed were determined.A total of 12 students were selected from the experimental groups 1 and 2 according to the scores obtained from the science and mathematics achievement tests.The focus group interview was conducted in two sessions with two group of 6 people.

Data collection tools
The study utilized achievement tests developed to measure students' science and mathematics course achievements and an interview form developed to identify stu-dent views about STEM training.

Science Achievement Test (SAT). The "Science Achievement
Test" was developed to identify the learning level of students in the science course.For validity purposes, the views of a class teacher and an elementary education expert (with a doctoral degree in the field of science education) were obtained, and their feedback was used to create a draft form.For reliability purposes, the draft form was implemented on 118 5th graders; difficulty and distinguishing indices were identified for each item; and faulty or deficient questions were spotted.After corrections, the KR-20 reliability coefficient of the 20-item Science Achievement Test was .80;its mean score difficulty was .65 and standard deviation was 4.33.Tekin (1996) writes that the mean score difficulty of a test should be approximately .50 and KR-20 value should be close to 1. Tan and Erdoğan (2001) argue that a KR-20 reliability coefficient that is lower by "0.50" is enough for 10-15 item multiple choice tests.Therefore, SAT is reliable enough to be used in the study.

Mathematics Achievement Test (MAT).
This test was developed to measure 4th graders' learning levels in the mathematics class.In order to ensure the appropriateness of the contents and quality of the draft MAT to measure the acquisitions of 4th grade, the views of a class teacher and an elementary education expert (with a doctoral degree in the field of mathematics education) were taken, and the fit between the experts was compared.For the reliability of the tool, the draft MAT was implemented on 117 fifth graders who were outside the study sample.The difficulty and distinguishing indices of each item were determined and faulty or deficient items were cleared.The KR-20 reliabilitycoefficient calculated from the item and test analyses was .80; the mean score difficulty of the test was .65 and its standard deviation was 4.33.Therefore, MAT was reliable enough to be used in the study.
A semi-structured interview form was prepared to identify students' perception about STEM activities and STEM practice.In the process of preparing the form, firstly the literature was searched and form a draft was prepared in the direction of the obtained data.The questions are prepared so that students can not be bored and can be answered easily.The draft form, which has been corrected according to expert opinion, consist of 8 questions.

The experimental process
Prior to designing theSTEM activities, the literature was surveyed.In line with the STEM lesson plan criteria prepared by Maryland State University Department of Education (2012), a total of 6 lesson plans were drawn with 14 science and 10 mathematics objectives.In order to ensure the appropriateness and feasibility of the lesson plans in an elementary school, views were obtained from five experts (two class teachers, one science teacher, one science education expert and one elementary education expert).Later, the following activities were finalized: 1. Activity 1: Let's Make a Street Lamp 2. Activity 2: Let's Get Heard 3. Activity 3: We Build Our City 4. Activity 4: Microbe Hunters 5. Activity 5: Clean Enough to Drink? 6.Activity 6: Let's Light Up Our City In experimental groups 1 and 2, classes were implemented with STEM activities while the control group followed the textbooks and worksheets recommended by the Ministry of Education.The instructional process was run by the class teacher in Experimental group 1 and by the researcher in Experimental group 2. The control group was taught by the class teacher.
Prior to the implementation, the class teacher of Experimental group 1 was informed in detail by the researcher about the goals and importance of STEM and the lesson plans.The instructional process in Experimental group 2 was run by the researcher.In line with the lesson plans, the students were presented with a global or daily life problem.They first articulated the problem, then listed their personal solutions and finally decided in groups on the most appropriate solutions.Based on their solution proposals, they decided which of the materials provided by the teacher they would need, and in what quantity.Following this, groups undertook their designs.In the next stage, the designs were tested, amended, if necessary replanned and re-designed, and thus finalized.During the process, groups prepared presentations about their work and presented them during the evaluation stage.
In addition, focus group interviews were held to identify the views of 12 experimental students with different achievement levels about the experiment.This was done to reveal whether the students had internalized the engineering design process and their views about STEM activities throughout the process.
The process in the control group was run by the class teacher.They used textbooks printed by publishers endorsed by the ministry.Throughout the process, observation was done in the control group to check whether the teacher was indeed following the textbook.

Data analysis
SPSS 15.0 (Statistical Package for Social Sciences) was used in the study to analyze the quantitative data obtained from the tools.The Science Achievement and Mathematics Achievement pre and post test mean scores of the groups were compared by using Two Factor ANOVA for Mixed Measurements and the significance level was set at 0.05.At the same time, interview questions were analyzed by using the content analysis technique.

Findings
This section includes findings from the "Science Achievement Test", "Mathematics Achievement Test" and interviews with experimental students about STEM training.

Findings about the achievement tests
Findings about the experimental and control groups' Science Achievement Test (SAT) and Mathematics Achievement Test (MAT) pre test-post test mean scores are given below.
Science achievement test results of the groups.The Science Achievement Test (SAT) results of experimental and control groups are shown in Tables 1 and 2. Table 2 shows the results of the two-factor ANOVA performed to identify whether the increase in the science achievement posttest scores of experimental and control students differed significantly.
Table 2 shows that there was a significant difference in the science achievement of two STEM experimental groups and the control group from the beginning to the end of the experiment, F(2, 65)= 17.20, p<.05.In other words, a significant increase was observed in student achievement in all three groups.This finding shows that teaching via STEM activities and via ministry endorsed textbooks have different effects on increasing science achievement.The Bonferroni test conducted to see the groups that differed showed that a significant difference existed between Experimental group 1 and the Control group in favor of the former, and also between Experimentalgroup 2 and the Control group in favor of the former.It may therefore be stated that STEM training, offered by a researcher or class teacher, improves scienceachievement more than regular instruction.
Mathematics Achievement Test Results of the Groups.Mathematics achievement Test (MAT) results of experimental and control groups are presented in Tables 3 and 4. Table 4 shows the results of the two-factor ANOVA performed to identify whether the increase in the mathematics achievement pretest and posttest scores of experimental and control students differed significantly.
As can be seen in Table 4, the mathematics achievement of experimental and control students changed from the beginning of the experiment to the end.There was no

Qualitative findings on experimental students' STEM training views
This section presents the findings of focus group interviews with 12 experimental students with different achievement levels who were selected by criterion sampling.The aim of conducting focus group interviews was to support other findings pertaining to the experiment, and to reveal the strengths and weaknesses of the experimental process from the students' perspectives.Data from the interviews were analyzed based on the questions.Student views about what their previous classes were like, their opinions on classes based on STEM activities, and the easy and challenging parts of the activities can be seen in Table 5 as views before, during and after STEM training.Their thoughts about interdisciplinary education, group work and what they would like their future training to be like are presented in Figure 1 through Figure 3.
Table 5 shows that the experimental students were previously taught via books (6) and teacher-centered approaches (5).In this experiment, students voiced opinions about the activities (Street lamp, purifier, electric circuits, etc.) and particularly emphasized in-class practices.While 12 students referred to the activities as challenging, 8 claimed that they were easy.In addition, they frequently stated the positive opinions that they liked group work (5), made joint decisions (5) and cooperated with each other (5).This shows that particularly group work created a positive effect on students during STEM activities.
After the experiment, students mostly referred to the activities as fun (24).They also mentioned projects (8), having learned (8), wanting to learn with STEM in the future (7) and learning both science and mathematics at the same time (6).One commonly emphasized concept after the study was doing experiments.In addition, some students emphasized that they may choose engineering as a future career both during and after the experiment.This suggests that STEM training encourages students to steer towards technical sciences as a profession.
Below are examples of student responses from different achievement levels.
" …previously our science and mathematics courses only involved writing.Now we do projects, and feel like we're working in a lab..." (Ramazan) "…during activities we had fun and learned and became brighter."(Berat) "…it was more fun.We built models and prepared presentations.We learned how to make presentations.We learned how to work like an engineer."(Sıla)  As can be seen in Figure 1, experimental students stated that integrating science and mathematics enabled them to have discussions during the process, have fun, felt better, and encouraged decision-making.They also stated that they practised engineering and improved their technology skills.
Below are sample responses from students with different achievement levels about integrating science and mathematics courses.Findings about student responses to the question "How did you feel about group work during activities?" are presented in Figure 2. Students stated that they mostly enjoyed the environments of STEM training; they completed activities faster and easier; and made joint decisions in the process.At the same time, they emphasized cooperation and class discussions.However, some students also said that individual work would be better (3).Overall, it is possible to state that group work had a positive effect on learning.
Below are the views of students from different achievement levels about group work "Group was a good idea, everybody chipped in their ideas.
If they were good, we used them.If not, we didn't.We decided together."(Esra) "Individual work instead of group work would be better as everyone would do something unique."(Asude) "I loved group work.I wouldn't be able to do this on my own.My group mates helped me.Their ideas helped with the model."(Berk) Figure 3 shows the findings about responses to the question "What would you like your future courses to be like?Why?"As can be seen inFigure 3, students frequently stated that they would like their future courses to be similar (to STEM activities).They also emphasized processes such as having more time to deal with the topics, making designs, creating projects, engaging in activities for manual skills development.In addition, students emphasized being active as opposed to merely listening and stated that the instructional process should be enjoyable.
Below are sample responses from students with different achievement levels about how they would like their courses to be in the future.
"I had a lot of fun.We remember better when we do things and it's more fun."(Mükremin) "I'd want future courses to be similar.We learned both science and mathematics.Meanwhile Turkish also came into play, such as reading.Sometimes social studies also came into play.Therefore we can do it, and it's fun."(Kemal Mert) "I'd want all courses to be like this because we learn more and we have fun with the projects."(Mustafa) It may be stated that experimental students with different achievement levels generally have positive views about STEM practices, enjoy interdisciplinary problem solution and project-based work, can learn more easily, may choose engineering as a career in the future, and would like to see STEM-like activities in their future science and mathematics courses.

Results and Discussion
This study was conducted to identify the effects of STEM training on elementary pupils' science and mathematics achievement levels and find out their views about STEM.There were two experimental groups and a control group.
In the first and second experimentalgroups, STEM activities were implemented by the class teacher and the researcher respectively, while the control group followed the regular instructional program.The present study started with equivalent experimental and control groups, as documented by their science and mathematics pre test results.At the end of the study, both science and mathematics achievement scores of the experimental and con-trol groups differed significantly, in favor of the experiment group.This finding mirrors those of previous studies (Ceylan, 2014;Judson, 2014;McClain, 2015;Olivarez, 2012;Pekbay, 2017;Wade-Shepherd, 2016;Yamak, et al., 2014;Yıldırım and Altun, 2015).
When science and mathematics achievement are examined separately, it is evident that the increase in science achievement was higher than that in mathematics for both experimental and control groups at the end of the study.This may have been because experimental students were not used to the interdisciplinary approach, and may have at times perceived the course as purely science despite the links between the two subject areas.Based on researcher observations, the limited treatment of topics only in the mathematics course in the control group, and the lack of a context may have resulted in a lower mathematics achievement level in this group.However, it may still be argued that treating different disciplines together must influence student achievement.Wade-Shepherd (2016) attempted to identify the effects of a STEM curriculum on secondary school students and found that students who received STEM training had significantly higher science and mathematics achievement scores than those who received traditional instruction.Having also studied secondary school students, Olivarez (2012) concluded that a STEM program elevates success in mathematics, science and reading.
An increase in the science achievement of the experimental group both within and across groups was also the case in Wendell and Rogers' ( 2013) study with elementary school pupils.They found that a curriculum based on engineering design enhanced elementary school pupils' science knowledge.Similarly, another study conducted with high school students concluded that the academic performance brought by STEM increased students' science and mathematics achievement (Wosu, 2013).Yet another study by Vollstedt et al. (2007) at the secondary school level aimed to enhance student knowledge in STEM fields by using robots, and found an increase at the end of the study in the fields of science, mathematics, engineering, robotics, and computer programming.Ricks (2006), as a result of his study during a summer camp based on STEM activities, also concluded that they advanced secondary school students'science knowledge.
While no STEM studies at elementary school level have been conducted in Turkey, those conducted in other educational levels in the country obtained parallel results to those of international studies.Yamak et al. (2014) used STEMactivities in a project over the summer term, and found that these activities enhanced secondary school students' science achievement.Similarly, different studies conducted with secondary students showed that instruction with STEM activities was found to enhance students' science achievement (Ceylan, 2014;Yıldırım, 2016).Therefore, the findings of the present study are parallel to those listed above.
Considering students' mathematics achievement, even though their achievement in this field may have been lower than that in science, the mean scores of the groups still differed significantly.By the end of the process, the mathematics achievement of Experimental I, Experimental II and control group students differed significantly, in favor of the first two groups.Instruction built upon STEM based activities in the experimental group increased students' mathematics achievement.In the control group, on the other hand, students'achievement levels decreased.
The Elementary School Mathematics Education Program states that students should be able to make links between their own lives and mathematics and construct mathematical meaning through concrete experiences (MEB, 2015).In this study, experimental students were exposed to real life problems in the activities, made links with their previous knowledge to solve the problems, and learned through concrete experiences.Observations of the control group, however, showed that classes in this group were taught in an abstract way, by not making many associations.This may have led to an increase in experimental students' achievement while it resulted in a decrease in that of the control group.Similarly, aiming to explore the effects of STEMtraining on 4th graders' mathematics achievement, McClain (2015) concluded that the mathematics achievement of students at STEM schools was significantly higher than that of students in other schools.Judson (2014) also concluded that STEM training increased elementary school students' mathematics achievement.
The fact that mathematics achievement in this study was not as pronounced as science achievement, coupled with the findings of Kager (2015) and Tolliver (2016), suggests that the influence on mathematics achievement was not as significant.
The views of experimental students on STEM training showed that they generally felt positive, enjoyed the practices and developed a positive attitude towards STEM fields.This may be attributed to the fact that classes were learner-based and students actively participated in the sessions and created things.The results were similar both at the elementary school (Saleh, 2016) and secondary school levels (Kager, 2015;Pekbay, 2017).Despite these positive views, Talbot (2014) found that extracurricular STEM activities with elementary school pupils did not change much in their attitudes.This may have been because those pupils also participated actively in other courses and found this to be common practice.
The experimental group students who learned with STEM activities stated that they advanced both their mathematics and science knowledge.Worker and Mahacek (2013) concluded that after-school STEM training involved students actively in science, engineering and technology, and strengthened their conception of science and mathematics in the design process.The students in their study stated that they might choose engineering as a future career.Tseng, et al. (2013) wrote that extracurricular programs might encourage students to become interested in engineering fields as a future profession, and that the knowledge to be gained in these programs would benefit them in their future careers.Şahin (2013) also stated that students who participated in science fairs and STEM clubs tended to more commonly choose STEM areas after their secondary education.Similarly, Apedoe, et al. (2008) concluded that STEMtraining encouraged students to consider engineering as a future profession.Studies in Turkey likewise showed that students wished to go into these fields in the future (Baran, et al., 2016;Gökbayrak and Karışan, 2017;Gülhan & Şahin, 2016).
In the interviews, experimental students stated that they wished to see more STEM activities in their future courses.This finding is in line with Gökbayrak and Karışan's (2017) results.They also found that 6th graders wanted to have more STEM activities and found such courses to be more motivating and mentally challenging.The findings of this study revealed that classes based on problem solution and project-based integrated STEM activities enhanced elementary school pupils' science and mathematics achievement, and enabled them to sense these fields as a whole.At the same time, it was found that students got bored when they did not actively participate in courses; they enjoyed classes based on STEM activities; and they might choose one of these fields for their future careers.
Therefore, it appears that STEM training may be particularly beneficial at the elementary school level.Considering the positive effects of STEM training and the problems that the teacher experienced throughout the study, it is recommended that class teachers receive both pre-service and in-service training on how to use STEM activities.
In addition, considering the significant effect of integrated STEM training at elementary school level (Becker & Park, 2011;Murphy and Mancini-Samuelson, 2012;Lamb, et al., 2015) and the lack of such studies in Turkey, it would be useful to emphasize STEM training studies in elementary schools.Future studies may also focus on the effects of STEM training on higher level thinking skills.
Figure 1 below shows findings from student responses to the question "How did you feel integrating science with mathematics?"

Figure 2 .
Figure 2. Student Views about Group Work

Figure 3 .
Figure 3. Student views about how they would like their future courses to be

Table 1 .
Experimental and control groups' SAT mean scores and standard deviations

Table 1 ,
Experimental group 1 (where STEM activities were taught by the class teacher) had the science achievement mean score of 9.04 before the experiment, which rose to 15.38 afterwards.The second STEM training group, Experimental group 2 (where classes were taught by the researcher), had the science achievement mean score of 10.36 prior to the experiment, which rose to 15.92 afterwards.The science pre and post test mean scores of the control group, which followed the regular curriculum specified by the Ministry of Education, were 9.13 and 10.31, respectively.Considering that the highest possible score from the test is 20, it may be stated that the science achievement test mean scores of both experimental and control students increased in the post test.

Table 3 .
Mathematics Academic Test (MAT) Mean Scores and Standard Deviations

Table 2 .
ANOVA results on science achievement test (SAT) pretest-posttest mean score

Table 4 .
Mathematics achievement test (MAT) pre test-post test ANOVA results

Table 5 .
Experimental students' views about STEM training before and after the experiment